Opportunities and Challenges When Using the Electronic Health Record for Practice-Integrated Patient-Facing Interventions: The e-Assist Colon Health Randomized Trial

Background Even after a physician recommendation, many people remain unscreened for colorectal cancer (CRC). The proliferation of electronic health records (EHRs) and tethered online portals may afford new opportunities to embed patient-facing interventions within clinic workflows and engage patients following a physician recommendation for care. We evaluated the effectiveness of a patient-facing intervention designed to complement physician office-based recommendations for CRC screening. Design Using a 2-arm pragmatic, randomized clinical trial, we evaluated the intervention’s effect on CRC screening use as documented in the EHR (primary outcome) and the extent to which the intervention reached the target population. Trial participants were insured, aged 50 to 75 y, with a physician recommendation for CRC screening. Typical EHR functionalities, including patient registries, health maintenance flags, best practice alerts, and secure messaging, were used to support research-related activities and deliver the intervention to enrolled patients. Results A total of 1,825 adults consented to trial participation, of whom 78% completed a baseline survey and were exposed to the intervention. Most trial participants (>80%) indicated an intent to be screened on the baseline survey, and 65% were screened at follow-up, with no significant differences by study arm. One-third of eligible patients were sent a secure message. Among those, more than three-quarters accessed study material. Conclusions By leveraging common EHR functionalities, we integrated a patient-facing intervention within clinic workflows. Despite practice integration, the intervention did not improve screening use, likely in part due to portal-based interventions not reaching those for whom the intervention may be most effective. Implications Embedding patient-facing interventions within the EHR enabled practice integration but may minimize program effectiveness by missing important segments of the patient population. Highlights Electronic health record tools can be used to facilitate practice-embedded pragmatic trial and patient-facing intervention processes, including patient identification, study arm allocation, and intervention delivery. The online portal-embedded intervention did not improve colorectal cancer (CRC) screening uptake following a physician recommendation, likely in part because portal users tend to be already highly engaged with healthcare. Relying on patient portals alone for CRC screening interventions may not alter screening use and could exacerbate well-known care disparities.


Introduction
Despite multiple effective screening tests, colorectal cancer (CRC) screening remains underutilized relative to other cancer-screening tests. 1 Our research has shown that a driving factor behind this underutilization among insured individuals is the gap that exists between physician recommendation and patient receipt of care. While the overwhelming majority (93%) of patients who are due for CRC screening receive a physician recommendation during routine primary care office visits, only 54% of patients are screened in the following year. 2 The gap in adherence to CRC screening following a physician recommendation is likely fueled by multiple factors, including suboptimal patient-provider communication during screening recommendations and inadequate logistical support in completing screening once patients leave the physicians' office. [3][4][5][6][7][8][9][10][11][12][13][14][15][16][17] How to support patients in obtaining evidence-based screening (or other services) once they have a physician recommendation remains a challenge that has implications not only for patient well-being but also for the organizations responsible for delivering their care. Using individual health navigators holds promise, especially for low-literacy patients, but costs associated with such ''high-touch'' interventions limit scalability. 18,19 Numerous prior studies have found that stand-alone patient decision aids result in improvements in patient screening knowledge, risk perceptions, and related outcomes but also limited (if any) changes in screening behaviors [20][21][22][23][24] and impracticalities for practice integration. [25][26][27][28] Similarly, while patient reminders and the removal of structural barriers can increase screening use, such interventions remain disconnected from existing clinic processes 18,[29][30][31][32][33][34][35][36][37][38][39][40] and often are not sustainable outside a research environment. The effectiveness and impact of previously tested CRC screening interventions may be limited, in part, by such practice integration challenges. The proliferation of electronic health records (EHRs) and accompanying patient portals may afford new opportunities to economically engage and support patients at the time of a CRC screening or other physician recommendation for care in ways that connect with existing clinic processes yet extend beyond the boundaries of traditional office visits.
Using the patient portal functionality now commonly available within EHRs, we developed an EHR-embedded patient-facing intervention, e-Assist colon health, to assist in obtaining care following a physician recommendation for CRC screening. To ensure the intervention's fit with practice, we partnered with clinicians and other health system staff during the design and implementation of the intervention. 41 We hypothesized that people who received the e-Assist intervention would be more likely to complete CRC screening (primary outcome) relative to those who received links to health education documents on CRC and CRC screening (enhanced usual care). In this article, we report findings from the randomized trial used to evaluate the intervention's effectiveness and implementation.

Study Design and Trial Eligibility
For the primary outcome, we evaluated the intervention's effect on CRC screening using a 2-arm practiceembedded pragmatic trial. The trial was conducted within a large integrated health system servicing Detroit, Michigan, and the surrounding tricounty area. Trial participants included insured patients aged 50 to 75 y who were eligible for CRC screening (as recommended by the US Preventive Services Task Force at the time of study recruitment 42 ). As previously reported, 43 the trial targeted adults of average risk for CRC and thus excluded those with a personal or family history of CRC, colon polyps, inflammatory bowel disease, familial adenomatous polyposis, or hereditary nonpolyposis as well as those for whom an accelerated screening schedule had been documented in their medical record. Because the intervention was designed to address common gaps in patient-provider conversations at the time of CRC screening recommendation, trial eligibility was also limited to those with a primary care office visit that included a physician recommendation for CRC screening (as indicated by an open access colonoscopy referral or stool test order). As described below, because the intervention was embedded within the EHR, trial enrollment was also limited to patients with an activated portal account. All aspects of the trial were approved by the health system's Institutional Review Board.

Identifying Trial Eligible Patients
Two common EHR tools, the patient registry function and health maintenance flags, were used to identify and allocate study-eligible patients to the appropriate study arm. For study purposes, the EHR registry function was used to identify patients who, should they have a primary care visit with a recommendation for CRC screening, would meet the study inclusion criteria. As such, when a primary care physician electronically signed the notes, or ''closed'' an office visit encounter within the EHR, if that visit was with a patient previously identified as study eligible (i.e., the patient was included within the EHR registry) and included a stool test order or a colonoscopy referral (i.e., physician recommendation for CRC screening), the patient became study eligible.
An additional EHR tool, a health maintenance flag, was used to assist with study randomization. Because it was not technically feasible to randomize patients within the EHR in real time as they became study eligible, as we have described elsewhere, 43 we randomized all patients included in the EHR registry, storing their allocation as a health maintenance flag that was not visible to clinicians or others accessing the EHR for patient care. This hidden flag was used to electronically allocate the appropriate intervention or enhanced usual care material to patients once they became study eligible via a primary care physician electronically signing an office visit encounter that included a CRC screening recommendation.

Delivering Study Material
When a patient became study eligible, a series of programming rules automated the sending of study recruitment material and either the intervention or enhanced usual care material. This was achieved using the best practice advisory/alert (BPA) system within the EHR. While BPAs are generally used to send automated notifications or reminders to alert physicians and other clinical staff to important patient needs, they can be made silent or not appear as a pop-up message or task when clinical staff are using the EHR. Instead, they can operate in the background, routing notifications to specific user-owned in-boxes that can be checked at the user's discretion.
For study purposes, when a physician electronically signed (i.e., closed) an office-visit note that contained a CRC screening order with a study-eligible patient, BPA programming logic led to a secure inbox message being sent to that patient's online portal account. While the content of the secure message inviting the patient to participate in the trial was identical regardless of randomization allocation, the programming logic behind the BPA used information on the patient's randomization allocation (as stored in the study's health maintenance flag) to send patients in the 2 trial arms a different message attachment. The attachment was an online questionnaire that contained the informed consent material, a 6-item pretrial survey, and either the e-Assist intervention or links to the enhanced usual care material. The informed consent and pretrial survey material embedded within the attached questionnaire were identical regardless of study arm. The content of the remainder of the attached questionnaire differed by study arm (i.e., either the e-Assist intervention content or links to enhanced usual care material) and could be accessed only upon study consent.

Intervention Design and Content
As described in detail elsewhere, 43 we used the health belief model 44 to guide the overall intervention design, including the use of a physician recommendation for CRC screening as the cue for engagement with the intervention, and self-determination theory to guide the overall tone of written messages to ensure they were autonomy supporting. 45 People allocated to the intervention received information specifically designed to complement typical physician office-based recommendations for CRC screening. As such, e-Assist content filled 3 informational gaps that patients endorse as important yet frequently missing from patient-physician office visit discussions of CRC 3-17 : 1) information central to informed and shared decision making (i.e., information on when CRC screening is recommended, the different screening modalities and associated benefits/risks, and comparing available screening tests), 2) addressing barriers to screening, and 3) an understanding of the logistics of testing and assistance completing testing. The intervention content, all of which we were able to embed within the questionnaire function within the EHR, included text, pictures, and links to short videos. All users were presented with information reiterating that their doctor had recommended CRC screening and the benefits of CRC. User responses to questions facilitated progression through intervention content and the branching logic that determined what content was visible. The latter enabled users to select and filter what information was viewed and thus enabled the user to tailor content based on relevance and interest. For example, to ensure message salience, embedded questions inquired about the user's readiness to be screened. 46 Those indicating they were ready to be screened were provided with tips for completing their preferred screening test, while those indicating they were not ready to be screened were offered suggestions for how to overcome common barriers to screening. Similarly, users who indicated they were undecided about how to screen were provided with information regarding the pros and cons of different test options. Additional questions further determined the user's preferences for additional information and thus determined the level of detail regarding those options that users saw.
Any trial participant who did not indicate they had completed CRC screening at the end of the questionnaire received a follow-up module. The content of the followup module was tailored to information collected during the initial module regarding screening test preferences and readiness to screen. Content was designed to use messaging consistent with a motivational counseling approach, 47,48 to facilitate a CRC screening decision among undecided users, and to assist in test completion among those expressing a readiness to screen. Most follow-up modules were sent 2 wk after the initial module (questionnaire) was submitted. The trial's protocol paper provides additional details regarding the intervention's design and content. 43 People allocated to enhanced usual care also received an EHR questionnaire. The content within that questionnaire was identical to that received by those in the intervention group in terms of consent material, pretrial survey items, and introductory material reiterating that CRC screening had been recommended for them and outlining the benefits of screening, but instead of the text, pictures, and short videos, it contained links to 4 webpages that were stored within the health system's patient portal's health information library. That material, distributed by Healthwise at the time of the study, 49 contained educational information on the etiology, symptoms, and treatment of CRC as well as screening modalities and the interpretation of screening results. The 2-wk follow-up module sent to trial enrollees in this arm included a welcome screen and a link to the National Cancer Institute's CRC screening website. Recipients of this information were provided with links to the entire packets of information, regardless of relevance or preference.

Data Sources and Measures
The primary outcome for the trial was receipt of CRC screening as documented in the EHR. We considered any documented receipt of CRC screening, regardless of physician recommendation or modality (i.e., colonoscopy or stool test), in the 12-mo period following the date of the office visit encounter that resulted in the patient's trial eligibility.
In addition to compiling CRC screening receipt from the EHR, we also used data available within the EHR to obtain preenrollment characteristics of trial participants. These included patient age, race, sex, marital status, insurance coverage, Charlson Comorbidity Score, 50 and whether English was the person's preferred language.
Among those completing the pretrial survey, participants' responses were used to further describe study participants at the time of enrollment and to evaluate hypothesized effect modification. Survey items included a measure of health literacy, 51 CRC screening decision stage, 52 decision-making preference, 53 CRC-related worry, 54 perceived CRC susceptibility, 55 and CRC screening history. 56 In addition to some broad measures of program reach (e.g., estimates of screening-eligible patients who do v. do not have active portal accounts) for which we have previously published results, 57 we used the benchmark reporting function within the EHR to track two, more narrow, measures of reach among trial eligible patients.
Among patients in the EHR registry (and thus due for CRC screening and with an activated portal account), we tracked the proportion who became study eligible by virtue of having a primary care visit with a CRC screening order and, among those, the proportions who opened the study messages sent to their portal inbox, interacted with the questionnaire attached to that message, and consented to study participation. These custom reports could be generated at any time by research staff, enabling realtime participant tracking. From these reports, we developed 2 measures to assess the degree to which the intervention or enhanced usual care material reached the study's target population. 58 First, among EHR registry patients (i.e., those of average risk and due for CRC screening with an activated portal account), we derived the proportion sent a secure message containing the intervention or enhanced usual care material, and second, among those sent a secure message, we derived the proportion who opened the message and thus saw that they were invited to the trial.
In addition to constructing these measures, for the individuals who consented to trial enrollment, we were able to compile additional information regarding implementation timing by using the date/time stamp associated with their interactions with study material. For example, we tracked the number of days between the office visit that triggered the person's study eligibility (i.e., the date of CRC recommendation) and when the secure message that contained the study invitation was sent. Gaps in days between these 2 dates are due to routine variability in workflow processes, because not all physicians electronically sign the notes from each of their office visit encounters on the day of the visit. Second, we tracked the time in days between when a secure message was sent (i.e., the date the physician electronically signed the note) and when it was accessed by the patient in the online portal. Gaps in days between these 2 dates are due to routine variability in the frequency with which a person chooses to view secure messages and/or access his or her portal account.

Statistical Analysis and Power
Continuous and categorical pretrial participant characteristics are summarized as mean (s) and counts (%), respectively. We estimated the effects of the characteristics on study arm assignment and intervention exposure. The effects of continuous, binary, and categorical characteristics were tested by Wilcoxon rank-sum test, Pearson chi-squared test, and Fisher exact test, respectively.
For the primary evaluation of effectiveness, we used an intent-to-treat (ITT) analysis among all people consenting to study participation. For the primary outcome, EHR-documented CRC screening receipt within 12 mo, we estimated the ITT effect by a 2-level hierarchical logistic regression model in which patients are nested within ordering physicians. 59 As a secondary assessment of effectiveness, we limited the study sample to those trial enrollees who completed the pretrial survey and thus were exposed to the intervention or enhanced usual care material. With N = 1,800 patients seen by 150 physicians, we estimated having 0.86 power to detect a 7% change in CRC screening rates between the intervention and enhanced usual care arms (66% v. 59%, respectively). 43 We also tested whether health literacy, decision stage, and decision-making preference moderated the intervention's effectiveness, as planned by study protocol. 43 The significance of effect moderation was tested by likelihood ratio tests. The latter could be tested only among those completing the baseline survey (i.e., the exposed sample).

Study Participants
Between June 14, 2017, and September 15, 2019, 6203 people who were prerandomized to the e-Assist intervention or enhanced usual care trial arm had a primary care office visit with a physician order for CRC screening and thus became trial eligible (Figure 1). Among these, 4,761 accessed the trial invitation that was sent via patient portal message, and 1,825 consented to trial participation (919 in the e-Assist intervention arm and 906 in the enhanced usual care arm). A total of 170 primary care physicians electronically signed 1 or more of the CRC screening orders with a person who consented to trial participation.
On average, participants who consented to trial participation (N = 1,825; i.e., the ITT sample) were 59.7 y of age, 62.5% were female, and 30.1% were Black ( Table  1). The proportion of trial enrollees who were married differed significantly by study arm, with 68.5% of those enrolled in the intervention arm married compared with 63.0% in enhanced usual care. In addition, those assigned to the intervention arm were significantly more likely to have a recommendation for colonoscopy screening (as opposed to stool testing; 83.4% v. 77.8%) at the time of trial enrollment. No other pretreatment differences were observed across the 2 study arms.
Among those consenting to trial participation, 1419 (78%) completed the baseline survey and were therefore exposed to the intervention/enhanced usual care material. This included 720 (78.3%) of the patients in the intervention arm and 699 (77.2%) of the patients in the enhanced usual care arm. With the exception that women who consented to trial participation were more likely to go on to complete the baseline survey and therefore be exposed to the intervention/enhanced usual care material, we did not find statistically significant differences among those who consented to trial participation by exposure status (Supplementary Appendix Table A-1.).
As within the ITT sample, among the exposed sample, pretrial characteristics did not differ significantly across study arms, except for marital status (68.6% married in the e-Assist arm v. 60.9% married in the enhanced usual care arm) and receipt of a colonoscopy referral, which remained significantly higher in those allocated to the intervention compared with enhanced usual care (84.2% v. 77.3%, respectively; data not shown). Pretrial survey responses among the exposed sample did not differ by study arm (Table 2).

Intervention Effectiveness
Sixty-five percent (65.1%) of trial enrollees who consented were screened for CRC in the 12 mo following the office visit that triggered their trial eligibility (i.e., a primary care office visit with a CRC screening recommendation), 65.7% in the intervention and 64.5% in the enhanced usual care arm. Most of these screenings occurred within the first 6 mo after that office visit: 47.1% of trial enrollees were screened within 3 mo of their office visit and 60.1% within 6 mo of their office visit. This is in comparison with an overall CRC screening rate of 62% among the health system's general population. Among those exposed to either the intervention or enhanced usual care material, screening rates were 67.8% in the intervention-exposed group and 66.7% in the enhanced usual care exposed group. Most screenings within the exposed groups similarly occurred within 6 mo of their office visit, 49.8% within 3 mo of their visit, and 62.2% within 6 mo. None of these differences were statistically significant (i.e., P . .10), nor were 12-mo screening rates (the study's primary outcome) statistically different once we adjusted for pretreatment covariates that differed significantly across the 2 study arms ( Table 3).
In addition, within the exposed sample, we found no statistically significant effect moderation by any of the 3 a priori hypothesized patient characteristics (Table 4). It should be noted, however, that 83.9% of participants reported either having been screened (2.9%) or intending to be screened for CRC in the next 6 mo (81.0%) at the time they completed the baseline survey (i.e., before being exposed to the intervention or enhanced usual care material; Table 2). Similarly, almost two-thirds of exposed participants endorsed being of high health literacy and preferring to be responsible for making decisions regarding screening ( Table 2).

Intervention Implementation
We randomized a total of N = 19,745 people who, per data contained in the EHR, were insured, aged 50 to 75 y with an activated online portal account, and of average risk and due for CRC screening. Between June 14, 2017, and September 15, 2019, almost one-third (n = 6,203 or     Variance of physician-specific random effect; ICC, intraphysician correlation coefficient t=(t + p 2 =3). To enable model convergence, the decision stage was grouped into 3 categories (1: screened within the past 6 mo, 2: intent to screen within 6 mo, and 3: otherwise [i.e., intent to screen, but not within 6 mo; undecided regarding screening; or decided against screening]). 31.4%) became study eligible because they had a primary care visit with a recommendation for CRC screening.
Among those sent a message, more than three-quarters (n = 4761 or 76.8%) accessed the trial invitation. Overall, this resulted in the intervention reaching approximately one-quarter (24.1%) of the study-eligible patients ( Figure 2). Among study-eligible people who accessed the trial invitation, 38.3% enrolled in the trial (N = 1825), enabling us to meet study enrollment goals. Those who enrolled in the trial accessed the secure message inviting them to participate in the trial at an average of 6.1 (s = 19.3) d after the primary care visit in which they received a recommendation for CRC screening (Figure 3). This time delay is a result of both physician and patient actions. First, the secure message inviting the patient to the trial was sent, on average, 1.3 (s = 14.1) d following the patient's office visit (i.e., the average time between an office visit and the physician electronically signing the office visit note). While this ranged from the same day to 586 d after the visit, 74% of visit notes were signed within 4 d, and only 61 patients (3.3%) were sent the secure message more than 7 d after their appointment. Second, once a secure message was sent, it was accessed by the patient on average 5.0 d later, although this too ranged widely (from 0 to 352 d). This resulted in a handful of patients (n = 69) consenting to trial participation after they completed CRC screening. In addition, some trial enrollees (n = 44) consented to trial participation without the study team capturing a message read date. This happened either because the patient used the questionnaire page within their portal to access the questionnaire that contained the trial consent, baseline survey, and intervention/enhanced usual care material (as opposed to using the attachment provided with the study invitation secure message) or because the message and/or questionnaire were accessed via a linked proxy account. None of these nuanced situations differed significantly by study arm. Nor did accounting for them alter the effectiveness results or conclusions (data not shown).

Discussion
By leveraging the functionality of an EHR, we were able to automate both the identification of patients and the delivery of an intervention designed to engage and support patients to complete CRC screening after a physician recommended it. Doing so ensured the intervention's integration with existing primary care workflows without requiring busy primary care providers and staff to alter current practices. Despite this successful practice integration, patients' use of the intervention did not improve CRC screening rates. This may have been at  least in part because the use of an EHR-tethered patient portal limited our ability to reach those patients for whom the intervention may have been most effective. This latter group, for example, may include those who are aware that screening is recommended, have an interest in learning more, but have not yet made a commitment to acting, or those who may want to act, but who may face challenges in following through on their doctor's recommendation for care. As such, our results highlight some of the current challenges in improving CRC screening rates among primary care patients as well as both the opportunities and challenges in using the functionalities of the EHR to do so.
Importantly, given providers' interest in leveraging their investments in EHR technology, we found that using the EHR enabled us to efficiently identify and automate the sending of a patient-facing intervention following a physician recommendation for care. While this ability is potentially important to integrating patientfacing interventions into clinical care delivery, there are significant limitations in such interventions reaching patients when relying on patient portals as the sole mode of delivery. Using data from this study's sampling process, we have previously shown how only 28% of adults aged 50 to 75 y who are due for CRC screening have an activated portal account as well as how, among this population, Black adults are substantively and significantly less likely to have an active portal account. 57 Thus, although we ultimately reached only a quarter of study-eligible patients, we reached a substantially smaller proportion of all primary care patients due for CRC screening, and those we did not reach were disproportionately Black. Furthermore, as indicated by responses to the pretrial survey, those reached by the intervention reported generally high levels of health literacy, a strong preference to make their own decisions regarding CRC screening, and a strong intent to be screened for CRC at the time of study enrollment. This selectivity is consistent with prior studies that found an association between primary care visit frequency 60 and CRC screening as well as an association between cancer screening and portal use. 61 Multiple studies that have been published since this intervention was conceived have shown how Black and racially or otherwise minoritized adults are not only less likely to use the patient portal 62-66 but also to use each aspect of the portal less. 67,68 As such, relying on patient portals alone to reach patients due for CRC screening (or any other service) may not only limit the effectiveness of the intervention but also further contribute to existing disparities and exacerbate existing structural racism.
Despite the limitations in overall reach when using portals for patient-facing decision support, our results illustrate how EHR-tethered portals can be used as a low-cost means to engage certain patients. In this study, more than three-quarters of patients who were sent a secure message via their online portal not only accessed that message but did so in a timely manner (i.e., within 1 wk of their office visit). This finding is consistent with those of others researchers who successfully engaged portal users around the time of a scheduled visit to primary care. 69 Thus, for the subset of patients with an active portal account, secure messaging around the time of an office visit may be a good way to route supplemental material to patients that addresses common deficits in physician office-based recommendations for care. But given known racial disparities in portal use, including within the context of this trial, 57 such interventions must be used in combination with other modalities such as telephone calls, text messages, and/or print material. Regardless of modality, ongoing challenges integrating decision aids and shared decision making more generally into practice highlight the importance of continuing to explore how not only to embed patient-facing interventions within clinic workflows but how to do so in ways that reach all of those needing support. Doing so undoubtably will mean not relying on one communication channel but instead devising ways to meet patients where they are most comfortable and able to receive that support.
Regardless of challenges in intervention reach, for those reached, the provision of a theory-based intervention designed to complement physician office-based recommendations for CRC screening did not result in more screening than simply providing links to existing online educational material on CRC etiology, symptoms, screening, and treatment. Importantly, more than 80% of the people reached indicated that they had either been screened in the prior 6 mo or intended to be screened in the next 6 mo before they ever interacted with the intervention, and screening use among these people topped 75%. While the intervention may be relatively more effective in supporting screening among those who had less intentions to be screened, the portal-based intervention did not frequently reach such people. Whether the same intervention content might be effective among those still contemplating whether to be screened or whether a more targeted intervention that focused more heavily on barrier removal might be effective among the population reached here should be explored in the future.
Because few studies have reported how complex EHR functionalities such as BPAs, registries, and health maintenance flags can be used to support research activities, [70][71][72] our findings contribute to the literature on the use of EHRs in pragmatic trials and other types of practice-embedded research. While, as evidenced by our findings, caution needs to exist surrounding the sole reliance on portals to route patient-facing material to people, 57 it is important to note that the other EHR functionalities we leveraged work independently of the portal. In addition, most of the functionalities can be set to work silently, in the background of the EHR, and thus in ways that remain not visible or disruptive to clinic staff. Patient identification, follow-up, and even the collection of patient-reported outcomes via in-clinic use of such EHR-embedded tools 73 can therefore greatly facilitate pragmatic and other trial processes.
Study limitations include the sole reliance on EHR data to obtain CRC screening information, as such a reliance may have led to underreporting of CRC use. We do not, however, have reason to believe that this underreporting would have differed by study arm. As noted above, given the reliance on the patient portal, a primary study limitation was the limited ability to reach the targeted population. In addition to the challenges reaching the target population, among those who viewed the secure study invitation, there may have been additional selectivity bias, as enrolling in the trial required the ability not only to get to the secure message but open the attachment and navigate within the questionnaire. This may have been especially true for patients who prefer to receive information in a language other than English. In addition, it is possible that those consenting to study participation differed from those who may otherwise elect to engage with similar interventions outside of a research setting. It may also be that physicians selectively order CRC screening for only those patients who express an interest during the office visit (or, at least, selectively do not order screening for those verbally refusing screening), further limiting the reach of our intervention. Regardless, an intervention tied to an office-based physician order for care will never reach those who choose not to seek care, highlighting the importance of both inreach and outreach strategies to improve CRC screening rates in the primary care setting. Because our study did not collect reasons for nonenrollment among those who did not consent to study participation, we are not able to shed light on why people may have elected not to view the study material. Furthermore, without a sample of people who provided consent but were not exposed to anything (i.e., true usual care), we are not able to determine whether the simple provision of information on CRC and CRC screening might improve CRC screening use following a physician recommendation for care. Given what is known about the complexities of behavior change and that the CRC screening rates observed among trial enrollees (65%) were approximately equal to those observed among the health system's general population of primary care patients aged 50 to 75 y with an activated portal account around the same time (62%), this would seem highly unlikely. There may also be practice-level factors such as scheduling delays or the use of reminder calls, which could affect CRC screening timeliness and ultimately use. Although we did not compile such information within the context of this trial, most practice-level preventive health initiatives are conducted by a centralized (i.e., health system-level) population health management team within the health system within which the study was conducted, thus minimizing potential confounding.

Conclusions
While embedding patient-facing interventions within the EHR enables much needed practice integration, doing so likely minimizes program effectiveness by presenting challenges in reaching important segments of the patient population and may even exacerbate well-known racial and other disparities.