Analyzing User Engagement Within a Patient-Reported Outcomes Texting Tool for Diabetes Management: Engagement Phenotype Study

Background Patient-reported outcomes (PROs) capture patients’ views on their health conditions and its management, and are increasingly used in clinical trials, including those targeting type 2 diabetes (T2D). Mobile health (mHealth) tools offer novel solutions for collecting PRO data in real time. Although patients are at the center of any PRO-based intervention, few studies have examined user engagement with PRO mHealth tools. Objective This study aimed to evaluate user engagement with a PRO mHealth tool for T2D management, identify patterns of user engagement and similarities and differences between the patients, and identify the characteristics of patients who are likely to drop out or be less engaged with a PRO mHealth tool. Methods We extracted user engagement data from an ongoing clinical trial that tested the efficacy of a PRO mHealth tool designed to improve hemoglobin A1c levels in patients with uncontrolled T2D. To date, 61 patients have been randomized to the intervention, where they are sent 6 PRO text messages a day that are relevant to T2D self-management (healthy eating and medication adherence) over the 12-month study. To analyze user engagement, we first compared the response rate (RR) and response time between patients who completed the 12-month intervention and those who dropped out early (noncompleters). Next, we leveraged latent class trajectory modeling to classify patients from the completer group into 3 subgroups based on similarity in the longitudinal engagement data. Finally, we investigated the differences between the subgroups of completers from various cross-sections (time of the day and day of the week) and PRO types. We also explored the patient demographics and their distribution among the subgroups. Results Overall, 19 noncompleters had a lower RR to PRO questions and took longer to respond to PRO questions than 42 completers. Among completers, the longitudinal RRs demonstrated differences in engagement patterns over time. The completers with the lowest engagement showed peak engagement during month 5, almost at the midstage of the program. The remaining subgroups showed peak engagement at the beginning of the intervention, followed by either a steady decline or sustained high engagement. Comparisons of the demographic characteristics showed significant differences between the high engaged and low engaged subgroups. The high engaged completers were predominantly older, of Hispanic descent, bilingual, and had a graduate degree. In comparison, the low engaged subgroup was composed mostly of African American patients who reported the lowest annual income, with one of every 3 patients earning less than US $20,000 annually. Conclusions There are discernible engagement phenotypes based on individual PRO responses, and their patterns vary in the timing of peak engagement and demographics. Future studies could use these findings to predict engagement categories and tailor interventions to promote longitudinal engagement. Trial Registration Clinicaltrials.gov NCT03652389; https://clinicaltrials.gov/ct2/show/NCT03652389 International Registered Report Identifier (IRRID) RR2-10.2196/18554


Introduction
Background A patient-reported outcome (PRO) is defined by the National Quality Forum and Food & Drug Administration as a "report of the status of a patient's health condition that comes directly from the patient without amendment or interpretation of the patient's response by a clinician or anyone else." [1,2]. PROs include health-related quality of life [3], adherence to medical regimens, satisfaction with treatment, and elements of disease control [4]. Although innovations in medical technology have allowed the measurement of physical, physiological, or biochemical data with great accuracy, they are not able to provide the patient's perspectives on their treatment or disease [5]. These data can only be obtained directly from patients [6]. Thus, PROs in clinical trials provide a more holistic assessment of the benefits of the treatment or intervention under investigation [7]. With the advent of patient-centered health care systems, where a patient is considered the center of the health care system [8], patients and patient advocates have called for more patient-centered outcomes reporting (ie, PROs) in combination with other clinical and physiological outcomes [5].
Traditionally, PROs have been assessed using survey instruments [4,9]. However, recent advancements in mobile health (mHealth) technologies have enabled a wide variety [10] of tools and apps that can be used to collect PRO data. With mHealth technologies, PROs can be assessed electronically from PCs, from mobile solutions such as tablet PCs or smartphones using apps or texting tools, or through data entered via web browsers [11]. The use of mHealth technologies to collect PRO data offers several advantages over traditional survey-based methods, including real-time data collection, reduced time for documentation, automated algorithms and calculations, in-home symptom monitoring, immediate transfer of data for clinical use, enhanced patient engagement in care, and more informed clinical decision-making [12][13][14].
Uncontrolled type 2 diabetes (T2D) is a significant public health problem in the United States, especially among vulnerable populations (eg, low-income, racial, and ethnic minorities) [15,16]. Prior studies have recognized that patients play a central role in the management of T2D (eg, being aware of its signs and symptoms and engaging in daily self-care behaviors), and several national and local organizations have launched initiatives to support the development and use of PROs in the evaluation of T2D patient care [17][18][19][20][21]. However, existing research that incorporates PROs in T2D care has been mostly limited to clinical drug trials examining patient tolerance to new treatment regimens [22]. The few practice-based studies conducted on T2D used long lists of PRO measures and only had patients report PROs on a single occasion, typically before clinic visits [23,24]. Such reporting increases the risk of a recall bias. To address these shortcomings, a growing number of studies are using mHealth platforms that enable real-time collection of PROs outside the clinical environment [25][26][27][28][29][30][31].

Objective
Analysis of prior mHealth research revealed that most studies did not consider user engagement metrics when evaluating the design implications of the intervention on T2D patients' health outcomes [30]. Of the few studies that reported engagement data, most were limited by small sample sizes [32] and low response rates (RRs) [33]. Although prior research has found that consideration of user preference and personalization with mHealth PRO interventions are key aspects that influence user engagement [34], there is a lack of consensus regarding best practices for modifying mHealth PRO tools to optimize digital intervention and improve patients' engagement [35]. Thus, the ideal cadence of PRO collection to facilitate sustained engagement in an intervention is unclear and may vary according to user characteristics [36]. To address these gaps, this study reports on the analysis of longitudinal user engagement data from an ongoing randomized controlled trial (Investigating an mHealth texting tool for embedding patient-reported data in diabetes management [i-Matter]) evaluating the efficacy of a PRO mHealth texting tool for T2D management among 282 patients with uncontrolled T2D [31]. This paper discusses and compares patterns of engagement with the PRO tool among patients randomized to the i-Matter intervention and across sociodemographic characteristics to offer insights for future adaptation of the intervention based on patients' engagement.

Recruitment
Patients were recruited from a network of primary care practices at NYU Langone Health across New York City's 5 boroughs and Long Island. The details of our recruitment approach have been reported previously [31]. Briefly, to participate in i-Matter, patients must (1) have a diagnosis of T2D for ≥6 months, (2) have uncontrolled T2D defined as hemoglobin A1c >7% documented in the electronic health record (EHR) at least twice in the past year, (3) be fluent in English or Spanish, (4) be willing to send and receive text messages, and (5) be >18 years of age. Patients were excluded if they (1) refused or were unable to provide informed consent; (2) had acute renal failure, end-stage renal disease, evidence of dialysis, renal transplantation, or other end-stage renal disease-related services documented in the EHR; (3) participated in another T2D study; (4) had significant psychiatric comorbidity or reports of substance abuse (as documented in the EHR); (5) were pregnant or planning to become pregnant within 12 months; or (6) planned to discontinue care at the practice within the next 12 months. This paper focuses on 61 patients randomized to the i-Matter intervention who have either dropped out of the trial before the 12-month study visit (ie, noncompleters) or completed the trial (ie, completers). We excluded patients who were currently participating in the trial as their data were incomplete and would not provide a comprehensive view of how their engagement with the PRO messages may change over time.

The i-Matter Intervention
The "Investigating an mHealth texting tool for embedding patient-reported data into diabetes management" (i-Matter) trial is evaluating the efficacy of an innovative mobile PRO system that incorporates patients' perspective of their disease into the management of T2D in primary care practices. Patients participating in the trial were randomized to the i-Matter intervention or usual care (ie, standard diabetes care by the primary care provider) in a 1:1 ratio by the study statistician. The i-Matter intervention uses text messaging to capture patients' self-reported PROs in real time, provides data-driven feedback and motivational messages based on responses to the PROs, and creates dynamic visualizations of the PROs that are shared in personalized reports and integrated into the clinical EHR. We are currently conducting a randomized controlled trial to evaluate the efficacy of the i-Matter intervention versus usual care in reducing hemoglobin A1c levels and adherence to self-care behaviors at 12 months among 282 patients with uncontrolled T2D who receive care in resource-limited primary care practices.

Description of PRO Messages Embedded in the i-Matter Intervention
The details of the PRO system development have been reported elsewhere [31]. Briefly, we used a mixed method, user-centered design approach to select PROs that were integrated into the i-Matter intervention. Our approach included reviewing the existing literature on PRO measures for T2D, conducting interviews with primary care providers and patients to capture their experiences with T2D, and collecting survey data. These data sources were combined to identify the PROs that would be integrated into a beta version of i-Matter and refined with user testing among patients with T2D. Table 1 lists the final set of PROs, including their timing and response options, which were integrated into the i-Matter intervention. Daily daytime messages included PROs on sleep quality and healthy eating, whereas a daily nighttime message included a PRO on physical activity (Table 1 provides further details). Patients can choose when they would like to receive medication adherence [37] PRO based on their medication regimen in the afternoon, at night, or at both times. The remaining PROs were similarly timed for all patients. In addition, patients can choose one healthy living goal from a selected list of topics identified in user testing (Table 1). Questions on individualized healthy living goals and patients' diabetes quality of life were sent weekly, and the remaining messages were sent daily. PROs can be sent in either English or Spanish depending on the patients' language preferences.

Engagement Metrics
User engagement data were extracted from the patients' responses to the PROs embedded in the texting tool at the end of their participation in the 12-month study. The primary engagement metrics evaluated in this study are listed in Table  2, including the metrics of RR and response time (RT). RR represents the percentage of PRO questions that garnered any valid responses. The RR metric represents patient engagement with the individual PRO questions [36]. The RR of a message was measured as the time difference in seconds between when the PRO message was sent to a patient and the time the patient sent the corresponding response. Only valid responses to the PROs were used when measuring RT. The RT metric reflects how well the timing of PRO messages integrates into patients' everyday lives, which in turn is expected to affect their level of engagement.
The overall RR and RT were measured by taking the average of the corresponding measures over a time frame, either weekly or monthly. We also measured RRs and RTs by grouping the messages sent at similar times of the day (daytime, nighttime) or times of the week (weekdays and weekends) and those that required similar responses (Yes or No, 1-10). Depending on the patient's decision regarding when to receive the message, the medication adherence PRO was either used once or twice for the nighttime RR and RT measures.

Demographic Characteristics
At baseline, all patients completed a self-report instrument that was used to collect patient sociodemographic data, including sex, race or ethnicity, age, annual household income, education level, marital status, and employment status.

Analysis
To analyze user engagement with PRO messages, we first compared the engagement metrics of RR and RT between users who completed the 12-month study (ie, completers) and those who ended their participation before program completion (ie, noncompleters). In addition, we investigated the distribution of dropout times among noncompleters. The goal of this analysis was to identify participants who were likely to drop out of the program at the early stages and tailor the program to minimize dropouts in future iterations.
We compared the longitudinal data of user engagement from the completer group using latent class trajectory modeling (LCTM). The goal was to classify heterogeneous populations into homogeneous clusters or subgroups with distinct trajectories [38] based on similarities in their engagement behaviors. For the LCTM models, we experimented with user engagement measures at various time intervals, including weekly, biweekly (measured once every 2 weeks), monthly, and bimonthly (once every 2 months). We chose the monthly engagement measures for the final analysis, as they provided a balance between smaller weekly and biweekly units, where the difference in engagement between the classes would be less distinguishable, and the larger bimonthly engagement measures that would entail smaller sample sizes for trajectories.
Finally, we investigated the difference in engagement between the subgroups identified by the LCTM model from various cross-sections of time and PRO message types. The Shapiro-Wilk normality test was used to evaluate all engagement measures for each group separately before conducting comparisons. We also explored patients' overall sociodemographic characteristics and their distribution among the subgroups. The goal of the analyses was to further characterize the subgroups and identify patients who were likely to be part of a subgroup.

Ethics Approval
This study was approved by the NYU Langone Health Institutional Review Board (i18-01044).

Patient Engagement in the i-Matter Intervention
As of April 2022, a total of 61 patients completed their participation in the i-Matter intervention. Of the 61 participants, 42 (69%) completed the 12-month program, whereas the remaining 19 (31%) noncompleters ended their participation either by opting out on their own (10/61, 16%) or requesting the recruitment team to disenroll (9/61, 15%) from the program before the 12-month end point. Figure 1 shows the distribution of the time (in days) when the 19 patients ended their participation before the 12-month study visit. The average participation time in the program was 211 (SD 124.99; range 9-363) days. At least 53% (10/19) of patients dropped out of the program before the average participation time. Of the remaining patients, 4 ended participation in the last week (after 356 days from the day of enrollment) of the program. In all 4 cases, the overall RR was above 70% (mean 85.92%, SD 10.73%; median 90%, IQR 10.34%), suggesting that ending the program could have been unintended.

Overall Engagement Metrics in the Completers Group
The mean RR among the completers was 71.44% (SD 26.50%), and the median was 76.91% (IQR 32.72%). Figure 2 shows the distribution of RR for this group. Approximately 45% of patients had an RR below the mean value. The distribution of RRs was found to be nonnormal (P<.001).  Figure 3 shows the comparison of PRO engagement metrics between patients who ended the program early (noncompleters) versus those who completed the entire 12 months (completers). The RR of participants ( Figure 3A) who completed the program was much higher (mean 71.44%, SD 26.50%) than that of those who did not (mean 47.40%, SD 37.33%). In addition, the distributions of RTs in the 2 groups in Figure 3B suggest that completers, on average, were quicker (mean 1325, SD 3709 seconds) to respond to PRO messages than the noncompleters (mean 1359 seconds, SD 3754 seconds); however, the difference was not significant because of large variations in the RTs.

Description of the Engagement Subgroups Among the Completers
Initially, we constructed a scoping model that provisionally selected a plausible number of classes, K=2, for the LCTM. In the next step, we refined the preliminary working model by altering the number of classes (K=3) and exploring variations in latent class linear mixed models. We capped the number of classes at K=3 because of the sample size (ie, number of patients) in our study. The number of classes for the final model was determined based on the lowest Bayesian information criterion. On the basis of distribution of our outcome variable, RR in the completers group, we investigated both standard linear mixed models (using the hlme function) and latent process, latent class mixed models (lcmm function). Finally, we performed model adequacy assessments by examining the posterior probability of being assigned to each trajectory class and assigning each individual to the class with the highest probability. An average of these maximum posterior probabilities of assignments above 70% [39] in all classes was considered acceptable. We tested a total of 11 models (Multimedia Appendix 1), and in the end, a latent class linear mixed model with K=3 classes was chosen as the best fit. The results from the trajectories of the 3 classes are shown in Figures  4A and 4B. We defined the 3 classes as low engaged (red), moderate engaged (blue), and high engaged (green) subgroups. Spaghetti plots of individual-level data illustrate that initial RRs, combined with the timing and direction of changes in the engagement metrics, characterize the subgroups. For example, as shown in Figure 4B, the low engaged subgroup is characterized by the lowest RR at the initial weeks of the intervention, coupled with a sharp increase in RR until the midtrajectory (~5 months), followed by a steady decline in RR for the remainder of patient participation. In contrast, the RR of the moderate engaged subgroup begins above 80%, decreases at the midpoint of the intervention to 75%, and then steadily rises toward the final months to 79%. Finally, the high engaged subgroup showed a consistently high RR across the 12-month study period ( Figure 4B).  Table 3 displays the RR and RT across the 3 subgroups according to the day of the week and the time of day each PRO message was sent. As the distribution of RTs was found to be significantly nonnormal for the entire population and also for the 3 subgroups individually, we used Kruskal-Wallis 1-way ANOVA to compare engagement among the 3 subgroups, followed by post hoc Dunn tests with Bonferroni adjustments for pairwise comparisons. Overall, the results suggest that patients in the high engaged subgroup had a significantly higher overall RR (96.27%) than those in the moderate engaged (78.66%) or low engaged (54.35%) subgroups (Table 4). In addition, patients in the high engaged subgroup, on average, took a significantly shorter time (mean 559 seconds, SD 451 seconds) to respond to PRO messages than the moderate engaged (1564 seconds, SD 1138 seconds) or low engaged subgroups (2814 seconds, SD 5115 seconds). The differences in engagement between the 3 subgroups were consistent across most measures, regardless of the day of the week (weekdays and weekends) or timing of the messages (daytime vs nighttime; Table 4). In addition, the RRs of messages sent at night (highlighted in gray) were found to be higher than the overall RR for all 3 subgroups.  We also compared user engagement between subgroups for the 6 PRO messages individually. In addition, the PROs were grouped by response type (yes or no as binary, and the remaining messages as a Likert-type scale) and compared separately. As shown in Table 5, a significant difference in the RR was observed for all 6 messages. For example, the mean RRs of sleep quality PRO were 44.84%, 60.44%, and 93.28% in the low engaged, moderate engaged, and high engaged subgroups, respectively. Similarly, RT was also significantly different among the 3 subgroups for all messages, except for physical activity. Overall, the binary PROs (Yes or No) had a higher RR than the Likert-scale PROs for all 3 subgroups. In fact, patients' RR to binary PRO messages, medication adherence, and diet (healthy eating) were above average for all 3 subgroups. In contrast, patients were consistently less responsive to the Likert-scale PRO on fulfilling a healthy living goal (weekly at 2 PM), with only one of 3 messages receiving any valid response from the low engaged subgroup. Further details of pairwise comparisons of the subgroups are provided in Table 6.

Comparison of Patient Sociodemographic Characteristics Among the Engagement Subgroups
We examined patients' sociodemographic characteristics for those who completed the program (n=42) and compared their distribution among the 3 engagement subgroups. In 6.85% (23/336) of cases, at least one of the sociodemographic questionnaires was missing responses. As shown in Table 7, the overall patient sample was mostly female (12/42, 71%), non-Hispanic or Latino origin (22/28, 79%), fluent in English (38/41, 93%), completed a bachelor's degree or above (21/41, 52%), and married or living with a partner (18/41, 44%). The distribution of demographic data further shows that relative to the overall population distribution, a higher proportion of male Hispanic or Latino origin patients were more high engaged than females and non-Hispanic or Latino origin counterparts. In addition, the high engaged subgroup was composed of mostly older patients with the lowest variations in age range (between 53 and 68 years), and a higher percentage of Hispanic or Latino patients (3/10, 30%) were bilingual (2/16, 13%), and completed at least a graduate-level education (5/16, 31%).
In comparison, the low engaged subgroup was composed primarily of patients who were identified as African Americans (7/12, 58%) and reported the lowest annual income, with one out of every 3 patients earning US $20,000 or less annually. Finally, marital status was relatively consistent between low-engaged and high-engaged groups.

Principal Findings
Although achieving glycemic control is of clinical importance for patients with T2D, it is the daily experience of living with T2D that drives patients' perseverance to adhere to treatment regimens and become engaged in their care [40]. This study reports user engagement with PROs in the i-Matter trial, designed to incorporate the collection of real-time PRO data that are meaningful to both patients and providers in the clinical management of T2D. Our retrospective analysis of user engagement found discernible engagement phenotypes based on individual PRO responses: patients who dropped out from the program early (noncompleters) took more time to respond to PRO questions and were less likely to respond than the completers. Among the completers, the analysis of the longitudinal RRs identified 3 subgroups with significant differences in engagement. The completers from the lowest engagement subgroup had a significantly lower RR and longer RT than those from the other 2 completer subgroups. These results suggest that patients who have lower RR in combination with longer RTs are at risk of dropping out of the program or continuing with lower than average engagement with PRO questions. Future analyses will evaluate whether this pattern is associated with poorer adherence to self-management behaviors.
Our analysis further revealed that the engagement phenotypes among completers differed in the timing of peak engagement. The low engaged completers showed an almost normal distribution of average engagement over time, with peak engagement in the middle of the program, followed by a steady decline. The decline in engagement following the peak could be due to fatigue onset among patients from responding to multiple daily text messages across the 12-month study [41]. Further evaluation and analysis are required to determine how to maintain peak engagement in this subgroup in the latter half of the program. The moderate engaged subgroup showed an opposite trend in peak engagement compared with the low engaged subgroup: a decline in engagement in the first 7 months of the program, followed by a steady increase.
The high engaged subgroup consistently showed >90% RR and low RT throughout their participation. They also represent the largest sample among the 3 subgroups. The reasons for high engagement and whether this response pattern leads to better health outcomes (behavioral and clinical) among patients with T2D will be explored in future analyses. The variations in peak engagement timing suggest that longitudinal data on patients' motivations and self-care behavioral activities need to be analyzed for periodic changes and to evaluate their impact on user engagement.
Traditionally, sociodemographic characteristics, including older age, lower income, unemployment status, lower education status, minority racial and ethnic group membership, language barriers, and geographic barriers, have been associated with an elevated risk for poorer health outcomes [42]. Our analysis of the sociodemographic measures also showed that distributions of race, age, language, and income varied between the high-and low engaged subgroups of our intervention. The high engaged completers were predominantly older, of Hispanic descent, bilingual, and highly educated (graduate school or above).
These findings reflect the growing trends in mHealth research, which has shown that behavioral interventions, particularly those that leverage text messaging, have high rates of user engagement among Hispanic individuals with limited English proficiency [43,44]. For example, Cartujano-Barrera et al [44] reported high levels of interactivity (73%) and low disenrollment (20%) with a 12-week smoking cessation intervention delivered via text messaging in a sample of bilingual Hispanic individuals. Similarly, a text messaging intervention designed to improve diabetes management showed high levels of engagement (86% RR to at least one text message) and low dropout rates (6%) among low-income Hispanic patients who were followed in safety-net primary care practices [45]. This is in line with data published by the Pew Research Center [46], which showed higher cell phone ownership and use among Hispanic than among non-Hispanic White patients (100% vs 97%). Data from qualitative evaluations of text messaging studies in Spanish-speaking Hispanic individuals suggest that receiving and responding to text messages serves as a source of emotional support [45] for engaging in self-care behaviors to improve diabetes management. Prior research on disparities in self-care behaviors among patients with T2D reported consistent evidence of no disparities in exercise and some evidence of reverse disparities. Compared with non-Hispanic White patients, Hispanic patients with T2D had healthier diets [47], which likely manifested in higher engagement, especially with Healthy Eating PRO among Hispanic patients. Although our finding that older adults were more likely to be high engagers of the intervention may seem inconsistent with previous research [48][49][50], recent data show that more than 85% of adults aged ≥50 years communicate primarily via text messages [51]. Research on barriers to diabetes medication adherence has also found evidence that younger age is associated with motivational and behavioral barriers [37], which could have manifested in lower engagement among younger patients in our intervention.
Our analysis further suggests that African American patients with the lowest annual income are most likely to have low engagement with PRO messages. It is plausible that the initial development of our intervention may not have captured the unique needs and concerns of this patient population. Consequently, the content and delivery of messages may not have been suitable, resulting in low engagement rates. Previous research has also found barriers to low-income African American patients' engagement in text messaging interventions, including low ownership of personal cell phones, difficulty in responding to text messages, and reporting that the program was not helpful or relevant for improving self-management behaviors [37]. These findings suggest that future text message interventions must consider both the technical elements of the intervention as well as the contextual factors (eg, financial and cultural) that could affect user engagement.

Limitations
Although this study has many strengths, we note the following limitations that can be considered for future research. First, although our intervention enrolled patients with T2D, it is more common for patients to have 2 or more chronic diseases (ie, multimorbidity) than one disease in isolation (89.3% vs 8.5%, respectively) [52]. Recent research has demonstrated the negative impact of multimorbidity on PROs such as quality life, psychosocial health, self-efficacy, physical function, and self-management behaviors [53]. Thus, future research should examine whether adapting i-Matter for a multimorbid population would improve the engagement of patients and provider management of co-occurring chronic diseases rather than using a single disease focus that can cause inefficiencies and fragmentation in care. Second, we did not examine psychosocial factors that could impact patient participation in the intervention. Their motivation, knowledge, and self-efficacy behaviors before joining the program may affect their engagement with PROs [54]. Future analysis of i-Matter data will examine how user engagement metrics differ based on self-reported diabetes self-care behaviors, medication adherence, self-efficacy, and motivation for diabetes management. We also note a possible limitation of our data analysis because of the small sample size. A larger sample size would provide greater statistical power for detecting statistically significant differences in user engagement between demographics if there are true differences. Future analyses of i-Matter data will include the remaining patients who completed the trial.

Conclusions
In summary, we found that patients who dropped out from the intervention early (ie, noncompleters) had a lower RR to PRO questions and took longer to respond to PRO questions than completers. Among completers, analysis of longitudinal RRs identified 3 subgroups of engagement over time. Our results suggest that patient sociodemographics along with RT measures offer good predictability for patients who need further support to stay engaged in the intervention. Future trials should identify these patients early in the intervention and customize PRO messages and their timing to elicit higher engagement.
The i-Matter intervention was designed and developed through active involvement of patients and addresses difficulties with protocol compliance, lack of clinical integration in the EHR, and provider skepticism about the utility of PROs in practice, which were hallmarks of previous trials, thus increasing the likelihood of developing a sustainable approach [55]. Despite these efforts, the results showed that 31.1% of patients dropped out of the intervention before the final 12-month study visit, suggesting room for improvement in the PRO texting tool for future trials. For example, the messages sent during the daytime yielded the lowest engagement from this subgroup, which suggests that further customization of PRO message timing is needed to ensure that the cadence of messages fits within the daily lives of users to increase engagement. This analysis provides insights into how to make PROs more patient-centric in future iterations of the i-Matter intervention. The current research conducting qualitative interviews with patients who complete the program will be used to identify potential motivators that could be integrated into future versions of i-Matter.