Evaluation of the Implementation and Effectiveness of a Mobile Health Intervention to Improve Outcomes for People With HIV in the Washington, DC Cohort: Study Protocol for a Cluster Randomized Controlled Trial

Background Gaps remain in achieving retention in care and durable HIV viral load suppression for people with HIV in Washington, DC (hereafter DC). Although people with HIV seeking care in DC have access to a range of supportive services, innovative strategies are needed to enhance patient engagement in this setting. Mobile health (mHealth) interventions have shown promise in reaching previously underengaged groups and improving HIV-related outcomes in various settings. Objective This study will evaluate the implementation and effectiveness of a clinic-deployed, multifeature mHealth intervention called PositiveLinks (PL) among people with HIV enrolled in the DC Cohort, a longitudinal cohort of people with HIV receiving care in DC. A cluster randomized controlled trial will be conducted using a hybrid effectiveness-implementation design and will compare HIV-related outcomes between clinics randomized to PL versus usual care. Methods The study aims are threefold: (1) We will perform a formative evaluation of PL in the context of DC Cohort clinics to test the feasibility, acceptability, and usability of PL and tailor the platform for use in this context. (2) We will conduct a cluster randomized controlled trial with 12 DC Cohort clinics randomized to PL or usual care (n=6 [50%] per arm) and measure the effectiveness of PL by the primary outcomes of patient visit constancy, retention in care, and HIV viral load suppression. We aim to enroll a total of 482 participants from DC Cohort clinic sites, specifically including people with HIV who show evidence of inconsistent retention in care or lack of viral suppression. (3) We will use the Consolidated Framework for Implementation Research (CFIR) and the Reach Effectiveness Adoption Implementation Maintenance (RE-AIM) framework to measure implementation success and identify site, patient, provider, and system factors associated with successful implementation. Evaluation activities will occur pre-, mid-, and postimplementation. Results Formative data collection was completed between April 2021 and January 2022. Preliminary mHealth platform modifications have been performed, and the first round of user testing has been completed. A preimplementation evaluation was performed to identify relevant implementation outcomes and design a suite of instruments to guide data collection for evaluation of PL implementation throughout the trial period. Instruments include those already developed to support DC Cohort Study activities and PL implementation in other cohorts, which required modification for use in the study, as well as novel instruments designed to complete data collection, as guided by the CFIR and RE-AIM frameworks. Conclusions Formative and preimplementation evaluations will be completed in spring 2022 when the trial is planned to launch. Specifically, comprehensive formative data analysis will be completed following data collection, coding, preliminary review, and synthesis. Corresponding platform modifications are ready for beta testing within the DC Cohort. Finalization of the platform for use in the trial will follow beta testing. Trial Registration ClinicalTrials.gov NCT04998019; https://clinicaltrials.gov/ct2/show/NCT04998019 International Registered Report Identifier (IRRID) PRR1-10.2196/37748

The CONSORT-EHEALTH checklist is intended for authors of randomized trials evaluating web-based and Internet-based applications/interventions, including mobile interventions, electronic games (incl multiplayer games), social media, certain telehealth applications, and other interactive and/or networked electronic applications. Some of the items (e.g. all subitems under item 5 -description of the intervention) may also be applicable for other study designs.
The goal of the CONSORT EHEALTH checklist and guideline is to be a) a guide for reporting for authors of RCTs, b) to form a basis for appraisal of an ehealth trial (in terms of validity) CONSORT-EHEALTH items/subitems are MANDATORY reporting items for studies published in the Journal of Medical Internet Research and other journals / scientific societies endorsing the checklist.
As the CONSORT-EHEALTH checklist is still considered in a formative stage, we would ask that you also RATE ON A SCALE OF 1-5 how important/useful you feel each item is FOR THE PURPOSE OF THE CHECKLIST and reporting guideline (optional).
Mandatory reporting items are marked with a red *. In the textboxes, either copy & paste the relevant sections from your manuscript into this form -please include any quotes from your manuscript in QUOTATION MARKS, or answer directly by providing additional information not in the manuscript, or elaborating on why the item was not relevant for this study. yes: all primary outcomes were significantly better in intervention group vs control partly: SOME primary outcomes were significantly better in intervention group vs control no statistically significant difference between control and intervention potentially harmful: control was significantly better than intervention in one or more outcomes inconclusive: more research is needed Other: not submitted yet -in early draft status not submitted yet -in late draft status, just before submission submitted to a journal but not reviewed yet submitted to a journal and after receiving initial reviewer comments submitted to a journal and accepted, but not published yet If this is a JMIR submission, please provide the manuscript tracking number under "other" (The ms tracking number can be found in the submission acknowledgement email, or when you login as author in JMIR. If the paper is already published in JMIR, then the ms tracking number is the four-digit number at the end of the DOI, to be found at the bottom of each published article in JMIR) T   1  a  )  T  I  T  L  E  :  I  d  e  n  t  i  f  i  c  a  t  i  o  n  a  s  a  r  a  n  d  o  m  i  z  e  d  t  r  i  a  l  i  n  t  h  e  t  i  t  l  Identify the mode of delivery. Preferably use "web-based" and/or "mobile" and/or "electronic game" in the title. Avoid ambiguous terms like "online", "virtual", "interactive". Use "Internet-based" only if Intervention includes non-web-based Internet components (e.g. email), use "computer-based" or "electronic" only if offline products are used. Use "virtual" only in the context of "virtual reality" (3-D worlds). Use "online" only in the context of "online support groups". Complement or substitute product names with broader terms for the class of products (such as "mobile" or "smart phone" instead of "iphone"), especially if the application runs on different platforms.  T  R  A  C  T  :  S  t  r  u  c  t  u  r  e  d  s  u  m  m  a  r  y  o  f  t  r  i  a  l  d  e  s  i  g  n  ,  m  e  t  h  o  d  s  ,  r  e  s  u  l  t  s  ,  a  n  d  c  o  n  c  l  u  s  i  o  n  s   N  P  T  e  x  t  e  n  s  i  o  n  :  D  e  s  c  r  i  p  t  i  o  n  o  f  e  x  p  e  r  i  m  e  n  t  a  l  t  r  e  a  t  m  e  n  t  ,  c  o  m  p  a  r  a  t  o  r  ,  c  a  r  e  p  r  o  v  i  d  e  r  s  ,  c  e  n  t  e  r  s  ,  a  n  d  b  l  i  n  d  i  n  g  s  t  a  t  u  s  . subitem not at all important a  t  u  r  e  s  /  f  u  n  c  t  i  o  n  a  l  i  t  i  e  s  /  c  o  m  p  o  n  e  n  t  s  o  f  t  h  e  i  n  t  e  r  v  e  n  t  i  o  n  a  n  d  c  o  m  p  a  r  a  t  o  r  i  n  t  h  e  M  E  T  H  O  D  S  s  e  c  t  i  o  n  o  f  t  h  e  A  B  S  T  R  A  C  T Mention key features/functionalities/components of the intervention and comparator in the abstract. If possible, also mention theories and principles used for designing the site. Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "We aim to enroll and complete baseline assessments with a total of 482 participants from Cohort clinic sites..." Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Use data will be evaluated as one of several implementation outcomes: "A preimplementation evaluation was performed to identify relevant implementation outcomes. Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "...4) speaks and reads English and/or Spanish at fourth grade level or above..." "...and the participating Cohort site research assistants will be assigned to oversee patient enrollment, training on PL use..." a c e a s s e s s m e n t s : Open vs. closed, web-based vs. face-to-face assessments: Mention how participants were recruited (online vs. offline), e.g., from an open access website or from a clinic, and clarify if this was a purely webbased trial, or there were face-to-face components (as part of the intervention or for assessment), i.e., to what degree got the study team to know the participant. In online-only trials, clarify if participants were quasi-anonymous and whether having multiple identities was possible or whether technical or logistical measures (e.g., cookies, email confirmation, phone calls) were used to detect/prevent these. Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Both focus groups met online during COVID-19 surges in D.C. ...all members were invited to participate and provided informed consent." "The site research assistant will administer baseline surveys using REDCap software ...6 and 12-month surveys will be administered at a clinic visit, or over the telephone. " "Given the nature of the intervention, neither researchers nor participants will be blinded to the outcome of randomization of clinic sites." Information given during recruitment. Specify how participants were briefed for recruitment and in the informed consent procedures (e.g., publish the informed consent documentation as appendix, see also item X26), as this information may have an effect on user self-selection, user expectation and may also bias results. Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Eligible PWH were those...could provide legal informed consent, and could participate virtually..." "...all members were invited to participate and provided informed consent." "Informed consent to participate in the implementation evaluation phase of the study will be obtained. Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "All focus groups and interviews were audio-recorded and transcribed verbatim." "...and paradata metrics collected automatically by the app .." "The site research assistant will administer baseline surveys using..." "...research assistants designated to collect and export patient...data into a central database..." "...This site assessment form will be re-distributed electronically via REDCap..." "A suite of data collection instruments...were developed..." Clearly report if outcomes were (self-)assessed through online questionnaires (as common in web-based trials) or otherwise. Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "A pre-and post-participation survey will be performed..." ""...baseline surveys will be completed by trial participants..." "...PL post-training feedback survey follows providers' completion of learning modules..." "Provider follow up surveys, for example, probe for.  Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study The PL app tailored for use in the DC Cohort will not include a specific institutional affiliation within the platform. Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "The development team made modifications agreed on by the team. Beta testing...will then be conducted..." "... will review all output from beta testing and make final app modifications." Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Coding will be performed by at least two independent research assistants..." "After user testing, final beta testing to detect any bugs, glitches, or data loss issues will be conducted..." Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study " Figure 1: Screenshots of home page and several PositiveLinks features." Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Application webpage is provided in this checklist which will be published as appendix. Screenshots will be provided in Figure 1.
Access: Describe how participants accessed the application, in what setting/context, if they had to pay (or were paid) or not, whether they had to be a member of specific group. If known, describe how participants obtained "access to the platform and Internet" [1]. To ensure access for editors/reviewers/readers, consider to provide a "backdoor" login account or demo mode for reviewers/readers to explore the application (also important for archiving purposes, see vi Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Participants were remunerated with a $25 gift card..." "Patients who do not own a smartphone on enrollment will be provided..." Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "PositiveLinks (PL) is a clinic-associated mHealth platform...features were designed using... delivers appointment reminders, daily queries ("check-ins").... (Figure 1)." Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Dose expected is as frequently as daily (check-in feature), or an 'as needed basis' depending on the feature: "The PL platform delivers ...daily queries ("check-ins")... (Figure 1)." Clarify the level of human involvement (care providers or health professionals, also technical assistance) in the e-intervention or as co-intervention (detail number and expertise of professionals involved, if any, as well as "type of assistance offered, the timing and frequency of the support, how it is initiated, and the medium by which the assistance is delivered". It may be necessary to distinguish between the level of human involvement required for the trial, and the level of human involvement required for a routine application outside of a RCT setting (discuss under item 21 -generalizability). Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Clinic providers ...will have access to the provider online learning management system ...remote assistance provided by the PL program team. On-site administration will be supervised by site research assistants." Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Primary outcomes to be evaluated at 12 months following participation include...

5
The primary hypotheses to be tested are...The study will also test the impact of PL versus usual care on secondary outcomes...relevant implementation-centered outcome measures identified during the pre-implementation evaluation." "The site research assistant will administer baseline surveys using REDCap software ...surveys will be administered at a clinic visit, or over the telephone. " "All DC Cohort sites have research assistants designated to collect and export patient ...data into a central database. Informed consent procedures (4a-ii) can create biases and certain expectations -discuss e.g., whether participants knew which intervention was the "intervention of interest" and which one was the "comparator". Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "To compare the primary outcomes...at 12 months between clusters, we will perform logistic regression using a mixed effects model (MEM), accounting for...The comparisons between the conditions will be adjusted for both individual-level...and cluster-level characteristics). Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Please see response to 4a-iii for specific excerpts from the manuscript. Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study N/A -this manuscript is for a research protocol. Planned enrollment is detailed in 'Sample size determination' as well as Figure 2 (Methods) Strongly recommended: An attrition diagram (e.g., proportion of participants still logging in or using the intervention/comparator in each group plotted over time, similar to a survival curve) or other figures or tables demonstrating usage/dose/engagement. Copy and paste relevant sections from the manuscript or cite the figure number if applicable (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Planned enrollment is detailed in 'Sample size determination' as well as Figure 2 (Methods). Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Interviews and focus groups for the formative phase were completed between April 2021 and December 2021." "Recruitment for beta testing with PWH is currently underway." "Patient recruitment for the cluster randomized trial is planned to start in the spring of 2022." Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study N/A -this manuscript is for a research protocol and randomized trial activities have not been initiated yet. Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study N/A -this manuscript is for a research protocol and randomized trial activities have not been initiated yet. Report multiple "denominators" and provide definitions: Report N's (and effect sizes) "across a range of study participation [and use] thresholds" [1], e.g., N exposed, N consented, N used more than x times, N used more than y weeks, N participants "used" the intervention/comparator at specific pre-defined time points of interest (in absolute and relative numbers per group). Always clearly define "use" of the intervention. In addition to primary/secondary (clinical) outcomes, the presentation of process outcomes such as metrics of use and intensity of use (dose, exposure) and their operational definitions is critical. This does not only refer to metrics of attrition (13-b) (often a binary variable), but also to more continuous exposure metrics such as "average session length". These must be accompanied by a technical description how a metric like a "session" is defined (e.g., timeout after idle time) [1] (report under item 6a). Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Usage metrics and specific implementation outcomes will be specifically analyzed for the intervention arm. Remaining outcomes including the primary outcomes and other relevant implementation outcomes separate from the intervention will be analyzed across both arms, (see sections 'Study Aim 2: Cluster Randomized Controlled Trial,' 'Study Aim 3: Implementation Evaluation' -Methods). Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study The planned/conducted processes for obtaining qualitative feedback are described in 'Formative evaluation,' 'Formative Data analysis,' 'Mid-and post-implementation evaluations' and 'Analysis of implementation outcomes' sections. Further details of the qualitative feedback obtained during the formative phase will also be published separately following analysis.
subitem not at all important Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study As a hybrid effectiveness-implementation design, the goal is to demonstrate outcomes when the intervention is disseminated across various treatment settings without formal incentives for participation in the cluster randomized controlled trial. A small number of participants may receive smartphones to support their participation in the study. Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study To be discussed in planned future publications following trial completion. We aim to study these questions in detail (ie program execution at each step of implementation) through our third study aim, and feel we will be poised to address this question following analysis of trial data.