Remote Evidence-Based Programs for Health Promotion to Support Older Adults During the COVID-19 Pandemic and Beyond: Mixed Methods Outcome Evaluation

Background Evidence-based programs (EBPs) for health promotion were developed to reach older adults where they live, work, pray, and play. When the COVID-19 pandemic placed a disproportionate burden on older adults living with chronic conditions and the community-based organizations that support them, these in-person programs shifted to remote delivery. While EBPs have demonstrated effectiveness when delivered in person, less is known about outcomes when delivered remotely. Objective This study evaluated changes in remote EBP participants’ health and well-being in a national mixed methods outcome evaluation in January 1, 2021, to March 31, 2022. Methods We used the RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) for equity framework to guide the evaluation. We purposively sampled for diverse remote EBP delivery modes and delivery organizations, staff, and traditionally underserved older adults, including people of color and rural dwellers. We included 5 EBPs for self-management, falls prevention, and physical activity: videoconferencing (Chronic Disease Self-Management Program, Diabetes Self-Management Program, and EnhanceFitness), telephone plus mailed materials (Chronic Pain Self-Management Program), and enhanced self-directed mailed materials (Walk With Ease). Participant and provider data included validated surveys, in-depth interviews, and open-ended survey questions. We used descriptive statistics to characterize the sample and the magnitude of change and paired t tests (2-tailed) and the Fisher exact test to test for change in outcomes between enrollment and 6-month follow-up. Thematic analysis was used to identify similarities and differences in outcomes within and across programs. Joint display tables facilitated the integration of quantitative and qualitative findings. Results A total of 586 older adults, 198 providers, and 37 organizations providing EBPs participated in the evaluation. Of the 586 older adults, 289 (49.3%) provided follow-up outcome data. The mean age of the EBP participants was 65.4 (SD 12.0) years. Of the 289 EBP participants, 241 (83.4%) were female, 108 (37.3%) were people of color, 113 (39.1%) lived alone, and 99 (34.3%) were experiencing financial hardship. In addition, the participants reported a mean of 2.5 (SD 1.7) chronic conditions. Overall, the remote EBP participants showed statistically significant improvements in health, energy, sleep quality, loneliness, depressive symptoms, and technology anxiety. Qualitatively, participants shared improvements in knowledge, attitudes, and skills for healthier living; reduced their social isolation and loneliness; and gained better access to programs. Three-fourths of the providers (149/198, 75.2%) felt that effectiveness was maintained when switching from in-person to remote delivery. Conclusions The findings suggest that participating in remote EBPs can improve health, social, and technological outcomes of interest for older adults and providers, with benefits extending to policy makers. Future policy and practice can better support remote EBP delivery as one model for health promotion, improving access for all older adults.


Background
One in 6 adults living in the United States is an older adult (aged ≥60 y).This number is expected to double in the next 40 years [1].While older adults possess the wisdom of experience and are often actively involved in taking care of themselves, their family members, and their communities, they also face a wide range of unique health challenges that come with aging [2].A disproportionate number of older adults live with chronic health conditions: 85% have 1 chronic health condition, and 60% report managing at least two [3].One in 4 older adults reports at least 1 fall every year, and falls remain a leading cause of death and injury among this age group [4].These health issues-combined with social determinants of health such as older adults' built environment, social context, and access to medical care-put older adults at risk for premature death and poorer quality of life [5].While only accounting for 17% of the population, older adults make up 35% of health care costs, according to 2019 Medical Expenditure Panel data [6].Furthermore, 86 cents of every dollar of health care spending goes toward chronic conditions [7,8], and the burden of chronic disease is unevenly borne by women; older adults; people of color; and people living in poverty who experience disparities in access to, and quality of, care [9].
To address these challenges, many health promotion programs have been created based on the Chronic Care Model [10], to improve access and quality of care through community-based programs that teach knowledge, skills, and self-efficacy to enhance older adults' health and well-being in their daily lives.Among these are evidence-based programs (EBPs) that have been researched and recognized by national and federal agencies, such as the Administration for Community Living (ACL) and the Centers for Disease Control and Prevention (CDC) Arthritis Management and Well-Being Program, as being effective in promoting health outcomes through standardized interventions [11,12].

Investigating the Effectiveness of Remote EBPs
Before the COVID-19 pandemic, many of these EBPs were primarily offered in person.This was not possible during the pandemic due to safety guidelines regarding physical distancing.At the same time, there was an increased need for programs because older adults became more isolated and less physically active, and they had less access to an overtaxed medical system [13,14].Seeing this need, EBPs quickly pivoted to remote delivery by mail, telephone, videoconferencing, or a combination of these modes.While researchers suggest that EBPs will work as intended using different forms as long as core functions are not modified [15,16], it is unknown whether program effectiveness is maintained when switching from in-person to remote delivery.To date, there have been limited studies investigating the effectiveness of remote EBPs that were originally designed to be offered in person [17][18][19][20][21].As such, we conducted a longitudinal national outcome evaluation of several EBPs to assess changes in older participants' health.

Framework and Design
We used the RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) for equity framework [22] to evaluate the potential impact of remote EBPs on older adults' health and well-being.The equity lens means that in addition to evaluating impact, we looked at outcomes across programs, sampled organizations that reach older adults who are underserved, reported EBP reach, and assessed whether there were any unintended consequences.As such, we used mixed methods to "give voice to participants as well as report statistical trends" [23].We conducted a multisite single-group pre-post evaluation.Multiple sites were selected to facilitate the generalizability of the findings, and a single-group design was chosen to make it feasible to conduct the evaluation during the COVID-19 pandemic and due to the descriptive nature of this study.

Ethical Considerations
This study was considered exempt from University of Washington Institutional Review Board review because the activities fell under category 2 with regard to quality improvement and program evaluation (STUDY00011549).Participants who completed the preprogram survey were given a US $10 gift card, and participants who completed the follow-up survey were given a US $20 gift card (we provided electronic gift cards unless participants requested a physical card).Providers who completed the survey received a US $10 electronic gift card.

Selected EBPs
EBPs are health promotion programs that have been evaluated and proven to be effective.We included 5 EBPs in our evaluation: Chronic Disease Self-Management Program (CDSMP), Diabetes Self-Management Program (DSMP), EnhanceFitness (EF), Chronic Pain Self-Management (CPSMP), and Walk With Ease (WWE).These EBPs are all currently recognized by the national social services agency for older adults (ACL) and public health agencies (CDC) [11,12].Before the COVID-19 pandemic, these programs were offered in person in group formats; they were adapted as follows for remote delivery in response to the COVID-19 pandemic: videoconferencing (CDSMP, DSMP, and EF), telephone plus mailed materials (CPSMP), and self-directed mailed materials plus enhanced support (WWE).These 5 EBPs were selected because they had sufficient program reach and represented different remote modes (telephone, videoconferencing, mail, or a combination of these modes) and health topics (chronic disease self-management, falls prevention, and physical activity).The adaptation process largely involved planned changes to modify delivery context (eg, smaller class size; adding a cofacilitator to support engagement with, and the usability of, technology; and providing telephone options for people without access to reliable internet or videoconferencing technology) as well as some unplanned changes that occurred organically during field experience in consultation with the program leads (rather than changes to program content) [24].It should be noted that WWE had created an enhanced self-directed program before the pandemic; we included this program in the evaluation to assess different modes of remote EBP delivery, given partners', policy makers', and older adults' interest in diverse ways to engage in remote health promotion.The enhanced self-directed WWE is delivered via a cohort, with a trained leader providing web-based motivation and support to individual participants during the program period.More information about the included programs is provided below (Figure 1) and on each program's website, including guidance for remote delivery.

Recruitment
In early 2021, we recruited organizations delivering remote EBPs with a brief web-based interest survey and webinars through several networks: EBP training listserves, the CDC Arthritis Program and ACL grantees, the Evidence-Based Leadership Collaborative, and regional EBP networks.The organizations included social services, public health, and health care agencies that were largely community based, although some were located in clinical settings.We used maximum variation purposive sampling [25] to identify organizations of diverse sizes and types, varied provider characteristics, and from different geographic areas to aid the generalizability of the evaluation findings.This sampling prioritized organizations engaging older populations with low-income status with multiple chronic conditions who are vulnerable to both COVID-19 infection and poor health outcomes and care (eg, people of color, those with disabilities, and those living in rural areas).The eligibility criteria for organizations was the delivery of at least 1 of the 5 remote EBPs from January 2021 through March 2022.

RenderX
We then used convenience sampling to invite all remote EBP participants to take part in the evaluation.Participant surveys were primarily conducted on the web using REDCap (Research Electronic Data Capture; Vanderbilt University) [26], with options to complete surveys by telephone or by mail.Participants were surveyed when they enrolled in the program and again 6 months after program enrollment (regardless of when they finished the program).At both time points, a link to the survey was emailed to participants, and they received up to 3 additional reminder emails and 3 reminder calls during the month of survey eligibility.In addition, community-based organizations encouraged baseline survey completion as part of program intake and orientation.Furthermore, all EBP providers at participating organizations were invited to take part in a 1-time survey eliciting their perspectives regarding the impact of remote EBPs on participants as well as providers.The EBP providers included leaders (people who delivered remote programs directly) and managers (people who coordinated and oversaw program delivery).
We also collected several program-specific outcomes [37] related to their health focus: self-efficacy for CDSMP, hypoglycemia for DSMP, and pain and the use of opioid medications for CPSMP.In addition, the preprogram survey included questions about remote EBP participant demographics: age, gender, race, ethnicity, living alone, caregiving, and chronic conditions.Rurality was defined using the participant's zip code and several federal criteria for rural funding [38].The 6-month follow-up survey included 3 additional quantitative items: two examining the usability of and anxiety about technology using the Senior Technology Acceptance Measure [39] and the other calculating a single-item net promoter score [40,41] to assess acceptability [42].
Provider survey data included multiple-choice questions about demographics, experience delivering EBPs, and the impact of remote EBPs on both participants and providers.Response choices were created from open-ended responses to an earlier Evidence-Based Leadership Collaborative remote EBP web-based survey in 2020 (L, Steinman, personal communication, December 2020).

Qualitative
The 6-month follow-up participant survey included open-ended items about program acceptability and unintended consequences or impacts.Participants who completed the follow-up survey over the telephone were also asked 2 additional qualitative questions about participating in the evaluation.Providers were also asked an open-ended question about any additional benefits for both remote EBP participants and providers.

Quantitative
Data were managed in REDCap [26] and analyzed using R software (R Foundation for Statistical Computing) [43].Our primary aim was to evaluate changes in remote EBP participants' health and well-being (self-rated health, fatigue or energy, exercise, pain, sleep, depression, anxiety, loneliness, social isolation, and program-specific outcomes) between program enrollment and 6-month follow-up.Only participants who included both baseline and 6-month follow-up data were included in these analyses.We assessed the magnitude of the changes using descriptive statistics, percentage change, and Cohen d effect sizes and tested for statistical significance of the changes using paired t tests (2-tailed).Percentage change was reported for both people who improved-or maintained-outcomes between remote EBP enrollment and 6-month follow-up, given the importance of maintaining health in an aging population [44].Effect sizes were calculated as mean (SD) [45].We also used descriptive statistics to characterize the sociodemographic characteristics of remote EBP participants and providers.We chose not to use the Bonferroni correction to adjust the P values because we carried out tests on multiple outcomes of interest across programs without a priori hypotheses (rather than 1 primary outcome of interest) [46].

Qualitative
Audio-recorded data were transcribed into Microsoft Word documents.These text data from transcripts and open-ended survey questions were analyzed using Dedoose software (SocioCultural Research Consultants, LLC) [47].For qualitative data, we used thematic analysis [48,49] to understand similarities and differences across and within remote EBP acceptability and benefits or unintended consequences.Two researchers (KC and LS) created a codebook to organize important text for comparison, using deductive codes from the interview guide and inductive codes from an initial read of the transcripts [50].We conducted 2 rounds of reliability coding, adjusted the codebook codes and definitions as needed, then completed coding using 1 coder.Coded exports were then organized in interpretation memos to summarize possible explanations for what was happening, including a summary of findings, key distinctions and counterexamples, and further points for consideration [51].

Integration
We used joint display tables [52] and compared quantitative and qualitative results to see where the findings converged, diverged, or expanded [23].

Participants
A total of 586 older adults participated in the evaluation over the study period, of whom 289 (49.3%) completed the 6-month follow-up survey (n=25, 8.7% completed the survey over the telephone).The primary reason for noncompletion was our inability to contact the older adults by telephone or email; only 10 (1.7%) of the 586 older adults declined to participate in the follow-up survey after being contacted.

Overview
Outcomes are reported by specific program and across the 5 programs included in our evaluation (Tables 2 and 3; Figure 2; Multimedia Appendix 2).

CDSMP Participants
When testing for whether these changes over time were not due to chance, CDSMP participants showed statistically significant improvements in health (mean change 0.32, 95% CI 0.117-0.530;P=.003; t 67 =3.

EF Participants
EF participants demonstrated better sleep quality (Fisher exact test: P=.001), with a large effect size of −0.66.Participants also showed strong improvements in health (effect size: −0.58) and technology usability (effect size: 0.60).From program enrollment to 6-month follow-up, 42% (5/12) of the participants showed improvement, and 33% (4/12) of the participants XSL • FO RenderX maintained the number of days on which they were physically active.

Across Remote EBPs
Across programs, remote EBP participants showed statistically significant improvements in their health (mean change 0. People with clinically significant depression at baseline (52/289, 18%) also significantly reduced their depression symptom severity from mean 14.7 (SD 4.2) to mean 11.5 (SD 5.1).
The percentage change is reported in Multimedia Appendix 2. Across programs, 26.7% (66/247) to 49.8% (136/273) of the participants improved health outcomes over time, and 17.4% (41/235) to 53.8% (133/247) of the participants maintained their health.Most effect sizes (Table 3) were small (<0.3), except for change in health over time, which had an effect size of −0.37.

Participants
Overall, participants shared that they liked and enjoyed participating in the remote EBPs and identified several ways in which the program impacted their lives.First, they reported changes in their knowledge, attitudes, and practices in promoting their health and well-being.Participating in remote EBPs helped older adults manage their chronic conditions, become more active, and feel more confident and better equipped to take care of themselves:

The program helped me understand how exercise can improve my mobility, and it encouraged me to remain active. [Male WWE participant #1346; aged 63 years; living with arthritis and asthma]
Likewise, the remote EBP participants stated that they learned a lot from the program, including from the materials, leaders, and other participants.This was true across programs: WWE participants learned about new walking spots in their communities from other walkers in their cohort, while self-management program participants learned new ways and tips for managing their condition (expanded their sense of their own options) and broadened their understanding of what life was like at different severity levels of their condition.The group-based formats of the remote EBPs helped provide accountability and motivation and also provided a variety of perspectives and ideas.For some participants, the remote program helped them accept the reality and seriousness of their condition and the changes needed to manage it.In addition, participants believed that others could benefit from the program as well.
The participants also reported social benefits from being part of the programs.Many participants felt less alone, gained a sense of comfort from talking with others struggling with the same conditions, and made friends over the course of the program.A sense of camaraderie was reported often: I liked the interaction with other people, it's helpful to find out how others are going through.To know that there's other people out there with a lot of pain and they're struggling with it, made me feel not alone.Some are worse and some are better.It feels isolating a lot with pain, so that was really nice to experience, seeing others.[Female CPSMP participant #1344; aged 74 years; living alone and managing multiple chronic conditions] Furthermore, remote EBP participants from various programs who were grappling with new or existing chronic conditions, changes in mobility and function due to aging, and the challenges of physical distancing during the pandemic, emphasized the value of learning together and feeling less alone.Even participants in the remote WWE program (self-directed tool kit enhanced with a leader virtually supporting a cohort of participants) found social benefits: I did appreciate the opportunity to meet with the group assigned to me and get the encouragement to get out and walking.[Female WWE participant #1249; aged 65 years; living with arthritis and a mental health condition] In addition, older adults shared how remote EBPs improved their access to the programs.The remote format made accessing these programs during the pandemic both safe and very convenient, in particular for people living with chronic pain or disabilities.Some participants would not have been able to participate if the programs had only been offered in person.That said, a few participants did not like the remote format and found it more difficult to access.For these participants, the XSL • FO RenderX downsides of not being able to meet and connect in person or the challenges with participating via telephone, mail, or videoconferencing outweighed the benefits of participating in class.Examples include issues with the technology itself (eg, poor internet connections that made the videoconferencing software freeze up) and discomfort with using technology (eg, unfamiliarity with navigating Zoom functions or unease using a mobile phone where they cannot see other participants).It should be noted that technology encompasses using a telephone (landline, smartphone, or other mobile phone) in addition to laptop computers, tablet devices, and PCs; dial-up or broadband internet; and videoconferencing platforms such as Zoom and Webex.
Finally, many participants shared that they wanted to take part in the remote EBP again, and some participants shared that they desired more follow-ups after the program ended.For these participants, there was a sense of having missed or forgotten some of what had been taught and wanting to refresh their knowledge.Others felt that they needed the motivation of continued check-ins to keep using what they had learned.In addition, some were not sure how to get their questions answered after the program ended, how to sign up for other programs offered by the organization, or whether they were allowed to take part in the program again.This points to opportunities for future supports and services after remote EBP engagement, such as monthly check-ins via telephone, videoconferencing, or social media to "keep the feelings of motivation and community after the program ends...[to] meet or discuss what folks are doing and what works and encourage each other to keep going" (female DSMP participant #1190; aged 61 years; living with multiple chronic conditions [arthritis, hypertension, diabetes, heart troubles, and a mental health condition]).Post-EBP supports and services could also provide a way to reinforce and deepen knowledge and skills that are learned and practiced during a relatively brief program and offer ways for family, friends, and caregivers to support the maintenance of program gains as well as widen program benefits to other people in the participants' communities.

Providers
Table 4 summarizes provider's perspectives on the impact of remote EBPs.Some of these impacts were expected; for example, 3 in 4 providers (149/198, 75.2%) reported improved health outcomes for older remote EBP participants, and reducing social isolation and loneliness emerged as the most common benefit for both participants and EBP providers (leaders and managers).In addition, half of the providers (102/198, 51.5%) noted that connections to other supports and services were a participant benefit; while this typically occurs during in-person EBP delivery, remote EBP delivery allowed for sharing timely and ever-changing information about testing for COVID-19 infection and recommended safety protocols, as well as referrals to services that may have paused or been shifted due to pandemic-related closures or physical distancing requirements.• Kept participants safe while also allowing them to access the benefits of these programs.

•
The program was still effective, and the benefit it had on participants could be seen.

•
For leaders without access, some stopped delivering the program.
a N/A: not applicable (either the benefit was not one of the multiple-choice answer choices in the provider survey or did not emerge during the interview data analysis).
Some of the impacts were positive but unintended; for example, providers reported improved comfort using technology as a benefit for remote EBP participants.Increasing the usability of technology and reducing anxiety about technology are not original outcomes that in-person EBPs strived to impact, but in remote EBP delivery, opportunities arose for some participants to become more comfortable using technology (telephone, videoconferencing, or tool kits) for engaging in, and receiving, other supports and services.Comfort using technology was a more prevalent impact than access to technology, which aligns with other findings that access was a challenge even when organizations provided software or hardware [53].Other unexpected positive impacts from the providers' perspectives include that delivering EBPs remotely allowed them to continue providing the program to older adults throughout the COVID-19 pandemic, reach participants they had not reached before, connect participants to each other, learn how to use technology, and be able to continue working or volunteering during the pandemic.
We also gathered data via surveys and interviews on the unintended negative consequences of delivering EBPs remotely, an important aspect when evaluating the public health impact of these programs with an equity lens.For some leaders, teaching remotely was too difficult or disconnecting due to not being able to see people's nonverbal cues and having to work harder to teach technology, engage people, or address emotional issues such that they felt that "something was lost."Strategies for mitigating this reduced impact included reducing class sizes or duration (for both telephone and videoconferencing sessions) and requiring people attending videoconferencing sessions to

XSL • FO
RenderX have their camera on and having 2 leaders taking part so that one could focus on engagement while one managed technology.

Principal Findings
Our evaluation found that remote EBP participants showed improvements on various outcomes from program enrollment to 6-month follow-up, including their self-rated health, energy, sleep quality, loneliness, depressive symptoms, and technology anxiety, within and across programs.Some program participants also reduced their anxiety, pain, pain interference, physical activity and self-efficacy, and enhanced their technology usability.The quantitative findings yielded mainly small effect sizes.This may be due to the heterogeneity of the participants; in particular, participants who did not enter the program as lonely or inactive would have no room to improve over time.
The qualitative findings suggest that remote EBP participants improved knowledge, attitudes, and skills on how to live healthier lives; reduced social isolation and loneliness; and gained better access to programs.In addition, providers shared that they too benefited from delivering programs remotely by staying connected, having access to technology, and improving their own health and well-being.
In some cases, the findings support previous research on the effectiveness of the remote EBPs that were part of our evaluation; for example, 1 study of 213 videoconferencing CDSMP participants in rural and remote Ontario, Canada, found similar improvements in self-rated health, energy, and psychological well-being (a measure related to depressive symptoms) 4 months after their last class [54].Another study of 97 telephone plus tool kit CPSMP participants in Cleveland, Ohio, United States, also reported better pain outcomes immediately after program completion [17].Furthermore, a study of self-directed WWE participants (n=270) in rural and urban North Carolina, United States, who were living with arthritis found that participants also reduced fatigue at follow-up 1 year after program enrollment [21].Our findings are also comparable to those of similar health promotion programs, such as increased physical activity and reduced depression for a remote DSMP [55].
In other cases, our findings contrast with those of previous research; for instance, 1 study of remote-delivered EF [20] found that the participants (n=15) decreased their knee pain as measured by the Knee Injury and Osteoarthritis Outcome Score; however, all study participants had symptomatic knee osteoarthritis, and the postprogram outcome was measured directly after active intervention at 12 weeks.It may be that our evaluation participants did not significantly improve pain outcomes due to their less severe baseline pain.In addition, the study of CPSMP participants mentioned previously [17] found statistically significant improvements in sleep and depression; our evaluation participants too improved their sleep quality and reduced their depressive symptom severity, but this change was not statistically significant.Our differing findings may be due to our smaller sample size (47 vs 97), which lowered our power to detect significant change, or a longer follow-up period (6 months vs 6 weeks), during which improvements may have been attenuated.
Furthermore, it should be noted that our outcome evaluation was not designed to compare in-person EBP delivery with remote EBP delivery.We used a common set of outcome measures of interest to older adults, EBP delivery organizations, and policy makers; as such, some of these measures look at similar constructs as the in-person EBP effectiveness studies but use different instruments, and some of these measures that are newly being evaluated (eg, loneliness and depression) were not evaluated in the in-person EBP research studies.In addition, we used a different time period compared to studies that evaluated the effectiveness of in-person EBP delivery: we looked at changes between program enrollment and 6-month follow-up, regardless of program duration, whereas prior effectiveness research looked at pre-post change over time based on program duration [21,[56][57][58][59].Our sample sizes were also smaller than those of previous in-person EBP research studies; therefore, the lack of statistical significance may be due to a lack of power to detect change.That said, it may be helpful to contextualize our remote EBP outcome evaluation findings with those from in-person EBP outcome evaluations.Remote CDSMP evaluation participants reported improvements in self-efficacy, health, energy (fatigue), sleep quality, physical activity, and depression, all outcomes that were also reported in previous studies of in-person CDSMP [37,[60][61][62].Furthermore, remote CDSMP evaluation participants did not show the significant improvements in pain outcomes that were seen in research with in-person program participants.Remote CPSMP evaluation participants reported outcome improvements similar to those of in-person CPSMP participants regarding pain and pain interference [56,63], while our evaluation added evidence for remote CPSMP participant improvements in energy (fatigue), anxiety, and physical activity; these 3 outcomes were not assessed in in-person CPSMP studies.However, remote CPSMP participants did not report significant improvements in self-efficacy, as was reported in previous in-person CPSMP studies.Looking at the DSMP, both remote and in-person delivery participants showed reduced hypoglycemia symptoms, health, and depression [57].Earlier research with in-person program participants found improved self-efficacy as well, and remote program participants showed improvement in terms of their physical activity days.For EF, both remote and in-person program participants reported improvements in self-rated health as well as physical activity [64,65].This evaluation adds novel findings on improved sleep quality and the usability of technology for remote program participants.In addition, in-person EF participants have reported reduced depression in other studies [64].Looking at WWE, both in-person and remote program participants have demonstrated improvements in health, fatigue, and physical activity [21].Remote WWE participants reduced their loneliness, which was not assessed in previous studies of in-person WWE [21].Previous research of in-person WWE also found participant improvement in pain and self-efficacy [21].
Although EBPs shifting to remote delivery was necessitated by the pandemic-related lockdown and other restrictions [24], the findings suggest that remote delivery can improve access to XSL • FO RenderX quality health promotion programs outside the COVID-19 pandemic context; for example, a caregiver for a person living with dementia can join a physical activity program from home to support their own health while not having to find respite care for their loved one, an older adult living with chronic pain can join a class even if they are experiencing elevated pain levels that would make it hard to leave even their bedroom, and a program leader can deliver the program across a region in Spanish to participants who may not have access to language-specific health promotion.This is similar to other studies of remote EBPs (eg, the value for people living with cancer to be able to participate from a distance despite living with weakened immune systems [66]).
Adapting the format and channels through which an EBP is delivered [67] is considered an appropriate modifiable intervention characteristic to better align with the needs and preferences of participants, as well as organizational and geographic contexts in which a remote option improves program feasibility and sustainability [68].Implementation science increasingly recognizes the need for adaptations [69] to improve the intervention-context fit (eg, increasing both practical and value fit [70]; increasing EBP feasibility to a given context [71]; and making sure that the EBPs can be delivered for different systems, organizations, providers, and participants than was originally studied [72]), support people living with multiple chronic conditions because comorbidity is the rule rather than the exception [73], and ensure EBP effectiveness by evaluating the intervention in different settings with varying provider and participant attributes [74].Our complementary process evaluation found that delivering in-person EBPs remotely did not require modifications to core program components [53], suggesting that fidelity to the active ingredients of the program models was maintained.That said, some of the observed lack of improvement in health outcomes by remote EBP participants may be due to failures in implementation (eg, inadequate intervention dose due to technology issues) that negatively impacted program effectiveness [42].

Strengths and Limitations
Our evaluation comes with several strengths.First, we conducted a pragmatic evaluation across the country in partnership with policy makers and delivery organizations so that the findings would have direct implications for improving older adults' health equity.Specifically, we built upon trusting relationships and took time to foster new relationships to engage people most impacted by implementing remote EBPs; we also gathered qualitative data to put quantitative outcomes in context for future quality improvement [75].Second, we included multiple brief, validated health outcomes across various EBPs and diverse participants and providers, centering outcomes that are important and meaningful to participants, providers, organizations, and policy makers that were not measured in previous studies [76].These include social factors such as social isolation and loneliness and mental health factors such as depression and anxiety, which lead to premature mortality for older adults [77,78].While measuring multiple outcomes across heterogeneous populations and programs made it harder to see large effect sizes, it better reflects how organizations work (delivering multiple programs) and how participants view health more holistically rather than as just 1 primary outcome.It may also be that small effect sizes suggest that these programs provided primary or secondary prevention to delay the onset of more impairing symptoms and conditions that was not picked up from our measures or relatively study time frame [79].Third, collecting qualitative as well as quantitative data from the perspectives of both participants and providers provides consistent measures to compare across studies as well as stories and unanticipated outcomes to explore more broadly in future research.
Looking at limitations, first, using a single-group design with the lack of a comparison group or randomization means that we cannot attribute change in health outcomes over time to participating in the remote EBP.Second, we recognize that the lack of statistical significance when assessing for whether change was due to chance or program participation may be due to small sample sizes that lack power.This is partly why we collected qualitative data from both providers and participants: we wanted to understand the magnitude of change from their perspectives.Third, our response rate was lower than is typical in controlled research studies, with only half of the remote EBP participants (289/586, 49.3%) providing follow-up data.This was due to pandemic-related logistical and methodological challenges faced by our evaluation team, program providers, and older adult participants; conducting an evaluation during the COVID-19 pandemic has been deemed more challenging than conducting evaluations in conflict areas [80].This suggests limitations to both internal validity through biases such as selection bias (eg, persons who liked the program or had access to a mobile phone or the internet to complete the evaluation were overrepresented) and external validity, with the COVID-19 pandemic being a unique context, because of which our findings may not be generalizable (eg, were improvements in social connection due to the high level of disconnectedness faced by everyone during the pandemic-related lockdown?).Finally, our convenience sample only included group-based programs and as such may not be applicable to one-on-one EBPs.We also did not have access to data on the entire remote EBP delivery population; therefore, we cannot comment on the representativeness of our outcome evaluation sample.Our process evaluation [24] suggests that persons who have been historically underserved by EBPs (eg, people of color, those with disabilities, and those living in rural areas) can be reached through remote delivery; however, care must be taken to ensure that remote delivery does not widen the health inequities caused by the digital divide.

Conclusions
In closing, the findings from this outcome evaluation suggest that participating in remote EBPs can improve health, social, and technology outcomes of interest for older adults, providers, and policy makers (Textbox 1).Future policy and practice can better support remote EBPs, improving access for all older adults (regardless of income, geography, and ability) and strengthening delivery organizations.

XSL • FO
RenderX provided the original work, first published in JMIR Aging, is properly cited.The complete bibliographic information, a link to the original publication on https://aging.jmir.org,as well as this copyright and license information must be included.
a CDSMP: Chronic Disease Self-Management Program.b CPSMP: Chronic Pain Self-Management Program.c DSMP: Diabetes Self-Management Program.d EF: EnhanceFitness.
a CDSMP: Chronic Disease Self-Management Program.b CPSMP: Chronic Pain Self-Management Program.c DSMP: Diabetes Self-Management Program.

Table 1 .
Demographics of remote evidence-based program evaluation participants.

Table 2 .
Participant health outcomes at enrollment and 6-month follow-up by remote evidence-based program.Pre: health outcomes at program enrollment; follow-up: health outcomes at 6-month follow-up from program enrollment.P<.05 (paired 2-tailed t tests, except for EF, which used the Fisher exact test).

Table 3 .
Six-month effect sizes for remote evidence-based program participant health outcomes.

Table 4 .
Perceived benefits of remote evidence-based program (EBP) delivery for participants (from providers' perspectives) and for providers (n=198).