Abstract
COVID-19 prompted distress and increased reliance on digital mental health interventions, which previously demonstrated low rates of retention and adherence. This single-arm trial evaluated whether self-guided, web-based, positive affect regulation skills (PARK) were engaging and associated with changes in well-being during the pandemic. Over 6 weeks, PARK delivers brief lessons and practices in skills designed to increase positive emotions: noticing positive events, savoring, gratitude, mindfulness, positive reappraisal, personal strengths, and self-compassion. Patient-Reported Outcome Measurement Information System (PROMIS) computer adaptive tests of anxiety, depression, social isolation, positive affect, and meaning and purpose were administered at baseline, post-intervention, and 6 months after baseline. Retention and usage of PARK were measured by the web-based assessment and intervention platforms. The sample (n = 616) was predominantly female, non-Hispanic, white, and well-educated. Of those who completed baseline, only 42% completed a follow-up assessment; 30% never logged into PARK. Among those who did, 86% used at least one skill, but only 14% completed PARK. Across retention and usage metrics, older age predicted more engagement. In multivariable models, people of color and people with greater baseline anxiety were more likely to complete PARK. All well-being indicators improved over time, with greater improvements in anxiety and social isolation among participants who accessed at least one PARK skill compared to those who did not. Retention and usage rates mirrored pre-pandemic trends, but within this select sample, predictors of engagement differed from prior research. Findings underscore the need for additional efforts to ensure equitable access to digital mental health interventions and research. Trials registration: NCT04367922.
Similar content being viewed by others
From the beginning of the COVID-19 pandemic, many recognized the potential for widespread changes in psychological well-being (Galea et al., 2020). Early US data showed that rates of distress increased sharply (Daly & Robinson, 2021), while happiness, life satisfaction, and other elements of flourishing declined (VanderWeele et al., 2020). To prevent deterioration of public mental health, evidence-based recommendations encouraged web-based dissemination of emotion regulation programs including reappraisal, mindfulness, and other coping skills (Park et al., 2021).
Positive psychological interventions (PPIs) specifically aim to increase emotional well-being and include many of these skills (Schueller et al., 2014). According to the Positive Pathways to Health Model, PPIs increase positive emotion, setting off a cascade of benefits such as improved coping, strengthened social connections, and decreased stress reactivity, resulting in better physical and psychological health (Moskowitz, Addington et al., 2019). Meta-analysis has confirmed that multicomponent PPIs decrease stress, anxiety, and depression; increase hedonic and eudaimonic well-being; and improve quality of life (Carr et al., 2020; Hendriks, Schotanus et al., 2019).
We previously demonstrated the efficacy of one such multicomponent PPI for adults facing health-related stress (Carrico et al., 2019; Moskowitz et al., 2017; Moskowitz, Cheung et al., 2019) and adapted it for self-guided, web-based delivery (Addington et al., 2019; Moskowitz et al., 2021). Given the likely need for approaches that would remain accessible during stay-at-home orders and other pandemic precautions, we used this as the basis for the web-based, self-guided Positive Affect Regulation sKills program (PARK), designed to improve coping with stress during the COVID-19 pandemic. PARK aims to increase daily experiences of positive emotion through brief, weekly lessons and daily practice of the following skills: noticing positive events, savoring, gratitude, mindfulness, positive reappraisal, personal strengths, attainable goals, and self-compassion. To evaluate the potential for PARK to improve psychological well-being among U.S. adults during a public health crisis, PARK was made available for a single-arm trial starting May 1, 2020.
Despite the relative low cost and potential for broad dissemination, rates of retention and adherence to digital mental health interventions (DMHI) tend to hover at or below 50% in randomized controlled trials (RCTs), and are even lower for open access sites/apps (Christensen et al., 2009; Fleming et al., 2018). For example, within self-guided apps available in the Google Play store to improve coping or emotional well-being, the median percentage of users who open the app 1 day after installation is around 70% and falls to or below 10% after 1 week (Baumel et al., 2019). In previous reviews, adult users of PPI and DMHI tend to be women, younger (vs. older adults), and people with higher socioeconomic status (SES; i.e., employed, higher education, and income), whereas mental health symptoms can interfere with usage (Borghouts et al., 2021; Christensen et al., 2009; Hendriks, Warren et al., 2019; Onyeaka et al., 2021). Meta-analysis has shown that greater usage of DMHI, in both self-guided and facilitator/therapist-guided formats, predicts better mental health outcomes (Gan et al., 2021). Thus, knowing who is likely to use an intervention is important for predicting who is likely to benefit from it.
The present study addresses this question in an open trial of a web-based PPI launched at the start of a global pandemic that significantly undermined well-being (Daly & Robinson, 2021; VanderWeele et al., 2020). Given homogenous samples in prior studies and low rates of engagement in many PPI and DMHI (Baumel et al., 2019; Christensen et al., 2009; Fleming et al., 2018; Hendriks, Warren et al., 2019), we examined baseline predictors of retention and usage, including demographic factors, perceived impact of the pandemic, and metrics of psychosocial well-being. We also assessed longitudinal change in well-being, according to pre-registered analytic plans (NCT04367922), and conducted exploratory tests of PPI usage as a moderator of well-being over time.
Method
Enrollment and Participation
Recruitment for the single-arm trial of PARK was conducted via online (e.g., ResearchMatch) and social media (e.g., Twitter) advertisements containing a link to an electronic screening form. Participants were eligible if they were age 18 or older, living in the USA, and able to access the internet and read English. People with an invalid email address were excluded. Eligible participants were able to automatically access the consent form, provide electronic documentation of consent, and directly begin the baseline (T1) assessment.
For all consenting participants who completed T1, study staff created their PARK accounts, which initiated automated emails providing access to the PARK website. Over 6 weeks, PARK delivers brief didactic material and practice in skills designed to increase daily experiences of positive emotion (see Table 1). Participants cannot skip ahead, but they can return to earlier skills and practices. The weekly lessons encourage participants to complete the current week’s practices each day, or as often as possible.
Participants receive daily emails asking them to report their levels of positive and negative emotions and positive and stressful events over the past 24 h (see “Self-reported Measures”); these daily emotion reporting emails also contain reminders to complete the skills practice. Additional features designed to increase PARK usage include (1) virtual badges: awards that participants can earn for activities such as logging into the PARK website, accessing the skills, and completing home practices; and (2) discussion boards that allow participants to share and discuss their home practice responses. No monetary incentives were provided for accessing PARK or completing assessments.
Self-reported Measures
Assessments were administered via REDCap at baseline (T1), post-intervention (T2, 8 weeks after baseline), and follow-up (T3, 26 weeks, approximately 6 months after baseline). Participants self-reported socio-demographics (e.g., age, race, ethnicity, and gender) and the impact of the pandemic on their financial status, access to resources, and psychological health (3 items each; rated 1–7; higher scores indicate greater impact). Patient-Reported Outcomes Measurement Information System (PROMIS) computer adaptive tests measured five well-being indicators: positive affect, meaning and purpose, depression, anxiety, and social isolation (Hahn et al., 2014; Pilkonis et al., 2011, 2014; Salsman et al., 2014). All five PROMIS outcomes are presented as T-scores (M = 50, SD = 10 in general population), with higher T-scores indicating more of the construct being measured (PROMIS Reference Populations, n.d.). PROMIS cut points [e.g., within normal limits, mild, moderate, and severe (PROMIS Score Cut Points, n.d.)] allow interpretion of clinical significance of T-scores, while a within-group change or between-group difference of 3 T-score points is accepted to represent a meaningful change or difference (Meaningful Change for PROMIS, n.d.).
During the intervention period, participants also receive daily surveys of emotions and events experienced in the past 24 h. Daily emotions are reported on the modified differential emotions scale, which asks participants to indicate how frequently they felt each of 20 positive and negative emotions (e.g., awe, glad, love; angry, sad, scared), using a scale ranging from 0 (never) to 4 (most of the time; Fredrickson, 2013). Participants additionally completed the 12-item daily inventory of stressful and positive events, where they indicated which events (e.g., being discriminated against; a positive interaction with someone) occurred and rated each one’s impact from 1 (not at all stressful/positive) to 4 (very stressful/positive; Almeida et al., 2002).
Objective Measures of Retention and Usage
Retention was calculated based on completion of REDCap-administered assessments, regardless of engagement with the PARK intervention website. Participants who completed baseline and T2 or T3 were classified as retained. Drop out was defined as completing only T1.
Usage was based on whether or not participants ever logged into the PARK website, number of skills accessed (0–10, see Table 1), number of different practices completed (0–9, see Table 1), and percent of total pages accessed within the PARK website. We defined completers as participants who accessed all 10 skills. We additionally report the mean number of days that participants completed the daily emotion reporting.
Analytic Strategy
These analyses include participants who consented between May 2, 2020, and August 26, 2021. We used independent t-tests and chi-square analyses to compare baseline variables between drop out vs. retained participants. Due to bimodal usage distributions (see “Results”; Figs. 1 and 2), we categorized usage into dichotomous variables: accessed at least 1 skill (yes vs. no), accessed at least 1 home practice (yes vs. no), accessed all 10 skills (yes vs. no), and accessed all 9 home practice assignments (yes vs. no). We examined bivariate relationships between baseline predictors (socio-demographic, COVID, well-being measures) and each of these four usage outcomes using independent t-tests for continuous predictors and chi-square or Fisher’s exact tests for categorical predictors. We then created multivariable logistic models for each usage outcome, adjusting for all baseline predictor variables; results are reported as odds ratios (OR).
To avoid over-estimation of missing data in the context of low rates of retention in assessments (see “Results”), longitudinal analyses of well-being include only participants who completed PROMIS measures at baseline (T1) and at least one of the follow-up points (T2 or T3). We first examined changes in well-being over time (T1-T2, T1-T3) using unadjusted longitudinal growth models with maximum likelihood estimation. Then we added the effect of usage (dichotomous: 0 vs. > 1 skills) and the interaction of usage and time (categorical, representing the 3 assessment points: 0, 8, 26 weeks). These adjusted models allow the slope to change between time points and examine whether linear change in well-being over time differed between participants who accessed none vs. any of the skills, allowing us to approximate a test of “control” vs. “intervention” effect in this single-arm trial.
Results
Enrollment and Retention
Of n = 1,070 who completed screening, 99% were eligible (Table 2). However, 36% never completed the consent form, 6% declined, and 2% withdrew after consenting. The final sample for this analysis includes n = 616 who consented and completed T1. The sample was predominantly female (74%), non-Hispanic (90%), white (75%), and well-educated (76% with college degree or higher; Table 3). Using PROMIS cut points, mean scores at baseline indicated moderate anxiety, mild depression, social isolation within normal limits, low levels of positive affect, and average levels of meaning and purpose.
More than half (n = 356, 58%) were non-respondents at both T2 and T3. Comparing this drop-out sample with those who completed T2 and/or T3 (Table 3), the retained sample was older and statistically less anxious; however, given that the between-group difference in anxiety was only 1.8 T-score points, this does not represent a meaningful difference. Retained participants demonstrated higher usage of PARK across all metrics.
Usage
Thirty percent of the baseline sample (n = 186 of 616) never accessed the PARK website after study staff created their account. Compared to those who logged in at least once, those who never accessed PARK were more likely to be single, separated, divorced, or widowed, rather than married/partnered (Table 4). They also reported greater pandemic-related impact on their resources, higher depression, higher anxiety, and lower meaning and purpose scores.
Usage is summarized in Table 3, and Figs. 1 and 2 show distributions of usage for number of skills accessed (0–10) and practiced (0–9). Of n = 430 who logged into PARK, n = 62 (14%) never accessed the intervention content. The same amount (n = 62, 14%) were completers, accessing all 10 skills. All usage metrics were highly correlated (number of skills and home practices: r = .85, p < .0001; number of skills and daily emotion reports: r = .70, p < .0001).
In bivariate analyses (Table 4), several baseline variables predicted PARK usage. Across metrics, older participants showed higher usage. In addition, non-Hispanic ethnicity and higher education predicted accessing > 1 (vs. 0) skills and higher social isolation predicted accessing > 1 (vs. 0) practice exercises. The only additional predictor of completion was that lower psychological impact of the pandemic predicted skill completion (accessing all 10 vs. < 10 skills).
In the multivariable logistic regression models predicting each usage outcome while controlling for all baseline variables (Table 5), accessing at least one PARK skill was more likely among participants who were married/partnered, but less likely among those without a college degree. Older age was the only significant predictor of accessing at least one practice. People of color and older participants showed higher odds of completing all 10 skills and all 9 practice assignments, while completion of home practice was also more likely among those with greater anxiety at baseline.
Change in Well-being
In unadjusted longitudinal growth models observing the change in PROMIS scores from baseline to post and from baseline to follow-up (Table 6), all five well-being metrics demonstrated statitistically significant improvements from baseline to post and from baseline to follow-up. Anxiety, depression, positive affect, and social isolation demonstrated meaningful change (> 3 T-score points) from T1 to T3; only meaning and purpose did not. Anxiety and depression were mildly elevated at baseline and fell within normal limits at follow-up. At all time points, social isolation was within normal limits, and both positive psychological outcomes (positive affect, meaning and purpose) were within the average range — although positive affect was just above the low to average threshold at baseline.
To approximate a test of intervention efficacy in this single-arm trial, we examined change in well-being over time among retained participants who accessed at least one skill (“intervention” group) compared to those who did not (“control”). The interaction between this usage group and time was significant only for anxiety and social isolation at T2 (Table 6). As shown in Figs. 3 and 4, participants who accessed at least one skill demonstrated significantly greater baseline to post-intervention decreases in anxiety and social isolation than participants who did not access any skills. There was no evidence that improvement in either of these well-being metrics from baseline to follow-up (T1 to T3) was moderated by usage. Further, we did not find evidence suggesting that improvements in any of the other well-being metrics (depression, positive affect, meaning and purpose) exhibited moderation effects by usage.
Discussion
At first glance, many findings from this single-arm trial of PARK, a self-guided, web-based program of positive emotion regulation skills designed to enhance coping with the stress of the COVID-19 pandemic, followed patterns seen in previous PPI and DMHI research. For example, the PARK sample primarily included non-Hispanic, white, well-educated women, mirroring socio-demographic characteristics seen in related trials (Hendriks, Warren, et al., 2019; Moskowitz et al., 2021). Like other studies describing psychological well-being during the pandemic (Daly & Robinson, 2021; VanderWeele et al., 2020), PARK participants reported clinically significant elevations in anxiety and depression and decrements in positive affect at baseline. With only 42% retained in assessments and 30% of participants never accessing the PARK website, retention and usage rates closely mirrored pre-pandemic trends of RCTs and publicly available apps (Baumel et al., 2019; Christensen et al., 2009; Fleming et al., 2018) — including another one specifically designed for coping with the pandemic, COVID Coach (Jaworski et al., 2021). Moreover, compared to participants who accessed the PARK website at least once, those who never logged in to PARK reported higher depression, anxiety, and impact of the pandemic on their resources, as well as lower meaning and purpose, reiterating earlier findings that psychological symptoms can deter DMHI usage (Borghouts et al., 2021).
Thus, even during a time that necessitated web-based delivery of most activities, this format can still fail to reach key segments of the population — namely, people of color, adults with lower socioeconomic status and limited access to resources, and those with a heavier psychological burden. Admittedly, our team did not strive to recruit a representative sample or engage in targeted recruitment and retention efforts. Psychological research and dissemination, including healthcare and the digitial health marketplace, should more actively heed guidelines for inclusive practices (Buchanan et al., 2021; Patalay & MacDonald, 2022).
Within the select sample who did access the PARK website, several predictors of usage ran counter to prior research. Across the variety of usage metrics we examined, we found greater usage of PARK among older participants. Whereas previous DMHI studies showed higher engagement among younger participants (Onyeaka et al., 2021), this contrary finding in PARK may stem from pandemic-related restrictions on in-person activities; for example, a web-based tool such as PARK might have been particularly accessible to middle-aged adults who were increasingly at home with children and older adults closely adhering to distancing precautions given their increased risk of COVID and its sequelae. Although we did not link individuals’ dates of participation with their local public health measures, pandemic-related reductions in the availability of in-person activities might also explain why usage of PARK home practice exercises was more common among participants with higher self-reported social isolation.
Participants who qualified as completers (i.e., they accessed all 10 skills or all 9 home practice exercises) also differed from trends in prior research. PARK completion was more likely among older adults, people of color, and participants with higher baseline levels of anxiety. These findings underscore the need to remove barriers to enrollment in clinical research and access to DMHI. They also parallel findings from COVID Coach, a publicly available app for coping with the pandemic, which demonstrated a positive association between psychological symptoms at baseline and days of app usage (Jaworski et al., 2021). Together, these findings indicate that, as the pandemic drove increased reliance on phone- and web-based mental health resources (Sorkin et al., 2021), it might also have shifted trends in who uses DMHI.
In simple longitudinal tests of PROMIS scores, all well-being metrics statistically significantly improved over time, with meaningful improvements seen in anxiety, depression, social isolation, and positive affect (i.e., all but meaning and purpose). Because PARK was not tested in a randomized controlled trial, these changes could have resulted from factors unrelated to the intervention. For example, studies have reported co-occurrence of declines in mental health and increases in pandemic-related stressors such as regional elevations in COVID prevalence and restricted mobility (Daly & Robinson, 2021; Santomauro et al., 2021). Psychological well-being in PARK participants could have fluctuated with local changes in the pandemic, which we did not measure.
Despite the single-arm study methodology, capitalizing on naturally occurring differences in PARK usage allowed us to approproximate a test of efficacy by comparing changes in well-being over time in participants who used at least some of the PARK skills vs. those who did not. These results provide preliminary evidence that using at least some of the PARK skills facilitated greater decreases in anxiety and social isolation. In particular, this was clinically significant for anxiety, where only participants who accessed PARK skills showed drops in anxiety from mildly elevated at baseline to within normal limits at follow-up.
In addition to the limitations of the single-arm design, this study did not account for all factors that could have influenced retention, usage, or participants’ well-being. For example, we did not measure factors previously demonstrated to predict DMHI usage, such as experience with technology and mental health literacy (Borghouts et al., 2021). Additionally, given that use of DMHI has been positively associated with COVID case rates (Sorkin et al., 2021), some attrition could have occurred during lulls in the pandemic. We did not thoroughly assess pandemic-related changes such as increased caregiving responsibilities, difficulty transitioning to remote work, and interpersonal conflict within the home, which contributed to global increases in depression and anxiety (Alzueta et al., 2021). However, in addition to PROMIS well-being scores, we included measures of COVID impact, which allow us to account for pandemic-specific perceived stress. In previous US samples, these individual-level responses to and appraisals of the pandemic were predictive of mental health outcomes, whereas objective threat of COVID was not (Nikolaidis et al., 2022).
Caveats about the select sample and relatively low engagement with PARK notwithstanding, we can draw several conclusions from this study. First, the stress of the pandemic led to a need for programs such as PARK, and many people have relied on DMHI to be nimble and responsive to this significant, global event (Park et al., 2021; Sorkin et al., 2021). Our team made PARK widely available by request, and among those who accessed it, multiple well-being metrics improved. Factors that previously were barriers to DMHI usage, such as older age and psychological symptoms (Borghouts et al., 2021; Onyeaka et al., 2021), in this case predicated greater engagement. Still, these findings from PARK alone are not proof that PPIs work and should be widely distributed. Instead the current results provide more information about how well-being may change among individuals enrolled in a PPI, along with individual (e.g., age) and contextual (e.g., social isolation) factors that may influence who is most likely to engage in programs like PARK. They also underline the importance of continuing to adapt our research practices and our interventions to ensure that science and health include everyone (Buchanan et al., 2021; Patalay & MacDonald, 2022).
References
Addington, E. L., Cheung, E. O., Bassett, S. M., Kwok, I., Schuette, S. A., Shiu, E., Yang, D., Cohn, M. A., Leykin, Y., Saslow, L. R., & Moskowitz, J. T. (2019). The MARIGOLD study: Feasibility and enhancement of an online intervention to improve emotion regulation in people with elevated depressive symptoms. Journal of Affective Disorders, 257, 352–364. https://doi.org/10.1016/j.jad.2019.07.049
Almeida, D. M., Wethington, E., & Kessler, R. C. (2002). The daily inventory of stressful events: An interview-based approach for measuring daily stressors. Assessment, 9(1), 41–55. https://doi.org/10.1177/1073191102091006
Alzueta, E., Perrin, P., Baker, F. C., Caffarra, S., Ramos-Usuga, D., Yuksel, D., & Arango-Lasprilla, J. C. (2021). How the COVID-19 pandemic has changed our lives: A study of psychological correlates across 59 countries. Journal of Clinical Psychology, 77(3), 556–570. https://doi.org/10.1002/jclp.23082
Baumel, A., Muench, F., Edan, S., & Kane, J. M. (2019). Objective user engagement with mental health apps: Systematic search and panel-based usage analysis. Journal of Medical Internet Research, 21(9), e14567. https://doi.org/10.2196/14567
Borghouts, J., Eikey, E., Mark, G., Leon, C. D., Schueller, S. M., Schneider, M., Stadnick, N., Zheng, K., Mukamel, D., & Sorkin, D. H. (2021). Barriers to and facilitators of user engagement with digital mental health interventions: Systematic review. Journal of Medical Internet Research, 23(3), e24387. https://doi.org/10.2196/24387
Buchanan, N. T., Perez, M., Prinstein, M. J., & Thurston, I. B. (2021). Upending racism in psychological science: Strategies to change how science is conducted, reported, reviewed, and disseminated. The American Psychologist, 76(7), 1097–1112. https://doi.org/10.1037/amp0000905
Carr, A., Cullen, K., Keeney, C., Canning, C., Mooney, O., Chinseallaigh, E., & O’Dowd, A. (2020). Effectiveness of positive psychology interventions: A systematic review and meta-analysis. The Journal of Positive Psychology, 1–21. https://doi.org/10.1080/17439760.2020.1818807
Carrico, A. W., Neilands, T. B., Dilworth, S. E., Evans, J. L., Gόmez, W., Jain, J. P., Gandhi, M., Shoptaw, S., Horvath, K. J., Coffin, L., Discepola, M. V., Andrews, R., Woods, W. J., Feaster, D. J., & Moskowitz, J. T. (2019). Randomized controlled trial of a positive affect intervention to reduce HIV viral load among sexual minority men who use methamphetamine. Journal of the International AIDS Society, 22(12), e25436. https://doi.org/10.1002/jia2.25436
Christensen, H., Griffiths, K. M., & Farrer, L. (2009). Adherence in internet interventions for anxiety and depression. Journal of Medical Internet Research, 11(2), e13. https://doi.org/10.2196/jmir.1194
Daly, M., & Robinson, E. (2021). Psychological distress and adaptation to the COVID-19 crisis in the United States. Journal of Psychiatric Research, 136, 603–609. https://doi.org/10.1016/j.jpsychires.2020.10.035
Fleming, T., Bavin, L., Lucassen, M., Stasiak, K., Hopkins, S., & Merry, S. (2018). Beyond the trial: Systematic review of real-world uptake and engagement with digital self-help interventions for depression, low mood, or anxiety. Journal of Medical Internet Research, 20(6), e199. https://doi.org/10.2196/jmir.9275
Fredrickson, B. L. (2013). Positive emotions broaden and build. In P. Devine & A. Plant (Eds.), Advances in experimental social psychology (Vol. 47, pp. 1–53). Academic Press. https://doi.org/10.1016/B978-0-12-407236-7.00001-2.
Galea, S., Merchant, R. M., & Lurie, N. (2020). The mental health consequences of COVID-19 and physical distancing: The need for prevention and early intervention. JAMA Internal Medicine, 180(6), 817–818. https://doi.org/10.1001/jamainternmed.2020.1562
Gan, D. Z. Q., McGillivray, L., Han, J., Christensen, H., & Torok, M. (2021). Effect of engagement with digital interventions on mental health outcomes: A systematic review and meta-analysis. Frontiers in Digital Health, 3, 764079. https://doi.org/10.3389/fdgth.2021.764079
Hahn, E. A., DeWalt, D. A., Bode, R. K., Garcia, S. F., DeVellis, R. F., Correia, H., Cella, D., & & PROMIS Cooperative Group. (2014). New English and Spanish social health measures will facilitate evaluating health determinants. Health Psychology, 33(5), 490–499. https://doi.org/10.1037/hea0000055
Hendriks, T., Schotanus-Dijkstra, M., Hassankhan, A., de Jong, J., & Bohlmeijer, E. (2019). The efficacy of multi-component positive psychology interventions: A systematic review and meta-analysis of randomized controlled trials. Journal of Happiness Studies. https://doi.org/10.1007/s10902-019-00082-1
Hendriks, T., Warren, M. A., Schotanus-Dijkstra, M., Hassankhan, A., Graafsma, T., Bohlmeijer, E., & de Jong, J. (2019). How WEIRD are positive psychology interventions? A bibliometric analysis of randomized controlled trials on the science of well-being. The Journal of Positive Psychology, 14(4), 489–501. https://doi.org/10.1080/17439760.2018.1484941
Jaworski, B. K., Taylor, K., Ramsey, K. M., Heinz, A., Steinmetz, S., Pagano, I., Moraja, G., & Owen, J. E. (2021). Exploring usage of COVID Coach, a public mental health app designed for the COVID-19 pandemic: evaluation of analytics data. Journal of Medical Internet Research, 23(3), e26559. https://doi.org/10.2196/26559
Meaningful Change for PROMIS. (n.d.). https://www.healthmeasures.net/score-and-interpret/interpret-scores/promis/meaningful-change
Moskowitz, J. T., Carrico, A. W., Duncan, L. G., Cohn, M. A., Cheung, E. O., Batchelder, A., Martinez, L., Segawa, E., Acree, M., & Folkman, S. (2017). Randomized controlled trial of a positive affect intervention for people newly diagnosed with HIV. Journal of Consulting and Clinical Psychology, 85, 409–423. https://doi.org/10.1037/ccp0000188
Moskowitz, J. T., Addington, E. L., & Cheung, E. O. (2019). Positive psychology and health: Well-being interventions in the context of illness. General Hospital Psychiatry, 61, 136–138. https://doi.org/10.1016/j.genhosppsych.2019.11.001
Moskowitz, J. T., Cheung, E. O., Snowberg, K. E., Verstaen, A., Merrilees, J., Salsman, J. M., & Dowling, G. A. (2019). Randomized controlled trial of a facilitated online positive emotion regulation intervention for dementia caregivers. Health Psychology, 38(5), 391–402. https://doi.org/10.1037/hea0000680
Moskowitz, J. T., Addington, E. L., Shiu, E., Bassett, S. M., Schuette, S., Kwok, I., Freedman, M. E., Leykin, Y., Saslow, L. R., Cohn, M. A., & Cheung, E. O. (2021). Facilitator contact, discussion boards, and virtual badges as adherence enhancements to a web-based, self-guided, positive psychological intervention for depression: Randomized controlled trial. Journal of Medical Internet Research, 23(9), e25922. https://doi.org/10.2196/25922
Nikolaidis, A., DeRosa, J., Kass, M., Droney, I., Alexander, L., Di Martino, A., Bromet, E., Merikangas, K., Milham, M. P., & Paksarian, D. (2022). Heterogeneity in COVID-19 pandemic-induced lifestyle stressors predicts future mental health in adults and children in the US and UK. Journal of Psychiatric Research, 147, 291–300. https://doi.org/10.1016/j.jpsychires.2021.12.058
Onyeaka, H., Firth, J., Kessler, R. C., Lovell, K., & Torous, J. (2021). Use of smartphones, mobile apps and wearables for health promotion by people with anxiety or depression: An analysis of a nationally representative survey data. Psychiatry Research, 304, 114120. https://doi.org/10.1016/j.psychres.2021.114120
Park, C. L., Finkelstein-Fox, L., Russell, B. S., Fendrich, M., Hutchison, M., & Becker, J. (2021). Psychological resilience early in the COVID-19 pandemic: Stressors, resources, and coping strategies in a national sample of Americans. American Psychologist, 76(5), 715–728 no pagination specified-no pagination specified. https://doi.org/10.1037/amp0000813.
Patalay, P., & MacDonald, A. W. (2022). New titles can give new perspectives: Reflections on language and equity in clinical science. Journal of Psychopathology and Clinical Science, 131(1), 1–3. https://doi.org/10.1037/abn0000739
Pilkonis, P. A., Choi, S. W., Reise, S. P., Stover, A. M., Riley, W. T., Cella, D., & PROMIS Cooperative Group. (2011). Item banks for measuring emotional distress from the Patient-Reported Outcomes Measurement Information System (PROMIS®): Depression, anxiety, and anger. Assessment, 18(3), 263–283. https://doi.org/10.1177/1073191111411667
Pilkonis, P. A., Yu, L., Dodds, N. E., Johnston, K. L., Maihoefer, C. C., & Lawrence, S. M. (2014). Validation of the depression item bank from the Patient-Reported Outcomes Measurement Information System (PROMIS) in a three-month observational study. Journal of Psychiatric Research, 56, 112–119. https://doi.org/10.1016/j.jpsychires.2014.05.010
PROMIS Reference Populations. (n.d.). https://www.healthmeasures.net/score-and-interpret/interpret-scores/promis/reference-populations
PROMIS Score Cut Points. (n.d.). https://www.healthmeasures.net/score-and-interpret/interpret-scores/promis/promis-score-cut-points
Salsman, J. M., Lai, J.-S., Hendrie, H. C., Butt, Z., Zill, N., Pilkonis, P. A., Peterson, C., Stoney, C. M., Brouwers, P., & Cella, D. (2014). Assessing psychological well-being: Self-report instruments for the NIH Toolbox. Quality of Life Research, 23(1), 205–215. https://doi.org/10.1007/s11136-013-0452-3
Santomauro, D. F., Herrera, A. M. M., Shadid, J., Zheng, P., Ashbaugh, C., Pigott, D. M., Abbafati, C., Adolph, C., Amlag, J. O., Aravkin, A. Y., Bang-Jensen, B. L., Bertolacci, G. J., Bloom, S. S., Castellano, R., Castro, E., Chakrabarti, S., Chattopadhyay, J., Cogen, R. M., Collins, J. K., et al. (2021). Global prevalence and burden of depressive and anxiety disorders in 204 countries and territories in 2020 due to the COVID-19 pandemic. The Lancet, 398, 1700–1712. https://doi.org/10.1016/S0140-6736(21)02143-7
Schueller, S., Kashdan, T., & Parks, A. (2014). Synthesizing positive psychological interventions: Suggestions for conducting and interpreting meta-analyses. International Journal of Wellbeing, 4(1), Article 1. https://internationaljournalofwellbeing.org/index.php/ijow/article/view/310
Sorkin, D. H., Janio, E. A., Eikey, E. V., Schneider, M., Davis, K., Schueller, S. M., Stadnick, N. A., Zheng, K., Neary, M., Safani, D., & Mukamel, D. B. (2021). Rise in use of digital mental health tools and technologies in the United States during the COVID-19 pandemic: Survey study. Journal of Medical Internet Research, 23(4), e26994. https://doi.org/10.2196/26994
VanderWeele, T. J., Fulks, J., Plake, J. F., & Lee, M. T. (2020). National well-being measures before and during the COVID-19 pandemic in online samples. Journal of General Internal Medicine, 36, 248–250. https://doi.org/10.1007/s11606-020-06274-3
Acknowledgements
The authors are grateful to research team members Amanda Summers, Caroline Leong, and Veronika Grote for enrolling participants and creating participant accounts on the PARK website.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Funding
Research reported in this publication was supported, in part, by the National Institutes of Health’s National Center for Advancing Translational Sciences, Grant Number UL1TR001422. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Competing Interests
The authors declare no competing interests.
Data Availability
The datasets generated during and/or analyzed during the current study are available in the OSF repository, https://osf.io/q845h/.
Ethics Approval
This study was approved by the Northwestern University IRB.
Informed Consent
All participants consented to the research before beginning the study.
Additional information
Handling Editor: Wendy Berry Mendes
Rights and permissions
About this article
Cite this article
Addington, E.L., Cummings, P., Jackson, K. et al. Exploring Retention, Usage, and Efficacy of Web-Based Delivery of Positive Emotion Regulation Skills During the COVID-19 Pandemic. Affec Sci 4, 118–130 (2023). https://doi.org/10.1007/s42761-022-00135-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s42761-022-00135-4