Efficacy of a Mobile App-Based Intervention for Young Adults With Anxiety Disorders

Key Points Question Does the efficacy of a self-guided cognitive behavioral therapy mobile app for young adults with anxiety disorders, differ by incentive conditions used to promote user engagement? Findings In this randomized clinical trial of 59 young adults, change in anxiety from baseline to end of the intervention did not differ significantly between groups incentivized by loss of points, gain of points, or gain of points plus social support. However, there was a large and significant decrease in anxiety across incentive conditions. Meaning The findings of this study suggest that self-guided mobile cognitive behavioral therapy apps can be efficacious in improving anxiety symptoms, regardless of the type of incentive strategy used to promote engagement.


Introduction
Anxiety disorders have been increasing among 18-to 25-year-old individuals more rapidly than in any other adult age range, with recent past-month prevalence estimates reaching 15%. 1,2A US Surgeon General advisory 3 emphasizes that although the rate of change accelerated during the COVID-19 pandemic, psychiatric diagnoses among young people were already increasing before the pandemic.
6][7][8][9][10] These disorders can be treated effectively by psychotherapeutic interventions, and a majority of patients, particularly young adults, express a preference for psychotherapy over psychotropic medication. 11Cognitive behavioral therapy (CBT) is considered a gold standard for treatment of anxiety 12 on the basis of abundant evidence of efficacy. 13,14However, despite the existence of efficacious interventions, more than 25% of young adults perceive an unmet need for mental health treatment, 15,16 in part because of challenges in accessing such treatment.[18] Digital interventions, such as smartphone applications, allow treatment at home, are more affordable than paying for ongoing traditional mental health care, and do not rely on the ability to access a trained, local therapist.The increased prevalence of anxiety has not been met by an increased supply of mental health professionals, and digital access to evidence-based, self-guided CBT represents a scalable alternative.Self-guided CBT apps have demonstrated efficacy in adults with depression, [19][20][21] but few studies have investigated their efficacy in individuals with anxiety disorders. 22,23lf-guided mental health apps have been stymied by 2 limitations.First, most self-guided apps provide an incomplete representation of the psychotherapies on which they are based.In particular, they do not emphasize behavioral strategies (eg, exposure to anxiety-provoking situations), 24,25 a key therapeutic component of interventions for anxiety. 26,27Second, low user engagement diminishes their potential impact. 28,29Engagement may be improved by involving social support in the intervention 29 or by incorporating behavioral economics and gamification. 30,31e Maya app is a digital CBT intervention targeting anxiety in young adults that addresses the limitations of existing apps.The app teaches a comprehensive array of CBT skills that target common features of anxiety, with a focus on behavioral skills often omitted from other apps.1][32][33][34] Material is presented in an interactive format to encourage active participation, which leads to better learning and greater engagement than passive viewing of information. 35In addition to these features, the app can be used in conjunction with a text-based, gamified incentive system to further encourage engagement.

JAMA Network Open | Psychiatry
Mobile App-Based Intervention for Young Adults With Anxiety Disorders The primary aim of the current study was to compare the effects of 3 incentive strategies on the efficacy of the app in improving anxiety in adults aged 18 to 25 years with anxiety disorders.Given the powerful impact of social feedback in adolescence and early adulthood, [36][37][38] our primary hypothesis was that anxiety would decrease more for participants incentivized by receipt of reward points combined with social support than for participants incentivized by either gain or loss of reward points alone. 29,39,40The secondary aims of the study were to evaluate engagement and selfreported satisfaction with the app and to compare these results among the incentive conditions.

Study Design
This randomized clinical trial examined improvement of anxiety during the intervention accompanied by 1 of 3 incentive conditions to promote engagement.2][43] However, the full sample was not recruited due to disruption of research procedures during the COVID-19 pandemic.Participants were individuals aged 18 to 25 years with anxiety disorders, who were assigned to use the app for 6 weeks and were randomized to a loss-framed, gain-framed, or gain-social support incentivization.Anxiety was assessed at baseline, week 3 (midpoint), week 6 (end of intervention), and week 12 (follow-up).
Change in anxiety (Hamilton Anxiety Rating Scale [HAM-A] 44 ) at week 6 (end of intervention) was the primary end point.Approval for the trial protocol (Supplement 1) was obtained from the Weill Cornell Medicine Institutional Review Board before study initiation.Written consent was obtained remotely using an electronic version of the informed consent form that followed federal, state, and local regulations, as applicable.This report follows the Consolidated Standards of Reporting Trials (CONSORT) reporting guideline for randomized clinical trials.

Participants
Participants were recruited through online advertisements and the outpatient psychiatry clinics of Weill Cornell Medicine.Eligible participants had a primary anxiety disorder diagnosis, determined by a score of 4 or higher on the clinical severity rating of the Anxiety and Related Disorders Interview Schedule for DSM-5 (ADIS-5) 45 and were fluent in English.Race and ethnicity were self-reported by participants from categories defined by investigators to provide information about the representativeness of the sample.Participants were excluded if they were already participating in CBT, had changed psychotropic medication dosage in the past 12 weeks, or endorsed suicidal ideation with intent and/or plan (Figure 1).

Study Intervention
The content of the app was developed by clinicians and researchers at Weill Cornell Medicine, and the app is free to download.The current publicly available version of the app includes the same core skills as the version used for the current study but includes language with a greater focus on wellness and resilience, has fewer required exercises, and allows for greater user choice of activities.Participants were assigned to complete 12 sessions of the intervention during 6 weeks (2 sessions per week).
Modules included psychoeducational videos and quizzes, cognitive restructuring exercises, exposure exercises, mindfulness, and additional CBT skills (eFigure in Supplement 2).Each session included homework to be completed before the next session.

Incentive Conditions
The incentive conditions consisted of the following: (1) a loss-framed condition in which participants began each week with a specified number of points that they could lose for not completing assigned sessions on time; (2) a gain-framed condition in which participants could earn points for completing

Randomization
Randomization was conducted through the Way to Health platform, which generated the random allocation sequence.Participants were randomized to 1 of 3 incentive conditions according to a simple 1:1:1 nonblocked design through the Way to Health platform.Study research assistants (M.M.S.) enrolled participants and were masked to the randomization assignment.Because targeted enrollment was not achieved due to the COVID-19 pandemic and the 1:1:1 randomization scheme did not specify block randomization, participants were allocated unevenly to the 3 conditions.

Measures
The ADIS-5 45 was administered to determine psychiatric diagnosis and clinical severity rating at baseline and end point.The HAM-A 44 score was the primary measure of anxiety.The Anxiety Sensitivity Index (ASI) 51 and Liebowitz Social Anxiety Scale (LSAS) 52 scores were secondary measures of anxiety.The 24-item Hamilton Depression Rating Scale 53 was the measure of depressive symptoms (exploratory efficacy outcome).App engagement was measured in terms of the total number of sessions completed.Retention was assessed at week 6 (end of intervention) and week 12 (follow-up).Participant satisfaction was measured with the User Version of the Mobile Application Rating Scale (uMARS). 54Detailed descriptions of measures and training procedures are provided in the eMethods in Supplement 2.

Procedure
Study visits were conducted remotely via a Health Insurance Portability and Accountability Act-compliant videoconferencing platform (Zoom, versions 5.6.7-5.12.9;Zoom Video Communications Inc).The baseline assessment included the measures described above and a session to assist the participant with app setup.App programs were set to begin the Sunday after enrollment.Outcome measures and qualitative feedback questionnaires about the app were collected at weeks 3 (midpoint), 6 (end of intervention), and 12 (follow-up).The uMARS was assessed beginning at week 1 rather than at baseline to allow participants to use the app before rating it.The week 3 and 6 visits also included troubleshooting of technical difficulties.

Statistical Analysis
Statistical analyses were conducted with Jamovi, version 2.3.2.0, 55 a software package based on the R statistical language (R Foundation for Statistical Computing).To evaluate the effect of the intervention on primary (HAM-A) and secondary (ASI and LSAS) measures of anxiety, we used linear mixed-effects models with subject intercept as a random effect.All participants with baseline data were included following an intent-to-treat approach.Linear mixed-effects models used the restricted maximum likelihood estimation method, which accounts for missing values without removing entire participants from the analysis.We tested the main effect of time (baseline [week 0], midpoint [week 3], end point [week 6], and follow-up [week 12]) as a fixed effect, incentive condition (gain-social support, gain-framed, and loss-framed) as a fixed effect, and a time × incentive condition interaction.
To compare participant engagement (sessions completed) among the incentive conditions, because of the imbalance in sample size across conditions, we used a nonparametric Kruskal-Wallis test.To evaluate participant satisfaction with the intervention, we computed a linear mixed-effects model with a random effect of participants, fixed effects of time (week 1, midpoint, end point, and follow-up) and incentive condition (gain-social support, gain-framed, and loss-framed), a time × incentive condition interaction, and the total uMARS App Quality score as the outcome.For descriptive purposes, we calculated the percentage of the sample who rated the app quality as a 4 or greater on the uMARS, based on the published threshold for app satisfaction.
Post hoc 2-tailed, paired t test analyses of change in assessment scores from baseline to midpoint, end point, and follow-up used the Holm method to correct for multiple comparisons.
Effect sizes are reported in terms of Cohen d, in which d = 0.2 represents a small effect, d = 0.5 represents a medium effect, and d = 0.8 represents a large effect. 56A P value threshold of .05 was used to determine significance.Data analysis was performed from December 21, 2022, to June 14, 2024.Exploratory analyses are reported in the eMethods in Supplement 2.
Participants presented most frequently with generalized anxiety disorder (33 [56%]) followed by social anxiety disorder (24 [41%]).The mean (SD) baseline HAM-A score was 15.0 (6.5).Demographic and clinical characteristics of the sample are provided in Table 1.No participants reported serious adverse events.

Efficacy on Anxiety
There was a significant main effect of time on the primary measure of anxiety (HAM-A scores): anxiety was significantly lower at midpoint (week 3 mean difference, −3.For the secondary measures of anxiety, there was a significant main effect of time on the ASI and the LSAS.There were significant decreases in severity of anxiety sensitivity at midpoint (week 3 mean difference, −3.69; 95% CI, −6.22 to −1.17 (Figure 2).Effect sizes were moderate to large.There were no statistically significant main effects of incentive or time × incentive condition interactions for the ASI or LSAS (Table 2).

Engagement
Overall, 58 of the 59 participants (98%) completed the week 6 end point assessment, and 55 of 59

Discussion
Contrary to our primary hypothesis, we did not find evidence of a difference in efficacy among the 3 incentive conditions.However, after 6 weeks of use of the app, there was a clinically and statistically significant decrease in anxiety symptoms across conditions.This decrease in anxiety persisted at the week 12 follow-up.Study retention, app engagement, and participant satisfaction with the app were high across incentive conditions and did not differ by condition.All but 1 participant remained in the study during the intervention period, and a majority completed all intervention sessions.At least 68% of participants assigned a mean app quality rating greater than 4 of 5 at each time point.
The small size of comparator groups may have limited our ability to detect group differences by incentive condition.The absence of a group difference may also reflect a ceiling effect on the engagement and efficacy of the app.In addition to the incentive strategies used in conjunction with the app, the app itself includes numerous features intended to enhance both engagement and efficacy: a visually appealing user interface, a progress monitoring feature, interactive modules with activities designed to maintain users' attention, and comprehensive coverage of CBT and adjunctive skills.Moreover, all 3 incentive conditions involved either a gain-framed or a loss-framed component, both of which are efficacious in changing health-related behaviors. 400][61] Secondary analyses indicated that anxiety remained lower than baseline at a week 12 follow-up assessment (6 weeks after the end of the intervention) and did not differ between the end of the intervention and follow-up, suggesting potential persistence of benefit.This finding is consistent with prior work that found sustained effects of traditional therapist-led CBT after treatment termination. 62,63milar to overall anxiety, both anxiety sensitivity (fear of fear) and social anxiety decreased from baseline to the end of the intervention, and improvement persisted at follow-up.Our findings suggest that, as intended, the intervention may target specific anxiety symptom clusters in addition to overall anxiety.It remains to be determined whether individuals with clinical profiles characterized by anxiety sensitivity, social anxiety, or other symptoms would respond equally well to a more streamlined app that includes only the most relevant modules.
We measured high levels of engagement and participant satisfaction with the app across incentive conditions, supporting the feasibility of the app for young adults with anxiety.There was a 98% retention rate compared with the 50% retention reported for other randomized clinical trials of mental health apps, 64 and 64% of participants completed all required sessions compared with the 35% reported in the literature. 65The app could be scaled up for wider use because it does not require significant financial resources on the part of the user or a clinician.It is free to download and uses incentive systems based on points rather than monetary incentives.Additional discussion points are given in the eDiscussion in Supplement 2.

Limitations
This study had several limitations.Because the full sample could not be recruited as intended and we did not use block randomization, participants were not allocated evenly to the 3 incentive groups.
The unbalanced allocation and reduced sample size decrease our capacity to detect group differences. 66Another limitation was the absence of a control condition in which participants did not receive the app or received the app alone without additional incentives.We elected not to include such a control group to allow for the inclusion of 3 incentive conditions within the constraints of the available resources.However, this study design does not allow us to estimate the extent to which symptom changes reflect regression to the mean or placebo-like effects.That said, although untreated individuals show spontaneous improvement of anxiety, the effect size of this improvement is typically small. 67It is therefore unlikely that the moderate to large reductions of anxiety observed in the current study were spurious.Finally, the sample was predominantly female, college educated, and of Asian or White race and non-Hispanic ethnicity, limiting conclusions about generalizability to more diverse groups.

Conclusions
The

47 - 49
Text messages included reminders, gamification incentives (points and badges), and, where relevant, updates to the social support person with suggested messages of support (eg, "[Participant] has completed the third week of the program!Let me tell you about some achievements.[Participant] earned 1800 points for completing sessions on time.Encourage [Participant] to keep this up.").
This trial was intended to begin in March 2020, but study initiation was delayed until June 2021 after nonessential research activities at Weill Cornell Medicine were paused due to the COVID-19 pandemic.Therefore, enrollment was between June 16, 2021, and November 11, 2022.Our recruitment target of 120 participants was amended as a result, which reduced power and led to imbalanced comparison groups.The trial concluded when the end of the preplanned study period was reached.We report methods and results in accordance with guidelines for clinical trials affected by the pandemic.50

Figure 1 .68 Assessed for eligibility 9 7 not meet inclusion criteria 2 to participate 59 Randomized 30 Randomized to gain condition 30 Received intervention 3 30 Analyzed 2 0
Figure 1.CONSORT Diagram Illustrating Number of Participants Included at Each Point in the Study From Enrollment to Analysis

Figure 2 .
Figure 2. Results of the Linear Mixed-Effects Model Demonstrating Change in Outcome Measure by Time and Incentive Condition
a Data are presented as number (percentage) of participants unless otherwise indicated.bd Although all participants had a principal diagnosis of an anxiety disorder, anxiety disorders were considered a comorbidity if the participant had a second anxiety disorder in addition to the principal diagnosis.no

Table 2 .
Results of Linear Mixed Models for the Main Effect of Time Demonstrating Estimates of Mean Change by Time JAMA Network Open.2024;7(8):e2428372.doi:10.1001/jamanetworkopen.2024.28372(Reprinted) August 20, 2024 9/15 Downloaded from jamanetwork.comby guest on 08/31/2024 results of this study demonstrate the efficacy and feasibility of a comprehensive, scalable, selfguided mobile CBT intervention for young adults with anxiety, which did not differ significantly by incentive condition.The Maya app includes design features, an engaging user interface, and broad coverage of CBT techniques that address the limitations of existing digital mental health apps.In the context of a mental health crisis in young adults and numerous barriers to traditional mental health services, digital interventions represent a promising step toward wider dissemination of high-quality, evidence-based interventions.Such interventions have the potential to help with shortages of mental health professionals and long wait times and to address geographic and socioeconomic disparities in access to care.Treatment programs that incorporate apps may be a viable strategy to address the persistent lack of access to adequate mental health care 3 that exists even in resourcerich settings.64.Torous J, Lipschitz J, Ng M, Firth J. Dropout rates in clinical trials of smartphone apps for depressive symptoms: a systematic review and meta-analysis.J Affect Disord.2020;263:413-419.doi:10.1016/j.jad.2019.11.167 65.Linardon J, Fuller-Tyszkiewicz M. Attrition and adherence in smartphone-delivered interventions for mental health problems: a systematic and meta-analytic review.J Consult Clin Psychol.2020;88(1):1-13.doi:10.1037/ccp0000459 66. Hey SP, Kimmelman J.The questionable use of unequal allocation in confirmatory trials.Neurology.2014;82 (1):77-79.doi:10.1212/01.wnl.0000438226.10353.1c67.Scott AJ, Bisby MA, Heriseanu AI, et al.Understanding the untreated course of anxiety disorders in treatmentseeking samples: a systematic review and meta-analysis.J Anxiety Disord.2022;89:102590.doi:10.1016/j.janxdis.Therapeutic Themes, Symptom Targets, and Modules Included at Each Week of the Maya Program eMethods.Descriptions of Measures and Training Procedures eResults.Linear Mixed Model Results by Outcome eDiscussion.