Severely Burdened Individuals Do Not Need to Be Excluded From Internet-Based and Mobile-Based Stress Management: Effect Modifiers of Treatment Outcomes From Three Randomized Controlled Trials

Background: Although internet-based and mobile-based stress management interventions (iSMIs) may be a promising strategy to reach employees suffering from high chronic stress, it remains unknown whether participants with high symptom severity of depression or anxiety also benefit from iSMIs or should be excluded. Objective: This study aimed to evaluate the efficacy of iSMIs in subgroups with high symptom severity and to test whether baseline symptom severity moderates treatment outcome. Methods: Data from three randomized controlled trials (N=791) were pooled to identify effect modifiers and to evaluate efficacy in subgroups with different levels of initial symptom severity. The outcomes perceived stress (Perceived Stress Scale, PSS), depression severity (Center for Epidemiological Depression Scale, CES-D), and anxiety (Hospital Anxiety and Depression Scale, HADS) symptom severity were assessed at baseline, 7-week postassessment, and 6-month follow-up. Potential moderators were tested in predicting differences in the change of outcome in multiple moderation analyses. Simple slope analyses evaluated efficacy of the iSMI comparing the intervention group with the waitlist control group in subgroups with low, moderate, and severe initial symptomology based on means and SDs of the study population. In addition, subgroups with clinical values of depression (CES-D≥16) and anxiety (HADS≥8) at baseline were explored, and response rates (RRs; 50% symptom reduction) and symptom-free (SF) status (CES-D<16, HADS<8) were reported. Results: Individuals with high stress (PSS≥30), depression (CES-D≥33), anxiety (HADS≥15), and emotional exhaustion (MBI≥5.6) benefited significantly from the intervention with great reductions of stress (dpost=0.86-1.16, dFU=0.93-1.35), depression (dpost=0.69-1.08, dFU=0.91-1.19), and anxiety (dpost=0.79-1.19, dFU=1.06-1.21), and effects were sustained at 6-month follow-up. Symptom severity moderated treatment outcomes, as individuals with higher symptom severity at baseline benefited significantly more from the intervention than individuals with lower symptom severity. Furthermore, 82.9% (656/791) of individuals had J Med Internet Res 2018 | vol. 20 | iss. 6 | e211 | p.1 http://www.jmir.org/2018/6/e211/ (page number not for citation purposes) Weisel et al JOURNAL OF MEDICAL INTERNET RESEARCH


Introduction Background
High chronic stress is linked to adverse psychological health outcomes. Left untreated, individuals suffering from high occupational stress can develop common mental disorders such as depression or anxiety [1]. Stress, depression, and anxiety are associated with productivity loss and absenteeism, and can negatively affect workplace safety [2,3].
Meta-analytic evidence on occupational interventions of the last two decades, aiming to improve mental health, has found varying evidence of benefit ranging from nonsignificant to moderate effect sizes depending on, for example, type of intervention, intervention content, and outcome categories [4,5]. In a systematic review, Martin et al [6] found overall small positive effects in pooled data of 17 studies, investigating whether different types of health promotion interventions in the workplace reduce depression (standardized mean difference, SMD=0.28, 95% CI 0.12-0.44) and anxiety (SMD=0.29, 95% CI 0.06-0.51). A systematic review and meta-analysis from 2014 identified nine workplace-based randomized controlled trials (RCTs) aimed at reducing the level of depression symptoms. Pooled effect size estimates showed the interventions to be superior to the control groups by a small positive effect (SMD=0.16, 95% CI 0.07-0.24) [7,8]. Richardson et al found a moderate effect size across studies in a meta-analysis of stress management interventions (SMIs) in occupational settings (d=0.53, 95% CI 0.36-0.69), significant moderate intergroup effects for anxiety (d=0.68), and a small to moderate effect for mental health (d=0.44) [9]. Although the range of effect size varies, occupational interventions and SMIs seem to bear the potential to improve psychological health outcomes. However, the previous studies did not examine whether there were differences in efficacy between subgroups with varying initial symptom severity. It therefore remains an open research question whether individuals with high clinical symptomology, that is, depression and anxiety, also benefit from SMIs.
One specific form of SMIs, which has proven to be effective in certain contexts, is internet-based and mobile-based stress management interventions (iSMIs) [5,10]. Advantages of iSMIs include the following: (1) individuals can avoid stigmatization by participating anonymously, (2) such trainings are flexible and adaptable to any work and life situation, (3) material can be revised as often as desired, (4) access to treatment and treatment uptake is facilitated by not having waiting times nor a limitation of resource distribution, and (5) fostering self-efficacy of participants.
Occupational iSMIs could reach individuals who would likely not seek psychological treatment [11,12]. This includes severely burdened individuals who, thus, have developed clinical profiles with high symptoms of depression and anxiety. Most individuals with depression and anxiety do not seek treatment, for example, because of fear of stigma [13]. SMIs may bear the potential to attract individuals who would not make use of mental health interventions explicitly labeled for targeting mental health, that is, depression or anxiety. However, it remains unknown whether individuals with high symptom severity also benefit from low threshold iSMIs, as its exploration, to date, is lacking.
It seems plausible that individuals with severe symptomology, for example, those who experience clinical levels of depression or anxiety, are too burdened to substantially improve their mental health through occupational SMIs not specifically designed to treat depression or anxiety. It may be that methods and techniques delivered in SMIs are not sufficient, as highly burdened participants may need more therapeutic support than generally provided in (i)SMIs. In addition, with regard to iSMIs, some individuals with high symptom severity may be overwhelmed and unable to apply psychological self-help strategies effectively into their daily lives.
Allowing severely affected individuals to participate in (i)SMIs, who are unlikely to benefit, could be problematic. Participation may result in aggravation, hopelessness, and deterioration of symptoms, and may delay, or in the worst case, inhibit affected individuals from seeking appropriate treatment in time, hence contributing to a chronification of symptoms [14]. In addition, this may result in unnecessary intervention delivery costs. Thus, it is crucial to investigate differential effects of SMIs, and to test whether such approaches are also effective in severely affected populations, or whether these individuals should rather be excluded from SMIs and referred to clinical treatment for psychological disorders.
Through moderation analysis, differential treatment effects can be investigated. However, primary studies are generally solely powered to detect overall treatment effects, and thus underpowered to adequately perform reliable subgroup and moderator analyses [15]. To overcome this issue, data can be pooled from single studies by adding individual participant data in one dataset for common analyses [16].

Objective
This study aims to investigate the effects of an iSMI in subgroups of individuals who experience severe levels of stress, depression, anxiety, emotional exhaustion, or insomnia at baseline, and to test whether these baseline indicators of clinical impairment moderate the intervention efficacy in the reduction of stress, depression, and anxiety. If proven effective for severely burdened individuals, iSMIs could be a crucial component in the amelioration of mental health in occupational health settings, irrespective of the severity level of clinical impairment populations experience.

GET.ON Stress Intervention
Secondary analyses were conducted based on pooled individual participant data from three RCTs evaluating the same iSMI (GET.ON Stress). The three studies comparing an iSMI to a waitlist control group (WLC) were identical in design, differing only in respect to guidance intensity [17]. The iSMI is based on the Lazarus and Folkman's transactional model of stress [18], and core components focus on problem solving [19,20] and emotion regulation [21,22]. The studies were approved by the ethics committee of the Philipps University Marburg (2013-20K, AZ 2013-35K, AZ 2012-43K) and registered in the German Clinical Trial Register under DRKS00005112, DRKS00005384, and DRKS00004749. Further details on the intervention can be found in a published study protocol [23].

Sample
Participants of the three studies were recruited via the occupational health program of a large health insurance company in Germany (company website, newspaper articles, and advertisements in the membership magazine), a study website, mass media (newspapers and television), and announcements by the Ministry of Education. Recruitment was open to the general working population and not restricted to members of a certain health insurance company. In total, 791 participants were included. Inclusion and exclusion criteria are provided in Textboxes 1 and 2, respectively. A standard procedure was followed when individuals showed a notable suicidal risk. They were advised to seek help from their general practitioner, local psychiatric emergency room, or to contact the official emergency number. Telephone numbers and information on relevant institutions were provided via email. The cut-off on the Perceived Stress Scale (PSS) was chosen to include participants with heightened stress based on 1 SD (6.2) above the mean (15.3) found in a large working population [26]. There was no participation cut-off for individuals with severe impairment because of critically high symptom severity of depression or anxiety. Figure 1 shows the flow of participants.

Measures
The outcomes stress, depressive symptom severity, and anxiety symptom severity were collected by self-report at baseline (T1), at postassessment 7 weeks after randomization (T2), and at 6-month follow-up (T3). Furthermore, potential moderators, emotional exhaustion and insomnia severity, were collected at baseline.

Primary Outcome
The primary outcome of the primary studies was perceived stress measured by the German version of the PSS (10 items; score range: 0-4; total score range: 0-40; alpha T1 =.75) [22,24]. Higher scores on the PSS indicate more severe perceived stress.

Missing Data
Treatment outcomes included perceived stress, depressive symptom severity, and anxiety symptoms. The dropout rate ranged from 7.5% (59/791) in the primary outcome stress at postassessment to 13.8% (109/791) at 6-month follow-up in the outcome insomnia severity. Overall, these are low rates of intervention dropout. Comparing the dropout rates between the iSMI and the WLC, it becomes evident that the outcome dropout rates are higher in the intervention condition (12.4%-21%) compared with the WLC (2.5%-6.6%).
Missing data of all outcomes were handled with multiple imputations (MIs) in accordance with the intention-to-treat principle [34,35]. Prerequisites of MI include that the data are missing at random and that the data are normally distributed. Using Little's missing completely at random (MCAR) test, it was determined that the missing data were MCAR. The test was not significant (χ 2 50 =40.2, P=.84). However, the outcome data were not normally distributed, which can potentially lead to implausible low or negative values after using MI. To verify post hoc that this did not affect the imputed data, it was tested whether outliers in the new dataset corresponded with missing data in the original dataset, which was not the case.
Missing data were handled by using the MI procedure in SPSS. Estimations of missing data were based on all available data for all outcome measures at all assessment points (T1, T2, and T3), as well as age, gender, and intervention condition (iSMI and WLC). In addition, 100 imputed datasets were created and aggregated into 1 dataset for the final analysis. Significance level was set to <.05.

Multiple Moderation Analyses
To identify subgroups with different levels of initial symptom severity and to assess the efficacy of the intervention for populations with severe impairment on changes in outcomes (stress, depressive, and anxiety symptoms), multiple moderation analyses (MMAs) were performed with the SPSS macro PROCESS [36]. MMAs are separate multiple regression models each containing 3 elements that are tested in 3 steps predicting the outcome: (1) main effect of the baseline variable, (2) main effect of the treatment condition (iSMI and WLC), and (3) interaction effect (baseline×treatment condition). If the interaction effect is significant in predicting the outcome, it is considered to be an effect modifier moderating the outcome. Effect modifiers of treatment outcome indicate that populations benefit significantly different from each other depending on initial symptom severity. Moderators of the difference in change of the outcomes will be reported for postassessment and 6-month follow-up.
To explore the direction of an interaction effect, improve interpretability, and provide specific estimations for the investigated subgroups of interest, follow-up simple slope analyses were performed. Baseline data on perceived stress, depression, anxiety, emotional exhaustion, and insomnia severity were split into 3 subgroups based on distributional values, mean (M), and SD of the study population (mean-1SD; mean; mean+1SD), and outcomes in each subgroup were explored. To compare efficacy of the iSMI and WLC in the respected subgroups at postassessment and 6-month follow-up, Cohen d, 95% CI, and the number needed to treat (NNT) were calculated. Baseline differences were controlled for by using the difference in changes of outcome from baseline to postassessment (T1-T2) and from baseline to 6-month follow-up (T1-T3) as dependent variable. The study variable was not included in the final analysis, as it was not associated with the primary outcome stress at 6-month follow-up. Cohen d can be interpreted based on 3 approximations: small effects (d=0.2), medium effects (d=0.5), and large effects (d=0.8) [37]. All continuous variables were standardized for interpretability.

Response Rates and Symptom-Free Status in Clinical Subgroups
In addition, effects in subgroups with clinical values of depression (CES-D≥16) and anxiety (HADS≥8) at baseline were explored. Results were reported as mean between-group differences, Cohen d, and NNT while controlling for baseline severity by using change scores from baseline to postassessment. Improvements at an individual level were investigated by assessing the number of participants who achieved a response rate (RR) through 50% symptom reduction and symptom-free (SF) status (CES-D<16, HADS<8). Differences between the intervention condition and control group were tested in a chi-square (χ 2 ) test.

Descriptive Statistics
Participants were aged 42.6 years (SD 9.7) on average at baseline, and the majority were female (76.7%, 607/791). Overall, the sample was highly educated with a minimum of 12 years of schooling and a degree (71.9%, 569/791) or mid-level education with 10 years of schooling and a degree (25.4%, 201/791), with only 2.5% (21/791) of persons having low education levels with no school degree or 9 years of schooling. Participants had an average value of 25.6 (SD 4.3) on the PSS. The mean of depressive symptomology (CES-D) was 23.9 (SD 8.6). Anxiety symptoms (HADS) were at an average of 11.1 (SD 3.4). The average of emotional exhaustion (MBI) was 4.7 (SD 0.7). Insomnia severity (ISI) was at a mean of 13.7 (SD 6.1). Most participants had clinically relevant levels of depression (82.9%, 656/791) and anxiety (85.3%, 675/791) at study uptake. Further details on baseline characteristics can be found in Table 1.
The classification in subgroups with low, moderate, and severe initial symptom severity is based on means and SDs of the study population. Means and SDs for all investigated subgroups  Table 2. All subgroups showed a reduction of symptom severity in favor of the iSMI in comparison with the WLC, and effects were sustained at 6-month follow-up.

Primary Outcome
Effect sizes for differences in change of perceived stress at postassessment were medium to large. The smallest effect was observed for individuals with low emotional exhaustion at baseline (d=0.57; 95% CI 0.22-0.91; NNT=3.18), and greatest effects were observed for those with high emotional exhaustion (d=1.16; 95% CI 0.78-1.54; NNT=1.7). Moderators of change in the outcome stress at postassessment were stress (beta=1.04, P=.01) and emotional exhaustion (beta=1.12, P=.008). Highly stressed individuals and individuals with high emotional exhaustion profited significantly more from the iSMI than individuals with low symptom severity (low: PSS T1-T2 d=0.65, MBI T1-T2 d=0.57; high: PSS T1-T2 d=0.98, MBI T1-T2 d=1. 16). Figure  2 shows the estimated course of symptom change in the iSMI compared with the WLC at postassessment and 6-month follow-up.
Looking at the effects at 6-month follow-up, moderate to large effects were found in change of stress, the smallest effect for stress (d=0.66; 95% CI 0.3-1.02; NNT=2.78) and the largest effect for emotional exhaustion (d=1.35; 95% CI 0.96-1.75; NNT=1.52). Stress (beta=0.96, P=.049) was the only moderator for change in stress at 6-month follow-up. Highly stressed individuals showed greater reduction in stress through the intervention than low-stressed individuals (low: PSS T1-T3 d=0.66, high: PSS T1-T3 d=1.27).   Table 3 shows the effects on change in the outcomes stress, depressive symptoms, and anxiety from baseline to postassessment for the 3 subgroups divided by baseline psychopathology (mean-1SD; mean; mean+1SD) of stress, depression, anxiety, emotional exhaustion, and insomnia severity. The table also displays the P values of the standardized regression coefficient of the interaction effect between group (iSMI and WLC) and potential moderators. P values below .05 indicate that symptom severity of a certain characteristic is a moderator of treatment outcome.

Mean+1SD
Mean Mean-1SD Moderators at postassessment     Table 4. Effects on change in perceived stress, depressive symptoms, and anxiety from baseline to 6-month follow-up for different subgroups of baseline psychopathology severity (mean-1SD, mean, and mean+1SD).

Principal Findings
This study aimed to explore whether iSMIs are effective in severely burdened employees and tested whether baseline indicators of impairment moderated treatment outcome. Highly burdened participants who showed high levels of stress, depression, anxiety, emotional exhaustion, or insomnia severity profited substantially from the intervention on all outcome measures with moderate to large intergroup effect sizes compared with the control condition, both at postassessment and at 6-month follow-up. Moreover, higher impairment (ie, depression, anxiety, perceived stress, and emotional exhaustion) was associated with greater symptom improvement over time. These findings are in line with (1) studies showing that internet-based self-help interventions can be effective in clinical populations [38][39][40], (2) studies showing that internet-based occupational health interventions specifically designed to target depression can be effective [41,42], and (3) moderator analyses showing that higher symptom severity is not associated with worse treatment outcome in low-threshold self-help interventions [19,43]. This study extends these findings by showing that highly burdened participants with high levels of stress, depression, anxiety, insomnia severity, or emotional exhaustion can also substantially profit from an intervention that is labelled and specifically designed to reduce negative consequences of occupational stress. Reasons may include that the iSMI contains techniques based on cognitive behavioral therapy (ie, problem-solving and emotion regulation), which have shown to be effective in treating depression and anxiety [21,44].
Many of the participants were first-time help-seekers, indicated by 86.5% having no prior experience with health-related training and less than half (39.3%) having prior experience with face-to-face therapy, although the majority of participants had already reached clinically relevant levels of depression (82.9%) and anxiety (85.3%) before study uptake. This indicates that stress management may be a valid entry point to reach highly burdened individuals who otherwise may not seek treatment. Stress is less stigmatized than depression and anxiety, and online treatment provides the necessary anonymity for uncertain individuals to initially seek help.

Limitations
The study also has some limitations. First, as common for randomized trials, there was an elaborate screening process at study entry. This may have caused individuals with lower self-efficacy and less motivation to dropout before study uptake, and those most likely to profit to continue. Second, considering the description of the study population, the variance was rather low, reducing the heterogeneity of variance to explore differences within subgroups of the study sample. Third, subgroups were created based on the mean and 1 SD above or below the mean, which is typical for discerning simple slopes. Ranges are therefore based on this study population and may vary for other populations. Fourth, overall participants had elevated symptoms of impairment because of the study inclusion criterion heightened stress (PSS≥22). Therefore, there was no exploration of individuals with very low symptom severity. Fifth, all exploration of subgroups was based on the same iSMI GET.ON Stress. Therefore, the findings may not be generalizable directly to other interventions, especially as the latest meta-analysis on iSMIs indicated high heterogeneity between studies and interventions [5]. Finally, the main finding of this study that individuals with severe impairment profit greatly from the iSMI cannot be generalized to individuals who show indications of suicidality, as the studies excluded individuals with high suicide risk at baseline. Unfortunately, this remains an unresolved issue many internet interventions face, how to adequately deal with at-risk individuals who show active interest in participation but whose monitoring throughout a low-threshold intervention, often, cannot be guaranteed.
Strengths of this study include the strong methodology of RCTs, and the pooling of individual participant data from different studies investigating the same intervention, hereby overcoming the issue of studies generally being underpowered to explore subgroups [16].

Conclusions
This study contains important implications for research and practice. First, populations who experience high stress, are clinically depressed, or have high anxiety can alleviate their disease burden and reduce symptom severity by participating in iSMIs. Second, occupational stress and stress management are effective entry points for initial contact with mental health interventions, and therefore, severely burdened individuals should be targeted for participation. Third, future studies should explore whether tailoring of iSMIs to clinical profiles leads to superior outcomes in severely affected individuals compared with standardized approaches. Finally, as employers benefit from healthy employees, they should consider offering iSMIs in routine occupational settings and encourage participation. Severely burdened employees, especially those with high levels of depression and anxiety, should not be excluded but rather motivated to participate in iSMIs.