Economic evaluations of eHealth interventions targeting mental health problems in the workplace: a systematic review

Background: work-related mental health problems impose significant economic and personal burdens. eHealth interventions may offer low-cost, practical solutions, but guidance on their cost-effectiveness in workplace mental health is limited. Objective: the objective of this study was to systematically review economic evaluations of workplace eHealth interventions for mental health, offering insights into methodologies and cost-effectiveness outcomes. Methods: Adhering to PRiSMA guidelines, searches were conducted in embase, MeDliNe, web of Science, Google Scholar, cochrane library, Psycinfo and econlit databases in May 2022, selecting peer-reviewed papers that performed economic evaluations on workplace eHealth interventions for adult mental health. Quality was assessed using the Drummond checklist. Results: From 3213 references, eight met the inclusion criteria. these studies varied in economic perspective, types of economic analysis type, primary outcome measures, intervention focus (e.g. stress, alcohol, insomnia & return-to-work) and direct non-medical costs. Five eHealth interventions were found to be cost-effective and/or have a positive return on investment, with seven studies rated as high quality according to the Drummond checklist. Conclusions: the study outcomes unveiled the potential cost-effectiveness of eHealth interventions targeting mental health issues, particularly these focusing on workplace stress. However, generalization is challenging due to variations in the methodologies across studies.


Introduction
Mental health problems substantially contribute to both absenteeism and reduced productivity in the workplace.In 2018, an estimated one in nine adults, constituting 11% of the population on average across European Union (EU) countries, reported experiencing a mental health problem (OECD & European Union, 2020).These mental health challenges incurred costs exceeding 4% of the Gross Domestic Product (OECD & European Union, 2018).Throughout the course of the COVID-19 pandemic, there has been a significant increase in the prevalence of mental health problems (Wu et al., 2021).Within this figure, a growing occurrence of work-related mental health problems among the working population has been observed (European Commission, 2016).Estimates from 2019 indicate that 15% of working-age adults were affected by a mental disorder (World Health Organization, 2022).In the United Kingdom alone, the annual toll of work-related stress, depression or anxiety results in the loss of over 12.8 million working days (Buckley, 2019).The foremost reasons cited for work-related stress, depression or anxiety encompassed issues such as excessive workload, lack of support and changes at work (Harvey et al., 2017).According to an Australian study, employees avoid discussing work-related mental health problems with their managers (Direct Health Solutions, 2014).In a survey conducted in the USA, it was revealed that employees who did discuss work-related mental health concerns with their line manager, received assistance in only 40% of the cases.This often resulted in referrals to mental health treatment (26%) or relaxation training.Typically, work-related issues are inadequately addressed.(Anxiety and Depression Association America, 2005) According to a European survey, half of the surveyed European workers (50%) identified work-related stress as a prevalent issue in their workplace.This phenomenon has been associated with various challenges, including presenteeism, elevated accident levels (particularly notable in industries such as construction), increased absenteeism, disability (European Parliamentary Assembly Committee on Social Affairs, Health and Sustainable Development, 2018), and, in the long run, unemployment (Hilton et al., 2008;Maehlisen et al., 2018).Mental illness at work and work-related stress can also lead to physical problems such as obesity, diabetes mellitus, breast cancer, heart disease (Bajorek & Bevan, 2019;Hassard et al., 2018), and a higher risk of premature death (OECD & European Union, 2018).The associated burden and economic costs of this population to society have been significant (Li et al., 2021).Although common mental disorders such as anxiety and depressive disorder are treatable or even preventable (National Institute for Health and Care Excellence, 2022;Nederlands Huisartsen Genootschap, 2019, 2022), barriers have held people back accessing mental healthcare.Examples of these barriers include the limited availability of physical resources (e.g.therapists) (World Health Organization, 2013) and the associated stigma (Wahl, 2012).Hence, many organizations advertise eHealth interventions as a practical, low-cost way to alleviate mental problems among their staff (Gajarawala & Pelkowski, 2021).They are often available in guided or self-help formats through the internet and therefore hold the potential to streamline the delivery of care, leading to time and cost savings.This encompasses a reduction in both treatment and patient costs (Eells et al., 2014).Additionally, the anonymity afforded by these interventions proves advantageous in addressing the associated stigma related to mental health concerns (Postel et al., 2008).
A growing number of effectiveness studies and economic evaluations are available on eHealth interventions applied in the field of mental healthcare.A previously conducted systematic review has revealed that eHealth interventions designed for the general population targeting depression, anxiety, smoking cessation, and alcohol consumption, are often more effective and less costly than conventional care approaches (e.g.cognitive behavioral therapy) (Donker et al., 2015).The growing incidence of work-related mental health problems has spurred the development of eHealth interventions specifically designed to address mental health in the workplace.These interventions target prevention and treatment within the workplace, as well as the promotion of return-to-work (RTW).Moreover, studies on effectiveness, such as the work by Stratton et al. (2017), have demonstrated that these interventions are frequently more effective than conventional care approaches.
While the application of eHealth interventions in the workplace has become more common, and economic evaluations of work-focused mental health interventions in general have increased (Gaillard et al., 2020), there is currently a lack of systematic insight into the cost-effectiveness of eHealth interventions in the workplace.Consequently, there is insufficient guidance on whether and which eHealth interventions prove to be cost-effective in addressing mental health problems in the workplace.Given the potential for these interventions to reduce costs and improve effectiveness, and acknowledging the existing gap in the literature, a systematic review on the costs, effectiveness and cost-effectiveness of eHealth interventions addressing mental health problems in the workplace is conducted.The information provided in this study can serve as valuable information for decision-makers involved in allocating healthcare investments, operating at the macro (national) and micro (organizational) level.

Methods
This systematic review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (Page et al., 2021).

Data sources and search strategy
The systematic review was performed in the Embase, MEDLINE, Web of Science, Google Scholar, Cochrane library, PsycINFO and EconLit databases.The databases were searched on 24 May 2022.In the search strategy terms indicative of economic evaluations (e.g.cost-effectiveness) mental health (e.g.anxiety), eHealth (e.g.internet) and work (e.g.occupation) were included.Moreover, to be comprehensive, no restrictions were based on publication year.The final search strategy for each database is included in the Supplementary File.The results of the search strategy were imported into the reference management tool RefWorks.

Eligibility criteria
Papers were eligible for inclusion if they met all the following inclusion criteria: • English-written and published articles from inception to 24 May 2022 • Reporting on an economic evaluation of an eHealth intervention targeting mental health problems in the workplace, including interventions targeted at the prevention or treatment, and/or interventions targeted at the promotion of RTW after absence due to mental health problems.• Peer-reviewed • Participants were adults (18 years and older) and paid employees.• Clinical studies with both a control and intervention group Systematic reviews with or without meta-analyses, study protocols and studies without full-text availability were excluded from the systematic review, as were duplicates after completing the search, and studies that did not report on the collection of empirical data and an empirical assessment.

Study selection
The study selection was conducted by two reviewers, using a standardized Excel workbook (VonVille, n.d.).First, the titles and abstracts of the identified publications were independently assessed for eligibility.Once consensus was achieved between the reviewers, the full-text versions of potentially eligible publications were retrieved and further examined.This was also executed for the publications for which the title and abstract provided insufficient detail to determine eligibility.Publications that met the inclusion criteria, as agreed upon by both reviewers, were selected.Furthermore, forward, and backward reference searching within the included publications was conducted to identify potential eligible publications that did not result from the search strategy.Reviewers' disagreements were either resolved through consensus or by the determination of a third impartial reviewer.

Data abstraction
The data abstraction was performed by two reviewers using an Excel-based data abstraction form.This data abstraction form (Supplementary File) included the recommended items for economic evidence (Wijnen et al., 2016).These were items on general study characteristics (e.g.author, year of publication, study perspective, type of economic evaluation), study methods (e.g.methods for identifying, measuring, and valuing costs and effect) and outcomes.The outcomes also contained information about the means and dispersions (i.e. standard deviations (SDs), standard errors (SEs) and 95% confidence intervals (CIs)) from the: (incremental) effectiveness, (incremental) costs per perspective incremental cost-effectiveness ratio (ICER), and incremental cost-utility ratio (ICUR)).

Data analysis and presentation
Data on incremental effectiveness (QALYs and disease-specific outcomes), costs, and cost-effectiveness were depicted in Figure 2 to summarize the study outcomes.The purpose was to investigate whether there was a collective pattern in the results.In order to create a unified presentation of the studies in a figure, recalculations were conducted.Regarding the costs (Figure 2(C)), this involved indexing the currencies to the reference year 2022 and converting them to the international standard currency (International dollar (Int$)), through the application of the Purchasing Power Parity (PPP) (International Monetary Fund, 2022).Concerning the disease-specific outcomes (Figure 2(B)), a distinction was made between effects observed in treatment responders and effects pertaining to the mean reductions in symptoms (or questionnaire scores).The effects observed in treatment responders had first to be recalculated in an Odds Ratio (OR), using the following formula (Szumilas, 2010): ) .
The cost-effectiveness results were expressed in Incremental Net Monetary Benefits (INB) (Figure 2(D)), representing the difference in net monetary benefit between the intervention and control groups (Noparatayaporn et al., 2021).A negative INB (INB < 0) indicated that an intervention was not as cost-effective as the control condition.Conversely, a positive INB (INB > 0) indicated that the intervention was cost-effective compared to the control condition (Noparatayaporn et al., 2021).The INB was calculated as follows: (Noparatayaporn et al., 2021).In this formula, K is the cost-effectiveness threshold, ΔC are the incremental costs, and ΔE are the incremental effects.The threshold was set on UK's maximum threshold per QALY: ₤30,000, or 44,643 Int$ after conversion to International dollars (Int$) (Donker et al., 2015).

Quality assessment
The quality of the included studies was assessed using the Drummond 35-item checklist (included in the Supplementary File) (Drummond & Jefferson, 1996).An item was scored with a "yes", "no", "not clear" or "not applicable" based on the information provided in the study.Items were scored "not clear" when reported, but with insufficient detail.If the hypotheses, research question or objectives/aims were mentioned, the study question (item 1) was rated favorably.Item 9 (regarding the details of the design and results of the effectiveness study) was rated favorably, if the authors sufficiently addressed; the selection of the study population, the method of allocation of subjects, blinding, whether analyzed by ITT, the effect size with Cis, and a reference to the published source.Items 10, 20 and 21 were considered not applicable because this systematic review only included trial-based economic evaluation studies and excluded meta-analyses (item 10) and modelling studies (item 20-21).
The results from the quality assessment were expressed as a percentage of the maximum possible score.Studies with a quality score of ≥75%, >50% to ≤75% or ≤50% were considered; high quality, moderate quality or low-quality, respectively (Hamberg-van Reenen et al., 2012).The quality assessment was executed by the authors.

Results
The systematic search of all databases resulted in 3213 references.After the removal of duplicates, a total of 1876 references remained, for which the titles and abstracts were screened.The title and abstract screening resulted in the exclusion of an additional 1876 references.Full-text versions of the remaining 47 publications were available, obtained and reviewed.Based on these full-text readings another 39 publications were excluded.The primary reason for exclusion was that studies did not examine eHealth interventions or did not target mental health problems in the workplace (85%; 1591/1876).The second reason for exclusion was that publications did not report on an economic evaluation (8%; 157/1876).These publications, for example, reported on costs or effects only.Another 6% (113/1876) of the publications were excluded because they did not use an empirical assessment.These publications included modelling studies, study protocols and systematic reviews.Finally, less than 1% (7/1876) of the publications were excluded because they were either not written in English or not peer-reviewed.
Backward citation searching yielded one additional publication (Thiart et al., 2016).After the quality assessment, eight publications were included.The flow chart of the study selection process is presented in Figure 1.

General characteristics of the included studies
The general characteristics of the included publications are presented in Table 1.Publication dates of the eight included papers ranged between 2014 and 2022, and the number of participants varied between 128 and 637.The eight included publications reported on a total of seven unique eHealth interventions.Two studies performed an economic evaluation alongside the same RCT in which an identical eHealth intervention, aiming to reduce work-related stress, was evaluated (Ebert et al., 2018;Geraedts et al., 2015).All studies performed a RCT, or cluster RCT.Six studies recruited participants from different working sectors (e.g.banking companies and research institutes) in the Dutch (Geraedts et al., 2015;Lokman et al., 2017), German (Buntrock et al., 2022;Ebert et al., 2018;Kählke et al., 2019) and British working population (Phillips et al., 2014).Two studies recruited participants employed in one working sector (nurses and schoolteachers respectively) (Noben et al., 2014;Thiart et al., 2016).One eHealth intervention aimed to prevent impairments in work-functioning by preventing and reducing multiple mental health problems (distress, work-related fatigue, risky drinking, depression, anxiety, and post-traumatic stress disorder) (Noben et al., 2014).Three eHealth interventions aimed to improve work-functioning by reducing and/or preventing work-related stress and included participants with moderate or more severe symptoms of stress and/or depression (Ebert et al., 2018;Geraedts et al., 2015;Kählke et al., 2019;Phillips et al., 2014).One eHealth intervention aimed to improve work-functioning by treating insomnia and included participants with moderate to severe symptoms of insomnia (Thiart et al., 2016).One eHealth intervention aimed to promote RTW after absence due to mental health problems and included participants with moderate to more severe symptoms of anxiety and/or depression (Lokman et al., 2017), and one eHealth intervention aimed to improve work-functioning by treating problematic alcohol use in participants with severe alcohol use disorder (Buntrock et al., 2022).The eHealth interventions were either specifically developed for, and used by employees (Ebert et al., 2018;Geraedts et al., 2015;Kählke et al., 2019;Lokman et al., 2017;Thiart et al., 2016) or not specifically developed for employees, but developed for the prevention and treatment of common mental health problems, and used by employees (Buntrock et al., 2022;Noben et al., 2014;Phillips et al., 2014).The interventions were mainly based on problem solving, emotion regulation and and/or cognitive behavior therapy.As comparative treatment, four studies used a waitlist control group (WLC) (Buntrock et al., 2022;Ebert et al., 2018;Kählke et al., 2019;Thiart et al., 2016) and another three studies used care-as-usual (CAU) (Geraedts et al., 2015;Lokman et al., 2017;Noben et al., 2014), in which two studies received personal feedback about their results (Geraedts et al., 2015;Noben et al., 2014) and in the other study usual sickness guidance was provided (Lokman et al., 2017).One study used an attentional control group, in which the intervention was partly provided; five websites offering general insights on mental well-being (Phillips et al., 2014).
It is important to note that two studies included two separate intervention conditions.One study included an occupational physician (OP) intervention and an eHealth intervention, both of which were compared to the control condition (Noben et al., 2014).Given the eligibility criteria of this study, the health economic results regarding the OP intervention were not included (as this is not an eHealth intervention).The other study included an unguided and a guided intervention group.Participants in the guided condition were supported by a trained psychologist (eCoach) (Buntrock et al., 2022).Because of the eligibility criteria of this study, the health economic results regarding the guided intervention were not included (as this influences the actual effect of the eHealth intervention).

Types and perspectives
Five of the included publications reported on a cost-benefit analysis, all of which were based on an employer's perspective (Buntrock et al., 2022;Ebert et al., 2018;Geraedts et al., 2015;Kählke et al., 2019;Thiart et al., 2016).One publication also included the healthcare, employee's and societal perspective (Lokman et al., 2017).Cost-effectiveness analyses were conducted in seven publications (Buntrock et al., 2022;Ebert et al., 2018;Geraedts et al., 2015;Kählke et al., 2019;Noben et al., 2014;Phillips et al., 2014;Thiart et al., 2016).Three of them performed the analyses from an employer's perspective (Ebert et al., 2018;Geraedts et al., 2015;Thiart et al., 2016), four out of a societal perspective (Buntrock et al., 2022;Geraedts et al., 2015;Kählke et al., 2019; Noben et al., 2014), and in one publication the perspective was not reported (Phillips et al., 2014).Thereby, four publications performed cost-utility analyses, two with a societal perspective (Buntrock et al., 2022;Kählke et al., 2019), one using both the employer's and the societal perspective (Geraedts et al., 2015), and in one publication the perspective was not reported (Phillips et al., 2014).The time horizon of the included economic evaluations varied between six and 12 months, except for one publication, which conducted a RCT for 12 weeks (Phillips et al., 2014).The methods used in each of the economic evaluations are presented in Table 2 Primary outcome measures The primary outcome measures used in the analyses were either disease-specific (e.g. the Perceived Stress Scale (PSS-10), the Alcohol Use Disorders Identification Test (AUDIT), or the Insomnia Severity Index (ISI), or generic, preference-based, multi-attribute utility instruments: the AQoL-8D (Buntrock et al., 2022), the EQ-5D-5L (Geraedts et al., 2015;Phillips et al., 2014), the EQ-5D-3L (Lokman et al., 2017) and the SF-6D (Kählke et al., 2019).The preference-based instruments were expressed in utilities and recalculated into QALYs.In the cost-benefit analyses, benefits were defined as the difference between study conditions in total monetized outcome measures.Conditional upon the chosen perspective, the total monetized outcome measures included productivity costs, direct medical care costs, direct non-medical care costs and/or health benefits.These health benefits were defined as the costs of QALY health gains.

Included costs
All economic evaluations (i.e.cost-benefit, cost-effectiveness and cost-utility analyses) performed from an employer's perspective included intervention and productivity costs.One publication also included occupational health costs in performing a cost-effectiveness and cost-utility analysis from an employer's perspective (Geraedts et al., 2015).The cost-benefit analysis performed from a healthcare perspective included intervention and direct medical costs.The cost-benefit analysis performed from an employee's perspective included direct non-medical costs and health benefits.These health benefits were defined as the monetary value of QALY health gains.The economic evaluations (i.e.cost-benefit, cost-effectiveness and cost-utility analyses) performed from a societal perspective included: intervention costs, productivity costs, direct medical costs and direct non-medical costs.The inclusion of intervention costs exhibited a degree of variability.For example, two publications (Ebert et al., 2018;Kählke et al., 2019), incorporated the costs related to intervention development, whereas the remaining publications either omitted these costs entirely, or the inclusion of such costs was unclear.An overview of the costs assessed as part of the intervention per study is presented in Table 3. Sources of the intervention costs differed between the publications.Three publications used pricing information directly sourced from the manufacturer of the eHealth intervention (Buntrock et al., 2022;Ebert et al., 2018;Kählke et al., 2019), one publication based the costs on invoices of the web site host (Geraedts et al., 2015), one publication conducted interviews with health care providers  (Saunders et al., 1993).b PSS-10: Perceived Stress scale (S.Cohen, 1988).c CeS-D: Centers for epidemiologic Studies Depression Scale (lewinsohn et al., 1997).d PHQ-9: Patient Health Questionnaire-9 (Kroenke et al., 2001).e GaD-7: General anxiety Disorder-7 (Spitzer et al., 2006).f nWfQ: the nurses Work functioning Questionnaire (Williams, 2017).g ISI: Insomnia Severity Index (Morin et al., 2011).• Direct medical costs (health care utilization, medication) • Direct non-medical costs (overthe-counter drugs, unpaid work, travel, out-of-pocket expenses) Intervention costs: • Get.on Institute Direct medical costs: • German unit cost prices (Gesundheidswesen) • German register for pharmaceutical drugs: rote liste Direct non-medical costs: • replacement cost method Productivity costs: • absenteeism: self-reported gross average income of participants per day • occupational health costs Societal perspective: • Intervention costs • Productivity costs (absenteeism, presenteeism) • Direct medical costs (health care utilization inc.occupational health care, medication) • Direct non-medical costs (domestic tasks) Intervention costs: • Invoices of the Web site host Direct medical costs: • Dutch standard costs prices for economic evaluations in healthcare • Direct medical costs (health care utilization, medication) • Direct non-medical costs (overthe-counter drugs, opportunity costs of leisure time, travel expenses, domestic help) Intervention costs: • Get.on Institute Direct medical costs: • German unit cost prices (Gesundheidswesen) • German register for pharmaceutical drugs: rote liste Direct non-medical costs: • German unit cost prices (Gesundheidswesen) • Self-reporting by patients Productivity costs: Healthcare perspective: • Intervention costs • Direct medical costs (health care utilization, medication) employee's perspective: • Direct non-medical costs (out-ofpocket travel expenses, domestic help) • Health benefits (QalYs €) d Societal perspective: • Intervention costs • Productivity costs (absenteeism, presenteeism) • Direct medical costs (health care utilization, medication) • Direct non-medical costs (out-ofpocket travel expenses, domestic help) Interventions costs: • not mentioned Direct-medical costs: • Dutch standard costs prices for economic evaluations in healthcare In the netherlands, QalY health gains are valued between €20.000 and €80.000 per QalY.one study used the lower bound of €20.000 to conduct the analysis under conservative assumptions.Table 2. Continued.about a potential market price (Thiart et al., 2016), and two publications did not mention the source (Buntrock et al., 2022;Noben et al., 2014).Overall, productivity costs were defined as absenteeism and presenteeism costs, although one publication assessed absenteeism costs only (Phillips et al., 2014).All publications valuated the absenteeism costs based on the human capital approach.Only in the study conducted by Geraedts et al. (2015), a combination of both the human capital and friction cost approaches were employed.Specifically, the human capital approach was adopted from the employer's perspective, while the friction approach was utilized from the societal standpoint.The publications diverged in their sources for estimating absenteeism and presenteeism costs.One part of the publications based the productivity costs on the self-reported gross average income (Buntrock et al., 2022;Ebert et al., 2018;Kählke et al., 2019;Thiart et al., 2016) and the other part on sex-and age-specific price weights based on standard cost prices (Geraedts et al., 2015;Lokman et al., 2017;Noben et al., 2014).One publication did not mention the source of the productivity costs (Phillips et al., 2014), and one publication did not mention the source of the presenteeism costs (Buntrock et al., 2022).The publications that calculated presenteeism costs based their calculations on the Osterhaus method (Osterhaus et al., 1992), except for one (Geraedts et al., 2015).The latter study based the presenteeism calculations on the WHO Health and Work Performance Questionnaire.Furthermore, there was variability in the inclusion of costs within the category of direct non-medical costs across the publications.All publications, except one (Noben et al., 2014) included domestic tasks or help costs, and all publications except one (Geraedts et al., 2015) included out-of-pocket travel expenses in the direct medical costs.Two publications also included the expenses for over-the-counter drugs (Buntrock et al., 2022;Kählke et al., 2019) and one publication included the opportunity costs of leisure time (Kählke et al., 2019).Cost data was mainly gathered through self-report questionnaires, such as the Treatment Inventory of Costs in Patients with Psychiatric Disorders (TiC-P).In all publications the direct medical costs prices were derived from the standard national cost prices of the corresponding country.

Results of the analyses reported in the included economic evaluations
An oversight of the different health economic results is given in Table 4.

Health economic results from cost-benefit analyses
In the studies included that did a cost-benefit analysis, the authors consistently performed their analyses from an employer's perspective, revealing cost-beneficial outcomes (Buntrock et al., 2022;Ebert et al., 2018;Geraedts et al., 2015;Kählke et al., 2019;Thiart et al., 2016).These interventions targeted stress, insomnia, alcohol disorder, and the promotion of RTW.In the instances of cost-beneficial outcomes from an employer's perspective, the incremental productivity gains exceed the costs associated with the intervention.Two publications reported a positive Net Benefit (NB > 0), Benefit-Cost Ratio (BCR > 1) and Return on Investment (ROI > 1) (Buntrock et al., 2022;Thiart et al., 2016).One publication reported a positive Incremental Net Benefit (INB > 0), BCR (BCR > 1) and ROI (ROI > 1) (Lokman et al., 2017).One publication reported a positive NB (NB > 0), BCR (BCR >1), but a negative ROI (ROI < 1) (Geraedts et al., 2015), and one publication reported a positive NB (NB > 0), but a negative ROI (ROI < 1) (Buntrock et al., 2022).The intervention targeted at  (Buntrock et al., 2022;lokman et al., 2017) included set costs.It was not elaborated what costs were included in the set costs; therefore, the costs included were rated "not clear".b It was unclear whether this study included the costs for developing and providing the intervention (noben et al., 2014).c this study evaluated freely available eHealth interventions and did not report nor include the intervention costs in their analyses.therefore, the costs included were rated "not reported".Societal perspective promotion of RTW was also cost-beneficial from a societal and employee's perspective, but not from a healthcare perspective (Lokman et al., 2017).

Health economic results from cost-effectiveness analyses
From an employer's perspective, three publications demonstrated an ICER favoring the intervention condition with dominant results.In instances of dominance for the intervention condition, it showcases lower costs and higher effects compared to the control condition.These interventions were focused on the prevention and reduction of work-related stress (Ebert et al., 2018;Geraedts et al., 2015) and insomnia (Thiart et al., 2016).From a societal perspective, two publications aimed at the prevention and reduction of work-related stress and one at the treatment of alcohol disorders, showed dominant ICERs (Buntrock et al., 2022;Geraedts et al., 2015).One publication presented lower costs and a lower effectiveness for the control group (Noben et al., 2014).The publication that did not report on the perspective showed lower costs, but also lower effectiveness (Phillips et al., 2014).When comparing the results from different perspectives in studies that conducted analyses from both viewpoints, one publication revealed higher costs in the intervention condition than in the control condition, using an employer's perspective.Conversely, from a societal perspective, the costs of the control condition were higher than those of the intervention condition (Buntrock et al., 2022).In another publication, the societal perspective showed a more substantial difference in costs compared to the employer's perspective (Geraedts et al., 2015).

Health economic results from cost-utility analyses
No publications showed a dominant ICUR for the intervention condition from an employer's perspective, despite the fact that the conducted studies reported positive results in terms of costs (see Figure 2(D)) (Buntrock et al., 2022;Geraedts et al., 2015).Most of the publications showed lower incremental costs for the intervention group (see Figure 2(C)), but no differences in effectiveness (see Figure 2(B)).This led to INBs of 475,36 Int$ and 879,81 int$ respectively (Buntrock et al., 2022;Geraedts et al., 2015).Using a societal perspective, one publication showed dominant results for the intervention condition based on the ICUR, where the QALYs were higher, and the costs were lower for the intervention group.This publication targeted the reduction and prevention of work-place related stress and showed an INB of 1218,48 Int$ (Kählke et al., 2019).The remaining publications also reported positive results in terms of costs, although they did not exhibit dominance (see Figure 2(D)) (Buntrock et al., 2022;Geraedts et al., 2015).In alignment with the employer's perspective, lower incremental costs (see Figure 2(C)) were demonstrated, accompanied by an almost negligible difference in effects (see Figure 2(B)).This yielded INB's of: 971,29 Int$ and 974,29 Int$, respectively (Buntrock et al., 2022;Kählke et al., 2019).When comparing disease-specific with generic outcomes (see Figure 2(A,B)), in the majority of publications that included both, the differences in effects between the intervention and control groups were notably smaller for the generic outcomes than for the disease-specific outcome measures (Buntrock et al., 2022;Geraedts et al., 2015;Kählke et al., 2019)

Main findings
The aim of this systematic literature review was to provide an overview of the available literature on economic evaluations of eHealth interventions targeting mental health problems in the workplace.Eight publications met the eligibility criteria and were included in this review, with seven deemed high-quality publications.All publications focused on a specific target group of employees, selected by the symptoms of mental health problems.Overall, the economic evaluations equally took an employer's and societal perspective.The most common form being cost-effectiveness analyses using a specific outcome measure, followed by cost-benefit and cost-utility analyses.The cost-benefit analyses were nearly all expressed in ROI, NB and BCR.Publications predominantly used disease-specific outcome measures, which varied across studies.The primary outcome measures indicated a greater treatment responder effect for the intervention group in two of the three eHealth interventions targeting work-related stress, the publication addressing alcohol treatment, and the intervention targeting insomnia.Additionally, two eHealth interventions for work-related stress and the publication on work functioning improvement showed a greater decrease in symptoms for the intervention group.Preference-based instruments demonstrated minimal effect outcomes, nearly approaching zero.The cost-benefit analyses showed that all examined interventions were cost-beneficial from an employer's perspective.This implies that eHealth interventions addressing workplace-related stress, insomnia, return to work, and problematic alcohol use, seem to lead to higher benefits than costs in monetized terms for the employer.This emphasis on the employer could possibly suggest that cost-benefit analyses were perceived more important for the employer than for society.Nevertheless, the employer's perspective showed smaller cost differences than the societal perspective.These interventions thus appear to have the potential not only to reduce costs for the employer, but also for society.
Cost-effectiveness and cost-utility analyses also showed results in favor of the intervention condition.From a disease-specific viewpoint, the interventions targeting prevention and reduction of work-related stress, together with the intervention targeting insomnia, were cost-effective.Both interventions reported publications with a dominant ICER.In the case of a dominant ICER, the effectiveness is higher, and the costs are lower for the intervention group compared with the control group.The study targeting the promotion of RTW, demonstrated positive INB's INB from both an employer's and societal perspective, suggesting potential cost-effectiveness of the corresponding intervention.In general, the publications included comparable cost categories from each perspective.For example, from the employer's viewpoint, these categories encompassed intervention and productivity costs, while from the societal perspective, they incorporated the costs from the employer's perspective in addition to direct medical and/or direct non-medical costs.The inclusion of costs within the intervention category and the sources used to determine intervention costs, differed among studies.Calculating the productivity losses, all publications included absenteeism and, except for one (Phillips et al., 2014), presenteeism costs.To valuate absenteeism, all the publications made use of the human capital approach.One publication used both the human capital approach and the friction cost approach (Geraedts et al., 2015).In doing the latter, the costs were truncated to the time span in which organizations need to replace a worker (Koopmanschap et al., 1995).In applying the human capital approach, the number of lost workdays was multiplied by the average gross daily wage (van den Hout, 2010).However, there was a discrepancy in how the average gross daily salary was determined across the publications.In some cases, it relied on self-reported data provided by the patients regarding their salaries (Buntrock et al., 2022;Ebert et al., 2018;Kählke et al., 2019;Thiart et al., 2016), while in other cases, it was calculated using sex-and age-specific price weights, derived from standard cost prices (Geraedts et al., 2015;Lokman et al., 2017;Noben et al., 2014).The same division was observed in the calculation of the presenteeism costs.The majority of the papers (Buntrock et al., 2022;Ebert et al., 2018;Kählke et al., 2019;Lokman et al., 2017;Noben et al., 2014;Thiart et al., 2016) applied the "Osterhaus method" to estimate the costs of presenteeism by multiplying the number of workdays with reduced functioning by a corresponding self-reported inefficiency score (Osterhaus et al., 1992).The unit prices for valuing presenteeism, represented by the average gross daily salary, were similar to the estimates used for absenteeism.For the valuation of the direct medical costs all studies used the standard unit cost prices of the corresponding countries.

Limitations
Some limitations to the study should be noted.Despite the comprehensive search strategy, only a limited number of studies were included.This limits the generalizability of the results.Less restrictive eligibility criteria could have increased the number of relevant studies.For example, limiting the search strategy to published studies that are peer-reviewed and written in English, might have resulted in relevant studies being missed.Moreover, due to the focus on economic evaluations and, consequently, the exclusion of studies that reported on effects or costs only, this study cannot draw strong conclusions on the effectiveness of eHealth interventions targeting mental health problems in the workplace.Finally, the comparison of the results was limited due to heterogeneity in type of eHealth intervention, study population and methodology of economic evaluations.

Comparison with prior work and implications
The results of this study about the potential of eHealth interventions being cost-effectiveness was in line with the literature conducted out of a broader perspective (Deady et al., 2017;Gentili et al., 2022;Massoudi et al., 2019).No publication in the literature presented differences between the societal and employer's perspective, however, the importance of economic evaluations from a societal perspective was addressed (Jönsson, 2009).Moreover, the various inclusion of cost categories, measuring of costing methods, and usage of outcome measures, underpin the need for further development of guidelines for cost categories, costing methods and outcome measures for mental eHealth care in the workplace.The importance for development of these guidelines was stressed in other publications, in which the results provided heterogeneity in terms of the design and a number of challenges specific to eHealth interventions, including the estimations of all costs and outcomes (Jankovic et al., 2021).This development is first of all needed to increase the ability to compare.The different disease specific outcomes, the lesser use of preference-based instruments and the differences in the inclusions of costs within the cost-categories hampered the comparability between studies.This low usage of preference-based measurements is, based on the literature, possibly the result of a lack of sensitivity (Lamers et al., 2006).Secondly, the moment of assessment could have had an impact on the publication results.Existing literature indicates that the true impact of an eHealth intervention can be accurately accessed only after its implementation, rather than during the implementation phase itself (Granja et al., 2018).For instance, the study that concentrated on the return to work (RTW) (Lokman et al., 2017), observed the most significant cost differences in the final month of the 12-month trial.This suggests that the interventions might yield different results over a time horizon extending beyond 12 months.Thirdly, all the studies employed absenteeism and/or presenteeism to calculate productivity losses.It is important to note that presenteeism, which refers to reduced productivity while at work due to health issues, can be challenging to identify and measure accurately.This difficulty in detecting presenteeism can potentially introduce significant inaccuracies into the calculations (Homrich et al., 2020).The interpretation of the health economic results necessitates caution due to the substantial uncertainty associated with the cost estimates.Furthermore, it's worth noting that a significant portion of the studies relied on self-report data to quantify costs.Self-report data can be influenced by factors such as social desirability and recall bias, potentially leading to discrepancies between the reported costs in the publications and the actual, underlying costs (Khare & Vedel, 2019).Hence, the true costs might differ from the costs reported in the publications, and further research in the validity of productivity estimates based on self-report data is recommended.Fourthly, it is recommended to include a preference-based, disease specific outcome measure such as the Mental Health Quality of Life (MHQoL) questionnaire (van Krugten et al., 2022).This generic outcome measure is more sensitive to the effects of mental health problems (van Krugten et al., 2022).Finally, considering the promising outcomes from this study, which suggest that eHealth interventions in the workplace can yield cost-effectiveness or cost savings, there arises a need for future research.These studies should not only continue to explore eHealth interventions but also extend their focus to assess cost-effectiveness in relation to other critical mental health aspects, including anxiety and depression.The current study showed the potential to further assess eHealth interventions in the workplace, promoting well-being and economic efficiency.

Conclusions
Our findings suggest that economic evaluations focusing on eHealth interventions in the workplace equally take a societal and employer's perspective, with the most common form being cost-effectiveness analyses using a disease-specific outcome measure, followed by cost-benefit and cost-utility analyses.The outcomes of the studies revealed the potential of eHealth interventions to be cost-effective (especially when targeted at stress reduction).These results underline that next to company level initiatives, general policies targeting eHealth interventions at the workplace are recommended.

Disclosure statement
No potential conflict of interest was reported by the author(s).

.
control group n non treatment responders in control After which it was converted into Cohen's d (Sánchez-Meca et al., 2003): Cohen sd OR The effects pertaining to the mean reductions in symptoms could readily be computed into Cohen's d, and was calculated as follows (J

Figure 1 .
Figure 1.Prisma flow diagram of the study selection process.

Figure 2 .
Figure 2. Visualization of the health economic results.

Table 1 .
General characteristics of the included studies.
a auDIt: alcohol use Disorders Identification test

Table 2 .
Methods for cost calculations used in the included economic evaluations.

Table 3 .
Cost categories included in the intervention costs per study.
a the intervention costs in these studies

Table 4 .
Health economic results reported in the included economic evaluation.
a Intervention Group.b Control group.c Cost-effectiveness plane.d Willingness to pay.e northwest quadrant in the cost-effectiveness plane.f northeast quadrant in cost-effectiveness plane.g Southwest quadrant in cost-effectiveness plane.h Southeast quadrant in cost-effectiveness plane.