How Are Information and Communication Technologies Supporting Routine Outcome Monitoring and Measurement-Based Care in Psychotherapy? A Systematic Review

Psychotherapy has proven to be effective for a wide range of mental health problems. However, not all patients respond to the treatment as expected (not-on-track patients). Routine outcome monitoring (ROM) and measurement-based care (MBC), which consist of monitoring patients between appointments and using this data to guide the intervention, have been shown to be particularly useful for these not-on-track patients. Traditionally, though, ROM and MBC have been challenging, due to the difficulties associated with repeated monitoring of patients and providing real-time feedback to therapists. The use of information and communication technologies (ICTs) might help reduce these challenges. Therefore, we systematically reviewed evidence regarding the use of ICTs for ROM and MBC in face-to-face psychological interventions for mental health problems. The search included published and unpublished studies indexed in the electronic databases PubMed, PsycINFO, and SCOPUS. Main search terms were variations of the terms “psychological treatment”, “progress monitoring or measurement-based care”, and “technology”. Eighteen studies met eligibility criteria. In these, ICTs were frequently handheld technologies, such as smartphone apps, tablets, or laptops, which were involved in the whole process (assessment and feedback). Overall, the use of technology for ROM and MBC during psychological interventions was feasible and acceptable. In addition, the use of ICTs was found to be effective, particularly for not-on-track patients, which is consistent with similar non-ICT research. Given the heterogeneity of reviewed studies, more research and replication is needed to obtain robust findings with different technological solutions and to facilitate the generalization of findings to different mental health populations.


Introduction
The effectiveness of psychotherapy for the treatment of mental disorders has been supported by an impressive amount of evidence. However, some patients do not respond to treatment as expected, either because they do not show an improvement during the intervention or they discontinue it, or because they show a deterioration [1,2]. There might be several reasons explaining individual differences in response to psychological treatments, including unchallengeable patient characteristics 2 of 22 (e.g., age), their personality and behavioral profiles, treatment characteristics and context, and patient physical health status and life context, to name some examples [3]. While acknowledging the previous, an increased number of studies have pointed to methodological deficits, namely, in how patients are monitored during treatments, as key factors influencing current treatment effectiveness [4,5].
Specifically, it has been proposed that a paradigm shift in the practice of psychotherapy towards an ecological momentary assessment (EMA) is necessary in order to monitor patients repeatedly and frequently in their natural environment [6,7]. However, simply monitoring the patient does not appear to be enough to improve the patient's outcomes [8,9]. In this sense, therapist (and patient) feedback has been argued to be a fundamental aspect if patient monitoring during therapy is to be effective [1,10].
The aforementioned procedure is known by different terms in the literature, such as routine outcome monitoring, with outcome or continuous feedback; progress monitoring; or, probably the most popular, measurement-based care (MBC) [9,11]. For simplicity and readability, the latter will be preferred throughout this text.
MBC is defined as a periodic and recurrent assessment of the patient's status over the course of an intervention using standardized measures. Importantly, the evaluation is followed by immediate, frequent, and systematic feedback of the patient's information to the therapist [12,13]. This procedure has been argued to help therapists to assess actual patient progress, suggest necessary adjustments to the treatment, and identify patient deterioration or improvement trajectories, thus enhancing the patient's response to the intervention [1,[14][15][16]. According to the American Psychological Association's (APA) Division 28 Task Force on Empirically Supported Relationships, MBC may also lead to an improvement in the therapeutic alliance and may avoid premature treatment termination, because MBC encourages collaboration between patients and therapists, thus promoting engagement, dialogue and discussion of real-life, daily patient difficulties during sessions. Furthermore, giving feedback to patients raises awareness of their progress and makes them become more mindful of their symptoms, which may also enhance the quality of psychological interventions and the patient-therapist alliance [14].
Several studies have shown the efficacy of this systematic patient monitoring with progress feedback to the therapist in psychological interventions [10,17]. Research also indicates that feedback from outcome measures enhances treatment effectiveness, particularly in not-on-track patients (i.e., those who do not make the expected changes) or when it is provided both to clinicians and patients [8,10,13,14,17,18]. Specifically, providing the therapists with immediate feedback about the patient's symptoms appears to reduce the number of early dropouts and improve several treatment outcomes (depressive and anxiety symptoms, psychosocial functioning, psychosis, quality of life, therapeutic alliance, etc.) when compared against usual treatment [17,19]. Overall, medium effect sizes have been reported when using MBC [1,14].
What the existent literature suggests is that MBC is a promising methodology to enhance the effectiveness of psychological treatments. However, there are a number of flaws into the literature on MBC that might have limited the impact and dissemination of this procedure [20,21]. Traditionally, MBC has been conducted with self-report, paper-and-pencil questionnaires that patients complete before or after therapy sessions. Additionally, assessments are mostly retrospective and based on the patient's recalled experiences during the past week [11]. With this information, the therapist examines and discusses the results during the actual therapy session [22]. As noted earlier, while this practice has been shown to provide some relevant information, relying on paper-and-pencil retrospective reports only, where daily experiences are not reported, might result in recall bias, thus making it difficult to understand patient fluctuations over time [23,24]. Furthermore, focusing on self-reports exclusively is problematic, as more objective data (e.g., actual number of steps taken or time spent out of the home) is ignored or based on patient appraisal only.
Currently, the rapid growth of new technologies in our society has changed the way psychotherapy is conducted. For example, information and communication technologies (ICT) have been argued to allow therapists to evaluate and receive patient progress feedback in real time, thus minimizing patient recall bias [25]. Additionally, the use of ICT allows obtaining objective data of patient changes in natural settings, for example, using sensors (accelerometers, positioning system, or pedometers, among others) [22]. Therefore, the use of handheld ICT devices such as smartphones, tablets, or laptops might increase the effectiveness of MBC by facilitating EMA before, during, and after treatment, providing the information immediately to therapists and researchers, and making it easier to combine collection of objective and subjective patient data [11,21,25].
In order to investigate to what extent ICTs are being implemented to enhance psychological interventions and how their application is effectively improving outcomes, we have conducted a systematic review to explore how ICT is being used for MBC in face-to-face psychological treatments. In doing so, we have investigated: 1. what the different technologies and procedures used for MBC during psychological interventions are and 2. to what extent the use of ICT for MBC is feasible, acceptable, and effective.

Search Strategy
The search was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses [26]. The search was conducted in February 2020 and included published and unpublished studies from the electronic databases PubMed, PsycINFO, and SCOPUS. In addition, reference mining was performed by searching through bibliographies of relevant articles. The selection of these databases was motivated by previous research showing that PsycINFO has high sensitivity and specificity when retrieving intervention studies and is especially suitable for psychology research, and that SCOPUS offers about 20% more coverage than other important databases such as Web of Science. Additionally, it has been argued that Google Scholar provides inaccurate results, and PubMed is one of the preferred tools for biomedical research [27,28].
The search strategy included variations of the terms "psychological treatment", "progress monitoring", and "technology" (See Appendix A for the complete list of search terms and combinations). Due to the diversity of terms, a broad search strategy of terms was used. Synonyms, abbreviations, and spelling variations were identified for the three concepts and combined in the search using the "OR" Boolean operator, with non-synonymous concepts combined using "AND". These terms were searched in titles and abstracts. The references of included studies and relevant systematic reviews were searched to identify studies that were missed during the literature search. There were no restrictions regarding language or publication period, but the search was only conducted in English.

Inclusion Criteria
Included studies were psychological treatments enhanced by MBC using technology systems. Specifically, included studies 1. were clinical trials (either feasibility, case studies, and both randomized or non-randomized investigations); 2. included the use of technology during MBC (both for monitoring and feedback provision) while undergoing a face-to-face psychological intervention; and 3. involved feedback to the therapist or to both therapists and patients based on standardized measures.
To be considered MBC, the intervention must satisfy the following components: 1. routine assessment of a symptom, an outcome, or a process measure; 2. therapist review of data; and 3. therapist use of data to inform clinical decisions. Therefore, the study population can include patients with any mental disorder from all ages who are routinely monitored via validated outcome measures using technologies over the course of a face-to-face psychological treatment.

Exclusion Criteria
Studies in which technology systems were not used in the whole MBC process, including the assessment and feedback parts, or where only patients but not therapists were provided with feedback, were excluded. In addition, studies were excluded if they did not include a face-to-face psychological intervention.

Search and Screening
Initially, 193 publications were identified from database searches and screening of reference lists (see Figure 1 for the study diagram flow). After excluding duplicates (n = 63), 130 publications were retained for screening. After initial screening of titles and abstracts, 84 of these documents were excluded due to eligibility reasons. For the remaining 46 publications, full texts were retrieved. After eligibility assessment of the full texts, 28 publications were excluded. The majority of publications were excluded because they did not meet the eligibility criteria, resulting in a final sample that comprised 18 publications.

Search and Screening
Initially, 193 publications were identified from database searches and screening of reference lists (see Figure 1 for the study diagram flow). After excluding duplicates (n = 63), 130 publications were retained for screening. After initial screening of titles and abstracts, 84 of these documents were excluded due to eligibility reasons. For the remaining 46 publications, full texts were retrieved. After eligibility assessment of the full texts, 28 publications were excluded. The majority of publications were excluded because they did not meet the eligibility criteria, resulting in a final sample that comprised 18 publications.
The search, screening process, and data extraction were conducted independently by the first two authors (PGM and VMB). When in doubt, study eligibility was discussed with a third author (CSR). After the phase of study eligibility assessment, inter-rater agreement was calculated (Cohen's kappa). This coefficient showed a substantial overall agreement, represented by a kappa of 0.908 (SD = 0.064; 95% CI, 0.781, 1.000).

Data Extraction
The following data were extracted from each included study, using a standardized data-extraction form developed a priori: authors, study setting (geographical setting and type of clinic), sample size, study design, study participants (demographics and type of mental disorder), The search, screening process, and data extraction were conducted independently by the first two authors (PGM and VMB). When in doubt, study eligibility was discussed with a third author (CSR). After the phase of study eligibility assessment, inter-rater agreement was calculated (Cohen's kappa). This coefficient showed a substantial overall agreement, represented by a kappa of 0.908 (SD = 0.064; 95% CI, 0.781, 1.000).

Data Extraction
The following data were extracted from each included study, using a standardized data-extraction form developed a priori: authors, study setting (geographical setting and type of clinic), sample size, study design, study participants (demographics and type of mental disorder), type of psychological intervention, assessment characteristics (frequency and setting), primary outcomes, feedback characteristics (to whom, frequency, and setting), type of technology used, technology feasibility, and clinical effectiveness. Data was extracted from all full texts by the first author (PGM) and then discussed with another author (VMB) before it was reviewed by all co-authors.

Risk of Bias Assessment
All studies included in this review were independently rated for quality by two reviewers (PGM and VMB). If the rating differed, reviewers discussed the articles to reach consensus with a third reviewer (CSR). The Study Quality Assessment Tools from the National Heart Lung and Blood Institute [29] were used to assess study quality and risk of bias. This tool was preferred because it includes six types of studies and specific criteria according to the study design (i.e., controlled intervention studies; systematic reviews and meta-analyses; observational cohort and cross-sectional studies; case-control studies; before-after studies with no control group; and case series studies). This tool allows reviewers to rate studies as "good", "fair", or "poor". Total quality scores ranged from 9 to 14 points depending on the study design.

Synthesis of Results
Frequency tables were used to summarize the characteristics of individual studies. We conducted a systematic review and not a meta-analysis, because the emphasis was not on effect sizes, but on how MBC with technology was being conducted. Additionally, we anticipated that included studies would be very heterogeneous, because the review includes any form of psychological intervention, all types of technologies, several trial designs, and different types of mental disorders and outcomes. Thus, we performed a narrative synthesis only.

Additional Analyses
Factors affecting study heterogeneity included variations in the type of mental disorder (e.g., major depressive disorder or anxiety disorders), outcomes included, treatment characteristics (type, format, and duration), measures used (clinician-rated versus self-rated), study design, and differences in the means by which MBC was delivered. The description of the findings was sensitive about the aforementioned subgroups when possible.

Assessment Procedure Used to Track the Patient's Status
The characteristics of studies included in the review are described in Table 2. Most studies (n = 9 [30,31,33,35,[38][39][40]44,46]) monitored their patients daily. The remaining studies monitored their patients weekly (n = 4 [32,41,42,47]), before every therapy session, or both daily and weekly (n = 5 [34,36,37,43,45]). In the latter studies conducting assessments both daily and weekly, daily assessments usually included shorter questionnaires that evaluated therapeutic process outcomes, mood, or medication adherence, while longer outcome scales (i.e., measures of depressive or anxiety symptoms) were administered weekly. Treatment effectiveness was assessed most commonly with the primary outcome measures of interest for the investigation, which most often were the Patient Health Questionnaire-9 for depressive symptoms and the Generalised Anxiety Disorder-7 for anxiety symptoms (n = 4 [33,37,41,42]). Other outcomes included the frequency and amount of drug use (n = 1 [35]); the Subjective Experiences of Psychosis Scale, the Auditory Hallucinations subscale of the Psychotic Symptom Rating Scales for psychotic symptoms, and the Depression Anxiety Stress Scale for negative emotional symptoms (n = 2 [38,39]); the Outcome Questionnaire-45 (n= 1 [47]); the Therapy Process Questionnaire (n = 1 [44]); the World Health Organization's Wellbeing Index (n = 1 [40]) for well-being; the Symptom Checklist-90 for bulimia symptoms and the Intersession Experience Questionnaire for the psychotherapy process (n = 1 [45]); the Altman Self-Rating Mania scale and the 16-item Quick Inventory of Depressive Symptoms-Self Report scale (n = 1 [43]); and other clinical measures related to sleep, mood, medication use, or daily functioning (n = 6 [30][31][32]34,36,46]). T and P P: T regularly shared P progress in order to incorporate strategies in therapy sessions and treatment plan.T: They got patient information several times a week in a graph format to discuss with them during sessions, to encourage them and monitor them.
EMA and Text messages [31] Daily questions about recent emotional events (e.g., emotions, scenario, somatic symptoms, automatic thoughts) + answers on demand by the participant Skills entries and satisfaction with the treatment. P and T P: They received personalized feedback from therapists.T: Information and graphs from the portal about patients' progress were discussed in weekly CBT sessions with the patients.
Smartphone app: SmartCAT app + SmartCAT therapist portal.. [32] Online before every session STIC: set of questionnaires T and clinicians stakeholders T: On-demand graphs of patient progress were provided to the therapist through STIC STIC online [33] Once daily mood monitoring messages at random between 8 a.m. and 8 p.m.
Attendance to therapy, duration of therapy and PHQ-9 P and T P: They received feedback about their mood responsesT: T reviewed information from an online dashboard were patient progress is shown. T can periodically review the graphs, identify key aspects and address any important event during or between sessions.
Automated text messages and Web-based platform (HealthySMS) [34] Daily and weekly surveys (between 5 p.m. and 10:30 p.m.) Mood, medication use, socialization and conflict P and T P: They discussed their feedback with the T at every session.T: They reviewed and discussed patient information (plots of symptoms over time, etc.) on the dashboard with P during sessions and between sessions.
Smartphone app: RealLife Exp + web-based platform [35] Once daily call (HealthCall) for selfmonitoring Primary drug use (frequency and amount), use of other drugs, medication adherence, and mood P and T P: They received the feedback at 30 and 60 days, where their information was discussed with the T.T: At 30 and 60 days, T discussed with P the generated graphs based on HealthCall about their drug use, moods and health behaviors.
Phone IVR system [36] Daily surveys (at 5 p.m. until 11:55 p.m.), weekly surveys (Sundays at 10 a.m. until Monday 11:55) and monthly in-person psychosocial assessments with research staff Daily surveys assessing mood, medication adherence, and social interaction, weekly surveys assessing symptoms, sleep, and medication adherence T T: They received alerts from the dashboard when P scores were clinically significant and took the proper decisions according to the patient demand.
Ginger.io (software) = Smartphone app + Clinician dashboard [37] 3/4 times daily, weekly; 8/12 weeks PHQ-9 and GAD-7 P and T P: P received notifications about their progress in the app, becoming more aware of their symptoms.T: T reviewed patient-reported information via an online dashboard and visualized patient progress graphs.
Smartphone app + online platform [38] Session 1 and 2: 10 daily evening EMA for 6 days. Session 3 and 4: 8 evening daily EMA (monitor changes in voices and coping strategies) SEPS, PSYRATS-AH, and DASS-21 P and T P: In session 2, P received a summary sheet with their EMA progress.T: In session 2, EMA feedback was discussed with the P in order to guarantee understanding, detect predictors and avoid causation.
Smartphone app: MovisensXS + web-based platform SEPS, PSYRATS-AH, and DASS-21 P and T P: In session 2, P received a summary sheet with their EMA progress.T: In session 2, EMA feedback was discussed with the P in order to guarantee understanding, detect predictors and avoid causation.
Smartphone app: RealLife Exp + web-based platform [40] Daily self-reported measures of well-being Well-being (WHO-5) P and T P: They received routinely individualized information about their progress in group discussion with the therapist.T: T received daily automatic plots of each patient's outcomes within trajectories.
Touch-screen technology in therapy rooms [41] Weekly (session-by-session) PHQ-9 and GAD-7 P and T P: They received their feedback in session with the therapist, where the information was reviewed, discussed and used to guide the treatment plan.T: They had access to patient progress graphs and response curves from the monitoring system, and they were warned automatically when a P was not-on-track.
Computer PCMIS [42] Weekly (session-by-session) PHQ-9 and GAD-7 P and T P: They received their feedback in session with the therapist, where the information was reviewed, discussed and used to guide the treatment plan.T: They had access to patient progress graphs and response curves from the monitoring system, and they were warned automatically when a P was not-on-track.
Computer PCMIS [43] Twice a day (only during psychoeducation sessions) and once a week Daily: mood and sleepWeekly: QIDS, ASRM and mood management strategies questionnaire P and T P: P received their feedback at every session with the T.T: T reviewed patient progress from daily mood rating and weekly scales from the previous week and discussed with the P the relationship between his/her mood changes and stressors.
Phone text messages or e-mails (True Colours mood monitoring system) [44] Daily process monitoring (during evenings). TPQ T T: On demand. Feedback was used for individualizing therapeutic decisions. SNS [45] Daily measures of psychotherapy process. Weekly measure of therapy outcome.
IEQ daily, weekly SCL-90 (Bulimia) P, T and researchers P: They viewed their progress and estimated their moods and symptoms during the past day.T: They had access to P data from the system in order to adapt the intervention delivered.
Smartphone: DynAMo web app [46] Daily monitoring Sleep patterns, physical activity, and activities of daily living P, T and caregivers P and caregiver: They could see a proportionate share of the information adapted to their needs.T: Information collected was available at all times in order to design personalized interventions.

Feedback Procedure
The majority of studies provided feedback to both therapists and patients (n = 15). The remaining studies gave feedback to therapists only (n = 3 [32,36,44]). Most studies used feedback to track the patients' progress focusing on key aspects during treatment and to monitor responses in-between sessions. Even when the feedback was not directly provided to the patient, this information was used to discuss patient progress during treatment sessions or to take clinical decisions (e.g., emphasize a specific content during session).
Feedback included information about treatment evolution in progress charts, summary sheets, graphs of scores and curves, and plots of scores within trajectories. Feedback information was either sent periodically to the therapist in response to patient assessments or on-demand (weekly or daily). In some studies, feedback to the therapist appeared when the patients' responses were considered clinically significant according to pre-established criteria.

Technology Characteristics
The technology characteristics of included studies are presented in Table 2. Most studies included handheld technology, such as smartphone apps (n = 7 [31,34,[36][37][38][39]45]), touch-screen technologies (n = 2 [40,46]), or laptops (n = 5 [32,41,42,44,47]), together with a web-based platform for the therapist. The remaining studies used phone text messages (n = 2 [30,33]), e-mail (n = 1 [43]), or a phone interactive voice response system (n = 1 [35]). For example, in one study, automated text messages and a web-based platform (e.g., HealthySMS) were used [33]. In other investigations, authors implemented an Internet-based system, such as the Patient Case Management Information System (PCMIS), or an electronic clinical record system which includes outcome-monitoring graphs that chart depression and anxiety scores at every session [41,42]. A similar example was the Synergetic Navigation System, an Internet-based device for data collection (with web-compatible devices such as PCs, tablets, or smartphones) and data analysis that allows for the implementation of questionnaires at any chosen interval [44]. Another Internet-based system was the DynAMo web app, a piece of software that combines algorithm-based treatment planning, process monitoring, and outcome monitoring, which can be used by both researchers and clinicians to plan treatments and monitor psychotherapeutic processes [45]. One investigation used the Systemic Therapy Inventory of Change System, an online system that assesses and tracks changes in the patients' interpersonal system, as well as in the therapeutic alliance, and also feeds these data back to the therapists on demand [32]. A final example of an Internet-based system used in one of the included investigations was Ginger.io, an mHealth software program comprising a therapist dashboard and an app which can collect data from self-report surveys sent to the participant in addition to "passive" data from the participant's phone, such as number of calls and SMS messages, and movement patterns based on Global Positioning System data [36].

Technology Feasibility
Because of the large differences in sample sizes across investigations (Table 3), case studies with one to four participants [32,39,45,46] have been described in previous sections but will not be considered in the feasibility and effectiveness summaries. The remaining studies included at least 17 participants, and therefore will be discussed in detail here and in Table 3. Overall, the results demonstrated that enhancing MBC in psychological therapy with technology was generally feasible and acceptable (Table 3). This statement is supported by the high average response rate for daily (mean = 63.3%, range = 40%-81% [34,36,43]), weekly (mean = 73.0%, range = 39%-88% [34,36,37,43]), and monthly (92.7% [30]) symptom monitoring across studies, low average missing data rates amongst patients (13% [44]), and high completion rates (mean = 77.8%; range = 64.1%-90% [31,35,38,40,44]). In addition, several studies reported satisfaction with the technology used to improve MBC in the intervention delivered, and most of them revealed that patients and therapists would recommend the technology used as part of the treatment [31,34,38]. Completion rate was 82.8%. Patients reported the app being easy to use. All parents report treatment satisfaction and would recommend the program.
NA [33] 85 (40 cont. + 45 Tx) NA Technology-supported MBC significantly increased treatment adherence (median of 13.5 weeks before dropping out) compared to traditional CBT (median of 3 weeks before dropping out). Effect sizes of technology-supported MBC CBT on depressive symptoms' severity (z = −5.80) were larger than for traditional CBT (z = −3.12), but differences were not significant [34] 61 Tx Moderate survey completion (daily = 40%; weekly = 39%). In general, both T (66%) and P (85%) reported they would continue using the app as part of the treatment.
At 12-month follow-up, reductions in non-injection drug use were comparable in traditional and technology-supported MBC motivational interviewing and superior than in the control condition. In the subset of patients with drug dependence, drug use was significantly lower in the technology-supported MBC condition at 12 months post-treatment. At 60 days, treatment retention in the technology-supported MBC group (88.8%) was superior than in the motivational intervention only condition (81.8%) and the control condition (78.3%) [36] 76 Tx Feasibility and acceptability of the smartphone app as an adjunct treatment tool is supported by the high response rate sate (weekly surveys: 77%; daily surveys: 69%) NA [37] 17 Tx The feasibility and acceptability of the mobile platform is supported by the high early response rate (weekly = 88%). NA [38] 34 (17 cont. + 17 Tx) High completion rates (74%) of EMA questionnaires and good satisfaction of participants support the feasibility and acceptability of the study, respectively.
Compared with the usual treatment, the technology-supported MBC treatment resulted in large improvements in confidence in coping with voices (Hedges g = 1.45) and medium improvements in understanding of voices (Hedges g = 0.61) and in psychotic symptoms (Hedges g = 0.51). Both groups showed similar changes in the impact of psychosis. [40] 1308 (408 Tx Fb +, 439 nFb + 461 cont.) High rates of touch-screen questionnaire completion (over 90%).
Technology-supported MBC for NOT patients was more effective than traditional CBT or monitoring without feedback in reducing depressive symptoms and the impact of emotions on functioning, as well as on increasing vitality. By contrast, changes in well-being, anxiety, and stress were comparable across conditions.
Technology-supported MBC achieved comparable reductions in depression and anxiety compared to controls, but with significantly less time (adjusted mean = 10.25, SE = 0.45 vs. adjusted mean = 6.59, SE = 0.51) and cost (between £65.88 and £129.20 cost reductions per treatment). Cases in the control condition were twice as likely to become not-on-track patients compared to those in the technology-supported MBC.

Clinical Effectiveness
As summarized in Table 3, data about clinical effectiveness was described in seven studies, including four RCTs [35,38,42,47], two nRCTs [33,40], and a quasi-experimental pre-post investigation [41]. All RCTs included active controls (i.e., traditional face-to-face psychological treatment) without MBC. According to these investigations, the use of technology-supported MBC appears to significantly reduce symptom severity, and changes are sometimes larger than with traditional interventions [38], especially in patients at risk of poor response to treatment (i.e., not-on-track cases) [35,40,42,47], which is consistent with previous research [42,46]. Furthermore, one study showed that although traditional CBT and technology-enhanced MBC CBT were comparable in terms of treatment effectiveness, the latter significantly reduced therapy duration and cost of treatment [41]. Also in favor of technology-assisted MBC, another investigation revealed that patients stayed in therapy longer (i.e., higher adherence) in the experimental condition, that is, when MBC was supported by technology (group CBT with a text messaging adjunct) as opposed to traditional MBC without technology (group CBT without the text messaging adjunct) [33]. In sum, most studies suggest that technology-supported MBC has the potential to improve the efficacy and cost-effectiveness of psychotherapy, especially for not-on-track individuals.

Risk of Bias Assessment
As observed in Tables 4-7, studies included in this review could be placed in four of the categories proposed by the National Heart Lung and Blood Institute [29], namely, case studies, before-after studies; observational cohort and cross-sectional studies; and controlled intervention studies. Overall, the four studies classified as case studies had a "good" quality, with total scores ranging from 5 to 7 points out of a maximum of 9 points [32,39,45,46]. Even so, none of them had follow-up sessions, or these were not reported, and two studies did not include a complete case definition [32,45]. Secondly, the two before-after studies could also be rated as "good" quality investigations, as both obtained 9 points of a maximum of 12 [41,43]. The main issue with one of the studies was related to sample size, although this concern was justified, as this was a pilot study and authors reported the previous in the limitation section [43]. The six studies classified as observational, cohort and cross-sectional studies correspond to feasibility and acceptability studies, and some quality criteria, such as numbers 7, 8, 12, and 14, were not applicable [30,31,34,36,37,44]. Overall, feasibility and acceptability studies did not meet most criteria required in observational and cross-sectional studies (they met only seven or eight criteria of a maximum of 14), so their quality could only be rated as "fair". It is important to note that just two studies maintained 80% of the sample [30,31] and four studies did not meet the participation rate of 50% from eligible participant criteria [30,31,37,44]. Finally, two of the controlled intervention studies [33,40] were rated as "poor" quality as they were non-randomized and did not follow most of the criteria for controlled studies (i.e., randomization, blind allocation, or assessment). These two studies met, respectively, only two [40] and eight [33] criteria of a maximum of 14. The remaining four controlled intervention studies [35,38,42,47] were "good" quality investigations despite the lack of blinded allocation and assessment [35,42,47] and relatively high drop-out rates [42,47].  . Were outcome measures of interest taken multiple times before the intervention and multiple times after the intervention (i.e., did they use an interrupted time-series design)?
Yes Yes 12. If the intervention was conducted at a group level (e.g., a whole hospital, a community, etc.) did the statistical analysis take into account the use of individual-level data to determine effects at the group level?

Yes Yes
Total score (maximum 12 points) 9 9 Note: CD, Cannot Determine; NA, Not Applicable; NR, Not Reported.

Discussion
The aim of the present study was to systematically review evidence on how ICT is being used for MBC in psychological treatments. To the best of our knowledge, this is the first attempt to systematically examine the different technologies that have been used so far for MBC during psychological interventions and to explore to what extent the use of ICT for MBC is feasible, acceptable, and effective.
One important finding was that only 18 studies met our inclusion criteria, which suggests that this is a field that requires more research. Additionally, the included studies varied greatly in terms of study design, diagnoses, MBC characteristics, and technology used, which again suggests that more investigation and replication will be needed to obtain robust findings about different technological solutions for MBC and to facilitate the generalization of the results to different populations. Future research should also examine the implication of technology systems for MBC in specific populations (e.g., children and adolescents, who are more familiar with technology systems). Furthermore, most of the included studies focused on mood and anxiety disorders, so it would be interesting to investigate the effects of technology used for MBC in other mental disorders.
As noted in the results section, the included technologies to support MBC were frequently handheld ICTs, such as smartphone apps, tablets, or laptops, which were involved both in the patient monitoring process and in the feedback to the therapists. In this sense, while tablets and laptops might be more difficult to use for EMA, it is encouraging that smartphone apps, which might facilitate EMA to a greater extent than other technologies, are also being used as supporting technologies for MBC.
An important finding regarding technology was that, overall, the use of technology for MBC during psychological interventions appears to be feasible and acceptable. In addition, technology in MBC was found to be effective [38] and cost-effective [41], particularly for not-on-track patients [35,40,42,47], as revealed in previous studies using MBC without technology [1]. Importantly, treatment engagement (i.e., time until dropout) was also enhanced with technology-supported MBC [33]. While these findings should be interpreted with caution, due to the limited number of existing investigations and the reduced number of controlled trials comparing technology-supported and non-supported MBC, the results suggest that the use of ICT to support MBC should continue to be tested in the future.
It is important to acknowledge that technology-supported EMA for MBC is different to a similar concept, which is ecological momentary intervention (EMI). Specifically, while in EMI a given intervention is provided in response to EMA in a timely manner (e.g., providing therapeutic skills with an app based on patient responses), in MBC EMA is only used to enhance face-to-face psychological interventions.
It is also important to note that some studies were excluded from this systematic review for several reasons which should be mentioned here. For instance, in some investigations, the monitoring process was not used to guide therapeutic decisions, even though technology was used for monitoring (thus, this would not be considered MBC). Conversely, in other studies, the technology was only used in one part of the MBC process, most frequently during the feedback part, but not for EMA (e.g., assessments were made in a paper-and-pencil approach, but then the information was introduced in a computer and presented in graphs or charts to the therapists and/or patients) [16,[48][49][50].

Limitations
Some limitations should also be considered when interpreting the results of the present systematic review. As in previous similar reviews [13,18], the heterogeneity of studies with respect to sample size, measures used, and methodology, to name some examples, made it difficult to piece together the results and restricted the implementation of a meta-analysis which affects the generalizability and robustness of findings. Moreover, the majority of included studies were feasibility pilot studies, case studies, or non-RCTs. Although these designs can yield valuable information, RCTs, which have been rarer, are considered superior because of their higher internal validity and, therefore, higher robustness of the evidence indicating a (causal) relationship. Additionally, some factors might have biased the present systematic review findings, including the fact that only three databases were used for the search, and the possibility that studies where ICT was not feasible or did not add any value have not been published. Finally, it is important to note that this systematic review is limited to the interpretations of the authors who conducted the systematic review.

Conclusions
To conclude, this systematic review found preliminary support for the use of technology in MBC during psychological interventions. The use of ICTs in MBC has brought some encouraging contributions to the evolution of psychotherapy and its inclusion in routine care might significantly change the way psychotherapists work. Particularly, the provision of real-time information on symptom progress over the course of psychological interventions might help therapists detect and rapidly react to problems that might occur during treatment (e.g., exacerbation in symptomatology or low adherence to recommended practices). This would make current interventions more flexible and personalized [22] and should favor the psychotherapeutic relationship during face-to-face interventions. Additionally, this might increase the patients' awareness of their own progress.
In addition, technology was generally found to be a feasible and acceptable add-on tool for the MBC process. Therefore, the use of technology for improving the MBC process is mostly supported, as it might facilitate EMA and offer some potential for improving psychotherapy thanks to the real-time connection between patient assessment and therapist and patient feedback [32]. While the presented results are, overall, encouraging, especially for not-on-track patients, more research is required in this field, especially RCTs comparing technology-supported MBC with traditional MBC without technology.