Routine Outcome Monitoring in Mental Health Care and Particularly in Addiction Treatment: Evidence-Based Clinical and Research Recommendations

Routine Outcome Monitoring (ROM) concerns the repeated measurement of the progress of a patient’s treatment during the course of therapy. ROM is receiving mounting attention as an important quality tool, and because of increased emphasis on health care accountability and cost containment. ROM seems especially relevant for complicated chronic patients that require a long-term treatment. Although addictive patients often have these clinical characteristics, ROM is relatively less investigated within addiction treatment. The present article summarizes recent state-of-the-art information concerning the use of ROM in mental health care and particularly in addiction treatment. First, some basics about ROM in general are described. Next, evidence for the effectiveness of ROM in mental health care and addiction treatment is reviewed. Finally, some clinical & research recommendations are suggested for the use of ROM in addiction treatment.


Introduction
Routine Outcome Monitoring (ROM) refers to the periodic assessment of patient variables, such as symptom severity, functioning and well-being. Assessments take place before, during and after treatment. Outcomes of these variables, like total score against normscores and rate of change, are given to therapist and preferably also to the patient. This process is often referred to as ROM Feedback [1]. The use of ROM as a component of evidence-based practice in mental health care has increased widely during the past decade [2]. Given the large number of publications, it is evident that ROM is a hot topic [3][4][5][6]. In ROM, Patient Reported Outcome Measures (PROMs) assess the patients' self-report experiences of symptoms, functioning, health and/or quality of life. Although PROMs are most often used in ROM, other clinician-rated instruments may be used as well [7][8][9][10].
ROM can be an important quality tool for measuring the clinical response of patients during the course of treatment and thereby for optimizing clinical decisions of the therapist. Ultimately, when a patient is identified to be responding poorly only at the end of treatment, it is too late for a shift in treatment that may have resulted in a more favourable outcome [1,11]. This is all the more important, since empirical evidence indicates that about 30% of mental health patients fail to respond during clinical trials and 65% of patients in routine care leave treatment without a measured benefit [11]. Moreover, therapists appear to overestimate positive outcomes in their patients and they are poor at identifying patients at risk for a negative outcome [11]). These findings point to the need for repeatedly measuring the patient treatment response during treatment and for giving feedback to therapists about it.
In light of this, it is suggested that ROM and feedback methods should become a standard of practice in mental healthcare [11]. This seems particularly relevant for complicated patient groups, like those with multi-morbidity and/or receiving long-term treatment. Patients suffering from addiction are often characterized with such clinical characteristics [12,13]. Research shows that co-occurrence of addiction with other mental disorders (e.g. depression, anxiety-, eating-, personality-, psychotic disorders) as well as co-occurring addictions (i.e., tobacco, alcohol, drugs, problem gambling, pathological internet use) are the rule rather than the exception. Such diagnostic comorbidity may reflect underlying genetic vulnerabilities, personality patterns, or interactions between individual symptoms, which complicate treatment progress [14][15][16]. Elevation of comorbid symptomatology may evoke relapse or could trigger an increase in substance abuse. In addition to presenting a more complex array of problems, patients with co-occurring or chronic disorders are at risk of poorer outcomes following treatment, which add to the cost-of-illness [17,18]. Overall, these patients have comprehensive needs that require complex and long-term treatment, including several other services (e.g. rehabilitation, detox, supported housing; [19]). In these complex treatments, ROM may be useful for monitoring the versatile psychopathology, including other relevant aspects (e.g. trauma assessment) and treatment can be adjusted in a more timely manner. This is in agreement with recent studies regarding integrated treatments for co-occurring addictive and psychiatric disorders [20][21][22].
Despite the interest in, and potential benefits of ROM, its growth across the various psychiatric disorders has been rather uneven [3]. For instance, ROM is relatively less investigated in chronic psychiatric patients, like those with addictive disorders and possible co-occurring psychiatric disorders [6,11]. Therefore, the present article summarizes recent state-of-the-art information concerning the use of and evidence for ROM in mental health treatment. First, some basics about ROM in general are described. Next, evidence for the effectiveness of ROM in mental health care and specifically in addiction treatment is reviewed. Finally, some clinical and research recommendations are suggested for the use of ROM in addiction treatment.

Method
For the section on the effectiveness of ROM (including ROM feedback) with adult psychiatric patients, a literature search was carried out up to May 2017 according to guidelines [23,24]. Consequently, the most relevant databases/literature sources were used: e.g. Medline; Embase; PubMed; Cochrane Central Register of Controlled Trials; CINAHL; Web of Knowledge; Ovid PsycINFO; Web of Science; International trial registries; Google Scholar. Electronic searches were supplemented with manual scanning of reference lists of retrieved articles and reviews. Retrieved references were then screened for relevance. To be comprehensive, non-randomized but relevant studies (e.g. ROM in addiction treatment) were included. Main search terms were: routine outcome monitoring, (progress) feedback, treatment outcome, mental health/psychiatry, addiction, addictive disorders, substance use disorders, substance abuse, effectiveness, efficacy, cost-effectiveness. Excluded were publications concerning routine outcome management (merely benchmarking), children/ adolescents and non-mental healthcare.

Routine outcome monitoring
The routine measurement of outcomes has risen in prominence over the past years. ROM started originally in health care and somatic medicine (e.g. oncology, neurology), later it was also utilized in mental health care [1]. Implementation of ROM in mental health care was somewhat delayed, mainly because it proved to be more difficult than implementation in other areas of health care [25]. Several terms besides (routine) outcome monitoring are used in the context of ROM: e.g. (routine) outcome measurement, treatment progress monitoring, patient-reported outcomes monitoring, patient monitoring, personfocused outcome measurement, measurement-based care, practicebased evidence, measurement feedback system.  34,35]. Which ROM instrument(s)/system to choose are less important than the fact that ROM basically implies multiple measurements during treatment, and not solely a pre-and a post-treatment measurement. If necessary, follow-up measurements after treatment can also apply, especially with chronic psychiatric disorders such as addiction. Additionally, ROM implies providing ROM-feedback on patient progress to the therapist and preferably also to the patient. In fact, without these basic conditions, there is no ROM but merely assessment.
In this context, at least two conceptualizations can be considered in which links between processes and outcome of ROM feedback are theorized: Feedback Intervention Theory and Therapeutic Assessment [1,10]. Feedback Intervention Theory (FIT; [11, [36][37][38]) implies that the provision of feedback to a therapist is a critical factor in the enhancement of self-regulation and knowledge for a therapist. Feedback to therapists works best with characteristics such as: e.g. immediate, frequent, simple, unambiguous, data-driven, novel information [36,37]. ROM feedback may initiate important management decisions for the clinician such as changing treatment, giving extra advice or education to the patient, or referring to other professionals [10]. Feedback may also help clinicians engage in thinking more deeply about patients and it provides them with a greater sense of professional identity [39]. Therapeutic Assessment (TA; [40,41]) on the other hand is focused on the therapeutic effects of providing test feedback of questionnaires to patients. In this context, discussing test-derived inferences with patients is part of a therapeutic strategy rather than diagnostics. TA usually involves three sessions and it is a mini-treatment in its own right, because it functions as a mean of patient self-verification, self-enhancement, self-efficacy, self-discovery [1]. Repeated ROM measurement might also increase patients' symptom awareness, their ability to report relevant symptoms, their self-confidence and treatment compliance [39]. This could help the patient to improve more quickly [10]. Additionally, feeding ROM results back to both therapist and patient could enable shared decisionmaking and shared goals between therapist and patient, which may be beneficial for patient satisfaction and possibly also treatment outcome [42,43].
Potential benefits of ROM are multiple, e.g.: a) improving diagnostics, timely adjustment of treatment and treatment management; b) shared decision making and patient participation; c) better patient outcomes (e.g. symptoms, functioning, quality of life); d) better cost-effectiveness of care; e) better satisfaction with provided care; f) better patient-therapist communication and therapeutic alliance; g) better adherence to clinical directives by therapists; h) scientific research; i) benchmarking (internal/external). There may also be potential obstacles for ROM, e.g.: a) financial burden; b) time burden for patient and therapist; c) implementation challenges; d) applicability in daily clinical practice (i.e., the actual use of ROM feedback by therapists); e) therapist reservations against ROM (e.g. external control, internal benchmarking, confidentiality). In this context, there is also debate on whether ROM is valid for the use of group-level ROM data for external benchmarking [44]. ROM use for benchmarking pertains to health care accountability and cost containment, and thereby growing influence of insurance companies who emphasize more transparency regarding cost-effectiveness and patient satisfaction. When ROM data are used for benchmarking, outcomes of mental health institutions are compared, but this may be invalid because institutions differ considerably from each other ("comparing apples to oranges"; [43]). The discussion on external benchmarking with ROM is further complicated by concerns about the privacy of patient data and the restriction that only pre-post treatment ROM data are used. For more discussion on external benchmarking with ROM, we refer to some relevant publications [44][45][46][47][48][49].

Effectiveness of ROM in mental health care
As described above, there might be many potential benefits of ROM in mental health care. However, what evidence for these benefits is available so far? In summary, the empirical evidence on the effectiveness of ROM or ROM feedback in mental health treatment of adult psychiatric patients is as follows [1,10,11,39,[50][51][52][53][54][55][56][57][58][59].
In general, giving clinicians ROM feedback on the progress of their patients may improve certain outcomes of mental health therapy [6,11]. Research showed that ROM feedback to therapists was especially helpful for off-track patients who are at risk for treatment failure (i.e., signal cases). That is to say, ROM feedback made a marked difference for complicated patients who went significantly off-track during treatment (about 20-40% of treated patients). In accordance with the aforementioned theory of Therapeutic Assessment, it was found that giving ROM feedback to both therapist and patient was more effective than to therapist alone [11]. Research also suggests that providing feedback from multiple sources (both wellbeing and affective psychological distress) enhances patient outcomes more, compared to single source feedback ([only wellbeing; [43]). Furthermore, ROM may improve the patient-therapist communication and the process quality of treatment (i.e., presence of treatment plan) and ROM feedback to therapists can lead to fewer treatment sessions [42]. Noteworthy was the finding that giving ROM feedback to both therapist and his/her supervisor (including joint discussion about it) seemed not more effective than giving ROM feedback to therapists alone [6]. Mixed results were found regarding the cost-effectiveness of ROM and the positive impact of ROM on satisfaction of patients with provided care [10, 42,60].
Despite these mostly positive results of ROM progress feedback in mental health treatment, there are also some remarks. First, studies regarding the effectiveness of ROM feedback differed quite a bit in terms of overall effect sizes, which ranged from small (0.1) [10] to larger (0.3-0.7) [11,61]. With more severe mental health problems, compared to common or mild problems, effect sizes seem to be reduced [6]. Second, it is hard to draw general conclusions on the effectiveness of ROM feedback, because ROM studies are heterogeneous and methodologically not always optimal (e.g. blinding of assessors, attrition; [10]). Third, ROM feedback may also have adverse effects in very vulnerable and emotionally deregulated patients, especially when they are not yet experiencing any symptomatic relief. Such adverse effects of ROM feedback were found in for instance (partial) inpatients with certain personality disorders [62] and in emergency psychiatry patients [63]. Especially in crisis situations, immediate feedback to psychiatric patients could be counterproductive, because patients are not (yet) able to reflect on their treatment processes, and confronting them repeatedly with their low level of functioning may demoralise them. So, perhaps ROM feedback does not work as well with more disturbed patients in comparison with the less disturbed [63].

Effectiveness of ROM in addiction treatment
What about the specific effectiveness of ROM feedback within substance abuse treatment? On the whole, the above-mentioned effectiveness of ROM feedback in mental health treatment was found across clinical samples and treatment settings [6,11]. That is, although the effectiveness of ROM feedback research largely concerned patients with common psychiatric disorders (depression, anxiety), patients with more complex and chronic psychiatric disorders were also studied (e.g. substance abuse, eating disorders, somatoform disorders, and psychiatric inpatients [6,11, [64][65][66][67]). The following is a summary of the most relevant studies regarding ROM feedback in a substance abuse treatment context.
One of the first studies evaluating feedback in substance abuse treatment was the study of McCaul and Svikis [68], which examined the impact of providing feedback reports on patient's attendance data to clinicians in a substance abuse treatment clinic. The results showed improvements in attendance. Forman et al. [69] reported on the feasibility of implementing a web-based feedback system for monitoring patient ratings in outpatient substance abuse treatment clinics. Although the results confirmed the feasibility of the feedback system for therapeutic alliance and treatment satisfaction, further modifications of the system were needed to enhance its potential clinical usefulness (e.g. drug/alcohol use).
Crits-Christoph et al. [70] reported on the results of a randomized trial that examined the efficacy of providing caseload feedback (averaged patient outcomes on therapeutic alliance, treatment satisfaction, drug/alcohol use) to therapists who treat substance abuse. No effects of feedback were found, because therapists were relatively unmotivated by feedback reports that were oriented towards helping them with patients as a group rather than feedback reports for specific patients. Therefore, these results led to a modified version of the feedback system, consisting of weekly feedback to therapists on individual patient outcomes (including functioning, symptoms and treatment satisfaction), with special attention to patients who were not progressing in treatment. The modified feedback system was tested in a subsequent study, which showed that a ROM feedback system adapted to the treatment of substance use problems was promising [64]. In a related study, Crits-Christoph et al. [71] examined predictors of outcome of substance use treatment, in order to inform the development of recovery curves for feedback reports to therapists. The results showed, among other things, that baseline severity of symptoms/functioning, employment, and craving were positively associated with the rate of change in symptoms/functioning, although further research is recommended [71].
Schuman et al. [72] broadened the usefulness of ROM feedback for substance abuse from individual treatment to group treatment. More specifically, they showed the efficacy of client feedback in group psychotherapy with soldiers referred for substance abuse treatment. When patients received feedback about their symptom course, the overall treatment effect was more favourable. This result is important, because many substance use treatment facilities use a group format to provide treatment. Noteworthy was the fact that these feedback effects were present for all patients in the feedback condition, not just those at risk for negative treatment outcome. This seems to be in contrast with ROM feedback in individual treatment, which is mostly effective in at risk patients [11]. Finally, Johnston et al. [73] evaluated the applicability and acceptability of ROM feedback in an addiction medicine setting. Their results generally showed that both therapists and patients agreed that the ROM feedback helped them to identify areas of greatest concern and was useful for treatment planning.

Practical use of ROM in addiction treatment
What about the practical use of ROM feedback in substance abuse treatment by therapists? In this context, particularly the previously mentioned publications of Crits-Christoph et al. [64] and Schuman et al. [72] are inspiring examples. Both publications concern the application of ROM among addicted patients according to an evidence based and fairly easy implementable method. Moreover, these publications include both psychotherapeutic treatment modalities: individual treatment [64] as well as group treatment [72].
In the study of Crits-Christoph et al. [64], patient progress across individual addiction treatment was tracked using the well-established Outcome Questionnaire-45 (OQ-45) [28], adapted to include drug and alcohol use (Modified OQ-45). The original OQ-45, a validated instrument across a broad range of normal and client populations, is a 45-item self-report scale (5-point scale) developed specifically for tracking and assessing patient outcomes in a therapeutic setting. The original OQ-45 has three subscales that measure quality of interpersonal relations, social role functioning, and symptom distress. High scores on the OQ-45 indicate greater levels of symptoms and/or poorer functioning [28,64]. The Modified OQ-45 included two extra items measuring the number of days in the past week that the patients used (a) alcohol and (b) drugs. Crits-Christoph et al. [64] only used the total OQ-45 score, which provides a global assessment of functioning, and the individual alcohol and drug use items. The Modified OQ-45 was administered immediately prior to every treatment session, up to a maximum of 12 treatment sessions. All OQ-45 data were collected electronically by clinic receptionists, research assistants, or therapists. Data were captured electronically and stored in a database. First, immediate OQ-45 feedback reports were viewed electronically, so that increasing or decreasing drug/alcohol use of the patient would be evident to the therapist [64]. Second, when patients were identified as being off-track (not progressing as expected) based upon their OQ-45 scores, therapists were given the opportunity to use clinical support tools (CST) that provided suggestions about ways to improve treatment [64]. Upon being identified as off-track, patients were asked to complete the Assessment for Signal Clients (ASC) [11]; a 40-item self-report measure (5-point scale) designed to assess the severity of problems that may be impeding treatment progress within 4 evidence based domains: problems with therapeutic alliance, motivation, social support, and stressful life events [64]. Therapists were trained to examine the ASC feedback report, noting the domains and the specific items of concern. After the patient completed the ASC, the ASC feedback report is made available to the therapist. This web-based ASC feedback report contains a link to a list of interventions that target each problem domain identified by the ASC [64]. Some specific examples of treatment interventions for the "motivation domain" are the following: assess how important it is for the client to make target changes, having him or her rate the desire to make specific changes on a scale of 1-10; discuss the client's important values, goals, and aspirations, and ask open-ended questions to see if the client is ready to make a commitment to change [64]. If it appeared that difficulty within the four domains did not adequately address the patient's treatment failure (i.e., going off-track), therapists were trained to reassess the patient's diagnostic formulation and to consider whether an alternate kind of treatment could be more effective and/or whether a referral for a medication consultation is an appropriate avenue to pursue [11,64]. In conclusion, the study of Crits-Christoph et al. [64] showed that patient progress feedback to therapists appeared to successfully bring the offtrack patients back on-track. A next step in this regard may be to also inform patients about their treatment progress via their therapist, which could facilitate the mutual communication and shared decision making about treatment.
In the study of Schuman et al. [72], patient progress across group addiction treatment was tracked using another well-known and validated instrument, the Outcome Rating Scale (ORS; [27]). This is a self-report instrument designed to measure patient progress repeatedly throughout the course of therapy. Adapted from the OQ [28], the ORS assesses four dimensions: (i) Individual-personal or symptomatic distress or well-being. (ii) Interpersonal-relational distress or how well the patient is getting along in intimate relationships. (iii) Social-the patient's view of satisfaction with work/school and relationships outside of the home. (iv) Overall-general sense of well-being [27]. The ORS translates these four dimensions into a visual analogue format of four 10 cm lines, with instructions to place a mark on each line, with low estimates to the left and high to the right. The four 10-cm lines add to a total score of 40. The score is the summation of the marks made by the patient to the nearest millimetre (mm) on each of the four lines, measured by a centimeter (cm) ruler or template. A software program was used to collect data and provide the basis for feedback to the therapists [27,72]. The program used algorithms derived from previous ORS research and normative samples to predict the expected treatment response (ETR) for individual patients entering therapy with the same intake score. ORS scores from subsequent sessions are then compared against the ETR, which allows clinicians to identify patients who are making progress as expected as well as those at risk for premature termination or a negative outcome [27,72]. Therapists were provided with a graph that used four different colours (Green/Blue/Yellow/Red) to indicate one of four different stages of treatment progress corresponding to the patient's ORS score compared to the ETR: "Patient's functioning is in the normal range. Consider successful termination. " (Green); "Patient's functioning is in the adequate range. No change in the treatment plan is recommended. " (Blue); "Patient's functioning is less than adequate. Consider altering the treatment plan by intensifying treatment, shifting intervention strategies, and monitoring progress especially carefully. " (Yellow); and "Patient is not making the expected level of progress. Steps should be taken to carefully review this case and decide upon a new course of action such as referral or a higher level of care. The patient's readiness for change may need to be re-assessed. " (Red) [27,72]. Five sessions of group treatment (10 patients per group) were offered, because it served the primary function to identify at-risk clients. If patients continued to exhibit concerns that warranted further intervention, additional group sessions were offered [72]. Before each group session, patients completed the computerized version of the ORS in a private room. Once patients placed their marks on the appropriate lines, they would click the "done" button and proceed to the group therapy room [72]. The program automatically scored the instrument. Patients did not see the results generated by the computer program. Any results they may have seen would have been revealed by their therapist if he or she chose to share the results. At the conclusion of each group session, therapists were given progress graphs of patients [72]. In sum, Schuman et al. [72] showed that the benefits of patient feedback in individual psychotherapy appear to extend to group psychotherapy. However, they also point to a downside of the ORS: although psychometrically acceptable, the ORS does not directly assess substance use which was a primary focus of their group treatment [73].

Clinical and research recommendations for use of ROM in treatment
As with any chronic disease, the objectives for successful long-term management of substance use disorders feature regular screening, additional care as needed, and especially long-term monitoring [60]. Long-term monitoring implies the monitoring of treatment progress of patients both during and after treatment, which is particularly relevant for chronic mental diseases such as addiction. Therefore, several studies in the context of ROM and addiction were focused on the feasibility and validity of low-budget telephonic follow-up interviews in routine outcome monitoring of substance abuse treatment [74][75][76][77]. It appears that telephone case monitoring is a cost-effective strategy that increased the days of abstinence after treatment [60]. However, in this context, research also showed that tracking problems make it difficult to interview substance abuse patients at the intended followup time [76].
An important concern generally remains the implementation and the integration of ROM in clinical practice [26, [78][79][80]. ROM feedback can be considered as a communication tool, and research showed that ROM feedback discussions have a positive effect on treatment cooperation [81]. However, research showed that ROM feedback reports were often not used by therapists [11,66,82]. So, more attention may be needed for recurrent training of therapists on the use of ROM as an opportunity to discuss treatment progress, stagnation, or decline with the patient and to set common treatment goals together [4, 79,83]. Moreover, ROM results could be integrated in the treatment plan of a patient [25]. This integration, however, is far from evident which was demonstrated in a recent study of Tasma et al. [2]. This study investigated whether, with ROM identified (mental) health problems (i.e., symptoms, psychosocial problems, cardiovascular risk factors), were reflected in the treatment plans (Electronic Patient Files) of psychotic patients. They found a substantial discrepancy between the ROM measurements and the treatment plan: low rates of detection of problems in the treatment plans, even though these problems were identified with ROM and the other way around was also the case. Thus, ROM and clinical practice appear to be two separate processes, whereas ideally they should be integrated. Strong efforts should be made to integrate ROM and consequent treatment activities. Such integration may help to provide patients with adequate and customized care and simultaneously minimize under-and over-treatment [2].
A major limitation of many feedback researches so far is the absence of information on the specific types of clinical actions that were taken by clinicians in response to feedback reports about their client. Future research will need to address this topic, because it relates to the possible mechanisms of feedback effects [64]. Additionally, it may be of interest to examine the effects of feedback within subgroups defined by certain clinically relevant baseline variables. It may be, for example, that feedback has little impact among patients with long term substance use and comorbidity and more impact on less complex patients. So, further research regarding possible moderators or mediators of feedback effects in (addiction) treatment is required [71].
It is further recommended to supplement ROM treatment progress feedback with earlier mentioned Clinical Support Tools (CST). CST refers to an instrument assessment for signal cases, with detailed information to therapists about alarming scores of signal cases on specific subscales or items (e.g. therapeutic alliance, social support, readiness to change, diagnostic formulation, life events, need for medication referral, suicidality [11]). Research showed that, when feedback with CST is used for off-track patients, outcome improvement is greater than feedback without CST [11]. Future research is necessary to develop and compare different ROM clinical support tools. Such studies may also investigate what might be the optimal alert system for therapists regarding clinical deterioration of patients, thereby increasing therapeutic impact and efficiency [6].
Some final suggestions for future research on ROM are the following [6,10,11,59]. First, research is required regarding the positive impact of ROM on a broad range of all relevant patients' outcomes (e.g. symptoms, social functioning, quality of life). In addition, further research is needed concerning the positive impact of ROM on costs, treatment management (e.g. number and frequency of treatment sessions) and shared decision making, because these aspects are still insufficiently investigated to draw final conclusions. Second, possible differential effectiveness of ROM in different treatments (like psychotherapy versus medication) has not been investigated sufficiently [10]. Third, most studies of ROM feedback in mental health treatment showed its short term effectiveness, and less researched are the long-time effects of ROM. Therefore, more research regarding the beneficial effect of ROM feedback in the longer term [beyond six months; 10] is recommended. Fourth, additional research is needed on possible harmful effects of ROM feedback in very vulnerable psychiatric patients. Fifth, further research is necessary on the effectiveness of ROM feedback in group psychotherapy, since studies in this context showed inconsistent results for different patient groups [71,84].

Conclusion
The use of Routine Outcome Monitoring (ROM) as a quality tool for measuring outcome of treatment in mental healthcare is evidencebased. In general, promising results were found regarding the effectiveness of ROM feedback in the treatment of both common mental disorders and more chronic mental disorders like substance abuse. Despite the large amount of research on ROM feedback so far, better quality trials are needed that are more decisive regarding when and for whom ROM feedback is effective in both individual and group treatment. This is particularly the case with addictive disorders in which ROM feedback is relatively less investigated. It is also recommended to examine how therapists actually use the ROM feedback for the benefit of their clinical decisions and in their communication with the patient. Only in this way, ROM can be an integral and valuable part of routine practice.