The Results of Clinician-Focused Implementation Strategies on Uptake and Outcome of Measurement-Based Care in General Mental Health Care

BACKGROUND: Measurement-Based Care (MBC) is a promising way to improve outcomes in clinical practice, but the implementation of MBC is often problematic and the uptake by clinicians is low. METHODS: We used an effectiveness-implementation hybrid design based on Grol and Wensing’s implementation framework to examine the results of clinician-focused implementation strategies on both clinicians’ uptake of MBC and outcomes of MBC for clients in general mental health care. Primary outcomes were questionnaire completion rate and discussion of feedback. Secondary outcomes were treatment outcome, treatment length, and satisfaction with treatment. RESULTS: There was a signicant medium effect of the MBC implementation strategies on questionnaire completion rate (one part of the clinicians’ uptake), but no signicant effect on the amount of discussion of the feedback (the other part of the clinicians’ uptake). Neither was there a signicant effect on clients’ outcomes (treatment outcome, treatment length, satisfaction with treatment). CONCLUSIONS: Establishing and sustaining MBC in real world general mental health care is very complex. Our study helps to disentangle the effects of MBC implementation strategies on differential clinician uptake, but the effects of MBC implementation strategies on client outcomes need further examination. for such a systematic approach to implementation. It is based on both existing theories and models and practical experience. We used this framework in our study. The framework consists of different phases of tailored implementation: orientation, insight, acceptance, change, and maintenance. Each phase has a focus associated with it: (a) Making clinicians aware of the innovation and getting them interested, (b) providing insight into their current way of working and the innovation, (c) getting clinicians motivated and intended to change, (d) implementing the innovation and conrmation of the usefulness, and (e) integration in existing routines within the organization. Within each phase, different implementation strategies can be used. mental health care. The results showed only a medium effect of the MBC implementation strategies on questionnaire completion rate (one indicant of clinicians’ uptake). During implementation, the rate at which questionnaires were completed increased from 26–37% before they decreased to almost 28% during follow-up. The questionnaire completion rate in the Control Group was stable at about 7%. Effects were found for neither the degree of discussion of feedback nor clients’ outcomes (treatment outcome, treatment length, nor satisfaction with treatment).


Contributions To The Literature
Because implementation of MBC is often problematic, uptake by clinicians is low, and previous studies have often not described their implementation strategies, this study explicitly focused on implementation strategies and their effects.
We found that clinician-focused implementation had a signi cant medium effect on questionnaire completion rate, but no effect on the amount of discussion of feedback or on client outcomes of MBC.
Although our study helps to disentangle the effects of clinician-focused implementation strategies on differential uptake by clinicians, the effects on client outcomes could not be established because clinicians were not using the feedback more often.

Background
One promising intervention for improving outcomes in clinical practice is Measurement-Based Care (MBC). MBC is the routine administration of symptom, outcome or process measures, clinicians' review of the feedback from these measures, clinicians' discussion of the feedback with their clients, and collaborative evaluation of the treatment plan based on the feedback [1,2]. This evidence-based practice is also referred to as Routine Outcome Monitoring (ROM) or Feedback-Informed Treatment (FIT).
Extensive research has been conducted on the additional effects of MBC on treatment results in adult mental health care. The most comprehensive meta-analysis to date [2] reported a small positive effect of MBC on symptom reduction in the full sample (d = 0.15) and in the subsample of clients who were not progressing well through treatment (d = 0.17) and a small favorable effect on dropout rates (OR = 1.20), but no effect on the percentage of clients who had deteriorated by the end of treatment or on treatment duration.
The implementation of MBC has often been problematic [3] and uptake by clinicians low [4]. Research across different countries indicates that typically fewer than 20% of clinicians employ MBC in their day-to-day work, only 5% of them use MBC every session, and 61.5% of them have never used MBC [5]. In the Netherlands, the same trends have been reported. Implementation is problematic [6], and there has been limited use of MBC by clinicians in their daily practice [7].
There are many different theories, models, and frameworks for implementing interventions in mental health care, each of which is based on different assumptions about human and organizational behavior [8]. However, the evidence for the validity of these theories, models, and frameworks is mixed and, overall limited [9]. It can be concluded that the effective implementation of an intervention requires a systematic approach with good planning and adequate evaluation [10]. Grol and Wensing [11] offer a framework for such a systematic approach to implementation. It is based on both existing theories and models and practical experience. We used this framework in our study. The framework consists of different phases of tailored implementation: orientation, insight, acceptance, change, and maintenance. Each phase has a focus associated with it: (a) Making clinicians aware of the innovation and getting them interested, (b) providing insight into their current way of working and the innovation, (c) getting clinicians motivated and intended to change, (d) implementing the innovation and con rmation of the usefulness, and (e) integration in existing routines within the organization. Within each phase, different implementation strategies can be used.
Grol and Wensing [11] provide suggestions for implementation strategies. In addition, MBC research has identi ed barriers to implementing MBC on different levels (client, clinician, organization, and system). Accordingly, additional strategies have been suggested for improving the implementation of MBC [12]. It appears that multifaceted or blended strategies, which involve techniques for facilitating adoption, implementation, and sustainment, are needed [1]. These strategies include using relevant and valid measures; using measurement feedback systems; leveraging local champions and opinion leaders; forming implementation teams with representatives from all stakeholder groups; having good instructional materials, clinician training, ongoing consultation, and supervision; adaptation and integration of the innovation within the organization; and generating incentives [1,11,12]. Additionally, the effective and sustainable use of MBC might require systematic efforts over extended periods of time [13]. In fact, successful implementation of an intervention can last between three and seven years (e.g., [10]).
Studies that have evaluated the effects of MBC implementation are relatively new [14][15][16][17][18][19][20]. In addition, very few of the studies has explicitly described the underlying implementation strategies. As a result, the implementation and use of MBC seem to be used interchangeably, and research on the implementation of MBC (which focuses on clinicians' uptake) and clinical research on the effects of MBC (which focuses on clinical outcomes) seem to be intertwined. As far as we are aware, the direct effects of MBC implementation strategies on clients' outcomes have not yet been examined.
Hybrid designs that simultaneously provide information on implementation processes and clients' outcomes seem to be needed [21]. By using an effectiveness-implementation hybrid design, it would be possible to disentangle the effects of MBC implementation strategies on clinicians' uptake from clients' outcomes. This could lead to better understanding of whether and how MBC implementation strategies affect clinicians and clients, and what needs to be changed to achieve better results.
In this study, we used such a hybrid design, which was based on Grol and Wensing's [11] implementation framework. Because the framework aims to optimize clients' outcomes by improving how clinicians deliver their treatment, we call the strategies used within each phase clinician-focused strategies. The clinician-focused implementation strategies were outlined before the implementation period began, but the strategies were elaborated during the implementation period, and additional strategies were added when results were found to be lagging. The goals of the study were to investigate whether implementation strategies that were developed with clinicians and aimed at clinicians had an effect on (a) clinicians' uptake of MBC and (b) clients' outcomes of MBC. We expected that better implementation would lead to (a) a higher rate of questionnaire completion and more discussion of the feedback by the clinicians, and (b) to better treatment outcomes, clients' greater satisfaction with treatment, and (c) a shorter length of the treatment.

Methods
The study adhered to the Standards for Reporting Implementation Studies (StaRI checklist, [22]; Additional File 1) and the Transparent Reporting of Evaluations with Nonrandomized Designs (TREND checklist, [23]; Additional File 2).

Design
We examined the results of our clinician-focused implementation strategies on both uptake of and outcomes from MBC by using an effectiveness-implementation hybrid design, which was based on Grol and Wensing's [11] implementation framework.
The study was presented to the clinicians as a way to improve MBC and thereby the quality of the care they provided to their clients. It was conducted in a large organization for general mental health care in the Netherlands, which has clinics in several different regions. The clinician-focused implementation strategies were applied within one region of this organization (Implementation Group). The other regions did not receive any additional implementation support and were combined to form the Control Group.

Context
Since 2011, mental health care organizations in the Netherlands have been required to collect outcome data for benchmarking on a national level. In January 2014, a distinction was made between general mental health care (for mild problems) and specialized mental health care (for complex problems). For the general mental health care, this resulted in changes in the organizational structures, the form of treatment, and the way of working. As a result, questions arose about how MBC could become a structural part of treatment in general mental health care and what additional effects MBC has in general mental health care. A need emerged for both improvements in MBC implementation and research on the effectiveness of MBC. To meet this need, the present study was initiated.
The large general mental health care organization in which the study was conducted consisted of clinics in seven regions throughout the Netherlands. All of the clinics were familiar with MBC, but the degree of implementation differed considerably among the clinics. The clinician-focused implementation strategies were applied within one region in the eastern part of the Netherlands. This region was strongly focused on MBC and had an explicit desire to improve their implementation of MBC.
There were four clinics in the region and they were combined to form the Implementation Group. Three of the other regions were excluded either because they were not part of the organization during the entire implementation period or they did not have data available for the entire period. The three remaining regions, which were spread throughout the Netherlands and included eight clinics, formed the Control Group.
The clinician-focused implementation strategies were provided to all of the clinicians in the Implementation Group. There was a total of 44 clinicians at the start of the implementation period in 2015 and 42 clinicians at the end of the implementation period in 2018. However, due to high staff turnover, only 19 clinicians worked at the organization during the entire implementation period. At the start and end of the implementation period, there were no signi cant differences in the proportion of female clinicians (65.9%, n = 29 versus 64.3%, n = 27) nor in the primary discipline of the clinicians: psychologists (77.3%, n = 34 versus 69.0%, n = 29), nurses (15.9%, n = 7 versus 19.0%, n = 8), and psychiatrists (6.8%, n = 3 versus 19.0%, n = 8).
The treatments that the clinicians offered were divided in short-term treatments and long-term treatments for serious mental illness. All of the short treatments (maximum of 12 sessions) were included in the study. They comprised various kinds of evidence-based interventions (ranging from solution-focused to cognitive-behavioral therapy) and focused on various kinds of problems (e.g., anxiety, mood, and personality disorders).

Procedure
As part of routine care, all of the clients in both the Implementation and the Control Group who were referred for a short-term treatment were invited by email to ll out the MBC measures before the intake, during treatment, and at the end of treatment.
No formal arrangements were in place regarding timing of the measurements, number of measurements, actions to be taken when clients did not respond, nor how feedback would be discussed with the clients. As part of the clinician-focused implementation strategies, all clients in the Implementation Group were invited to complete the measures before intake, after every third or fourth session, and before the last session. Clients in both groups were asked to ll out the Outcome Questionnaire-45 (OQ-45 [24,25]) at every measurement and the Consumer Quality Index (CQI [26,27]) before the last session.

Implementation strategies
Consistent with Grol and Wensing's [11] framework, we divided the clinician-focused implementation strategies into different phases. Additionally, we shifted the focus of the strategies from general to speci c. We outlined the clinician-focused implementation strategies before the start of the implementation period, and the strategies were further adapted in consultation with the clinicians in the MBC implementation team (see below) based on their experiences. The clinician-focused implementation strategies were applied from February 2015 to January 2018 and with a follow-up that continued through December 2018.

Phase 1: General MBC strategies (February 2015 -February 2016)
The main purpose of Phase 1 was to gain insight into the importance and the current state of MBC within the clinics, to outline possibilities for improving MBC, and to get clinicians interested to improve their use of MBC. The main implementation strategies used in this phase were (1) delivering a global presentation, (2) forming an MBC implementation team, (3) holding an MBC theme meeting in each clinic, and (4) providing monthly reports.
The starting point was the global presentation. Because it was given on an internal training day, most of the managers and clinicians attended. During the presentation, the current status of MBC and the study were outlined and the clinicians were invited to ask questions and give feedback.
The MBC implementation team comprised seven members: one clinician from each clinic, one manager, a highly experienced MBC researcher, and the rst author of the current paper. The members were self-selected on the basis of conversations and their positive attitude toward and interest in MBC. The premise was that the members of the MBC implementation team would serve as local champions and hence opinion leaders. The MBC implementation team was responsible for optimizing the conditions under which MBC would be delivered, developing and executing the implementation strategies, and identifying barriers and nding solutions. Examples of actions that the team took were as follows: they developed enhancements in the feedback report, developed and disseminated the MBC instructional materials, provided the clinicians with MBC cards with reminders and an action plan, and insured a continuing focus on MBC during clinic meetings.
Each member of the MBC implementation team started by organizing a clinic-speci c MBC theme meeting in his or her own clinic. At this meeting, more information was given about the study and the MBC implementation team, and practical aspects of MBC were discussed (e.g., where to nd the feedback reports, how to interpret the feedback report, how to discuss the feedback with the client).
The monthly report indicated the questionnaire completion rates, in both the preceding month and the course over time. The report was sent to the members of the MBC implementation team, and the managers and leading experts of each location. In the accompanying email, the cross-clinic conclusions were summarized, and clinics with more than 80% of the measures completed at the start of treatment were congratulated and given a cake and an inspirational card. The main purpose of Phase 2 was to gain each clinic's acceptance of the implementation strategies and foster the clinicians' motivation to change their behaviour. The main implementation strategies used in this phase were introduction of and training in MBC case consultation. Additionally, the members of the MBC implementation team continued with their activities.
During the MBC case consultation, the clinician was supposed to discuss with a colleague the feedback of those clients who were not improving (the not-on-track clients) while using a standardized format for the case consultation. The idea was that the clinician would choose a colleague with personal relevance for him or her and whose advice he or she would trust. The case consultation format included questions about how to explain negative results, implications for the treatment, and advice from the colleague. The format was designed to be completed within 15 minutes and it was integrated into the client's electronic health record. Thus, through MBC case consultation, both treatments could be improved and the clinician's use of feedback could be monitored.
Before starting with the MBC case consultation, the principal researcher provided each clinic with a one-hour training session.
The training consisted of a repetition of general and practical information about MBC, explanation of the MBC case consultation and the case consultation format, practice in conducting a MBC case consultation, and discussion of the clinicians' experiences. After the initial training, the monthly reports were supplemented with an overview of the not-on-track clients and the proportion of case consultation in each clinic. Also, in the MBC client-feedback report, a reminder was added to discuss within the MBC case consultation when a client was not-on-track. After four months, a one-hour, clinic-tailored booster session was provided in which the number of MBC case consultation to date were discussed; barriers were identi ed and resolved where possible; and clinicians were invited to suggest improvements by de ning their own MBC action points.
Between and after the training sessions, the MBC implementation team was available for additional support.

Phase 3: Individual-focused strategies (April 2017 -December 2017)
The main purpose of Phase 3 was to encourage clinicians who lagged behind to improve. The main implementation strategies included: (1) having an MBC implementation team at each clinic, (2) individual-focused monthly reports with additional rewards for achievements, and (3) one-on-one MBC appointments.
The MBC implementation team at each clinic included the member of the general MBC implementation team, the clinic manager, and a local champion of MBC. The aim of the local team was to get more clinicians involved, to specify and expand the implementation strategies, and to nd solutions for local barriers.
The monthly report was supplemented by results from each individual clinician. Clinics received a cake when they conducted case consultations for more than 50% of the not-on-track clients, and individual clinicians received owers when they met the criterion that was set by the local MBC implementation team (e.g., case consultation for at least 80% of their not-on-track clients).
To support clinicians who were struggling with MBC, each clinician with a questionnaire completion rate below 50% was offered a one-on-one meeting with the member of the MBC implementation team. During this meeting, possible obstacles were identi ed and solutions were considered. When the clinician was unable to improve the questionnaire completion rate during the following months, the clinician was invited to a meeting with the manager to consider additional steps for improving the MBC (e.g., coaching or supervision).

Phase 4: Follow-up (2018)
In the fourth phase, we examined whether MBC had been integrated into the clinicians' day-to-day practice and whether the changes were being maintained in the clinics. No additional implementation strategies were conducted during this phase.

Study variables
Background variables Baseline characteristics of clients in both the Implementation Group and the Control Group were collected at the start of treatment. These characteristics were: age, gender, education, household composition, marital status, diagnosis, and total di culties score on the Outcome Questionnaire-45 ( [28]; see below).

Uptake by clinicians
The rst goal of this study was to investigate whether the implementation strategies that were developed with clinicians and aimed at clinicians improved the clinicians' uptake of MBC. Thus, the primary outcome measures were questionnaire completion rate and extent to which the feedback had been discussed with the clients.
Because routine administration of the measures is a requirement for MBC, we de ned questionnaire completion rate as the proportion of clients who had completed the measures at both the beginning of treatment, at least once during treatment, and at the end of treatment. The questionnaire completion rate was derived from the electronic health record system.
The discussion of feedback was the extent to which the clinicians discussed the feedback reports with their clients. This was based on clients' answers to Question 12 on the Consumer Quality Index (CQI) for Mental Health care and Addiction Services, Version 1.0 [27], which was administered at the end of treatment. Question 12 on the CQI states, "Before and perhaps during treatment, you or your clinician(s) completed one or more measures about how you were doing at that time. Were the results discussed with you?" The response options were: no, not at all (1); a little bit (2); partly (3); largely (4); yes, completely (5), and not applicable (no lists completed).

Client outcomes
The second goal of this study was to investigate whether the implementation strategies that were developed with clinicians and aimed at clinicians improved clients' outcomes of MBC. Therefore, secondary outcome measures were treatment outcome, treatment length, and satisfaction with treatment.
Treatment outcome was the degree of change in symptoms and functioning as measured by the Outcome Questionnaire-45 (OQ-45 [28]). Treatment outcome was, therefore, available only for clients who lled out the measures at the start and end of treatment. The OQ-45 consists of 45 items, which are scored on a ve-point scale ranging from never (0) to nearly always (4). The total di culties score (range: 0-180) re ects symptom distress, problems with interpersonal relationships and problems with the social role. Change was de ned as the total score at the start of the treatment minus the total score at the end of the treatment. The RCI for the OQ-45 total score is 14, so a client must improve at least 14 points in order to show reliable change.
The Dutch OQ-45 has been found to have adequate and similar psychometric properties as the original OQ-45 [24].
Treatment length was measured as the number of days between registration and deregistration at the general mental health care institution.
Satisfaction with treatment was measured with Question 13 on the CQI: What rating do you give the treatment? Response options were 0 to 10, where 0 means very poor and 10 means excellent. Only the clients who lled out the measures at the end of treatment could answer the question about satisfaction with treatment.

Missing values
The percentage of missing values from the outcome questionnaire completion rate and treatment length were 0 and 0.3, respectively. However, for discussion of feedback, change in symptoms, and satisfaction with treatment, the percentage of missing values was 89, 72 and 73, respectively. The question arose as to whether the cases with non-missing values for each of the three variables could be seen as a valid representation of the total sample. Accordingly, we examined whether the nonmissing cases differed from the missing cases on each of the seven background variables, separately for the three outcome measures. We ran t-tests for independent samples (for age and the OQ-45 total score at the start of treatment) and chi-square tests (for the other background variables). Because of the large sample size, almost all of the tests showed signi cant results. To help us decide whether the signi cant results were meaningful, we used Cohen's d (based on the t-test results) and phi coe cients (based on the chi-square test results) as effect sizes. Cohen's d with effect sizes of .2 and phi coe cients with effect sizes of .1 are considered small effect sizes [29]. Cohen's d varied between .02 and .15, and phi varied between .01 and .06. These very small effect sizes suggest that in spite of the signi cant differences, there are no substantial differences between the non-missing and missing cases. For this reason, we assumed that the group with non-missing values was representative of the entire group for each of the three variables.

Analysis
We used descriptive statistics to depict the baseline characteristics of the participants, and we tested differences between the Implementation and Control group using a t-test for independent samples (for age) and chi-square tests (for the other six background variables).
For each outcome variable (questionnaire completion rate, degree of discussion of the feedback, changes in symptoms, length of treatment, and satisfaction with treatment), we plotted changes across time in the Implementation and Control Groups.
When there were both increases and decreases in an outcome variable, we identi ed differential changes in the four phases of the implementation strategies. We tested differences in proportions between the two subsequent half-year periods using z-tests for two independent proportions.
To compare changes across time in the Implementation and Control Groups, we used linear regression analysis with time as the independent variable, each of the outcome variables as the dependent variable, and slope as the overall estimate of changes across time. Because the likelihood of nding signi cant slopes in a large sample is strong, we additionally calculated effect size f 2 with values of .02, .15 and .35 as the criteria for a small, medium and large effect size, respectively [29].
To evaluate the effects of the implementation, we compared the slope of the Implementation Group with the slope of the Control Group using z-tests for differences between two independent slopes (30, pp. 46-47). The effect size was de ned as the difference between two standardized slopes, which can be interpreted as a partial eta-squared effect size (PES) with .01, .06 and .14 representing small, medium, and large effect sizes [31].
All analyses were conducted using IBM SPSS statistics, Version 26 [32].

Results
The nal sample included 50,272 unique clients who received a total of 55,647 admissions to treatment. That is, 5,375 clients (9.7%) were referred more than once for treatment, or they received multiple treatments after the same admission. Because our implementation strategies were focused on clinicians, treatment was the unit for analysis. The Implementation Group had 8,458 (15.2%) treatments, and the Control Group had 47,189 treatments (84.8% of all the treatments). Clients' baseline characteristics are presented in Table 1. Although they show a signi cant difference between the Implementation Group and the Control Group on all variables, the effect sizes are negligible.    5 show changes across time for both clinicians' uptake of MBC (questionnaire completion rate and degree to which the feedback was discussed) and client outcomes of MBC (changes in symptoms, length of treatment, treatment satisfaction). In the Implementation Group baseline rate of questionnaire completion was higher and days in treatment were lower than in the Control Group. Figure 1 shows that in the Implementation group there was a signi cant decrease in questionnaire completion rate during the second part of Phase 1 (second half of 2015 to rst half of 2016, z = -3.81, p = .000), a signi cant increase in the second part of Phase 2 (second half of 2016 to rst half of 2017, z = 3.97, p = .000) and in Phase 3 (second half of 2017 to rst half of 2018, z = 3.53, p = .000), and a signi cant decrease in Phase 4 ( rst half of 2018 to second half of 2018, z = -3.86, p = .000). Overall, the questionnaire completion rate increased from 26-37% during the implementation phases, before decreasing to almost baseline level during follow-up (28%). The questionnaire completion rate in the Control Group was fairly stable and varied around 7%.
The other outcome variables (see Figures 2 to 5) were rather stable across time. For both groups, the degree to which the feedback was discussed varied between 4 and 4.5 (results were largely discussed) and the OQ difference score varied around 20 points (RCI = 14 points). Treatment length was almost at (approximately 125 days for the Implementation Group and 175 days for the Control Group). The same is shown for satisfaction with treatment (approximately 8 for both groups).  Table 2 shows results for the regression analyses and corresponding effect sizes. Intercepts (b0) can be interpreted as the estimated starting level, regression coe cients (b1) as mean change (increase or decrease) across time for each year. In both the Implementation and Control Group, all variables except degree of discussion of feedback signi cantly increased across time. The rate at which the questionnaires were completed in the Control Group was an exception. Here, there was a signi cant decrease (t = -4.02, p = .000), but effect sizes were very small (f 2 < .005).
Only for questionnaire completion was there a signi cant difference between the slopes for the Implementation and Control Groups (z = 4.98, p = .000) with a medium effect size (.06). This means that the increase in the rate at which the questionnaires were completed across time was signi cantly higher in the Implementation Group.

Discussion
We used an effectiveness-implementation hybrid design based on Grol and Wensing's [11] implementation framework to examine the effects of clinician-focused implementation strategies on both clinicians' uptake and clients' outcomes of MBC in general mental health care. The results showed only a medium effect of the MBC implementation strategies on questionnaire completion rate (one indicant of clinicians' uptake). During implementation, the rate at which questionnaires were completed increased from 26-37% before they decreased to almost 28% during follow-up. The questionnaire completion rate in the Control Group was stable at about 7%. Effects were found for neither the degree of discussion of feedback nor clients' outcomes (treatment outcome, treatment length, nor satisfaction with treatment).
It is intriguing yet concerning that despite all the strategies we used, there was only an increase in the number of clients who completed the questionnaires. We expected that this would lead to more discussion of the feedback between clinicians and clients, and the bene ts of using the feedback would increase both clients' outcomes and clinicians' integration of MBC into daily practice. However, such a self-enforcing loop [33,34] never emerged because clinicians only stimulated more clients to complete the questionnaires but did not change their own behaviour (discussion of feedback with clients and colleagues).
An initial explanation for why the self-enforcing loop did not occur might be that the clinicians had an insu cient sense of urgency. People with a sense of urgency have a vision of why change is needed, feel that immediate change is needed, and are committed to change (e.g., [35]). This should instil the motivation to abandon current behavioral patterns and adopt new ones.
There was urgency for MBC because many clients (up to 55%) do not progress or even deteriorate during treatment [36] and clinicians are unable to identify clients who are likely to deteriorate (e.g., [37]). Through the combination of this urgency, the explicit desire of the implementation region to improve MBC implementation, and our multifaceted implementation-strategy with an increasing focus on the individual clinician, we anticipated that clinicians would be motivated to improve their use of MBC. Previous research has reported positive outcomes for clients when clinicians were self-selected to participate in MBC [38]. However, we apparently were unable to adequately motivate and move the clinicians to use MBC.
In addition, the clinicians' encouragement of clients to complete the questionnaires seemed externally instead of internally motivated. Beginning in 2011, Dutch mental health care organizations received nancial incentives for submitting MBC data to the mental health benchmark foundation (SBG), and health insurers used the results to establish purchasing policies [39].
Many clinicians had built up resistance against SBG, and consequently did only 'what was required' (achieve a su cient questionnaire completion rate). This probably explained the lack of requests for additional training, poor completion of the case consultation format, and MBC not being su ciently integrated into daily practice to become a routine (e.g., [40]).
A second explanation is that Grol and Wensing's [11] framework did not provide su cient guidance for implementing MBC in general mental health care. The effectiveness of a multifaceted implementation plan is determined by the effectiveness of the separate underlying strategies and their interaction [11]. Perhaps we (a) chose ineffective strategies for mental health care or for MBC, (b) combined too many strategies in too short a period of time, or (c) paid insu cient attention to speci c clinicians' or organization's characteristics (41,42).
Some additional limitations should also be acknowledged. First, we used a quasi-experimental design to closely match the desire of the implementation region. This region was strongly focused on MBC and already had a higher questionnaire completion rate than the other regions, but expressed a desire to further improve implementation of MBC. The clinicians, however, might have been grown indifferent to MBC after years of pressure to collect MBC data. In addition, we were confronted with low response rates for some outcome measures, which might have biased the results. Also, we were faced with a high rate of staff turnover. This is common when complex evidence-based practices are being implemented [42], but it is a hindrance to the implementation and sustainability of the results [43].
Other studies of MBC implementation in mental health care are not completely comparable with our study, because most of them appear to have interchanged MBC implementation with the use of it or have intertwined implementation with clinical research [14][15][16][17][18][19][20]. Although these studies have reported positive effects on use of MBC by clinicians and outcomes for clients, some of them have also experienced problems with implementation [15] and high rates of staff turnover [16]. Additionally, they have reported that client outcomes improved only when clinicians enhanced their use of MBC [17].
Despite the results and limitations of our study, we are unaware of any other studies that have used a hybrid design with multifaceted clinician-focused strategies based on an implementation framework to investigate the effects of MBC implementation strategies on both clinicians' uptake and clients' outcomes. We are also unaware of other studies within general mental health care that have been as large as our study. Our study helps to disentangle the effects of MBC implementation strategies on clinicians' differential uptake. However, because the self-enforcing loop did not occur, we were unable to examine the effects of MBC implementation strategies on clients' outcomes. Future research is needed to establish more guidance for implementing MBC in mental health care and to determine which possibilities can best help to establish the self-enforcing loop (e.g., the use of personal incentives, [44]) and to increase the sense of urgency (e.g., as with motivational interviewing, [45]). Here, the use of mid-range theories speci c to MBC might be helpful [46]. Mid-range theories are limited in scope, less abstract, address speci c phenomena or mechanisms of action, and re ect practice. They consist of relatively concrete, operationally de ned concepts and propositions that can be tested empirically. Examples of mid-range theories are the Theory of Planned Behavior and the Theory of Self-E cacy.
Finally, we recommend that future research make greater use of step-by-step evaluation of each strategy in both process and outcome [47]. Stopping, evaluating, and improving or otherwise changing a speci c strategy when needed, might lead to more information about the suitability and potential effects of each strategy. This could result in measurement-based implementation of measurement-based care.

Conclusions
We have examined the effects of clinician-focused implementation strategies on clinicians' uptake of MBC and the resulting outcomes for clients in general mental health care in the Netherlands. We found a signi cant medium effect for questionnaire completion rate, but not for the degree to which the feedback was discussed and consequently not for clients' outcomes. The results appear to have been limited due to clinicians' lack of a sense of urgency or insu cient guidance of the underlying framework. We are, however, unaware of any other studies that have investigated the direct effects of MBC implementation strategies on clinicians' uptake as well as clients' outcomes or other studies of this size within general mental health care. The current study helps to disentangle the effects of MBC implementation strategies on differential uptake by clinicians. Our main suggestion for organizations that implement MBC and researchers is to systematically use speci ed mid-range theories as a basis for step-by-step implementation and evaluation.

Declarations
Ethics approval and consent to participate The study was designed as a quality improvement project for the implementation of MBC and the care that clinicians provided to their clients. Because there was no infringement on the clinicians' physical or psychological integrity and the clients' data are anonymous, the study was not subsumed under the Medical Research Involving Human Subjects Act (WMO) and did not require ethical approval.

Consent for publication
Not applicable.

Availability of data and materials
The datasets that were generated and analyzed during the current study are available from the corresponding author upon request.

Competing interests
The authors declare that they have no con icts of interests. Discussion of feedback1 1In 2015, the question about discussion of the feedback was only lled out for 10 treatments (7 treatments within the Implementation group and 3 treatments in the Control group). Therefore, the results of 2015 and the rst half of 2016 were merged for the analysis.