Skip to main content

Advertisement

SpringerLink
Log in

Multilevel Mechanisms of Implementation Strategies in Mental Health: Integrating Theory, Research, and Practice

  • Original Article
  • Published:
Administration and Policy in Mental Health and Mental Health Services Research Aims and scope Submit manuscript

Abstract

A step toward the development of optimally effective, efficient, and feasible implementation strategies that increase evidence-based treatment integration in mental health services involves identification of the multilevel mechanisms through which these strategies influence implementation outcomes. This article (a) provides an orientation to, and rationale for, consideration of multilevel mediating mechanisms in implementation trials, and (b) systematically reviews randomized controlled trials that examined mediators of implementation strategies in mental health. Nine trials were located. Mediation-related methodological deficiencies were prevalent and no trials supported a hypothesized mediator. The most common reason was failure to engage the mediation target. Discussion focuses on directions to accelerate implementation strategy development in mental health.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Aarons, G. A., Ehrhart, M. G., Farahnak, L. R., & Sklar, M. (2014). Aligning leadership across systems and organizations to develop a strategic climate for evidence-based practice implementation. Annual Review of Public Health, 35, 255–274. doi:10.1146/annurev-publhealth-032013-182447.

    Article  PubMed  PubMed Central  Google Scholar 

  • Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011). Advancing a conceptual model of evidence-based practice implementation in public sectors. Administration and Policy in Mental Health, 38, 4–23. doi:10.1007/s10488-010-0327-7.

    Article  PubMed  Google Scholar 

  • Atkins, M. S., Frazier, S. L., Leathers, S. J., Graczyk, P. A., Talbott, E., Jakobsons, L., et al. (2008). Teacher key opinion leaders and mental health consultation in low-income urban schools. Journal of Consulting and Clinical Psychology, 76, 905–908. doi:10.1037/a0013036.

    Article  PubMed  Google Scholar 

  • Baer, J. S., Wells, E. A., Rosengren, D. B., Hartzler, B., Beadnell, B., & Dunn, C. (2009). Agency context and tailored training in technology transfer: A pilot evaluation of motivational interviewing training for community counselors. Journal of Substance Abuse Treatment, 37, 191–202. doi:10.1016/j.jsat.2009.01.003.

    Article  PubMed  PubMed Central  Google Scholar 

  • Baron, R. M., & Kenny, D. A. (1986). The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51, 1173–1182. doi:10.1037/0022-3514.51.6.1173.

    Article  CAS  PubMed  Google Scholar 

  • Barwick, M. A., Schachter, H. M., Bennett, L. M., McGowan, J., Ly, M., Wilson, A., et al. (2012). Knowledge translation efforts in child and youth mental health: A systematic review. Journal of Evidence Based Social Work, 9, 369–395. doi:10.1080/15433714.2012.663667.

    Article  PubMed  PubMed Central  Google Scholar 

  • Bauer, D. J., Preacher, K. J., & Gil, K. M. (2006). Conceptualizing and testing random indirect effects and moderated mediation in multilevel models: New procedures and recommendations. Psychological Methods, 11, 142–163. doi:10.1037/1082-989x.11.2.142.supp.

    Article  PubMed  Google Scholar 

  • Berwick, D. M. (2008). The science of improvement. Journal of the American Medical Association, 299, 1182–1184. doi:10.1001/jama.299.10.1182.

    Article  CAS  PubMed  Google Scholar 

  • Biesanz, J. C., Falk, C. F., & Savalei, V. (2010). Assessing mediational models: Testing and interval estimation for indirect effects. Multivariate Behavioral Research, 45, 661–701. doi:10.1080/00273171.2010.498292.

    Article  PubMed  Google Scholar 

  • Chan, D. (1998). Functional relations among constructs in the same content domain at different levels of analysis: A typology of composition models. Journal of Applied Psychology, 83, 234–246. doi:10.1037/0021-9010.83.2.234.

    Article  Google Scholar 

  • Chaudoir, S. R., Dugan, A. G., & Barr, C. H. (2013). Measuring factors affecting implementation of health innovations: a systematic review of structural, organizational, provider, patient, and innovation level measures. Implementation Science, 8, 22. doi:10.1186/1748-5908-8-22.

    Article  PubMed  PubMed Central  Google Scholar 

  • Chen, H. (1990). Theory-driven evaluations. Newbury Park, CA: Sage.

    Google Scholar 

  • Collins, P. Y., Patel, V., Joestl, S. S., March, D., Insel, T. R., & Daar, A. S. (2011). Grand challenges in global mental health. Nature, 475, 27–30. doi:10.1038/475027a.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4, 50. doi:10.1186/1748-5908-4-50.

    Article  PubMed  PubMed Central  Google Scholar 

  • Davies, P., Walker, A. E., & Grimshaw, J. M. (2010). A systematic review of the use of theory in the design of guideline dissemination and implementation strategies and interpretation of the results of rigorous evaluations. Implementation Science, 5(14), 5908-5. doi:10.1186/1748-5908-5-14.

    Google Scholar 

  • Doss, B. D. (2004). Changing the way we study change in psychotherapy. Clinical Psychology: Science and Practice, 11, 368–386. doi:10.1186/1748-5908-5-14.

    Google Scholar 

  • Edwards, J. R., & Lambert, L. S. (2007). Methods for integrating moderation and mediation: A general analytical framework using moderated path analysis. Psychological Methods, 12, 1–22. doi:10.1037/1082-989x.12.1.1.supp.

    Article  PubMed  Google Scholar 

  • Enders, C. (2013). Centering predictors and contextual effects. In M. A. Scott, J. S. Simonoff, & B. D. Marx (Eds.), The SAGE handbook of multilevel modeling (pp. 89–108). Thousand Oaks, CA: Sage Publications Inc.

    Chapter  Google Scholar 

  • Garland, A. F., Haine-Schlagel, R., Brookman-Frazee, L., Baker-Ericzen, M., Trask, E., & Fawley-King, K. (2013). Improving community-based mental health care for children: Translating knowledge into action. Administration and Policy in Mental Health, 40, 6–22. doi:10.1371/journal.pmed.1001434.

    Article  PubMed  PubMed Central  Google Scholar 

  • Garner, B. R., Godley, S. H., & Bair, C. M. L. (2011). The impact of pay-for-performance on therapists’ intentions to deliver high quality treatment. Journal of Substance Abuse Treatment, 41, 97–103. doi:10.1016/j.jsat.2011.01.012.

    Article  PubMed  PubMed Central  Google Scholar 

  • Glisson, C., Schoenwald, S. K., Hemmelgarn, A., Green, P., Dukes, D., Armstrong, K. S., & Chapman, J. E. (2010). Randomized trial of MST and ARC in a two-level evidence-based treatment implementation strategy. Journal of Consulting and Clinical Psychology, 78, 537–550. doi:10.1037/a0019160.

    Article  PubMed  PubMed Central  Google Scholar 

  • Glisson, C., & Williams, N. J. (2015). Assessing and changing organizational social contexts for effective mental health services. Annual Review of Public Health, 36, 507–523. doi:10.1146/annurev-publhealth-031914-122435.

    Article  PubMed  Google Scholar 

  • Godin, G., Belanger-Gravel, A., Eccles, M., & Grimshaw, J. (2008). Healthcare professionals’ intentions and behaviors: A systematic review of studies based on social cognitive theories. Implementation Science, 3(36), 1–12.

    Google Scholar 

  • Greenhalgh, T., Robert, G., MacFarlane, F., Bate, P., & Kyriakidou, O. (2004). Diffusion of innovations in service organizations: A systematic review and recommendations. Milbank Quarterly, 82, 581–629. doi:10.1111/j.0887-378x.2004.00325.x.

    Article  PubMed  PubMed Central  Google Scholar 

  • Grol, R. P., Bosch, M. C., Hulscher, M. E., Eccles, M. P., & Wensing, M. (2007). Planning and studying improvement in patient care: The use of theoretical perspectives. Milbank Quarterly, 85, 93–138. doi:10.1111/j.1468-0009.2007.00478.x.

    Article  PubMed  PubMed Central  Google Scholar 

  • Hayes, A. F., & Scharkow, M. (2013). The relative trustworthiness of inferential tests of the indirect effect in statistical mediation analysis: Does method really matter? Psychological Science, 24, 1918–1927. doi:10.1177/0956797613480187.

    Article  PubMed  Google Scholar 

  • Holth, P., Torsheim, T., Sheidow, A. J., Ogden, T., & Henggeler, S. W. (2011). Intensive quality assurance of therapist adherence to behavioral interventions for adolescent substance use problems. Journal of Child and Adolescent Substance Abuse, 20, 289–313. doi:10.1080/1067828x.2011.581974.

    Article  PubMed  PubMed Central  Google Scholar 

  • Insel, T. R. (2009). Translating scientific opportunity into public health impact: A strategic plan for research on mental illness. Archives of General Psychiatry, 66, 128–133. doi:10.1001/archgenpsychiatry.2008.540.

    Article  PubMed  Google Scholar 

  • Institute of Medicine. (2001). Crossing the quality chasm: A new health system for the 21st century. Washington, DC: Author.

    Google Scholar 

  • Judd, C. M., & Kenny, D. A. (1981). Process analysis: Estimating mediation in treatment evaluations. Evaluation Review, 5, 602–619. doi:10.1080/1067828x.2011.581974.

    Article  Google Scholar 

  • Kauth, M. R., Sullivan, G., Blevins, D., Cully, J. A., Landes, R. D., Said, Q., & Teasdale, T. A. (2010). Employing external facilitation to implement cognitive behavioral therapy in VA clinics: A pilot study. Implementation Science, 5, 75. doi:10.1186/1748-5908-5-75.

    Article  PubMed  PubMed Central  Google Scholar 

  • Kazdin, A. E. (2007). Mediators and mechanisms of change in psychotherapy research. Annual Review of Clinical Psychology, 3, 1–27. doi:10.1146/annurev.clinpsy.3.022806.091432.

    Article  PubMed  Google Scholar 

  • Kenny, D. A., Korchmaros, J. D., & Bolger, N. (2003). Lower level mediation in multilevel models. Psychological Methods, 8, 115–128. doi:10.1037/1082-989x.8.2.115.

    Article  PubMed  Google Scholar 

  • Kessler, R. C., Aguilar-Gaxiola, S., Alonso, J., Chatterji, S., Lee, S., Ormel, J., et al. (2009). The global burden of mental disorders: An update from the WHO World Mental Health (WMH) Surveys. Epidemiologia e Psichiatria Sociale, 18(1), 23–33. doi:10.1017/s1121189x00001421.

    Article  PubMed  PubMed Central  Google Scholar 

  • Klein, K. J., Dansereau, F., & Hall, R. J. (1994). Levels issues in theory development, data collection, and analysis. Academy of Management Review, 19, 195–229. doi:10.5465/AMR.1994.9410210745.

    Google Scholar 

  • Kozlowski, S. W. J., & Klein, K. J. (2000). A multilevel approach to theory and research in organizations: Contextual, temporal, and emergent properties. In K. J. Klein & S. W. J. Kozlowski (Eds.), Multilevel theory, research, and methods in organizations: Foundations, extensions, and new directions (pp. 3–90). San Francisco: Jossey-Bass.

    Google Scholar 

  • Kraemer, H. C., Kazdin, A. E., Offord, D. R., Kessler, R. C., Jensen, P. S., & Kupfer, D. J. (1997). Coming to terms with the terms of risk. Archives of General Psychiatry, 54, 337–343. doi:10.1001/archpsyc.1997.01830160065009.

    Article  CAS  PubMed  Google Scholar 

  • Kreft, I. G. G., de Leeuw, J., & Aiken, L. S. (1995). The effects of different forms of centering in hierarchical linear models. Multivariate Behavioral Research, 30, 1–21. doi:10.1207/s15327906mbr3001_1.

    Article  CAS  PubMed  Google Scholar 

  • Krull, J. L., & MacKinnon, D. P. (2001). Multilevel modeling of individual and group level mediated effects. Multivariate Behavioral Research, 36, 249–277. doi:10.1207/S15327906MBR3602_06.

    Article  CAS  PubMed  Google Scholar 

  • Landsverk, J., Brown, C. H., Reutz, J. R., Palinkas, L., & Horwitz, S. M. (2011). Design elements in implementation research: A structured review of child welfare and mental health studies. Administration and Policy in Mental Health, 38, 54–63. doi:10.1007/s10488-010-0315-y.

    Article  PubMed  PubMed Central  Google Scholar 

  • LeBreton, J. M., & Senter, J. L. (2008). Answers to 20 questions about interrater reliability and interrater agreement. Organizational Research Methods, 11, 815–852. doi:10.1177/1094428106296642.

    Article  Google Scholar 

  • Lochman, J. E., Boxmeyer, C., Powell, N., Qu, L., Wells, K., & Windle, M. (2009). Dissemination of the coping power program: Importance of intensity of counselor training. Journal of Consulting and Clinical Psychology, 77, 397–409. doi:10.1037/a0014514.

    Article  PubMed  Google Scholar 

  • MacKinnon, D. P. (2008). Introduction to statistical mediation analysis. New York: Lawrence Erlbaum Associates.

    Google Scholar 

  • MacKinnon, D. P., Fairchild, A. J., & Fritz, M. S. (2007). Mediation analysis. Annual Review of Psychology, 58, 593–614. doi:10.1146/annurev.psych.58.110405.085542.

    Article  PubMed  PubMed Central  Google Scholar 

  • MacKinnon, D. P., Lockwood, C. M., Hoffman, J. M., West, S. G., & Sheets, V. (2002). A comparison of methods to test mediation and other intervening variable effects. Psychological Methods, 7, 83–104. doi:10.1037/1082-989X.7.1.83.

    Article  PubMed  PubMed Central  Google Scholar 

  • MacKinnon, D. P., Lockwood, C. M., & Williams, J. (2004). Confidence limits for the indirect effect: Distribution of the product and resampling methods. Multivariate Behavioral Research, 39, 99–128. doi:10.1207/s15327906mbr3901_4.

    Article  PubMed  PubMed Central  Google Scholar 

  • Mathieu, J. E., DeShon, R. P., & Bergh, D. D. (2008). Mediational inferences in organizational research: Then, now, and beyond. Organizational Research Methods, 11, 203–223. doi:10.1177/1094428107310089.

    Article  Google Scholar 

  • Mathieu, J. E., & Taylor, S. R. (2007). A framework for testing meso-mediational relationships in organizational behavior. Journal of Organizational Behavior, 28, 141–172. doi:10.1002/job.436.

    Article  Google Scholar 

  • McHugh, R. K., & Barlow, D. H. (2012). The reach of evidence-based psychological interventions. In R. K. McHugh & D. H. Barlow (Eds.), Dissemination and implementation of evidence-based psychological interventions (pp. 3–15). New York: Oxford University Press.

    Google Scholar 

  • Michie, S., van Stralen, M. M., & West, R. (2011). The behavior change wheel: A new method for characterizing and designing behavior change interventions. Implementation Science, 6, 42. doi:10.1186/1748-5908-6-42.

    Article  PubMed  PubMed Central  Google Scholar 

  • Neuhaus, J. M., & Kalbfleisch, J. D. (1998). Between- and within-cluster covariate effects in the analysis of clustered data. Biometrics, 54, 638–645. doi:10.2307/3109770.

    Article  CAS  PubMed  Google Scholar 

  • Novins, D. K., Green, A. E., Legha, R. K., & Aarons, G. A. (2013). Dissemination and implementation of evidence-based practices for child and adolescent mental health: A systematic review. Journal of the American Academy of Child and Adolescent Psychiatry, 52, 1009–1025. doi:10.1016/j.jaac.2013.07.012.

    Article  PubMed  PubMed Central  Google Scholar 

  • Pituch, K. A., Murphy, D. L., & Tate, R. L. (2010). Three-level models for indirect effects in school- and class-randomized experiments in education. Journal of Experimental Education, 78, 60–95. doi:10.1080/00220970903224685.

    Article  Google Scholar 

  • Powell, B. J., Proctor, E. K., & Glass, J. E. (2014). A systematic review of strategies for implementing empirically supported mental health interventions. Research on Social Work Practice, 24, 192–212. doi:10.1177/1049731513505778.

    Article  PubMed  Google Scholar 

  • Preacher, K. J., & Hayes, A. F. (2004). Asymptotic and resampling strategies for assessing and comparing indirect effects in multiple mediator models. Behavior Research Methods, 40, 879–891. doi:10.3758/brm.40.3.879.

    Article  Google Scholar 

  • Preacher, K. J., Rucker, D. D., & Hayes, A. F. (2007). Addressing moderated mediation hypotheses: Theory, methods, and prescriptions. Multivariate Behavioral Research, 42, 185–227. doi:10.1080/00273170701341316.

    Article  PubMed  Google Scholar 

  • Preacher, K. J., & Selig, J. P. (2012). Advantages of Monte Carlo confidence intervals for indirect effects. Communication Methods and Measures, 6, 77–98. doi:10.1080/19312458.2012.679848.

    Article  Google Scholar 

  • Preacher, K. J., Zyphur, M. J., & Zhang, Z. (2010). A general multilevel SEM framework for assessing multilevel mediation. Psychological Methods, 15, 209–233. doi:10.1037/a0020141.

    Article  PubMed  Google Scholar 

  • Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., et al. (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health, 38, 65–76. doi:10.1007/s10488-010-0319-7.

    Article  PubMed  Google Scholar 

  • Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis (2nd ed.). Thousand Oaks, CA: Sage publications.

    Google Scholar 

  • Rohrbach, L. A., Graham, J. W., & Hansen, W. B. (1993). Diffusion of a school-based substance abuse prevention program: Predictors of program implementation. Preventive Medicine, 22, 237–260. doi:10.1006/pmed.1993.1020.

    Article  CAS  PubMed  Google Scholar 

  • Rousseau, D. M. (1985). Issues of level in organizational research: Multi-level and cross-level perspectives. In L. L. Cummings & B. M. Straw (Eds.), Research in organizational behavior (Vol. 7, pp. 1–37). Greenwich, CT: JAI Press.

    Google Scholar 

  • Saloner, B., Carson, N., & Lê Cook, B. (2014). Episodes of mental health treatment among a nationally representative sample of children and adolescents. Medical Care Research and Review, 71, 261–279. doi:10.1177/1077558713518347.

    Article  PubMed  Google Scholar 

  • Sobel, M. E. (1982). Asymptotic confidence intervals for indirect effects in structural equation models. Sociological Methodology, 13, 290–312. doi:10.2307/270723.

    Article  Google Scholar 

  • Steel, Z., Marnane, C., Iranpour, C., Chey, T., Jackson, J. W., Patel, V., & Silove, D. (2014). The global prevalence of common mental disorders: A systematic review and meta-analysis 1980–2013. International Journal of Epidemiology, 43, 476–493. doi:10.1093/ije/dyu038.

    Article  PubMed  PubMed Central  Google Scholar 

  • Tabak, R. G., Khoong, E. C., Chambers, D. A., & Brownson, R. C. (2012). Bridging research and practice: Models for dissemination and implementation research. American Journal of Preventive Medicine, 43, 337–350. doi:10.1016/j.amepre.2012.05.024.

    Article  PubMed  PubMed Central  Google Scholar 

  • Taylor, A. B., MacKinnon, D. P., & Tein, J. Y. (2008). Tests of the three-path mediated effect. Organizational Research Methods, 11, 241–269. doi:10.1177/1094428107300344.

    Article  Google Scholar 

  • Weisz, J. R., Kuppens, S., Eckshtain, D., Ugueto, A. M., Hawley, K. M., & Jensen-Doss, A. (2013). Performance of evidence-based youth psychotherapies compared with usual clinical care: A multilevel meta-analysis. JAMA Psychiatry, 70, 750–761. doi:10.1001/jamapsychiatry.2013.1176.

    Article  CAS  PubMed  Google Scholar 

  • Williams, N. J. (2015). Mechanisms of change in an organizational culture and climate intervention for increasing clinicians’ evidence-based practice adoption in mental health (Unpublished doctoral dissertation). University of Tennessee, Knoxville. http://trace.tennessee.edu/utk_graddiss/3482.

  • Williams, N. J., & Glisson, C. (2014). The role of organizational culture and climate in the dissemination and implementation of empirically-supported treatments for youth. In R. S. Beidas & P. C. Kendall (Eds.), Dissemination and implementation of evidence based practices in child and adolescent mental health (pp. 61–81). New York: Oxford University Press.

    Google Scholar 

  • Williams, J. R., Williams, W. O., Dusablon, T., Blais, M. P., Tregear, S. J., Banks, D., & Hennessy, K. D. (2014). Evaluation of a randomized intervention to increase adoption of comparative effectiveness research by community health organizations. Journal of Behavioral Health Services and Research, 41, 308–323. doi:10.1007/s11414-013-9369-4.

    Article  PubMed  Google Scholar 

  • Zhang, Z., Zyphur, M. J., & Preacher, K. J. (2009). Testing multilevel mediation using hierarchical linear models: Problems and solutions. Organizational Research Methods, 12, 695–719. doi:10.1177/1094428108327450.

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported by funding from the National Institute of Mental Health under Award Number F31MH099846 to Nathaniel J. Williams. The content is solely the responsibility of the author and does not necessarily represent the official views of the National Institutes of Health.

Author information

Authors and Affiliations

  1. School of Social Work, Boise State University, 1910 University Dr., Boise, ID, 83725-1940, USA

    Nathaniel J. Williams

Authors
  1. Nathaniel J. Williams
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nathaniel J. Williams.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Williams, N.J. Multilevel Mechanisms of Implementation Strategies in Mental Health: Integrating Theory, Research, and Practice. Adm Policy Ment Health 43, 783–798 (2016). https://doi.org/10.1007/s10488-015-0693-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10488-015-0693-2

Keywords

Search

Navigation