Skip to main content

Advertisement

SpringerLink
Log in

Virtual Learning Collaborative Compared to Technical Assistance as a Strategy for Implementing Health Promotion in Routine Mental Health Settings: A Hybrid Type 3 Cluster Randomized Trial

  • Original Article
  • Published:
Administration and Policy in Mental Health and Mental Health Services Research Aims and scope Submit manuscript

Abstract

Background

Despite widespread use of learning collaboratives, few randomized trials have evaluated their effectiveness as a strategy for implementing evidence based practices. This randomized trial evaluated the effectiveness of a virtual learning collaborative (VLC) in the implementation of a health promotion program for persons with serious mental illness (SMI) aimed at reducing cardiovascular risk reduction in routine mental health settings, compared to routine technical assistance (TA).

Methods

Fifty-five mental health provider organizations were recruited to participate in a Hybrid Type 3 cluster randomized implementation-effectiveness trial of the InSHAPE health promotion program for persons with SMI. Sites were stratified by size and randomized prior to implementation to an 18-month group-based VLC with monthly learning sessions or individual site TA with four scheduled conference calls over 18 months. Primary implementation and service outcomes were InSHAPE program fidelity, participation, and reach. Primary clinical outcomes were weight loss, cardiorespiratory fitness, and cardiovascular risk reduction (≥ 5% weight loss or > 50 m increase on the 6-Minute Walk Test). Program fidelity was assessed at 6, 12, and 24 months; program participation and participant-level outcomes were assessed at 3, 6, 9, and 12 months.

Results

VLC (N = 27) and TA (N = 28) sites were similar in organizational characteristics (all p > 0.05). At 12-month follow-up mean program fidelity score was higher in VLC compared to TA (90.5 vs. 79.1; p = 0.002), with over double the proportion with good fidelity (VLC = 73.9% vs. TA = 34.8%; p = 0.009). Over half of individuals in both VLC and TA achieved cardiovascular risk-reduction at 6-month follow-up (VLC: 51.0%; TA: 53.5%; p = 0.517) and at 12-month follow-up (62% VLC and TA; p = 0.912). At 12-month follow-up VLC compared to TA was associated with greater participation (VLC 69.5% vs. TA 56.4% attending at least 50% of sessions, p = 0.002); larger caseloads (VLC = 16 vs. TA = 11; p = 0.024); greater reach consisting of 45% greater number of participants receiving InSHAPE (VLC = 368 vs. TA = 253), and 58% greater number of participants achieving cardiovascular risk reduction (VLC = 150 vs. TA = 95).

Conclusion

Virtual learning collaboratives compared to routine technical assistance as an implementation strategy for evidence-based health promotion promote greater intervention fidelity, greater levels of intervention participation, greater reach, and a greater number of participants achieving clinically significant risk reduction outcomes, while achieving similarly high levels of intervention effectiveness for participants who completed at least 6 months of the program.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Data Availability

We do not wish to share the study data until our research team has completed data analysis related to each of the study aims outlined in the research plan. Questions about the data sources for this study can be directed to the corresponding author.

References

  1. National Institute of Mental Health. The Numbers Count: Mental Disorders in America 2013 [Available from: http://www.nimh.nih.gov/health/publications/the-numbers-count-mental-disorders-in-america/index.shtml

  2. Kessler, R. C., Chiu, W. T., Demler, O., & Walters, E. E. (2005). Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. Archives of General Psychiatry, 62(6), 617–627

    Article  Google Scholar 

  3. Walker, E. R., McGee, R. E., & Druss, B. G. (2015). Mortality in mental disorders and global disease burden implications: a systematic review and meta-analysis. JAMA psychiatry, 72(4), 334–341

    Article  Google Scholar 

  4. Bartels, S. J., Pratt, S. I., Aschbrenner, K. A., Barre, L. K., Jue, K., Wolfe, R. S., et al. (2013). Clinically significant improved fitness and weight loss among overweight persons with serious mental illness. Psychiatric services (Washington DC), 64(8), 729–736

    Article  Google Scholar 

  5. Bartels, S. J., Pratt, S. I., Aschbrenner, K. A., Barre, L. K., Naslund, J. A., Wolfe, R., et al. (2015). Pragmatic replication trial of health promotion coaching for obesity in serious mental illness and maintenance of outcomes. The American journal of psychiatry, 172(4), 344–352

    Article  Google Scholar 

  6. Daumit, G. L., Dickerson, F. B., Wang, N. Y., Dalcin, A., Jerome, G. J., Anderson, C. A., et al. (2013). A behavioral weight-loss intervention in persons with serious mental illness. New England Journal Of Medicine, 368(17), 1594–1602

    Article  CAS  Google Scholar 

  7. Green, C. A., Yarborough, B. J., Leo, M. C., Yarborough, M. T., Stumbo, S. P., Janoff, S. L., et al. (2015). The STRIDE weight loss and lifestyle intervention for individuals taking antipsychotic medications: a randomized trial. The American journal of psychiatry, 172(1), 71–81

    Article  Google Scholar 

  8. Bartels, S. J., Aschbrenner, K. A., Pratt, S. I., Naslund, J. A., Scherer, E. A., Zubkoff, L., et al. (2018). Implementation of a Lifestyle Intervention for People With Serious Mental Illness in State-Funded Mental Health Centers. Psychiatric services (Washington, DC). :appips201700368

  9. The Breakthrough Series. (2004). IHI’s Collaborative Model for Achieving Breakthrough Improvement. Diabetes Spectrum, 17(2), 97

    Article  Google Scholar 

  10. Ayers, L. R., Beyea, S. C., Godfrey, M. M., Harper, D. C., Nelson, E. C., & Batalden, P. B. (2005). Quality improvement learning collaboratives. Quality Management in Healthcare, 14(4), 234–247

    Article  Google Scholar 

  11. Nadeem, E., Olin, S. S., Hill, L. C., Hoagwood, K. E., & Horwitz, S. M. (2014). A literature review of learning collaboratives in mental health care: used but untested. Psychiatric services (Washington DC), 65(9), 1088–1099

    Article  Google Scholar 

  12. Schouten, L. M. T., Hulscher, M. E. J. L., Everdingen, J. J. E., Huijsman, R., & Grol, R. P. T. M. (2008). Evidence for the impact of quality improvement collaboratives: systematic review. BMJ (Clinical research ed), 336(7659), 1491–1494

    Article  Google Scholar 

  13. Zubkoff, L., Neily, J., & Mills, P. D. (2019). How to do a Virtual Breakthrough Series Collaborative. Journal of medical systems, 43(2), 27

    Article  Google Scholar 

  14. Van Citters, A. D., Pratt, S. I., Jue, K., Williams, G. E., Miller, P. T., Xie, H., et al. (2010). A pilot evaluation of the In SHAPE individualized health promotion intervention for adults with mental illness. Community Ment Health J, 46(6), 540–552

    Article  Google Scholar 

  15. Curran, G. M., Bauer, M., Mittman, B., Pyne, J. M., & Stetler, C. (2012). Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Medical Care, 50(3), 217–226

    Article  Google Scholar 

  16. Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., et al. (2011). Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Administration and policy in mental health, 38(2), 65–76

    Article  Google Scholar 

  17. Aschbrenner, K. A., Pratt, S. I., Bond, G. R., Zubkoff, L., Naslund, J. A., Jue, K., et al. (2019). A virtual learning collaborative to implement health promotion in routine mental health settings: Protocol for a cluster randomized trial. Contemporary clinical trials, 84, 105816

    Article  Google Scholar 

  18. Regan, J., Lau, A. S., Barnett, M., Stadnick, N., Hamilton, A., Pesanti, K., et al. (2017). Agency responses to a system-driven implementation of multiple evidence-based practices in children’s mental health services. Bmc Health Services Research, 17(1), 671

    Article  Google Scholar 

  19. Bond, G. R., & Drake, R. E. (2020). Assessing the Fidelity of Evidence-Based Practices: History and Current Status of a Standardized Measurement Methodology. Administration and Policy in Mental Health and Mental Health Services Research, 47(6), 874–884

    Article  Google Scholar 

  20. Balke, B. (1963). A simple field test for the assessment of physical fitness. Report Civil Aeromedical Research Institute (US), 53, 1–8

    Google Scholar 

  21. Larsson, U. E., & Reynisdottir, S. (2008). The six-minute walk test in outpatients with obesity: reproducibility and known group validity. Physiotherapy Research International, 13(2), 84–93

    Article  Google Scholar 

  22. Beriault, K., Carpentier, A. C., Gagnon, C., Menard, J., Baillargeon, J. P., Ardilouze, J. L., et al. (2009). Reproducibility of the 6-minute walk test in obese adults. International Journal of Sports Medicine, 30(10), 725–727

    Article  CAS  Google Scholar 

  23. Bittner, V., Weiner, D. H., Yusuf, S., Rogers, W. J., McIntyre, K. M., Bangdiwala, S. I., et al. (1993). Prediction of mortality and morbidity with a 6-minute walk test in patients with left ventricular dysfunction. Journal Of The American Medical Association, 270(14), 1702–1707

    Article  CAS  Google Scholar 

  24. Cahalin, L. P., Mathier, M. A., Semigran, M. J., Dec, G. W., & DiSalvo, T. G. (1996). The six-minute walk test predicts peak oxygen uptake and survival in patients with advanced heart failure. Chest, 110(2), 325–332

    Article  CAS  Google Scholar 

  25. Chetta, A., Aiello, M., Foresi, A., Marangio, E., D’Ippolito, R., Castagnaro, A., et al. (2001). Relationship between outcome measures of six-minute walk test and baseline lung function in patients with interstitial lung disease. Sarcoidosis Vasculitis and Diffuse Lung Diseases, 18(2), 170–175

    CAS  Google Scholar 

  26. Curtis, J. P., Rathore, S. S., Wang, Y., & Krumholz, H. M. (2004). The association of 6-minute walk performance and outcomes in stable outpatients with heart failure. Journal of Cardiac Failure, 10(1), 9–14

    Article  Google Scholar 

  27. Demers, C., McKelvie, R. S., Negassa, A., & Yusuf, S. (2001). Reliability, validity, and responsiveness of the six-minute walk test in patients with heart failure. American Heart Journal, 142(4), 698–703

    Article  CAS  Google Scholar 

  28. Kosak, M., & Smith, T. (2005). Comparison of the 2-, 6-, and 12-minute walk tests in patients with stroke. Journal of Rehabilitation Research and Development, 42(1), 103–107

    Article  Google Scholar 

  29. Miyamoto, S., Nagaya, N., Satoh, T., Kyotani, S., Sakamaki, F., Fujita, M., et al. (2000). Clinical correlates and prognostic significance of six-minute walk test in patients with primary pulmonary hypertension. Comparison with cardiopulmonary exercise testing. American Journal of Respiratory and Critical Care Medicine, 161(2 Pt 1), 487–492

    Article  CAS  Google Scholar 

  30. Pankoff, B. A., Overend, T. J., Lucy, S. D., & White, K. P. (2000). Reliability of the six-minute walk test in people with fibromyalgia. Arthritis Care and Research, 13(5), 291–295

    Article  CAS  Google Scholar 

  31. Redelmeier, D. A., Bayoumi, A. M., Goldstein, R. S., & Guyatt, G. H. (1997). Interpreting small differences in functional status: the Six Minute Walk test in chronic lung disease patients. American Journal of Respiratory and Critical Care Medicine, 155(4), 1278–1282

    Article  CAS  Google Scholar 

  32. Sciurba, F., Criner, G. J., Lee, S. M., Mohsenifar, Z., Shade, D., Slivka, W., et al. (2003). Six-minute walk distance in chronic obstructive pulmonary disease: reproducibility and effect of walking course layout and length. American Journal of Respiratory and Critical Care Medicine, 167(11), 1522–1527

    Article  Google Scholar 

  33. Rasekaba, T., Lee, A. L., Naughton, M. T., Williams, T. J., & Holland, A. E. (2009). The six-minute walk test: a useful metric for the cardiopulmonary patient. Internal Medicine Journal, 39(8), 495–501

    Article  CAS  Google Scholar 

  34. Wise, R. A., & Brown, C. D. (2005). Minimal clinically important differences in the six-minute walk test and the incremental shuttle walking test. COPD: Journal of Chronic Obstructive Pulmonary Disease, 2(1), 125–129

    Article  Google Scholar 

  35. Jennrich, R. I., & Schluchter, M. D. (1986). Unbalanced Repeated-Measures Models with Structured Covariance Matrices. Biometrics, 42(4), 805–820

    Article  CAS  Google Scholar 

  36. StataCorp (2017). Stata Statistical Software: Release 15. College Station: StataCorp LLC

    Google Scholar 

Download references

Acknowledgements

We gratefully acknowledge study participants who took their valuable time to participate in this study as well as Ms. Allison Kinney for her assistance with study coordination.

Funding

This study was funded by a grant from the National Institute of Mental Health (R01MH102325). The funder had no role in the design, collection, analysis, and interpretation of data; in the writing of the manuscript; and in the decision to submit this manuscript for publication.

Author information

Authors and Affiliations

  1. Massachusetts General Hospital, The Mongan Institute, Harvard Medical School, Boston, MA, USA

    Stephen J. Bartels

  2. Department of Psychiatry, Geisel School of Medicine at Dartmouth, Lebanon, NH, USA

    Kelly A. Aschbrenner, Sarah I. Pratt, Gail Williams & Rosemarie Wolfe

  3. Division of Preventive Medicine, University of Alabama at Birmingham, Birmingham, AL, USA

    Lisa Zubkoff

  4. Birmingham/Atlanta Geriatric Research Education and Clinical Center, Department of Veterans Affairs, Birmingham, AL, USA

    Lisa Zubkoff

  5. Ken Jue Consulting, Keene, NH, USA

    Kenneth Jue

  6. College of Health and Huamn Services, University of New Hampshire, Durham, NH, USA

    Marjorie M. Godfrey

  7. MJC Health Solutions, Amherst, NH, USA

    Michael J. Cohen

  8. Department of Humanities and Social Sciences, Indian Institute of Technology Bombay, Mumbai, Maharashtra, India

    Souvik Banerjee

  9. Department of Biomedical Sciences, Dartmouth College, Hanover, NH, USA

    Haiyi Xie

  10. Department of Global Health and Social Medicine, Harvard Medical School, Boston, MA, USA

    John A. Naslund

  11. Westat, Lebanon, NH, USA

    Gary R. Bond

Authors
  1. Stephen J. Bartels
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kelly A. Aschbrenner
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sarah I. Pratt
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lisa Zubkoff
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kenneth Jue
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Gail Williams
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Marjorie M. Godfrey
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Michael J. Cohen
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Souvik Banerjee
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Haiyi Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Rosemarie Wolfe
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. John A. Naslund
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Gary R. Bond
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

SJB led the conception, design, and execution of the study and was closely involved in data analysis and interpretation, and wrote the manuscript. KA directed the study, was closely involved in data analysis and interpretation, and revised the manuscript critically for important intellectual content. SB conducted the data analysis and prepared data tables reviewed by HX. RW managed the data and reviewed data tables. GB was closely involved in data analysis and interpretation and revised the manuscript critically for important intellectual content. SP, KJ, GW, MG, LZ, MC, and JN were involved in the design and execution of the study, participated in data interpretation, and revised the manuscript critically for important intellectual content. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Kelly A. Aschbrenner.

Ethics declarations

Ethics approval and consent to participate

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. The Committee for the Protection of Human Subjects at Dartmouth College approved all study procedures (approval no. 00028067). A waiver of informed consent was approved to collect de-identified participant level clinical data at the study sites. Verbal informed consent was obtained from all mental health organization staff who participated in the research interviews.

Competing interests

The authors declare that they have no competing interests. CONSORT guidelines for reporting clinical trials were followed in the preparation of this manuscript.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bartels, S.J., Aschbrenner, K.A., Pratt, S.I. et al. Virtual Learning Collaborative Compared to Technical Assistance as a Strategy for Implementing Health Promotion in Routine Mental Health Settings: A Hybrid Type 3 Cluster Randomized Trial. Adm Policy Ment Health 49, 1031–1046 (2022). https://doi.org/10.1007/s10488-022-01215-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10488-022-01215-0

Keywords

Search

Navigation