Abstract
This study reports the results of a pragmatic effectiveness–implementation hybrid trial of the Family Check-Up (FCU) conducted in three community mental health agencies with 40 participating therapists. Seventy-one families with children between 5 and 17 years of age participated. Intervention fidelity and level of adoption were acceptable; families reported high service satisfaction; and therapists reported high acceptability. Families in the FCU condition experienced significantly reduced youth conduct problems in comparison to usual care and completion of the FCU resulted in larger effects. This study provides promising evidence that implementing the FCU in community mental health agencies has the potential to improve youth behavior outcomes.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aarons, G. A. (2004). Mental health provider attitudes toward adoption of evidence-based practice: The Evidence-Based Practice Attitude Scale (EBPAS). Mental Health Services Research, 6(2), 61–74.
Aarons, G. A., & Sawitzky, A. C. (2006). Organizational climate partially mediates the effect of culture on work attitudes and staff turnover in mental health services. Administration and Policy in Mental Health and Mental Health Services Research, 33(3), 289–301.
Aarons, G. A., Sommerfeld, D. H., Hecht, D. B., Silovsky, J. F., & Chaffin, M. J. (2009). The impact of evidence-based practice implementation and fidelity monitoring on staff turnover: Evidence for a protective effect. Journal of Consulting and Clinical Psychology, 77(2), 270–280.
APA Presidential Task Force on Evidence-Based Practice. (2006). Evidence-based practice in psychology. The American Psychologist, 61(4), 271–285.
Chambless, D. L., & Hollon, S. D. (1998). Defining empirically supported therapies. Journal of Consulting and Clinical Psychology, 66(1), 7–18.
Connell, A. M., Dishion, T. J., Yasui, M., & Kavanagh, K. (2007). An adaptive approach to family intervention: Linking engagement in family-centered intervention to reductions in adolescent problem behavior. Journal of Consulting and Clinical Psychology, 75(4), 568–579.
Costello, E. J., Angold, A., Burns, B. J., Stangl, D. K., Tweed, D. L., & Erkanli, A. (1996). The Great Smoky Mountains Study of Youth: Goals, design, methods, and the prevalence of DSM-III-R disorders. Archives of General Psychiatry, 53, 1129–1136.
Curran, G. M., Bauer, M., Mittman, B., Pyne, J. M., & Stetler, C. (2012). Effectiveness–implementation hybrid designs: Combining elements of clinical effectiveness and implementation research to enhance public health impact. Medical Care, 50(3), 217–226.
Dempster, A., Laird, N., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society: Series B, 39(1), 1–38.
Department of Health and Human Services, U. P. H. S. (2000). Report of the Surgeon General’s Conference on Children’s Mental Health: A National Action Agenda. Washington, DC.
Dishion, T. J., Brennan, L. M., Shaw, D. S., McEachern, A. D., Wilson, M. N., & Jo, B. (2014). Prevention of problem behavior through annual Family Check-Ups in early childhood: Intervention effects from home to early elementary school. Journal of Abnormal Child Psychology, 42(3), 343–354.
Dishion, T. J., Kavanagh, K., Schneiger, A., Nelson, S. E., & Kaufman, N. (2002). Preventing early adolescent substance use: A family-centered strategy for public middle school. Prevention Science, 3(3), 191–201.
Dishion, T. J., Shaw, D. S., Connell, A., Gardner, F. E. M., Weaver, C., & Wilson, M. (2008). The Family Check-Up with high-risk indigent families: Preventing problem behavior by increasing parents’ positive behavior support in early childhood. Child Development, 79(5), 1395–1414.
Dishion, T. J., Smith, J. D., Knutson, N., Brauer, L., Gill, A., & Risso, J. (2014). Family Check-Up: COACH ratings manual. Version 2. Unpublished coding manual. Eugene, OR: Child and Family Center, University of Oregon.
Dishion, T. J., & Stormshak, E. A. (2007). Intervening in children’s lives: An ecological, family-centered approach to mental health care. Washington, DC: American Psychological Association.
Dishion, T. J., Stormshak, E. A., & Kavanagh, K. (2011). Everyday parenting: A professional’s guide to building family management skills. Champaign, IL: Research Press.
Forgatch, M. S., & Patterson, G. R. (2010). Parent management training-Oregon model: An intervention for antisocial behavior in children and adolescents. In J. R. Weisz & A. E. Kazdin (Eds.), Evidence-based psychotherapies for children and adolescents (pp. 159–178). New York: Guilford Press.
Glasgow, R. E., Vogt, T. M., & Boles, S. M. (1999). Evaluating the public health impact of health promotion interventions: The RE-AIM framework. American Journal of Public Health, 89(9), 1322–1327.
Goodman, R. (1997). The Strengths and Difficulties Questionnaire: A research note. Journal of Child Psychology and Psychiatry, 38(5), 581–586.
Goodman, R., Meltzer, H., & Bailey, V. (1998). The Strengths and Difficulties Questionnaire: A pilot study on the validity of the self-report version. European Child and Adolescent Psychiatry, 7(3), 125–130.
Hogan, M. F. (2003). New Freedom Commission report: The president’s New Freedom Commission: Recommendations to transform mental health care in America. Psychiatric Services, 54(11), 1467–1474.
Institute of Medicine. (2001). Crossing the quality chasm: A new health system for the 21st century. Washington, DC: National Academy Press.
Kataoka, S. H., Zhang, L., & Wells, K. B. (2002). Unmet need for mental health care among U.S. children: Variation by ethnicity and insurance status. American Journal of Psychiatry, 159, 1548–1555.
Little, R. J. A. (1988). A test of missing completely at random for multivariate data with missing values. Journal of the American Statistical Association, 83, 1198–1202.
McEachern, A., Dishion, T. J., Weaver, C. M., Shaw, D. S., Wilson, M. N., & Gardner, F. E. M. (2012). Parenting Young Children (PARYC): Validation of a self-report parenting measure. Journal of Child and Family Studies, 21(3), 498–511.
McFadyen-Ketchum, S. A., Bates, J. E., Dodge, K. A., & Pettit, G. S. (1996). Patterns of change in early childhood aggressive-disruptive behavior: Gender differences in predictions from early coercive and affectionate mother-child interactions. Child Development, 67(5), 2417–2433.
McHugh, R. K., & Barlow, D. H. (2010). The dissemination and implementation of evidence-based psychological treatments: A review of current efforts. American Psychologist, 65(2), 73–84.
McKay, M. M., Lynn, C. J., & Bannon, W. M. (2005). Understanding inner city child mental health need and trauma exposure: Implications for preparing urban service providers. American Journal of Orthopsychiatry, 75(2), 201–210.
Miller, W. R., & Rollnick, S. (2002). Motivational interviewing: Preparing people for change (2nd ed.). New York: Guilford Press.
Miller, S., Wampold, B. E., & Varhely, K. (2008). Direct comparisons of treatment modalities for youth disorders: A meta-analysis. Psychotherapy Research, 18(1), 5–14.
Muthén, B. O., & Satorra, A. (1995). Complex sample data in structural equation modeling. In P. V. Marsden (Ed.), Sociological methodology (pp. 267–316). Oxford: Blackwell.
Nguyen, T. D., Attkisson, C. C., & Stegner, B. L. (1983). Assessment of patient satisfaction: Development and refinement of a service evaluation questionnaire. Evaluation and Program Planning, 6(3–4), 299–313.
Patterson, G. R., Reid, J. B., & Dishion, T. J. (1992). Antisocial boys. Eugene, OR: Castalia.
Proctor, E. K., Landsverk, J. A., Aarons, G. A., Chambers, D., Glisson, C., & Mittman, B. (2009). Implementation research in mental health services: An emerging science with conceptual, methodological, and training challenges. Administration and Policy in Mental Health and Mental Health Services Research, 36(1), 24–34.
Selby, P., Brosky, G., Oh, P., Raymond, V., & Ranger, S. (2012). How pragmatic or explanatory is the randomized, controlled trial? The application and enhancement of the PRECIS tool to the evaluation of a smoking cessation trial. BMC Medical Research Methodology, 12(1), 101.
Shaw, D. S., Dishion, T. J., Supplee, L., Gardner, F. E. M., & Arnds, K. (2006). Randomized trial of a family-centered approach to prevention of early conduct problems: 2-Year effects of the Family Check-Up in early childhood. Journal of Consulting and Clinical Psychology, 74(1), 1–9.
Shaw, D. S., Gilliom, M., Ingoldsby, E. M., & Nagin, D. S. (2003). Trajectories leading to school-age conduct problems. Developmental Psychology, 39(2), 189–200.
Shrout, P., & Fleiss, J. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86, 420–428.
Smith, J. D., & Dishion, T. J. (2013). Mindful parenting in the development and maintenance of youth psychopathology. In J. T. Ehrenreich-May & B. C. Chu (Eds.), Transdiagnostic mechanisms and treatment for youth psychopathology (pp. 138–160). New York, NY: Guilford Press.
Smith, J. D., Dishion, T. J., Moore, K. J., Shaw, D. S., & Wilson, M. N. (2013a). Video feedback in the Family Check-Up: Indirect effects on observed parent-child coercive interactions. Journal of Clinical Child & Adolescent Psychology, 42(3), 405–417.
Smith, J. D., Dishion, T. J., Shaw, D. S., & Wilson, M. N. (2013b). Indirect effects of fidelity to the Family Check-Up on changes in parenting and early childhood problem behaviors. Journal of Consulting and Clinical Psychology, 81(6), 962–974.
Smith, J. D., Dishion, T. J., Shaw, D. S., Wilson, C., Winter, C., & Patterson, G. R. (2014). Coercive family process and early-onset conduct problems from age 2 to school entry. Development and Psychopathology. doi:10.1017/S0954579414000169.
Smith, J. D., Knoble, N., Zerr, A. A., Dishion, T. J., & Stormshak, E. A. (2014b). Multicultural competence and the Family Check-Up: Indirect effect on adolescent antisocial behavior through family conflict. Journal of Clinical Child & Adolescent Psychology, 43(3), 400–414.
Stormshak, E. A., Connell, A. M., Véronneau, M.-H., Myers, M. W., Dishion, T. J., Kavanagh, K., et al. (2011). An ecological approach to promoting early adolescent mental health and social adaptation: Family-centered intervention in public middle schools. Child Development, 82(1), 209–225.
Stormshak, E. A., Dishion, T. J., Light, J., & Yasui, M. (2005). Implementing family-centered interventions within the public middle school: Linking service delivery to change in problem behavior. Journal of Abnormal Child Psychology, 33(6), 723–733.
Stormshak, E. A., Fosco, G. M., & Dishion, T. J. (2010). Implementing interventions with families in schools to increase youth school engagement: The Family Check-Up model. School Mental Health, 2(2), 82–92.
Thorpe, K. E., Zwarenstein, M., Oxman, A. D., Treweek, S., Furberg, C. D., Altman, D. G., et al. (2009). A pragmatic–explanatory continuum indicator summary (PRECIS): A tool to help trial designers. Canadian Medical Association Journal, 180(10), E47–E57.
United States Department of Health and Human Services. (2010). The HHS poverty guidelines for the remainder of 2010. Retrieved July 8, 2013 from http://aspe.hhs.gov/poverty/10poverty.shtml%5D.
Van Ryzin, M. J., Fosco, G. M., & Dishion, T. J. (2012a). Family and peer predictors of substance use from early adolescence to early adulthood: An 11-year prospective analysis. Addictive Behaviors, 37(12), 1314–1324.
Van Ryzin, M. J., Stormshak, E. A., & Dishion, T. J. (2012b). Engaging parents in the Family Check-Up in middle schools: Longitudinal effects through the transition to high school. Journal of Adolescent Health, 50(6), 627–633.
Wampold, B. E., Budge, S. L., Laska, K. M., Del Re, A. C., Baardseth, T. P., Flűckiger, C., et al. (2011). Evidence-based treatments for depression and anxiety versus treatment-as-usual: A meta-analysis of direct comparisons. Clinical Psychology Review, 31(8), 1304–1312.
Weisz, J. R., Jensen-Doss, A., & Hawley, K. M. (2006). Evidence-based youth psychotherapies versus usual clinical care: A meta-analysis of direct comparisons. American Psychologist, 61(7), 671–689.
Weisz, J. R., Ugueto, A. M., Cheron, D. M., & Herren, J. (2013). Evidence-based youth psychotherapy in the mental health ecosystem. Journal of Clinical Child & Adolescent Psychology, 42, 274–286.
Wells, K. B. (1999). Treatment research at the crossroads: The scientific interface of clinical trials and effectiveness research. American Journal of Psychiatry, 156, 5–10.
Acknowledgments
This research was supported by Centers for Disease Control Grant CE001389-01 to Elizabeth A. Stormshak. Justin Smith received support from research training Grant MH20012 from the National Institute of Mental Health, awarded to Elizabeth A. Stormshak, and from the National Institute on Drug Abuse through a pilot study grant awarded to Justin Smith by the Center for Prevention Implementation Methodology for Drug Abuse and Sex Risk Behavior (P30 DA027828). The authors gratefully thank Thomas Dishion, Amy Baker, Daryl Ford, and Whitney Nash for their contributions to this project; Sara Landes for comments on an earlier version of the manuscript; Cheryl Mikkola for editorial support; and the agencies, therapists, and families who generously participated in our research.
Author information
Authors and Affiliations
Corresponding author
Appendix: Family Check-Up Caregiver Satisfaction Survey
Appendix: Family Check-Up Caregiver Satisfaction Survey
Indicate how strongly you agree or disagree with the following statement concerning your experience with the Family Check-Up provider
Strongly disagree 1 | Disagree 2 | Neither 3 | Agree 4 | Strongly agree 5 | |
|---|---|---|---|---|---|
“My therapist…” | |||||
1. gave me new ways of looking at my problems | □ | □ | □ | □ | □ |
2. gave me realistic ideas for making changes | □ | □ | □ | □ | □ |
3. role played with me how to use new skills | □ | □ | □ | □ | □ |
4. let me decide on areas I wanted to work on | □ | □ | □ | □ | □ |
5. helped me identify my strengths as a parent | □ | □ | □ | □ | □ |
6. helped me set goals I could reach | □ | □ | □ | □ | □ |
7. respected me | □ | □ | □ | □ | □ |
8. understood my situation | □ | □ | □ | □ | □ |
9. was someone I liked talking with | □ | □ | □ | □ | □ |
Rights and permissions
About this article
Cite this article
Smith, J.D., Stormshak, E.A. & Kavanagh, K. Results of a Pragmatic Effectiveness–Implementation Hybrid Trial of the Family Check-Up in Community Mental Health Agencies. Adm Policy Ment Health 42, 265–278 (2015). https://doi.org/10.1007/s10488-014-0566-0
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10488-014-0566-0