Skip to main content

Advertisement

Log in

Factors Associated with Teacher Delivery of a Classroom-Based Tier 2 Prevention Program

  • Published:
Prevention Science Aims and scope Submit manuscript

Abstract

Teachers sometimes struggle to deliver evidence-based programs designed to prevent and ameliorate chronic problem behaviors of young children with integrity. Identifying factors associated with variations in the quantity and quality of delivery is thus an important goal for the field. This study investigated factors associated with teacher treatment integrity of BEST in CLASS, a tier-2 prevention program designed for young children at risk for developing emotional/behavioral disorders. Ninety-two early childhood teachers and 231 young children at-risk for emotional/behavioral disorders participated in the study. Latent growth curve analyses indicated that both adherence and competence of delivery increased across six observed time points. Results suggest that teacher education and initial levels of classroom quality may be important factors to consider when teachers deliver tier-2 (i.e., targeted to children who are not responsive to universal or tier-1 programming) prevention programs in early childhood settings. Teachers with higher levels of education delivered the program with more adherence and competence initially. Teachers with higher initial scores on the Emotional Support subscale of the Classroom Assessment Scoring System (CLASS) delivered the program with more competence initially and exhibited higher growth in both adherence and competence of delivery across time. Teachers with higher initial scores on the Classroom Organization subscale of the CLASS exhibited lower growth in adherence across time. Contrary to hypotheses, teacher self-efficacy did not predict adherence, and teachers who reported higher initial levels of Student Engagement self-efficacy exhibited lower growth in competence of delivery. Results are discussed in relation to teacher delivery of evidence-based programs in early childhood classrooms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. Two items assess rules: three to five rules are visible in classroom and teacher reviews rules, addresses rule violations. Only the second item was included in the current study as we focused on teacher-delivered practices; three to five rules are visible in classroom is a static item that did not change from observation to observation.

References

  • Achenbach, T. M., & Rescorla, L. A. (2000). CBCL/1, 5-5 & C-TRF/1, 5–5 profiles. Burlington: Research Center for Children, Youth, and Families.

    Google Scholar 

  • Baker, C. N., Kupersmidt, J. B., Voegler-Lee, M. E., Arnold, D., & Willoughby, M. T. (2010). Predicting teacher participation in a classroom-based, integrated preventive intervention for preschoolers. Early Childhood Research Quarterly, 25, 270–283. doi:10.1016/j.ecresq.2009.09.005.

    Article  PubMed  PubMed Central  Google Scholar 

  • Barber, J. P., Crits-Christoph, P., & Luborsky, L. (1996). Effects of therapist adherence and competence on patient outcome in brief dynamic therapy. Journal of Consulting and Clinical Psychology, 64, 619–622. doi:10.1037/0022-006X.64.3.619.

    Article  CAS  PubMed  Google Scholar 

  • Berlin, L. J., Brooks-Gunn, J., McCarton, C., & McCormick, M. C. (1998). The effectiveness of early intervention: Examining risk factors and pathways to enhanced development. Preventive Medicine, 27, 238–245. doi:10.1006/pmed.1998.0282.

    Article  CAS  PubMed  Google Scholar 

  • Bronfenbrenner, U. (1979). The ecology of human development: Experiments by nature and design. Cambridge: Harvard University Press.

    Google Scholar 

  • Bronfenbrenner, U. (2005). Making human beings human: Bioecological perspectives on human development. Thousand Oaks: Sage Publications.

    Google Scholar 

  • Bruhn, A.L., Lane, K.L., Hirsch, S.E. (2013). A Review of Tier 2 Interventions Conducted Within Multitiered Models of Behavioral Prevention. Journal of Emotional and Behavioral Disorders, 22, 171–189

  • Carroll, K. M., & Nuro, K. F. (2002). One size cannot fit all: A stage model for psychotherapy manual development. Clinical Psychology: Science and Practice, 9, 396–406. doi:10.1093/clipsy.9.4.396.

    Google Scholar 

  • Carroll, K. M., Nich, C., Sifty, R. L., Nuro, K. F., Frankforter, T. L., Ball, S. A., et al. (2000). A general system for evaluating therapist adherence and competence in psychotherapy research. Drug and Alcohol Dependence, 57, 225–238. doi:10.1016/S0376-8716(99)00049-6.

    Article  CAS  PubMed  Google Scholar 

  • Carter, A. S., Wagmiller, R. J., Gray, S. A., McCarthy, K. J., Horwitz, S. M., & Briggs-Gowan, M. J. (2010). Prevalence of DSM-IV disorder in a representative, healthy birth cohort at school entry: Sociodemographic risks and social adaptation. Journal of the American Academy of Child & Adolescent Psychiatry, 49, 686–698. doi:10.1016/j.jaac.2010.03.018.

    Google Scholar 

  • Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6, 284–290. doi:10.1037/1040-3590.6.4.284.

    Article  Google Scholar 

  • Conroy, M. A., Sutherland, K. S., Algina, J., Ladwig, C., Werch, B., Martinez, J., Jessee, G., & Gyure, M. (2017). BEST in CLASS: A professional development intervention fostering high quality classroom experiences for young children with problem behavior. Manuscript submitted for publication.

  • Conroy, M. A., Sutherland, K. S., Algina, J. J., Wilson, R. E., Martinez, J. R., & Whalon, K. J. (2015). Measuring teacher implementation of the BEST in CLASS intervention program and corollary child outcomes. Journal of Emotional and Behavioral Disorders, 23(3), 144–155.

    Article  Google Scholar 

  • Conroy, M. A., Sutherland, K. S., Vo, A. K., Carr, S. E., & Ogston, P. (2014). Early childhood teachers’ use of effective instructional practices and the collateral effects on young children’s behavior. Journal of Positive Behavior Interventions, 16, 81–92. doi:10.1177/1098300713478666.

    Article  Google Scholar 

  • Dane, A. V., & Schneider, B. H. (1998). Program integrity in primary and early secondary prevention: Are implementation effects out of control. Clinical Psychology Review, 18, 23–45. doi:10.1016/S0272-7358(97)00043-3.

    Article  CAS  PubMed  Google Scholar 

  • Domitrovich, C. E., Bradshaw, C. P., Poduska, J. M., Hoagwood, K., Buckley, J. A., Olin, S., Hunter Romanelli, L., Leaf, P. J., Greenberg, M. T., & Ialongo, N. S. (2008). Maximizing the implementation quality of evidence-based preventive interventions in schools: A conceptual framework. Advances in School Mental Health Promotion, 1, 6–28.

    Article  PubMed  PubMed Central  Google Scholar 

  • Domitrovich, C. E., Cortes, R. C., & Greenberg, M. T. (2007). Improving young children’s social and emotional competence: A randomized trial of the preschool “PATHS” curriculum. The Journal of Primary Prevention, 28, 67–91. doi:10.1007/s10935-007-0081-0.

    Article  PubMed  Google Scholar 

  • Domitrovich, C. E., Gest, S. D., Jones, D., Gill, S., & DeRousie, R. M. S. (2010). Implementation quality: Lessons learned in the context of the head start REDI trial. Early Childhood Research Quarterly, 25, 284–298. doi:10.1016/j.ecresq.2010.04.001.

    Article  PubMed  PubMed Central  Google Scholar 

  • Domitrovich, C. E., Pas, E. T., Bradshaw, C. P., Becker, K. D., Keperling, J. P., Embry, D. D., & Ialongo, N. (2015). Individual and school organizational factors that influence implementation of the PAX Good Behavior Game Intervention. Prevention Science, 16(8), 1064–1074. doi:10.1007/s11121-015-0557-8.

    Article  PubMed  Google Scholar 

  • Durlak, J. A. (2015). Studying program implementation is not easy but it is essential. Prevention Science, 16(8), 1123–1127.

    Article  PubMed  Google Scholar 

  • Durlak, J. A. (2010). The importance of doing well in whatever you do: A commentary on the special section, “Implementation research in early childhood education.”. Early Childhood Research Quarterly, 25, 348–357. doi:10.1016/j.ecresq.2010.03.003.

    Article  Google Scholar 

  • Durlak, J. A., Weissberg, R. P., Dymnicki, A. B., Taylor, R. D., & Schellinger, K. B. (2011). The impact of enhancing students’ social and emotional learning: A meta-analysis of school-based universal interventions. Child Development, 82, 405–432. doi:10.1111/j.1467-8624.2010.01564.x.

    Article  PubMed  Google Scholar 

  • Feil, E. G., Severson, H. H., & Walker, H. M. (1998). Screening for emotional and behavioral delays: The Early Screening Project. Journal of Early Intervention, 21, 252–266. doi:10.1177/105381519802100306.

    Article  Google Scholar 

  • Fixsen, D. L., Naoom, S. F., Blasé, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. Tampa: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231).

    Google Scholar 

  • Guo, Y., Connor, C. M., Yang, Y., Roehrig, A. D., & Morrison, F. J. (2012). The effects of teacher qualification, teacher self-efficacy, and classroom practices on fifth graders' literacy outcomes. The Elementary School Journal, 113, 3–24. doi:10.1086/665816.

    Article  Google Scholar 

  • Han, S. S., & Weiss, B. (2005). Sustainability of teacher implementation of school-based mental health programs. Journal of Abnormal Child Psychology, 33, 665–679. doi:10.1007/s10802-005-7646-2.

    Article  PubMed  Google Scholar 

  • Hemmeter, M. L., Corso, R., & Cheatham, G. (2006). A national survey of early childhood educators: Training needs and strategies. Paper presented at the Conference on Research Innovations in Early Intervention, San Diego, CA.

  • Hogue, A., Liddle, H. A., & Rowe, C. (1996). Treatment adherence process research in family therapy: A rationale and some practical guidelines. Psychotherapy: Theory, Research, Practice, Training, 33, 332–345. doi:10.1037/0033-3204.33.2.332.

    Article  Google Scholar 

  • Hogue, A., Henderson, C. E., Dauber, S., Barajas, P. C., Fried A., Liddle, H. A. (2008) Treatment adherence, competence, and outcome in individual and family therapy for adolescent behavior problems. Journal of Consulting and Clinical Psychology, 76, 544–555

  • Little, M. A., Sussman, S., Sun, P., & Rohrbach, L. A. (2013). The effects of implementation fidelity in the Towards No Drug Abuse dissemination trial. Health Education, 113, 281–296. doi:10.1108/09654281311329231.

    Article  PubMed Central  Google Scholar 

  • McCabe, P. C., & Altamura, M. (2011). Empirically valid strategies to improve social and emotional competence of preschool children. Psychology in the Schools, 48, 513–540. doi:10.1002/pits.20570.

    Article  Google Scholar 

  • McLeod, B. D., Southam-Gerow, M. A., Rodriguez, A., Quinoy, A., Arnold, C., Kendall, P. C., & Weisz, J., R. (2016). Development and initial psychometrics for a therapist competence instrument for CBT for youth with anxiety. Journal of Clinical and Adolescent Psychology. Advance online publication. doi:10.1080/15374416.2016.1253018.

  • McLeod, B. D., Southam-Gerow, M. A., Tully, C. B., Rodriguez, A., & Smith, M. M. (2013). Making a case for treatment integrity as a psychological treatment quality indicator. Clinical Psychology: Science and Practice, 20, 14–32. doi:10.1111/cpsp.12020.

    Google Scholar 

  • Mendel, P., Meredith, L. S., Schoenbaum, M., Sherbourne, C. D., & Wells, K. B. (2008). Interventions in organizational and community context: A framework for building evidence on dissemination and implementation in health services research. Administration and Policy in Mental Health and Mental Health Services Research, 35(1–2), 21–37. doi:10.1007/s10488-007-0144-9.

    Article  PubMed  Google Scholar 

  • Mitchell, B. S., Stormont, M. A., & Gage, N. A. (2011). Tier 2 interventions within the context of school-wide positive behavior support. Behavioral Disorders, 36, 241–261.

  • Muthén, L. K., & Muthén, B. O. (1998-2017). Mplus user’s guide (Eighth ed.). Los Angeles: Muthén & Muthén.

    Google Scholar 

  • Nelson, J. R., Stage, S., Duppong-Hurley, K., Synhorst, L., & Epstein, M. H. (2007). Risk factors predictive of the problem behavior of children at risk for emotional and behavioral disorders. Exceptional Children, 73, 367–379. doi:10.1177/001440290707300306.

    Article  Google Scholar 

  • Newborg, J. (2005). Battelle developmental inventory, 2nd edition, examiner’s manual. Rolling Meadows: Riverside Publishing.

    Google Scholar 

  • Olejnik, S., & Algina, J. (2000). Measures of effect size for comparative studies: Applications, interpretations, and limitations. Contemporary Educational Psychology, 25, 241–286.

    Article  CAS  PubMed  Google Scholar 

  • Pas, E. T., Bradshaw, C. P., & Hershfeldt, P. A. (2012). Teacher-and school-level predictors of teacher efficacy and burnout: Identifying potential areas for support. Journal of School Psychology, 50, 129–145. doi:10.1016/j.jsp.2011.07.003.

    Article  PubMed  Google Scholar 

  • Pas, E. T., Waasdorp, T. E., & Bradshaw, C. P. (2015). Examining contextual influences on classroom-based implementation of Positive Behavior Support strategies: Findings from a randomized controlled effectiveness trial. Prevention Science, 16(8), 1096–1106. doi:10.1007/s11121-014-0492-0.

    Article  PubMed  Google Scholar 

  • Pianta, R., Howes, C., Burchinal, M., Bryant, D., Clifford, R., Early, D., & Barbarin, O. (2005). Features of pre-kindergarten programs, classrooms, and teachers: Do they predict observed classroom quality and child-teacher interactions? Applied Developmental Science, 9(3), 144–159. doi:10.1207/s1532480xads0903_2.

    Article  Google Scholar 

  • Pianta, R. C., Karen, M., Paro, L., & Hamre, B. K. (2008). Classroom assessment scoring system (CLASS) manual, pre-K. Baltimore: Paul H. Brookes Publishing Company.

    Google Scholar 

  • Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., Griffey, R., & Hensley, M. (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health, 38, 65–76.

    Article  PubMed  Google Scholar 

  • Sameroff, A. J. (1995). General systems theories and developmental psychopathology. In E. D. Cicchetti & D. J. Cohen (Eds.), Developmental Psychopathology, Vol. 1: Theory and Methods. Oxford: John Wiley & Sons.

    Google Scholar 

  • Sanetti, L. M. H., & Kratochwill, T. R. (2009). Toward developing a science of treatment integrity: Introduction to the special series. School Psychology Review, 38, 445–459.

    Google Scholar 

  • Schoenwald, S. K., Garland, A. F., Chapman, J. E., Frazier, S. L., Sheidow, A. J., & Southam-Gerow, M. A. (2011). Toward the effective and efficient measurement of implementation fidelity. Administration and Policy in Mental Health and Mental Health Services Research, 38, 32–43. doi:10.1007/s10488-010-0321-0.

    Article  PubMed  PubMed Central  Google Scholar 

  • Sutherland, K. S., Conroy, M. A., Algina, J., Ladwig, C., Jessee, G., & Gyure, M. (2017). Reducing child problem behaviors and improving teacher-child interactions and relationships: A randomized controlled trial of BEST in CLASS. Early Childhood Research Quarterly. in press.

  • Sutherland, K. S., Conroy, M. A., Vo, A., & Ladwig, C. (2015). Implementation integrity of practice-based coaching: Preliminary results from the BEST in CLASS efficacy trial. School Mental Health, 7, 21–33. doi:10.1007/s12310-014-9134-8.

    Article  Google Scholar 

  • Sutherland, K. S., McLeod, B. D., Conroy, M. A., Abrams, L., & Smith, M. M. (2014). Preliminary psychometric properties of the BEST in CLASS Adherence and Competence Scale. Journal of Emotional and Behavioral Disorders, 22, 249–259. doi:10.1177/1063426613493669.

    Article  Google Scholar 

  • Sutherland, K. S., McLeod, B. D., Conroy, M. A., & Cox, J. R. (2013). Measuring implementation of evidence-based programs targeting young children at risk for emotional/behavioral disorders conceptual issues and recommendations. Journal of Early Intervention, 35, 129–149. doi:10.1177/1053815113515025.

    Article  Google Scholar 

  • Tschannen-Moran, M., & Hoy, A. W. (2001). Teacher efficacy: Capturing an elusive construct. Teaching and Teacher Education, 17, 783–805. doi:10.1016/S0742-051X(01)00036-1.

    Article  Google Scholar 

  • Wanless, S. B., Rimm-Kaufman, S. E., Abry, T., Larsen, R. A., & Patton, C. L. (2015). Engagement in training as a mechanism to understanding fidelity of implementation of the Responsive Classroom Approach. Prevention Science, 16(8), 1107–1116.

    Article  PubMed  Google Scholar 

  • Webster-Stratton, C., Reid, M. J., & Hammond, M. (2004). Treating children with early-conduct problems: Intervention outcomes for parent, child, and teacher training. Journal of Clinical Child and Adolescent Psychology, 33, 105–124. doi:10.1207/S15374424JCCP3301_11.

    Article  PubMed  Google Scholar 

  • Williford, A. P., Wolcott, C. S., Whittaker, J. V., & Locasale-Crouch, J. (2015). Program and teacher characteristics predicting the implementation of banking time with preschoolers who display disruptive behaviors. Prevention Science, 16, 1054–1063. doi:10.1007/s11121-015-0544-0.

    Article  PubMed  Google Scholar 

  • Wolery, M. (2011). Intervention research: The importance of fidelity measurement. Topics in Early Childhood Special Education, 31, 155–157. doi:10.1177/0271121411408621.

    Article  Google Scholar 

Download references

Acknowledgements

This research was supported by a grant (R324A110173) from the U. S. Department of Education, Institute for Education Sciences, with additional support from an NIH/NCATS Clinical and Translational Science Award to the University of Florida UL1 TR000064 and Grant H325H140001 from the U.S. Department of Education, Office of Special Education Programs. The opinions expressed by the authors are not necessarily reflective of the position of or endorsed by the U. S. Department of Education.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Kevin S. Sutherland.

Ethics declarations

Funding Information

This research was supported by a grant (R324A110173) from the US Department of Education, Institute for Education Sciences, with additional support from an NIH/NCATS Clinical and Translational Science Award to the University of Florida UL1 TR000064. The opinions expressed by the authors are not necessarily reflective of the position of or endorsed by the US Department of Education.

Conflict of Interest

The authors declare that they have no conflict of interest.

Ethical Approval

All study procedures involving human participants were in accordance with the ethical standards of the researchers’ Institutional Review Boards and with the 1964 Helsinki Declaration and its later amendments of comparable ethical standards.

Informed Consent

Informed consent was obtained from all individual participants included in the study.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sutherland, K.S., Conroy, M.A., McLeod, B.D. et al. Factors Associated with Teacher Delivery of a Classroom-Based Tier 2 Prevention Program. Prev Sci 19, 186–196 (2018). https://doi.org/10.1007/s11121-017-0832-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11121-017-0832-y

Keywords

Navigation