Skip to main content
SpringerLink
Log in

PurposeFull People SEL and Character Education Program: A Cluster Randomized Trial in Schools Implementing Tier 1 PBIS with Fidelity

  • Original Paper
  • Published:
School Mental Health Aims and scope Submit manuscript

Abstract

Social–emotional learning (SEL), character education, and positive behavior intervention and supports (PBIS) are common approaches to Tier 1 universal social, emotional, and behavioral (SEB) supports in schools. There has been emerging evidence supporting the superior effect of integrated prevention that combines approaches to Tier 1 universal SEB programming compared to the traditional fragmented piecemeal approach. We conducted a cluster randomized trial to examine the additive effect of PurposeFull People (PfP; an SEL and character education program) in the context of schools implementing Tier 1 PBIS with fidelity. This study took place in five elementary schools implementing Tier 1 PBIS with fidelity. Twenty teachers/classrooms were randomly assigned to treatment (PfP combined with Tier 1 PBIS; ntreatment = 10) or control condition (Tier 1 PBIS only). An average of eight students were randomly recruited from each teacher’s classrooms (nstudent = 161; ntreatment student = 81). Four student SEB outcomes (behavior expectations, behavior discipline, character, and academic engaged time) were assessed at baseline and 4-month post-test. Multilevel ANCOVAs and generalized linear mixed models were used to assess the effectiveness of PfP and the effect variation across subgroups (i.e., student demographics and baseline status). Compared to PBIS alone, PfP combined with PBIS led to significantly larger improvements in students’ behavior expectations, behavior discipline, character, and academic engaged time. Cross-level interaction results indicated that the effectiveness of PurposeFull People on students’ SEB outcomes varied based on their baseline status, where students struggling most at baseline demonstrated the largest improvement. Limitations and implications for future research and practice are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Data Availability

The de-identified dataset is available via the Open Science Framework [https://doi.org/10.17605/OSF.IO/WPK4Q].

References

  • Algozzine, R. F., Barrett, S., Eber, L., George, H., Horner, R. H., Lewis, T. J., Putnam, B., Swain-Bradway, J., McIntosh, K., & Sugai, G. (2014). SWPBIS tiered fidelity inventory. OSEP Technical Assistance Center on Positive Behavioral Interventions and Supports. http://www.pbis.org

  • Baugh, A. D., Vanderbilt, A. A., & Baugh, R. F. (2019). The dynamics of poverty, educational attainment, and the children of the disadvantaged entering medical school. Advances in Medical Education and Practice, 10, 667.

    Article  PubMed  PubMed Central  Google Scholar 

  • Berger, E. (2019). Multi-tiered approaches to trauma-informed care in schools: A systematic review. School Mental Health, 11(4), 650–664.

    Article  Google Scholar 

  • Berkowitz, M. W., Bier, M. C., & McCauley, B. (2017). Toward a science of character education. Journal of Character Education, 13(1), 33–51.

    Google Scholar 

  • Bradshaw, C. P., Bottiani, J., Osher, D., & Sugai, G. (2013). Integrating positive behavioral interventions and supports (PBIS) and social emotional learning. In M. D. Weist, N. A. Lever, C. P. Bradshaw, & J. Owens (Eds.), Handbook of school mental health: Advancing practice and research (2nd ed., pp. 101–118). Springer.

    Google Scholar 

  • Brown, J. (2005). Character education: A resurgence to safety, a perspective in curricula trend. Academic Leadership: THe Online Journal, 3(2), 4.

    Google Scholar 

  • Berkowitz, M. W. (2002). The science of character education. Bringing in a New Era in Character Education, 508, 43–63.

    Google Scholar 

  • Calhoun, B., Williams, J., Greenberg, M., Domitrovich, C., Russell, M. A., & Fishbein, D. H. (2020). Social emotional learning program boosts early social and behavioral skills in low-income urban children. Frontiers in Psychology, 11, 561196.

    Article  PubMed  PubMed Central  Google Scholar 

  • Chafouleas, S. M., Kilgus, S. P., Riley-Tillman, T. C., Jaffery, R., & Harrison, S. (2011). Direct Behavior Rating Single Item Scales (DBR-SIS): An evaluation of the impact of various components to rater training on accuracy of rating academic engagement and disruption. Manuscript under review.

  • CASEL—Collaborative for Academic, Social, and Emotional Learning. (2013). The 2013 CASEL Guide: effective social and emotional learning programs—Preschool and Elementary School Edition. Author.

  • CASEL. (2022). CASEL guide: Effective social and emotional learning programs: Middle and high school edition. CASEL.

    Google Scholar 

  • Carroll, C., Patterson, M., Wood, S., Booth, A., Rick, J., & Balain, S. (2007). A conceptual framework for implementation fidelity. Implementation Science, 2(1), 1–9.

    Article  Google Scholar 

  • Cook, C. R., Frye, M., Slemrod, T., Lyon, A. R., Renshaw, T. L., & Zhang, Y. (2015). An integrated approach to universal prevention: Independent and combined effects of PBIS and SEL on youths’ mental health. School Psychology Quarterly, 30(2), 166.

    Article  PubMed  PubMed Central  Google Scholar 

  • Cook, C. R., Lyon, A. R., Locke, J., Waltz, T., & Powell, B. J. (2019). Adapting a compilation of implementation strategies to advance school-based implementation research and practice. Prevention Science, 20(6), 914–935.

    Article  PubMed  PubMed Central  Google Scholar 

  • Corcoran, R. P., Cheung, A. C. K., Kim, E., & Xie, C. (2018). Effective universal school-based social and emotional programs for improving academic achievement: A systematic review and meta-analysis of 50 years of research. Educational Research Review, 25, 56–72.

    Article  Google Scholar 

  • Dang, Z., Deng, C., Yang, X., Wei, K., & Huang, H. (2021). Nearest neighbor matching for deep clustering. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 13693–13702).

  • Domitrovich, C. E., Bradshaw, C. P., Greenberg, M. T., Embry, D., Poduska, J. M., & Ialongo, N. S. (2010). Integrated models of school-based prevention: Logic and theory. Psychology in the Schools, 47(1), 71–88.

    Article  PubMed  PubMed Central  Google Scholar 

  • Duncan, R., Washburn, I. J., Lewis, K. M., Bavarian, N., DuBois, D. L., Acock, A. C., Vuchinich, S., & Flay, B. R. (2017). Can universal SEL programs benefit universally? Effects of the positive action program on multiple trajectories of social-emotional and misconduct behaviors. Prevention Science, 18(2), 214–224.

    Article  PubMed  PubMed Central  Google Scholar 

  • Durlak, J. A., Weissberg, R. P., Dymnicki, A. B., Taylor, R. D., & Schellinger, K. B. (2011). The impact of enhancing students’ social and emotional learning: A meta-analysis of school-based universal interventions. Child Development, 82, 405–432.

    Article  PubMed  Google Scholar 

  • Eklund, K., Kilpatrick, K. D., Kilgus, S. P., & Haider, A. (2018). A systematic review of state-level social-emotional learning standards: Implications for practice and research. School Psychology Review, 47(3), 316–326.

    Article  Google Scholar 

  • Elias, M. J. (2014). The future of character education and social-emotional learning: The need for whole school and community-linked approaches. Journal of Character Education, 10, 37–42.

    Google Scholar 

  • Elias, M. J., Leverett, L., Duffell, J. C., Humphrey, N., Stepney, C., & Ferrito, J. (2015). Integrating SEL with related prevention and youth development approaches.

  • Elias, M. J., Parker, S. J., Kash, V. M., Weissberg, R. P., & O’Brien, M. U. (2008). Social and emotional learning, moral 176 Narvaez 176 education, and character education: A comparative analysis and a view toward convergence. In L. P. Nucci & D. Narvaez (Eds.), Handbook of moral and character education (pp. 248–266). Routledge.

    Google Scholar 

  • Fagan, A. A., Bumbarger, B. K., Barth, R. P., Bradshaw, C. P., Cooper, B. R., Supplee, L. H., & Walker, D. K. (2019). Scaling up evidence-based interventions in US public systems to prevent behavioral health problems: Challenges and opportunities. Prevention Science, 20(8), 1147–1168.

    Article  PubMed  PubMed Central  Google Scholar 

  • Filter, K. J., Johnson, L. D., Ford, A. L., Sowle, C. A., Bullard, S. J., Cook, C. R., Kloos, E., & Dupuis, D. (2022). An expert consensus process to distill Tier 1 PBIS into core practice elements essential to frontline implementation. Education and Treatment of Children, 45, 51–67.

    Article  Google Scholar 

  • Fixsen, D. L., Blase, K. A., Naoom, S. F., & Wallace, F. (2009). Core implementation components. Research on Social Work Practice, 19(5), 531–540.

    Article  Google Scholar 

  • Garner, P. W., Mahatmya, D., Brown, E. L., & Vesely, C. K. (2014). Promoting desirable outcomes among culturally and ethnically diverse children in social emotional learning programs: A multilevel heuristic model. Educational Psychology Review, 26(1), 165.

    Article  Google Scholar 

  • Gage, N. A., Leite, W., Childs, K., & Kincaid, D. (2017). Average treatment effect of school-wide positive behavioral interventions and supports on school-level academic achievement in Florida. Journal of Positive Behavior Interventions, 19, 158–167.

    Article  Google Scholar 

  • Greenberg, M. T. (2010). School-based prevention: Current status and future challenges. Effective Education, 2, 27–52.

    Article  Google Scholar 

  • Goodman-Scott, E., McMahon, G., Kalkbrenner, M. T., Smith-Durkin, S., Patel, S., Czack, A., & Weeks, N. (2021). An ex post facto study examining implementation of positive behavioral interventions and supports across school and community variables from an inclusive innovation perspective. Journal of Positive Behavior Interventions, 24(13), 109830072110137.

    Google Scholar 

  • Garwood, J. D. (2022). Special educator burnout and fidelity in implementing behavior support plans: A call to action. Journal of Emotional and Behavioral Disorders, 31(4), 106342662210992.

    Google Scholar 

  • Gettinger, M., Kratochwill, T. R., Eubanks, A., Foy, A., & Levin, J. R. (2021). Academic and Behavior Combined Support: Evaluation of an integrated supplemental intervention for early elementary students. Journal of School Psychology, 89, 1–19.

    Article  PubMed  Google Scholar 

  • Greenberg, M. T., & Abenavoli, R. (2017). Universal interventions: Fully exploring their impacts and potential to produce population-level impacts. Journal of Research on Educational Effectiveness, 10(1), 40–67.

    Article  Google Scholar 

  • Hamaker, E. L., & Muthén, B. (2020). The fixed versus random effects debate and how it relates to centering in multilevel modeling. Psychological methods, 25(3), 365.

    Article  PubMed  Google Scholar 

  • Hayashi, A., Liew, J., Aguilar, S. D., Nyanamba, J. M., & Zhao, Y. (2022). Embodied and social-emotional learning (SEL) in early childhood: Situating culturally relevant SEL in Asian, African, and North American contexts. Early Education and Development, 33(5), 746–763.

    Article  Google Scholar 

  • Hagermoser Sanetti, L. M., Williamson, K. M., Long, A. C., & Kratochwill, T. R. (2018). Increasing in-service teacher implementation of classroom management practices through consultation, implementation planning, and participant modeling. Journal of Positive Behavior Interventions, 20(1), 43–59.

    Article  Google Scholar 

  • Heck, R. H., & Thomas, S. L. (2020). An introduction to multilevel modeling techniques: MLM and SEM approaches. London: Routledge.

    Book  Google Scholar 

  • Horner, R. H., & Sugai, G. (2015). School-wide PBIS: An example of applied behavior analysis implemented at a scale of social importance. Behavior Analysis in Practice, 8(1), 80–85.

    Article  PubMed  PubMed Central  Google Scholar 

  • Ialongo, N. S., Domitrovich, C., Embry, D., Greenberg, M., Lawson, A., Becker, K. D., & Bradshaw, C. (2019). A randomized controlled trial of the combination of two school-based universal preventive interventions. Developmental Psychology, 55(6), 1313.

    Article  PubMed  Google Scholar 

  • Irvin, L. K., Tobin, T. J., Sprague, J. R., Sugai, G., & Vincent, C. G. (2004). Validity of office discipline referral measures as indices of school-wide behavioral status and effects of school-wide behavioral interventions. Journal of Positive Behavior Interventions, 6(3), 131–147.

    Article  Google Scholar 

  • Jeynes, W. H. (2019). A meta-analysis on the relationship between character education and student achievement and behavioral outcomes. Education and Urban Society, 51(1), 33.

    Article  Google Scholar 

  • Jolivette, K., & Swoszowski, N. C. (2020). Improving youth and staff supports: FW-PBIS framework implementation in juvenile correctional facilities. Remedial and Special Education, 41(2), 67–69.

    Article  Google Scholar 

  • Kim, J., McIntosh, K., Mercer, S. H., & Nese, R. N. (2018). Longitudinal associations between SWPBIS fidelity of implementation and behavior and academic outcomes. Behavioral Disorders, 43(3), 357–369.

    Article  Google Scholar 

  • Lee, M. K., Basford, J. R., Heinemann, A. W., & Cheville, A. (2020). Assessing whether ad hoc clinician-generated patient questionnaires provide psychometrically valid information. Health and Quality of Life Outcomes, 18, 1–17.

    Article  Google Scholar 

  • Li, P., & Redden, D. T. (2015). Comparing denominator degrees of freedom approximations for the generalized linear mixed model in analyzing binary outcome in small sample cluster-randomized trials. BMC Medical Research Methodology, 15(1), 1–12.

    Article  Google Scholar 

  • Lickona, T., Schaps, E., & Lewis, C. (2007). CEP’s eleven principles of effective character education. Character Education Partnership.

    Google Scholar 

  • Loader, C. (2012). Smoothing: local regression techniques. In Handbook of computational statistics (pp. 571–596). Springer, Berlin

  • Low, S., Cook, C. R., Smolkowski, K., & Buntain-Ricklefs, J. (2015). Promoting social–emotional competence: An evaluation of the elementary version of Second Step. Journal of School Psychology, 53(3), 463–477. https://doi.org/10.1016/j.jsp.2015.09.002

    Article  PubMed  Google Scholar 

  • Marhayani, D. (2016). Development of character education based on local wisdom in indegenous people Tengahan Sedangagung. Journal of Education, Teaching and Learning, 1(2), 66.

    Article  Google Scholar 

  • McIntosh, K., & MacKay, L. D. (2008). Enhancing generalization of social skills: Making social skills curricula effective after the lesson. Beyond Behavior, 18, 18–25.

    Google Scholar 

  • McIntosh, K., Massar, M. M., Algozzine, R. F., George, H. P., Horner, R. H., Lewis, T. J., & Swain-Bradway, J. (2017). Technical adequacy of the SWPBIS tiered fidelity inventory. Journal of Positive Behavior Interventions, 19(1), 3–13.

    Article  Google Scholar 

  • Moher, D., Hopewell, S., Schulz, K. F., Montori, V., Gøtzsche, P. C., Devereaux, P. J., ... & Altman, D. G. (2012). CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. International journal of surgery, 10(1), 28–55.

  • Morris, S. B. (2008). Estimating effect sizes from pretest-posttest-control group designs. Organizational Research Methods, 11(2), 364–386.

    Article  Google Scholar 

  • Napolitano, C. M., Sewell, M. N., Yoon, H. J., Soto, C. J., & Roberts, B. W. (2021). Social, emotional, and behavioral skills: An integrative model of the skills associated with success during adolescence and across the life span. Frontiers in Education, 6, 679561.

    Article  Google Scholar 

  • Nisar, H., Elgin, D., Bradshaw, C., Dolan., V., Frey, A., Horner, R., Owens, J., Perales, K., & Sutherland, K. (2022). Promoting social and behavioral success for learning in elementary schools: practice recommendations for elementary school educators, school and district administrators, and parents. 2M Research Services. Contract No. 92990019F0319. Arlington, TX.

  • O’Neill, J. M., Clark, J. K., & Jones, J. A. (2011). Promoting mental health and preventing substance abuse and violence in elementary students: A randomized control study of the Michigan Model for Health. Journal of School Health, 81, 320–330.

    Article  PubMed  Google Scholar 

  • Osher, D., Kidron, Y., Brackett, M., Dymnicki, A., Jones, S., & Weissberg, R. P. (2016). Advancing the science and practice of social and emotional learning: Looking back and moving forward. Review of Research in Education, 40(1), 644–681.

    Article  Google Scholar 

  • Osher, D., Mayer, M. J., Jagers, R. J., Kendziora, K., & Wood, L. (2019). Keeping students safe and helping them thrive: A collaborative handbook on school safety, mental health, and wellness (Vols. 2). Praeger/ABC-CLIO.

  • Pattaro, C. (2016). Character education: Themes and researches. An academic literature review. Italian Journal of Sociology of Education, 8(1), 6–30.

    Google Scholar 

  • Predy, L., McIntosh, K., & Frank, J. L. (2014). Utility of number and type of office discipline referrals in predicting chronic problem behavior in middle schools. School Psychology Review, 43(4), 472–489.

    Article  Google Scholar 

  • Parker, K., Nunns, M., Xiao, Z., Ford, T., & Ukoumunne, O. C. (2021). Characteristics and practices of school-based cluster randomised controlled trials for improving health outcomes in pupils in the United Kingdom: A methodological systematic review. BMC Medical Research Methodology, 21(1), 1–17.

    Article  Google Scholar 

  • Reinke, W. M., Herman, K. C., Darney, D., Pitchford, J., Becker, K., Domitrovich, C., & Ialongo, N. (2012). Using the classroom check-up model to support implementation of PATHS to PAX. Advances in School Mental Health Promotion, 5(3), 220–232.

    Article  Google Scholar 

  • Rhoads, C. H. (2011). The implications of “contamination” for experimental design in education. Journal of Educational and Behavioral Statistics, 36(1), 76–104.

    Article  Google Scholar 

  • Rocque, M. (2010). Office discipline and student behavior: Does race matter? American Journal of Education, 116(4), 557–581.

    Article  Google Scholar 

  • Smith, B. H. (2013). School-based character education in the United States. Childhood Education, 89(6), 350–355.

    Article  Google Scholar 

  • Smith, R. L., Eklund, K., & Kilgus, S. P. (2018). Concurrent validity and sensitivity to change of Direct Behavior Rating Single-Item Scales (DBR-SIS) within an elementary sample. School Psychology Quarterly, 33, 83–93.

    Article  PubMed  Google Scholar 

  • Sprague, J. R., & Walker, H. M. (2021). Safe and healthy schools: Practical prevention strategies. Guilford Publications.

    Google Scholar 

  • Sprague, J. R., Whitcomb, S., & Bear, G. (2019). Mechanisms for promoting and integrating school-wide discipline approaches. In M. Mayer & S. R. Jimerson (Eds.), School safety and violence prevention: Science, practice, policy (pp. 95–120). APA.

    Chapter  Google Scholar 

  • Spybrook, J., Zhang, Q., Kelcey, B., & Dong, N. (2020). Learning from cluster randomized trials in education: An assessment of the capacity of studies to determine what works, for whom, and under what conditions. Educational Evaluation and Policy Analysis, 42(3), 354–374.

    Article  Google Scholar 

  • Sutherland, K. S., Conroy, M. A., McLeod, B. D., Kunemund, R., & McKnight, K. (2019). Common practice elements for improving social, emotional, and behavioral outcomes of young elementary school students. Journal of Emotional and Behavioral Disorders, 27(2), 76–85.

    Article  Google Scholar 

  • Teerenstra, S., Melis, R. J. F., Peer, P. G. M., & Borm, G. F. (2006). Pseudo cluster randomization dealt with selection bias and contamination in clinical trials. Journal of Clinical Epidemiology, 59(4), 381–386.

    Article  PubMed  Google Scholar 

  • Thayer, A. J., Campa, D. M., Weeks, M. R., Buntain-Ricklefs, J., Low, S., Larson, M., & Cook, C. R. (2019). Examining the differential effects of a universal SEL program on student functioning through the dual continua model of mental health. The Journal of Primary Prevention, 40, 405–427. https://doi.org/10.1007/s10935-019-00557-0

    Article  PubMed  Google Scholar 

  • Tipton, E. & Miller, K. (2022) The Generalizer. Webtool hosted at https://thegeneralizer.org.

  • Treweek, S., Pitkethly, M., Cook, J., Fraser, C., Mitchell, E., Sullivan, F., Jackson, C., Taskila, T. K., & Gardner, H. (2018). Strategies to improve recruitment to randomised trials. Cochrane Database of Systematic Reviews, 22(2), MR000013.

    Google Scholar 

  • Wigelsworth, M., Verity, L., Mason, C., Qualter, P., & Humphrey, N. (2022). Social and emotional learning in primary schools: A review of the current state of evidence. British Journal of Educational Psychology, 92(3), 898–924.

    Article  PubMed  Google Scholar 

  • Wan, F. (2021). Statistical analysis of two arm randomized pre-post designs with one post-treatment measurement. BMC Medical Research Methodology, 21(1), 1–16.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Psychological and Quantitative Foundations, University of Iowa, 361 Lindquist Center, Iowa City, IA, 52242, USA

    Yanchen Zhang

  2. Department of Research and Development, CharacterStrong SEL Program, 4227 S. Meridian St., C #320, Puyallup, WA, 98373, USA

    Clayton R. Cook & Brian Smith

Authors
  1. Yanchen Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Clayton R. Cook
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Brian Smith
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yanchen Zhang.

Ethics declarations

Conflict of interest

YZ is an independent researcher responsible for program evaluation and declares that he has no competing or financial interests. CRC and BS are the developers of the PurposeFull People SEL and Character Education Program, but they did not participate or influence the data collection, analysis, or result reporting.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 30 kb)

Supplementary file2 (DOCX 59 kb)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, Y., Cook, C.R. & Smith, B. PurposeFull People SEL and Character Education Program: A Cluster Randomized Trial in Schools Implementing Tier 1 PBIS with Fidelity. School Mental Health 15, 985–1002 (2023). https://doi.org/10.1007/s12310-023-09600-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12310-023-09600-2

Keywords

Search

Navigation