A Randomized Impact Evaluation of Capturing Kids' Hearts, South Carolina, 2016-2018 (ICPSR 37954)
Version Date: Jul 29, 2021 View help for published
Principal Investigator(s): View help for Principal Investigator(s)
Thomas Hanson, WestEd;
Anthony Petrosino, WestEd
https://doi.org/10.3886/ICPSR37954.v1
Version V1
Alternate Title View help for Alternate Title
Summary View help for Summary
Funded by the National Institute of Justice, and in collaboration with the Flippen Group, the South Carolina Department of Education, and the districts of Charleston, Georgetown, Greenville, and Richland County Schools, WestEd conducted a randomized controlled trial of violence prevention and a school climate improvement program known as Capturing Kids' Hearts Campus by Design (CKH).
CKH is a skill intensive, systematic process, whole-school intervention, designed to strengthen students' connectedness to school through enhancing protective factors (strong bonds with teachers, clear rules of conduct that are consistently enforced) and targeting modifiable risk factors (inappropriate behavior, poor social coping skills). The CKH training program aims to address the mechanisms and processes of social skills instruction that lead to improved student behavior by promoting skills acquisition (i.e., modeling, coaching, and behavioral rehearsal), enhancing skills performance, removing competing behavior, and facilitating maintenance of social skills.
Components of CKH have been widely used throughout the United States. As of 2013, CKH training has been offered to over 200,000 staff in more than 7,000 schools. Although widely used, the whole package of CKH training and service has not been sufficiently subjected to a rigorous evaluation to assess the effectiveness of the program.
The purpose of this study is to test and evaluate the Capturing Kids' Hearts package of programs and processes offered in the school-wide model, CKH Campus by Design, which includes Capturing Kids' Hearts, Process Champions-Plus, Campus TrAction Pacs, Teen Leadership, CKH Recharged, and Leadership Blueprint.
Citation View help for Citation
Export Citation:
Funding View help for Funding
Subject Terms View help for Subject Terms
Geographic Coverage View help for Geographic Coverage
Smallest Geographic Unit View help for Smallest Geographic Unit
County
Restrictions View help for Restrictions
Access to these data is restricted. Users interested in obtaining these data must complete a Restricted Data Use Agreement, specify the reason for the request, and obtain IRB approval or notice of exemption for their research.
Distributor(s) View help for Distributor(s)
Time Period(s) View help for Time Period(s)
Date of Collection View help for Date of Collection
Study Purpose View help for Study Purpose
The purpose of this study is to conduct an evaluation of the Capturing Kids' Hearts (CKH) programs that will add to the literature on promising programs that improve school safety and climate, reduce bullying, and improve student social competencies. This randomized controlled trial provides the first rigorous test of the CKH package of programs to determine if its promise holds up in a rigorous evaluation.
The study is designed to address the following research question:
What are the impacts of CKH on violence perpetration and victimization, relationship bonds between and among students and teachers, and social competencies?
Specifically,
- Does Capturing Kids' Hearts enhance relationship bonds between and among students and teachers, increase students' social competencies, and reduce aggression and violence?
- Does Capturing Kids' Hearts reduce absenteeism, truancy, suspensions, and disciplinary referrals, and increase language arts and math scores?
Study Design View help for Study Design
To examine the impact of CKH, this study used a true, cluster-randomized, experimental design with repeated measures involving 27 middle schools (grades 6-8) served by four school districts in South Carolina - Charleston County, Georgetown County, Greenville County, and Richland County. Schools were randomly assigned to one of two treatment groups - a treatment group and a wait-listed control group. 15 schools were assigned to the treatment group, while 12 schools were assigned to the control group.
The treatment group received the CKH package, with training provided to all teachers and administrators - including the foundational CKH, Process Champions-Plus, Campus TrAction Pacs, CKH Recharged, and the Leadership Blueprint - and implemented the one-semester Teen Leadership course for grade 6 students. Schools in the control condition operated as "business as usual" during this time and received training following the two-year intervention and data collection period in July 2018.
Data collection started with the administration of an online survey to all school staff in May 2016 to assess baseline measures of school climate. CKH foundational training was delivered to all staff in treatment schools at the start of the school year (August 2016). Select teachers were trained to teach the one-semester Teen Leadership course for grade 6 students in fall 2016. Process Champion training and Campus TrAction workshops and consultations - which aim to enhance the capacity of schools to implement CKH with fidelity - was provided to select staff during the first implementation year. In addition, refresher training for intervention schools (Capturing Kids' Hearts Recharged) was provided at the start of year 2.
All students in grades 6-8 were surveyed in each school, with baseline data collected in September 2016 and at the end of each implementation year (May 2017 and 2018). The same survey instrument was used at baseline and subsequent administrations.
Data Collection Systems for Outcomes
Two data collection systems were implemented to facilitate outcome data collection: an incident tracking system for disciplinary referrals and an online survey data system. Data from these systems were merged using unique ID/access codes assigned to each student.
Incident Tracking
WestEd tracked discipline referrals in both treatment and control schools utilizing the districts' current systems for recording infractions. Site coordinators were responsible for "tagging" incident reports with the student ID codes for students involved in the incident so that the two data collection systems could be linked. Data on disciplinary referrals in the prior academic year (2015-16) were obtained to examine whether and how referrals changed once CKH was implemented. However, because the intervention will likely have an impact on disciplinary referrals, the incident data was not used to estimate program impacts.
Online Survey System
Students (and teachers) were assigned unique access codes that allowed WestEd to link baseline and outcome survey data, and link survey data to student involvement in referred incidents. The student rosters of the relevant grade 6-8 classrooms in participating schools were obtained from the participating school districts at the start of each academic year, and unique access coded were assigned to each student. Only this access code will be entered into the online system to access the survey and provide linkage to follow-up surveys and to involvement in referred incidents.
Sample View help for Sample
Staff Survey Data
All school staff were surveyed. School staff surveys were collected using the same survey instruments at three time periods: spring of 2016, 2017, and 2018. The data collected at the first time point, in May 2016, serves as the baseline staff survey data.
Student Survey Data
In all participating schools, all students with parental consent in grades 6-8 were surveyed over a three-year period. The same survey instrument was administered at all three time points with baseline data collected in spring 2016 and subsequent administration in May 2017 and 2018.
The sample consisted of 22,591 students which had data for at least one of the years.
Archival Record Data
Archival (i.e., administrative) data on students were provided by the districts of Charleston, Georgetown, Greenville, and Richland for the years 2015-16, 2016-17, and 2017-18. The archival data were received separately by district for each year, and the separate data were merged to create a single data file.
To prepare the archival record data for analysis, the archival record data were merged with the student survey data resulting in 25,836 total unique observations. Of those students, 21,242 had both archival data and CKH survey data in at least one year, while 3,245 students had archival data in at least one year but no survey data, and 1,349 students had CKH survey data in at least on year, but not archival data.
Time Method View help for Time Method
Universe View help for Universe
All students (grades 6-8), administrators, and staff on campus who interact with students at 27 middle schools served by four school districts in South Carolina - Charleston County, Georgetown County, Greenville County, and Richland County.
Unit(s) of Observation View help for Unit(s) of Observation
Data Type(s) View help for Data Type(s)
Mode of Data Collection View help for Mode of Data Collection
Description of Variables View help for Description of Variables
Staff Survey Data
The surveys asked staff to report their perceptions of students' connectedness to school, bonds between and among students and school staff, and students' personal and social competencies.
Student Survey Data
The surveys asked students to report information regarding demographics and various experiences at school, such as relationships with adults, connectedness to school, as well as aggressive or violent behaviors.
Archival Record Data
The archival data comprised of student-level demographics, achievement, discipline, and attendance data. Variables obtained from archival data and used as outcomes include the number of total absences, excused absences, unexcused absences, tardies, total suspensions, out-of-school suspensions, in-school suspensions, discipline referrals, as well as test scores for ELA and math assessments.
Two grouping variables were used to identify unique schools. The variable school ID 2 is a unique school identifier that represents the school a student attended in Year 1. The variable school ID 3 is a school identifier that represents the school identified by the school ID 2 variable but represents the school in Year 3 if the school is missing for the school ID 2 variable, but not missing in Year 3.
Response Rates View help for Response Rates
Student survey response rates:
- 72% in 2016
- 76% in 2017
- 86% in 2018
Staff survey response rates:
- 90% in 2016
- 87% in 2017
- 79% in 2018
Presence of Common Scales View help for Presence of Common Scales
Several Likert-type scales were used.
Some of the measures and/or subscales included the California School Climate, Healthy, and Learning Survey (Cal-SCHLS) and the Psychological Sense of School Membership (PSSM) scale (Goodenow, 1993).
HideNotes
The public-use data files in this collection are available for access by the general public. Access does not require affiliation with an ICPSR member institution.
One or more files in this data collection have special restrictions. Restricted data files are not available for direct download from the website; click on the Restricted Data button to learn more.