Whole-school interventions promoting student commitment to school to prevent substance use and violence: a systematic review

Objectives: Whole-school interventions that promote student commitment to school are a promising modality to reduce health inequalities through school-level change; however, evidence for the effectiveness of these interventions in improving policy-relevant health outcomes, such as substance use and violence, has not been comprehensively synthesised. Study design: This was a systematic review and meta-analysis. Methods: We searched 20 databases and a range of other sources to identify randomised trials meeting our intervention de ﬁ nition and reporting substance use and violence outcomes. Extracted effect estimates were meta-analysed using robust variance estimation with random effects, separating effects < 1 year from baseline and effects at or more than 1 year from baseline. Results: We included 18 evaluations with varying risk of bias. Pooled effects suggested signi ﬁ cant impacts on short-term (odds ratio [OR] ¼ 0.85, 95% con ﬁ dence interval [CI] 0.76, 0.96) and long-term (OR ¼ 0.79, 95% CI 0.65, 0.98) violence perpetration, short-term (OR ¼ 0.84, 95% CI 0.72, 0.98) and long-term (OR ¼ 0.85, 95% CI 0.73, 0.99) violence victimisation, and short-term (OR ¼ 0.83, 95% CI 0.70, 0.97) and long-term (OR ¼ 0.79, 95% CI 0.62, 0.998) substance use outcomes, with effects relatively stable between short-term and long-term analyses. Stratifying substance use meta-analyses by type (e.g. smoking, alcohol) did not impact results. All meta-analyses had substantial heterogeneity. Conclusion: Although diverse in content, interventions appear effective with respect to the review outcomes and as a form of universal prevention. Future research should consider contextual contin-gencies in intervention effectiveness, given considerable policy and practice interest in these interventions and the need to support schools in effective decision-making as to intervention choice. ©


Introduction
Whole-school interventions aim to modify the school environment to promote health. 1 There is increasing academic and policy interest in whole-school interventions that promote student commitment to school as a means of addressing complex public health problems as well as improving educational attainment.Interest in such interventions reflects awareness that health education lessons struggle to find a place in school timetables and have patchy results, which tend to dissipate with time.2e5 It also reflects interest in socioecological determinants of health, of which the school environment is one important element. 6If effective, these interventions might represent a pragmatic and efficient means of addressing multiple intercorrelated risk behaviours.
A subset of whole-school interventions aims to promote student commitment to school to prevent outcomes, such as substance use (i.e.tobacco, alcohol and other drugs) and violence, which are important, intercorrelated outcomes 7e10 often associated with disengagement from school. 11,12The theories of change underpinning these interventions postulate that interventions can build student commitment to school and therefore prevent substance use and violence by improving relationships within schools and between schools and local communities, 13 for example, via improving pedagogy, revising school policies, encouraging student volunteering or increasing parental involvement in school.Substance use and violence remain important outcomes to address, given the significant health burdens they create.Harms accruing from substance use in young people are socially stratified and a range of substance use behaviours are linked to poor health and long-term chronic and life-limiting illness.The prevalence, harms and costs of violence among young people mean that addressing this has long been a public health priority. 14,15The most recent evidence suggests that in England, 17% (24% boys vs 9% girls) of young people aged 11, 13 or 15 years report being involved in a physical fight two or more times in the last 12 months. 16he goal of this systematic review was to analyse the effectiveness of whole-school interventions that improve student commitment to school for reducing substance use or violence.

Methods
The review followed existing general criteria for review conduct and reporting of systematic reviews (Centre for Reviews and Dissemination; 17 Preferred Reporting Items for Systematic Reviews and Meta-Analyses 18 ).The protocol was registered with PROSPERO (International Prospective Register of Systematic Reviews) on 14 October 2019 (CRD42019154334).

Inclusion and exclusion criteria
We included interventions aligned with key theoretical constructs in whole-school interventions 19 and operationalised these as interventions including at least one of: modified teaching to increase student engagement in academic learning; enhanced studentestaff relationships; revised school policies with involving students and/or going beyond health or behaviour management policies; encouraged all students to volunteer in the community; or increased parental involvement in school.
We also included intervention trials compared against treatment-as-usual, no-treatment or other active-treatment groups; where children and young people aged between 5 and 18 years, attending mainstream school, were the target of interventions; and where either violence or substance use outcomes were reported.
We excluded studies of interventions that involved health or social and emotional skills curricula without whole-school components; targeted selected students or parents rather than being universal, whole-school interventions; only addressed behaviour management in the classroom or school-wide without addressing student engagement or commitment to school; involved students as peer educators or peer social marketers without students being involved in school policy-or decision-making; or involved revising policies or procedures relating purely to health or behaviour management without student input.We focus on randomised trials in this specific review.
We included violence measures defined as interpersonal physical, emotional or social abuse.Where scales included antisocial behaviour more generally, violence comprised a majority of items.We included substance use defined as tobacco, alcohol or other drugs and considered these both separately and together.

Search and selection
We undertook an extensive search of 20 databases in January 2020 and updated searches in 13 databases in May 2021 (see Supplementary File 1, including search strings used for MED-LINE).We also searched three trials registers and a range of websites, contacted experts and scrutinised reference lists of included papers.Two reviewers piloted screening of successive batches of 50 records, meeting to discuss disagreements and involving a third reviewer where necessary.Once agreement of >90% was reached, remaining references were screened by a single reviewer.Full-text reports were then screened in duplicate and independently.

Extraction and appraisal
Two reviewers extracted data in duplicate and independently on basic study details, design and methods, outcome measures and outcome estimates.Randomised evaluations were appraised using the Cochrane risk of bias tool. 20Domains for sequence generation and allocation concealment related to the robustness of the randomisation used, blinding related to whether evaluators or participants were blinded, complete outcome data captured substantial attrition (>30%) or imbalanced attrition between arms, and no selective outcome reporting related to whether a clear account of all expected and measured outcomes was presented.The domain for clustering related to whether trials accounted for clustering in analysis.The domain for reducing other sources of bias related to whether trial design accounted for imbalance between arms and other effect modifiers via, for example, stratified randomisation or control for baseline factors.

Meta-analysis
We estimated separate models for substance use and violence, using odds ratio as the metric for all analyses with a logistic transformation to convert standardised mean differences.We examined substance use outcomes together in one analysis, as well as separated into smoking tobacco, drinking alcohol, other drug use and any 'omnibus' measures of substance use.We considered violence victimisation, violence perpetration and 'observed' violence separately.We regarded follow-up times of up to 1 year and more than 1 year postbaseline as different outcomes.Where cluster randomised trials did not include appropriate control for clustering, we checked if results were sensitive to an estimate of clustering (e.g. an intra-cluster correlation coefficient, ICC of 0.02).
All analyses were undertaken in Stata v17 (Statacorp, College Station, TX) using robust variance estimation with random effects and an intercorrelation parameter of 0.8 in order to accommodate multiple effect sizes per study.Robust variance estimation is a validated method for including all relevant information from a study where multiple effect sizes are available for a given outcome domain and improves on methods for handling dependent effect sizes, such as selecting one effect size or meta-analysing within studies before combining effects between studies.

Included studies
In total, 126,180 references were identified from the electronic literature searches run in January 2020 (see Fig. 1).Of these 63,438 were identified as duplicates and removed.The updated May 2021 search identified 105,777 results.Of these, 96,068 were duplicates or already retrieved by the earlier search.This left 9709 new references, giving a total of 72,451 references that were screened on title and abstract.We screened 419 references at full text and included 18 randomised trials.Study-level details are presented in Supplementary File 2.

Quality appraisal
Included randomised trials were mostly inadequate in their reporting of key randomisation details, such as sequence generation and allocation concealment (see Supplementary File 1).As is often the case in randomised trials of psychosocial interventions, blinding was an issue in nearly all included trials.More than half the included trials had appropriately complete outcome data.All trials were cluster randomised; only one trial (Going Places Program 21,22 ) did not adequately account for clustering, but analysis results were not sensitive to including an estimate of clustering.

Description of interventions
A total of 17 interventions were evaluated in 18 trials; Positive Action was evaluated twice, in Hawaii 23e25 and in Chicago.26e28 Considered against our inclusion criteria, eight interventions (Aban Aya, 29,30 Cooperative Learning, 31e34 Cyber Friendly Schools, 35 Going Places Program, 21,22 Good School Toolkit, 36,37 Positive Action, 23e28 SEHER, 38,39 whole-of-school intervention 40,41 ) sought to increase commitment by using more engaging learning methods in classrooms; for example, Positive Action sought to achieve this by improving pedagogy especially as related to social-emotional learning, whereas Aban Aya used a student centred, culturally inclusive curriculum.A further 11 interventions (Aban Aya, Cyber Friendly Schools, DASI, 42,43 Friendly Schools Friendly Families, 44 Friendly Schools Transition Programme, 45 Gatehouse Project, 46e48 Good School Toolkit, Learning Together, 49,50 Positive Action, Restorative Practice Intervention, 51 SEHER) sought to improve studenteteacher relationships across the school; for example, by the use of restorative practice and non-punitive discipline (Learning Together, Restorative Practices Intervention, SEHER) or by supporting students to collaborate on groups and committees.Nine interventions (Aban Aya, Cyber Friendly Schools, DASI, DARE Plus, 52,53 Good School Toolkit, Learning Together, Positive Action, SEHER, whole-of-school intervention) included student involvement in school-level policy, rules or other decisions, especially via group policy review.Eight interventions (Aban Aya, DASI, Friendly Schools, 54 Friendly Schools/Cool Kids Taking Control, 55 Friendly Schools Friendly Families, Positive Action, SEHER, whole-of-school intervention) sought to involve parents in school-level decisions or activities.Only one intervention, Positive Action, supported students to volunteer in the community.Violence perpetration Meta-analyses suggested that interventions promoting commitment to school generate small but statistically significant impacts potentially of public health significance in reducing violence perpetration at up to 1 year postbaseline (OR ¼ 0.85, 95% confidence interval [CI] 0.76, 0.96).This analysis drew on 17 effect sizes from eight studies 21,22,30,32,42,44,54,56 and included a moderate amount of heterogeneity with an I 2 of 57.4% (Fig. 2).

Violence victimisation
Meta-analyses suggested that interventions promoting commitment to school generate small but statistically significant and, potentially, public health significant impacts in reducing violence victimisation at up to 1 year postbaseline (OR ¼ 0.84, 95% CI 0.72, 0.98).This analysis drew on 22 effect sizes from eight studies 32,39,42,44,46,53,54,56 and included a substantial amount of heterogeneity with an I 2 of 81.1% (Fig. 3).
Interventions promoting commitment to school also generate small but statistically significant and, potentially, public health significant impacts in reducing violence victimisation at or more than 1 year postbaseline (OR ¼ 0.85, 95% CI 0.73, 0.99).This analysis drew on 71 effect sizes from 11 studies 35e38, 44,46,49e56 and included a substantial amount of heterogeneity with an I 2 of 80.8% (see Supplementary File 1).

Violence observed
Only one outcome evaluation 54 presented evidence for this outcome up to 1 year postbaseline.Three evaluations 23,50,54 presented evidence at or more than 1 year postbaseline.Due to the range of informants and measures for this outcome, we did not undertake meta-analysis.In the short-term, control group students in the control group of the Friendly Schools evaluation 54 were more likely to see someone being bullied (OR ¼ 1.36, 95% CI 1.03,1.81)than intervention group students.This pattern continued at the end of the second school year from baseline (OR ¼ 1.48, 95% CI 1.14, 1.92) and at the end of the third year from baseline (OR ¼ 1.67, 95% CI 1.25, 2.24).In the trial of Learning Together, 15,50 observed violence was measured by student observations of others' aggressive behaviour at 24 and 36 months postbaseline.At 24 months, students receiving Learning Together were not significantly different from students in control schools in reports of observing aggressive behaviour (MD ¼ À0.08, 95% CI À0.18, 0.01).They were, however, significantly different at 36 months (MD ¼ À0.10, 95% CI À0.20, 0, P ¼ 0.049).In the Hawaii trial of Positive Action, 23 observed violence was measured by teacher report when students were in the fifth grade (year 6), corresponding to 3 or 4 years of exposure to the intervention.Teachers reported that Positive Action students engaged in 46% fewer violent behaviours (mean ratio ¼ 0.54, 90% CI 0.30, 0.77).

Discussion
Though diverse in their content and approach, whole-school interventions that aim to promote student commitment to school appear effective with respect to substance use, violence victimisation and violence perpetration.Meta-analyses did not suggest necessarily large effects but did suggest effects that would be public health significant; that is, effective as a form of universal prevention, as included school-based interventions in this review were.This is especially notable, given that activities included in our definition of included interventions primarily aimed to improve student commitment to school instead of, for example, providing psychoeducation relating to substance use and were thus primarily indirect in their impacts, targeting structural and interpersonal mechanisms.
Included interventions fulfilled our inclusion criteria in a range of ways, including foci on restorative practices and studentinvolved or student-led action groups, both of which qualitative evaluations 57,58 have found to be powerful tools for driving schoollevel change.In addition, mediational evidence from several included trials in this review 24,59,60 reinforces the role of improving student commitment to school as a 'realised', rather than theorised, pathway for achieving reductions in violence and substance use.This increases the plausibility of our focus on improving student commitment to school as a relevant target to reduce violence and substance use.
However, despite a consistent pattern of positive effects, all meta-analyses had non-trivial heterogeneity.This was partly by design, given the nature of interventions included was broad.A challenge to the interpretation of our findings is that the number of studies in each meta-analysis precluded any systematic examination of factors that might explain this heterogeneity.Future research should seek to understand what drives this heterogeneity.It is likely that this heterogeneity is driven by a combination of intervention features, contextual factors and the fit of interventions to local contexts.

Bullied at all
Moving forward, interventions might be further optimised by ensuring that they are well theorised.This would help ensure that interventions have the potential to promote student commitment to school by addressing the different mechanisms via which this might happen.These might include mechanisms, including reframing provision on measured and/or expressed student needs, improving relationships between staff and students through processes such as joint decision-making bodies or restorative practices, eroding boundaries between academic learning and broader development via innovative teaching methods; and eroding boundaries between schools and their local communities by students volunteering in communities and parents or other community members volunteering in schools.In the last instance, it is clear that this is an area for improving student commitment to school that remains underevaluated, given the few trials including this component.
Interventions should be co-theorised with school staff and students to ensure that theories of change and intervention materials use terminology, which is acceptable to schools, informed by our stakeholder consultation.Effectiveness is also likely to be improved by ensuring that interventions address not only 'upstream' institutional influences on violence or substance use, such as school environment, but also more 'downstream' individual influences, such as student knowledge, skills and norms.Interventions might also be optimised by designing them to be maximally implementable, for example, via ensuring good guidance, senior leadership, providing local needs data and developing collaborative coordinating bodies.
Key strengths of our review were the focus on theoretically driven inclusion criteria, the use of an exhaustive search approach and the statistical methods used to combine effect sizes.However, the risk of missed studies is always present, and we were unable to consider publication bias, given that methods for this are not welldeveloped in the context of robust variance estimation metaanalysis.Risk of bias in the included studies also poses a challenge to confidence in study findings.
In conclusion, whole-school interventions promoting student commitment to school are likely effective in reducing substance use and preventing violence victimisation and perpetration.Given the popularity and policy interest of this intervention approach, these findings are encouraging.However, the substantial heterogeneity in meta-analyses and the difficulty in accounting for this warrant further research to support decision-making in intervention implementation.