Effect of Summer Holiday Programs on Children’s Mental Health and Well-Being: Systematic Review and Meta-Analysis

Poor youth mental health is an area of global concern. Summer holiday programs may provide environments that support mental health when the structures and supports of school are not available. The aim of this review was to determine the effectiveness of summer holiday programs in improving the mental health, social–emotional well-being, and cognitive (non-academic) outcomes of children and adolescents. Studies of summer holiday programs for school-aged children (5–18 years) were included if they measured any mental, socio-emotional or cognitive (non-academic) outcome. Studies were excluded if they were published prior to 2000, targeted clinical populations or lasted less than five days. Six databases were searched (April 2023). Risk of bias was assessed using the PEDro tool. Study outcomes were grouped according to three main constructs: mental health (psychological well-being, anxiety, depression, distress, and self-perception including self-esteem, self-worth, self-concept, confidence, and competence); social–emotional well-being (behavior and social skills, e.g., communication, bullying, conflict resolution, empathy, and social skills); and cognitive function (memory, selective attention, and executive function). A fourth “other” group captured substance use, personality traits, character skills, and values. Effect sizes were calculated as the standardized mean difference between pre- and post-intervention scores. The synthesis involved a random-effects meta-analysis (presented in forest plots), where possible, with the remaining outcomes narratively synthesized. Twenty-six studies (n = 6812 participants) were included. The results of the meta-analysis suggested that summer programs showed a statistically non-significant trend toward reducing symptoms of anxiety and depression (k = 2 studies, SMD = −0.17, 95% CI −2.94, 2.60), psychological distress (k = 2 studies, SMD −0.46, 95% CI –1.71, 0.79), and no effect on self-esteem (k = 6 studies, SMD = 0.02, 95% CI −0.02, 0.06) or self-worth (k = 3 studies, SMD = 0.05, 95% CI 0.00, 0.11). Narrative syntheses indicated a pattern toward improvements in general mental health, self-perception, social–emotional outcomes, and cognition. Studies were generally small, with a high risk of bias. Summer holiday programs for children and adolescents show trends toward improving mental, social, emotional, and cognitive outcomes. Programs targeting disadvantaged children showed stronger patterns of improvement related to mental health and self-perception than programs targeting the general population. While effect sizes are small to negligible, they consistently indicate improvements. Summer programs present a promising avenue to promote mental health in children; however, further rigorously designed, clearly reported control-group studies are required to more fully understand their effects.


Background
A healthy childhood lays the foundation for lifelong health.Mental health, as defined by the World Health Organization, encompasses a state of well-being in which individuals recognize their abilities, can manage normal life stresses, work productively, and contribute to their community [1].The dual-factor model of mental health suggests that true mental health includes both the absence of mental illness (e.g., symptoms or disorders) and the presence of positive psychological well-being and emphasizes the importance of addressing both negative symptoms and fostering positive aspects such as life satisfaction, happiness, and personal growth [2,3].Mental health is an important aspect of overall health, and it is critical to healthy social and emotional development, and problem-solving and coping skills [4].Positive mental health allows children to more fully realize their academic potential [5].Conversely, mental health disorders such as anxiety and depression reduce children's quality of life across domains of psychological, physical, and social well-being [6,7].
Poor mental health of children and adolescents is a serious area of concern globally, as depression, anxiety, and behavioral disorders are the leading cause of illness and disability in 10-19-year-olds [8].The symptoms of poor mental health are not always clearly visible.They may manifest internally, such as through anxiety, depression, and psychological distress; or externally, such as through aggression and hyperactivity.Research indicates that internalizing symptoms are generally more prevalent in children and adolescents than externalizing symptoms [9].Compared to children with good mental health, children that suffer a mental health condition are six times more likely to suffer a mental health condition in adulthood [10] and go on to earn significantly lower income [11].
Environmental factors play an important role in the mental health of young people [12], and children spend a large proportion of their time in the school environment.School supports the development of positive mental health through consistent engagement in educational activities that positively impact students' cognitive development [13][14][15].Participation in classroom learning also requires children to practice executive functions, including memory, selective attention, and inhibitory control [16].Engagement in enjoyable school-based and extra-curricular activities can be rewarding in and of itself, providing opportunities to learn new skills and achieve mastery in meaningful areas, thus increasing self-worth.Meanwhile, tailored strategies can help enhance a more stable, intrinsic sense of self-acceptance, which is referred to as self-esteem [17].The social interaction and structured routines of the school day also help children develop social and emotional skills.Teachers can further provide emotional support and set clear expectations and boundaries that help children feel safe, ease anxiety, and develop a sense of belonging [18].Teachers can also identify students struggling with the social and cognitive demands of schooling, thus allowing for early intervention.School provides opportunities to meet peers and form friendships, thus playing an important role in social development [4,19] that can protect against social maladjustment [20].Furthermore, schools help children to display more favorable behavior patterns, including more time in physical activity and less time engaged in sedentary behaviors, like recreational screen-use [21,22].This can positively support children's mental health [23,24].For example, decreasing screen use helps to improve sleep habits, supporting, in turn, good mental health [12], while increasing physical activity in children has been shown to reduce anxiety and depression and improve self-esteem and cognitive function [25].
While school holidays offer a reprieve to students and families from the demands of the school term, extended periods of time spent away from the school setting reduces the supports provided by school, the absence of which may negatively impact children's mental health.Some evidence suggests that negative experiences over the summer holidays (like loneliness or food insecurity) can increase anxiety and negatively influence children's mental health, with disadvantaged children affected to a greater extent [26,27].Negative peer-group interactions over summer have been linked with increases in anti-social behaviors (bullying and victimization) [28], while reduced peer contact can reduce children's enjoyment of, and therefore time spent in, physical activity [29].
Summer programs have proven benefits in a range of other outcomes for children, and it is feasible that some of the beneficial features of summer programs also enhance mental, emotional, and social well-being.Previous reviews of summer programs looking at academic outcomes found improvements in mathematics and reading achievement [30][31][32], while reviews of health outcomes have found moderate reductions in sedentary behavior and small increases in moderate-to-vigorous physical activity [33,34].To date, no reviews have synthesized the evidence regarding the impact of summer programs on social, emotional, cognitive, and psychological outcomes.Thus, the objective of this systematic review was to determine the effectiveness of summer holiday programs in improving the mental health, social-emotional well-being, and cognitive (non-academic) outcomes of children and adolescents.The review questions were as follows: 1.
What are the effects of summer holiday programs on children and adolescents' mental health, well-being, and social-emotional and cognitive function?2.
Do the effects differ based on child characteristics (e.g., socioeconomic disadvantage and age)? 3.
Do the effects differ based on program characteristics (e.g., program content and duration)?

Protocol and Registration
A systematic review protocol was registered prospectively with PROSPERO (registration number: CRD42023409799) [35] and reported following the PRISMA 2020 guidelines [36].

Eligibility Criteria
Participants in the included studies were school-aged children (5-18 years) participating in a summer program (intervention) of at least five days' duration that was exclusively delivered during the summer period (i.e., not including afterschool or other holiday programs).Primary outcomes included any mental health (e.g., psychological well-being, depression, and anxiety), socio-emotional (e.g., social connectedness, sadness, loneliness, appropriate behavior), or cognitive (e.g., working memory and executive function) outcome.Academic outcomes were not included, because they have been the focus of other reviews [30,31].Studies were excluded if they were published before 2000 or if they targeted clinical populations (e.g., cancer and diabetes), special needs (learning or intellectual disabilities), or gifted/especially talented children.Study designs included both experimental studies with a control group, receiving "summer as usual" or a different summer program, and quasi-experimental (pre-post-intervention measures with no control group).No language limits were set.Full criteria are given in Supplementary File S1.

Information Sources and Search Strategy
Six databases were searched for peer-reviewed original articles: Embase, MEDLINE, JBI, PsychINFO (via OVID), ERIC, and Scopus (April 2023).A broad search strategy [37] was developed with an academic librarian, focusing on population and context terms (detailed in Supplementary File S2).Reference lists of included studies were searched using Citationchaser (version 0.0.3,Haddaway, Grainger and Gray 2021), and the corresponding authors of included studies were contacted to identify further relevant studies.

Selection Process
Search results were imported into Endnote 20 (Clarivate Analytics, Philadelphia, PA, USA), where duplicates were removed before being imported into ASReview [38] (version 1.1.1,ASReview LAB developers, Utrecht, The Netherlands), where the five same relevant studies identified during preliminary searches were used to train the search.Two independent reviewers (EE and BS) completed title and abstract screening, which was stopped once 10% of the total studies were screened and 100 consecutive, irrelevant titles were encountered.Full-text review was then completed using Covidence (Veritas Health Innovation, Melbourne, Australia).Screening was completed by EE and a second independent reviewer (TW or ZR), and disagreements were resolved through discussion.

Data Collection Process and Data Items
Charting tables were developed by the authorship team and piloted prior to use (example in Supplementary File S3).Fields for data extraction included study features (design, geographical location, and participant number); participant demographics (age, gender, and socioeconomic status); program design (structure, objectives, and environment); findings (measures of effect, certainty, and statistical significance, measurement tool, and timing of evaluation); and implementation outcomes (adverse events and attendance).Data extraction and risk-of-bias assessment were completed in duplicate using Covidence by two independent reviewers (EE, TW, ZW, JG, CS, and AG), with discrepancies resolved through discussion until consensus was reached.

Study Risk-of-Bias Assessment
Risk-of-bias assessment was completed in duplicate using Covidence by two independent reviewers (EE, TW, ZW, JG, CS, and AG), by employing the PEDro Risk of Bias Tool, recognized for its validity and reliability [39].The highest possible PEDro score of 10 signifies minimal risk of bias, with categories used to assign studies as low, moderate, and high risk of bias [39].However, the impracticality of blinding people delivering the program meant that this item was omitted, and the maximum score was revised to nine [40].The interpretation of scores in this paper was 7-9 (low risk of bias), 5-6 (moderate risk), and 0-4 (high risk of bias).

Effect Measures and Synthesis Methods
Studies were grouped with the assistance of a behavioral scientist according to three main constructs: mental health, social-emotional well-being, and cognition.Mental health included psychological well-being (e.g., anxiety, depression, distress, well-being) and self-perception (e.g., self-esteem, self-worth, self-concept, confidence, and competence).Social-emotional well-being covered behavior and social skills (e.g., communication, bullying, conflict resolution, empathy, and social skills), while cognitive function included outcomes related to memory, selective attention, and executive function.Outcomes falling outside of these main constructs were presented separately as "other" and included health-related behaviors (substance use), personality traits/character skills, and spirituality/values.Effect sizes were calculated as the standardized mean difference from pre-and post-intervention scores and were interpreted as small (0.2), moderate (0.5), or large (0.8) [41].Synthesis involved meta-analysis where possible, with the remaining outcomes narratively synthesized.Patterns were examined for each outcome when at least three outcomes were available.

Meta-Analysis
Meta-analyses were conducted for outcomes with sufficiently homogeneous data from at least two studies.Study authors were contacted for missing data.A randomeffects model was used for meta-analyses, considering heterogeneity in study design and outcomes reported.Meta-analyses were conducted using R software (version 4.3.1)with the meta, metafor, and dmetar packages [42][43][44][45][46]. Standardized mean differences and associated confidence intervals were used to indicate the size and precision of the effect estimate, with restricted maximum-likelihood and Knapp-Hartung adjustments [46].Meta-analysis results were presented using forest plots.Interpretation of results considered the direction, size, and precision of the effect with statistical significance set at p = 0.05.Patterns across studies were also considered.The I 2 statistic was used to evaluate statistical heterogeneity, with I 2 > 50% indicating substantial heterogeneity [41].Robustness of results (effect size, statistical significance, and heterogeneity) was explored via a sensitivity analysis, using leave-one-out analyses, and analyses were repeated, excluding studies possessing a high risk of bias.Publication-bias analysis was omitted due to the small study set [47].

Narrative Synthesis
Where meta-analysis was not possible, a narrative synthesis was conducted according to the synthesis without meta-analysis (SWiM) guidelines [48].Standardized measures of effect (e.g., odds ratio; mean difference; and standardized mean difference with statistical significance determined by the original study, usually p = 0.05) were used to code outcomes as improved, unchanged, or declined.To indicate the magnitude of intervention effect, a common effect metric was used: SMD (Cohen's D, Hedges' g) was prioritized, and when not available, it was calculated from baseline and post-intervention scores or converted from other effect metrics, such as eta-squared [49].

Certainty of Evidence
The overall evidence for each outcome was graded based on the Oxford Centre for Evidence Based Medicine's (OCEBM) 2011 Levels of Evidence [50,51].This approach evaluates an intervention's effectiveness using a hierarchy of evidence, considering study design, effect size, and consistency of effects across studies.First, studies' designs and effects were rated on a scale from one (highest) to five (lowest).Then, the evidence for each outcome was graded A-D: Grade A was assigned to outcomes with consistent level 1 studies (randomized controlled trials); Grade B for consistent level 2 studies (nonrandomized controlled studies); Grade C for consistent level 3 studies (non-randomized, controlled studies, and for this review, repeat-measure non-controlled studies); and Grade D for inconsistencies in the direction of effect.Grades were adjusted down for high risk of bias and inconsistency between studies and adjusted up for large and consistent effect sizes.A minus sign indicated considerable clinical or statistical heterogeneity (I 2 ≥ 50%).

Deviation from Registered Protocol
There were limited numbers of experimental trials with control groups; therefore, the original inclusion criterion of only experimental controlled study designs was broadened to include quasi-experimental designs (e.g., pre-post-intervention measures with participants acting as their own controls).Subgroup analyses based on child (age and SES) or program (duration and format) characteristics were undertaken narratively.

Study Selection
Database searching yielded 4226 studies, with reference-list searching adding 1014 more.After the removal of duplicates, 4347 titles and abstracts were screened with 181 full texts retrieved.Finally, 26 studies involving 6812 participants were included (Figure 1).A full list of studies excluded at the full-text screening stage and their reason for exclusion is presented in Supplementary File S4.

Study Characteristics
Included studies are summarized in Table 1.A summary of program and participant characteristics is presented in Table 2 (thematic coding is described in Supplementary File S5).Three studies were randomized controlled trials (RCTs) (k = 2 cluster RCTs [52,53], k = 1 individual RCT) [54], three were non-randomized controlled trials [55][56][57], and the remaining studies were within-subject, repeated-measures of groups all receiving an intervention (k = 2 studies had three groups [58,59], k = 1 study had two groups [60], and k = 17 studies had a single intervention group [61][62][63][64][65][66][67][68][69][70][71][72][73][74][75][76]).Sixty-two percent of the studies were from the United States (k = 16), with two studies from India and single studies from Australia, Canada, Italy, Japan, Spain, Switzerland, Turkey, and the United Kingdom.Ten of the eleven interventions that targeted children disadvantaged by low family SES or race were conducted in the US [52,59,63,[68][69][70][71]74,75], with the remaining study conducted in Spain [64].In 42% of studies (11/26 studies), program attendance was funded on behalf of the participants (e.g., trial funding, community organization, and scholarships) [52][53][54]58,62,64,68,69,71,73,75]; in two cases, the participants paid in full [55,72], and in another three studies, participants paid part of the fees with scholarships or third-party funding paying the gap [67,70,76].Only 38% of studies (10/26) reported attendance levels, and most of these (k = 7) reported high attendance (categorized as ≥66% of the sessions) [54,63,65,[69][70][71]73]. One study reported moderate attendance (51-65% of the sessions) [67], one reported low attendance (≤50% of the sessions) [77], and another reported different low-to-high attendance levels across three different sites [52].One study reported no adverse events [73], while all other studies did not mention adverse events.Thus, no studies described experiencing adverse events.A diverse range of outcomes were reported across the studies, and a brief definition of each broad construct is provided here for readers unfamiliar with this literature.First, "mental health" is used here to describe critical components of psychological well-being, encompassing symptoms of anxiety, depression, and psychological distress, as well as items related to self-perception, including self-esteem (the overall sense of value or worth) and self-worth (value derived from specific achievements or attributes) [78][79][80]."Socialemotional" outcomes relate to one's ability to comprehend and regulate emotions, form and sustain healthy relationships, and effectively interact within social contexts.This includes key social skills, such as communication, cooperation, assertion, problem-solving, responsibility, and self-control [81]."Cognitive" outcomes measure mental processes involved in gaining knowledge and understanding, and they encompass a range of cognitive functions, including memory, attention, executive function, and decision-making [82].
The content of interventions was primarily based around general enrichment (recreation, sports, and play-based programs), with some also targeted toward addressing specific mental health and well-being factors (e.g., overall mental health [53,68,74], stress reduction [59], development of social skills [62,64,77], or reduction in risk behaviors [53,54,66]).Some programs solely targeted physical health [55,75] (e.g., physical activity and weight control), while others combined physical and mental health components [52,76].Two broad studies covering multiple sites and programs did not report the program content [56,73].

Meta-Analysis
Figure 2 presents the meta-analysis results for each construct, and Supplementary File S6 provides the full results, along with sensitivity analyses.

Mental Health
A meta-analysis was conducted on combined anxiety and depression scores (n = 53) [59,74] (one study reported only the aggregate anxiety and depression scores [74]; thus, anxiety and depression scores had to be combined).The result indicated that summer programs had a small but statistically non-significant effect in reducing symptoms of anxiety and depression (SMD = −0.17,95% CI −2.94, 2.60, p = 0.0.58,I 2 = 13%).Both studies showed a trend toward improvement.Certainty of evidence: Grade C.
A meta-analysis was conducted on psychological distress in two studies (n = 56 participants) [59,69].The result indicated that summer programs had a small/moderate but statistically non-significant effect in reducing symptoms of psychological distress (SMD −0.46, 95%CI -1.71, 0.79, p = 0.26, I 2 = 51%).A sensitivity analysis identified Levy et al. (2020) [59] as an influential case, and when omitted from the analysis, the effect size reduced to small and remained statistically non-significant.All studies showed a trend toward an improvement.Certainty of evidence: Grade C.
A narrative synthesis was conducted on mental health data from four studies (n = 714) [52,54,58,65].Two studies measured general mental health and found either a small improvement [65] or no change [54].Two measured a positive/negative affect: one found improvements [52], and the other (which investigated various settings, curricula, and facilitation strategies across three different intervention programs) also measured life satisfaction and found mixed results for all outcomes [58].Overall, there was a trend toward improvements in general mental health.Certainty of evidence: Grade C-.

Meta-Analysis
Figure 2 presents the meta-analysis results for each construct, and Supplementary File S6 provides the full results, along with sensitivity analyses.

Mental Health
A meta-analysis was conducted on combined anxiety and depression scores (n = 53) [59,74] (one study reported only the aggregate anxiety and depression scores [74]; thus, anxiety and depression scores had to be combined).The result indicated that summer programs had a small but statistically non-significant effect in reducing symptoms of anxiety and depression (SMD = −0.17,95% CI −2.94, 2.60, p = 0.0.58,I 2 = 13%).Both studies showed a trend toward improvement.Certainty of evidence: Grade C.
A meta-analysis was conducted on psychological distress in two studies (n = 56 participants) [59,69].The result indicated that summer programs had a small/moderate but
A meta-analysis was conducted for self-worth, based on three studies (n = 424 participants) [67,75,76].The results indicated that summer programs had no effect on self-worth (SMD = 0.05, 95%CI 0.00, 0.11, p = 0.05, I 2 = 0%).The direction of effect was consistent across studies; however, the effect sizes were negligible.Overall, the effect of summer programs on self-worth is negligible.Certainty of evidence: Grade C.

Social-Emotional Well-Being
A narrative synthesis was conducted on social-emotional outcomes from fourteen studies (n = 4771 participants) [54,58,[61][62][63][64][66][67][68]71,73,74,76,77].For emotional and behavioral outcomes, an improvement was found in self-control [71], but no change was found in externalizing behaviors [71] or behavioral conduct [76].Mixed findings were also found when teachers simultaneously rated children's behaviors as deteriorating while parents rated them as improved [68].Regarding communication skills, a single study was available that was suggestive of an overall improvement [77].For pro-social behaviors, improvements were seen in empathy [64], social skills [63,73], social acceptance [76], social competence [67], relationships and coping with problems [66], and aspects of conflict resolution [62].Other studies found no effect on connection, caring [74], social competence, or belonging [61].One study had mixed results for social skills across intervention groups that received different curriculum and facilitation strategies [58].A final study looked at bullying and found improvements in victimization, but no change in perpetration [54].There was an overall trend toward improvements in social-emotional well-being across the heterogeneous measures of social-emotional well-being.Certainty of evidence: Grade C-.

Cognition
A narrative synthesis was conducted on cognitive outcomes, including memory [57,60] and executive function [58] (n = 274).All three studies found improvements, with effects ranging from small to large.Certainty of evidence: Grade C.

Other Outcomes
The remaining outcomes included health-related behaviors (substance use, k = 2 RCTs), personal attributes, and character skills (k = 3 studies) and values (k = 2 studies).For substance use, findings were conflicting, with one finding a small improvement [53] and the other reporting worsening [54].For personal attributes and character skills, one study each found improvements in physical and thinking skills (related to awareness and enjoyment of the environment) [73] and altruism [56], while two other studies found no changes in "horsemanship attributes" (which included responsibility and motivation) [72] or mindfulness [69].Certainty of evidence: Grade D-.

Subgroup Analysis
Subgroup analyses were performed using count-based methods considering participant characteristics (disadvantage and age) and program features (duration, format, content, funding, and daily contact hours) for each outcome category.The results are detailed in Supplementary File S7.

Participant Characteristics
A subgroup analysis based on disadvantage was conducted on all 26 studies.Studies that reported outcomes exclusively disadvantaged populations (i.e., low SES or racial minority, n = 18 outcomes) were compared to those from populations not identified as disadvantaged (i.e., mixed SES groups, SES/race not reported, n = 25 outcomes).Disadvantaged children showed more improvements in self-perception, with a trend toward improvements in mental health.Non-disadvantaged studies showed clearer improvements in social-emotional outcomes.A subgroup analysis based on participant age was conducted according to broad school categories (described previously, detailed in Supplementary File S5): primary school (n = 15 outcomes), middle school (n = 13 outcomes), and high school (n = 6 outcomes).There were no clear patterns based on children's age.
A subgroup analysis was conducted based on the funding model for children's attendance.Studies where children's attendance was funded in part or full by the child's family (n = 8 outcomes) [55,67,70,72,76] was compared to programs where children's attendance was externally funded (e.g., by the research trial, n = 18 outcomes) [52][53][54]58,62,64,69,73,75].Family-funded programs appeared to have clearer social-emotional improvements than programs for which students did not pay.

Discussion
This systematic review aimed to determine the effectiveness of summer holiday programs in improving the mental health, social-emotional well-being, and cognitive (nonacademic) outcomes of children and adolescents.It included 26 trials, involving a total of 6812 children.The results revealed that previous studies have measured a wide variety of different mental, social, and cognitive outcomes, though the evidence base for any particular outcome is quite small.The results from the meta-analysis revealed that, while the directions of effects were generally favorable, effect sizes were small and not statistically significant for mental health outcomes (anxiety/depression and distress) and negligible for self-perception outcomes (self-esteem and self-worth).The results from the narrative synthesis indicated trends toward improvements in general measures of mental health, self-perception, social-emotional well-being, and cognition.Disadvantaged populations showed greater improvement in mental health and self-perception compared to non-disadvantaged populations.
First, we can explore how these programs compare to mental health-promotion programs in other settings.A review of systematic reviews (without meta-analysis) of mental health-promotion interventions for children (up to the age of nineteen) found that school-based interventions improved broad constructs of general mental health but did not change problematic behaviors, such as delinquency, conduct disorder, or school nonattendance [83].More recently, a meta-analysis of school-based interventions that specifically targeted stress, anxiety, and depression found moderate (d = 0.62) improvements in depressive symptoms and no change in anxiety [84].Targeted programs showed larger effects than universal programs.The effects found in our review are in the same direction as the reviews of school-based interventions, but they are smaller in size.This is most likely because summer program interventions are much shorter than school-based interventions, which can continue for a year or more [85].Also, the studies included in our review were small and likely underpowered to reveal changes.
It is also important to consider that, in the context of the summer holidays, a lack of deterioration in outcomes may, in fact, indicate a positive finding.A vast majority of the included studies were pre-or post-intervention design, and while there were inconsistencies in the size of the effects across the outcomes, the directions of effects were almost never in the direction of deterioration.These findings should be overlaid on observational research that suggests that numerous outcomes decline during the summer holidays [86]; negative experiences over summer, like social isolation, hunger, and decreased physical activity, can have unfavorable consequences for children's mental well-being, particularly for disadvantaged children [26].Thus, a lack of decline may indicate that the summer programs actually did have a protective effect for children's mental well-being.However, until there are high-quality studies with control group design, this uncertainty will remain unanswered.
The subgroup analysis based on participant characteristics suggested greater improvement in mental health and self-perception in disadvantaged populations compared to non-disadvantaged populations.This may be because children and adolescents from disadvantaged backgrounds are more likely to suffer from poor mental health [87] and, therefore, have more scope to improve over the course of the intervention than children already scoring well at baseline.This was the case for Thurber and colleagues [73] in their study of over five thousand children attending one of 80 accredited summer camps across the United States (n = 3395 completed both baseline and immediate post-camp measures).They found that children with the lowest scores pre-camp made the greatest gains over the course of the summer program.This could reflect a sharper contrast between the out-of-school environments typically experienced by disadvantaged children compared to non-disadvantaged children and, therefore, the greater relative benefits gained from being within the supportive networks and environments of summer programs.
The subgroup analysis based on program characteristics suggested a somewhat surprising pattern.While targeted programs appeared better at improving social-emotional outcomes than general programs, the reverse was the case for mental health and cognitive outcomes, where general programs seemed more effective.When we explored this finding further, we found that the targeted programs tended to recruit populations living with a greater number of risk factors for mental health issues [88], such as children living with mentally ill parents [66] or in homeless shelters [68], as well as disadvantaged populations (racial minority and low socioeconomic status).Taken together, our findings could indicate that social-emotional outcomes are more amenable to the interventions offered, while mental health outcomes are harder to change in the face of persistent structural barriers outside the scope of the summer program.Conversely, our findings may highlight the value of play and physical activity in enhancing youth mental health, as the general programs frequently focused on sports, outdoor recreation, yoga, and increasing physical activity.
It is important to acknowledge the study's strengths and limitations.Importantly, this is the first review to explore the effect of summer programs on children's mental, emotional, or social well-being.This review used rigorous systematic review methodology, including a highly comprehensive search strategy, and used a meta-analysis to synthesize data.It also considered a wide range of mental, emotional, and social outcomes.However, the review has a number of limitations, mostly related to the current state of the evidence base on this topic.The majority of included studies had small sample sizes and were rated as having a high risk of bias.Less than a quarter of the studies used a controlled design, which seems particularly prudent here, where previous observational studies have demonstrated clear time effects.While many outcomes were measured, there were inconsistencies in the use and reporting of validated outcome measures, with some studies using self-designed questionnaires.The majority of the studies were from the United States, limiting the generalizability to other contexts.While this limits our ability to draw strong conclusions from these data, there are still valuable insights to be gleaned on the role that summer holiday programs could play in improving the mental health of young people.
There are several important implications from this review.First, it builds on our understanding of how summer programs can impact children's health in a holistic sense.Summer programs have demonstrated effectiveness in improving academic outcomes including reading, writing [31], and mathematics [89], and physical health outcomes, including reducing sedentary behavior by increasing moderate-to-vigorous physical activity [55,90,91].The current review highlights the quality and design issues of studies of summer programs targeting mental-emotional well-being.The same rigor that has been used to investigate health and academic outcomes should be adopted in future studies to explore the effect that summer programs have on mental and emotional well-being outcomes.Further research using rigorously designed and conducted RCTs is needed to test the effectiveness of specific mental health interventions delivered within (or as) summer holiday programs.Due to a lack of data on the mental health challenges children face and their "natural" trajectory over summer, control-group data will be vital to further help us understand the role that summer programs play in children's overall mental health and well-being.Future intervention studies should clearly describe the components and structure of their interventions and use consistent, validated outcome measures to detect changes in mental, social, and emotional outcomes so that the most effective program structure and content can be identified.
Summer programs may form part of a broader mental health-promotion strategy.Summer programs could complement school-based interventions by providing further small gains, or at least preventing regression, during a potentially high-risk time for declines.This could lead to cumulative effects when interventions in school and summer settings are run side by side [89].But in order to improve mental health outcomes for entire populations, interventions need to be delivered at scale in real-world settings, a task that has proven exceptionally difficult thus far [85].Taking an implementation science approach to future research is necessary and involves designing or adapting programs into potentially suitable large-scale delivery models.
There are policy implications for these findings.Ensuring equitable access to mental health interventions is crucial so that the populations in need (e.g., disadvantaged families, those living with a greater number of risk factors for mental health disorders) are able to access them.Some have highlighted the need for greater cross-disciplinary collaboration in the design and implementation of trials, as such cross-disciplinary collaboration is currently hindered by existing policies and practices related to training, trial funding, and publishing [85].Addressing these barriers requires concerted efforts to foster collaboration across various disciplines, ensuring comprehensive and inclusive mental health interventions.

Figure 1 .
Figure 1.PRISMA flow diagram of study identification, screening, and inclusion.

Figure 1 .
Figure 1.PRISMA flow diagram of study identification, screening, and inclusion.

Figure 2 .
Figure 2. Meta-analysis results for anxiety and depression, distress, and self-esteem and selfworth.

Figure 2 .
Figure 2. Meta-analysis results for anxiety and depression, distress, and self-esteem and self-worth.

Table 1 .
Characteristics of included studies.
Intervention description: Goal: the skills, outcomes, or behaviors targeted by the program.Description: curriculum and delivery features of the program.Dose: format (day/residential), number of program hours per day, days per week, and total week duration.Key: ACA, American Camp Association; CATCH, Coordinated Approaches to Child Health; COPE, Creating Opportunities for Personal Empowerment; CHAMPS, Children and Mentally Ill Parents; h, hours; K, kindergarten; NERF, Nutrition Education Recreation and Fitness (camp); OCEBM, Oxford Centre for Evidence Based Medicine Levels of Evidence (2011 (Levels A-D)) [50]; PA, physical activity; PE, physical education; RCT, randomized controlled trial; RN, registered nurse; SES, socioeconomic status; SFSP, Summer Food Service Program; STARS, Start Taking Alcohol Risks Seriously; USA, United States; USDA, United States Department of Agriculture.

Table 2 .
Summary of participant and program characteristics.

Table 2 .
Cont.NR: not reported.W: weeks.Participant characteristics.Age: range and mean values for the majority of participants.Primary school, kindergarten/reception to grade six (ages five to 11 years); middle school, grades 7-9 (ages 12-14 years); highschool students, grades 10-12 (ages 15 years and older).SES: socioeconomic status.Program characteristics.Format: residential (overnight stays), day (daytime only programs run outside the home), mixed (more than one setting).Setting: community (e.g., public spaces, non-profit community organizations), school (educational settings inc.schools and universities).Content: general (enrichment and recreation-based programs).Cost: free (no cost for participants), funded (payment made on behalf of the participants, e.g., trial funding), paid (participants paid attendance fees).Daily contact, contact hours per session: full day (7+ hours and residential programs), school day (6-7 h), half day (2-6 h), sessional (2 h or less).Intensity, frequency of sessions: occasional, less than once per week (e.g., home-based program with formal contact sessions at the start and end of the program).