Pediatric sport-related concussion education: Effectiveness and long-term retention of the head safety in youth sports (HSYS) program for youth athletes aged 11–16

Objective: The goals of this study are to evaluate the effectiveness of the curriculum for youth athletes and determine long-term retention in those who have previously participated. Design: Prospective cohort study. Setting: Middle schools. Participants: 887 male and female sixththrough eighth-grade Physical Education students, ranging from ages 11 to 16 entered and 858 students finished the HSYS Program. Subjects (n = 29) were excluded if they did not complete their pre-workshop or post-workshop test. Interventions: The program is designed to be a 35–45 min workshop. Main Outcome Measures: Tests were administered before and after workshop participation to evaluate knowledge of concussion and response to head injury. Results: The aggregated pre-workshop and post-workshop mean test scores for participants from all five middle schools were 68 ± 2% and 85 ± 3%, respectively, which demonstrated significant improvement in test scores after participation in the HSYS Program (paired t-test, p < 0.0001). Analyzed individually, each school (MS1–5) demonstrated significant improvement between preand post-workshop mean test scores (paired t-test, p < 0.0001). On evaluation of longterm retention of the HSYS Program, students (n = 78) from MS3 who had previously participated in the program scored 7% higher than those students (n = 145) who *Corresponding author: Ross-Jordon S. Elliott, Department of Neurology, Medical Arts & Research Center–MARC, San Antonio, Texas 78229, USA E-mail: ELLIOTTR@livemail.uthscsa.edu


ABOUT THE AUTHORS
Regarding key research activities, our group is currently exploring the effectiveness of the HSYS program utilizing different modalities versus our control data. We hope to have exciting results that affect how a significant number of youth athletes acquire knowledge concerning head safety.
Regarding how this paper relates to current issues: the HSYS program is important because American Football is one of the most popular sports in the United States, with over 2 million children and adolescents participating in tackle football to date. Injury prevention is paramount in order to decrease the incidence of head injuries and we believe education is one of the most effective and important methods to accomplish this task. Thus, the HSYS program and other research initiatives and programs with a similar paradigm are necessary in order to protect this vulnerable population from the burden of head trauma.

PUBLIC INTEREST STATEMENT
This paper describes the Head Safety in Youth Sports (HSYS) Program, a multifactorial curriculum that effectively educates students and provides long-term retention regarding concussion and head safety. Of note, in the United States, 65% of sport-related traumatic brain injuries occur in patients, 5 to 18 years of age. We believe that the HSYS program engages youth athletes in understanding the significance of head safety and creates an opportunity to initiate this discussion preemptively, and influence athletic behavior to improve head safety.

Introduction
The third International Conference on Concussion in Sport defines sports-related concussion as a "complex pathophysiological process affecting the brain, induced by traumatic biomechanical forces" (Marshall, 2012). The Centers for Disease Control and Prevention estimates an annual occurrence of 1.7 million traumatic brain injuries (TBI) in the United States (Centers for Disease Control and Prevention, 2011). With a growing number of youth participating in organized and other sports, reports of athletes sustaining this type of injury are increasing. Data from the National Electronic Injury Surveillance System-All Injury Program indicate that between 2001 and 2009, the total number of TBI related to sports and recreation increased from 153,375 to 248,418 (Centers for Disease Control and Prevention: Nonfatal traumatic brain injuries, 2011). Providing timely education to young athletes on prevention, recognition, and response to sports-related head injuries holds the potential to improve safety and post-injury management.
Athletes between the age of 5 and 18 constitute 65% of all sports-related traumatic brain injuries treated in US emergency departments (Centers for Disease Control and Prevention: Nonfatal traumatic brain injuries, 2007). Many sports associations, such as the National Football League, have begun actively promoting programs aimed at the prevention of head injuries (Kirkwood, Yeates, & Wilson, 2006;Schatz & Moser, 2011).
Currently, no uniform educational program exists that offers students a comprehensive overview of concussion prevention and management. Several federally funded and independently operated bodies have developed projects to address this deficit. The ThinkFirst program was developed by a group of neurosurgeons belonging to the Association of Neurological Surgeons and the Congress of Neurological Surgeons. The organization trains community-based injury prevention coordinators to facilitate informative, student-centered discussions with victims of traumatic brain or spinal cord injury. A study conducted at three Chicago suburban schools found that the ThinkFirst Program increased student knowledge related to potential for injury and improved attitudes about safe behavior practices (Gerhardstein, 2007). The Sports Legacy Institute is a Boston-based organization with a similar approach, but no published data exist regarding the efficacy of this program (Williamson et al., 2014). Upon evaluation of program participants' understanding of content, Bagley et al. reported an increased ability to pass a concussion-specific quiz after the presentation (Bagley et al., 2012).
In most states, formal education regarding the nature and risk of concussion and head injury is only required for coaches, employees, volunteers, and other adults involved with a youth athletic activity (Concussion Education Plan and Guidelines, 2015;Edwards & Bodle, 2014;Virginia Board of Education Guidelines for Policies, 2015). This trend can be attributed to the fact that most state concussion regulations are based on Washington's Zackery Lystedt Law, which requires youth athletes and a parent and/or guardian to sign and return a concussion information sheet on a yearly basis (Engrossed House Bill 1824). Alternatively, some states have opted to provide student athletes with formal training programs (House committee report version, 2009;Louisiana Youth Concussion Act, 2011;Parker, 2015).
Even with the implementation of state-mandated educational programs, a gap in knowledge about mild to moderate TBI and the associated symptoms continues to exist among athletes. In 2014, McAllister-Deitrick, Covassin and Gould (2014) found that youth athletes demonstrated a less than satisfactory level of knowledge about sports-related concussion symptoms. Another study conducted among 1,532 varsity high school football players revealed that limited understanding of the risks and potential consequences of a concussion contributed to underreporting of probable head injury (McCrea, Hammeke, Olsen, Leo, & Guskiewicz, 2004). Head injuries present with a high degree of variability and can be difficult to recognize if not properly trained (Marshall, 2012). This study was designed to determine the effectiveness of an educational program focused on increasing the understanding of sports-related head injuries among middle school athletes.

Materials and methods
The study was approved by The University of Texas Health Science Center at San Antonio Institutional Review Board under Exempt Protocol to utilize de-identified pre-workshop and post-workshop tests to analyze the effectiveness of the administration of the Head Safety in Youth Sports (HSYS) Program.

Participants
Nine Texas middle schools were contacted by email to inquire their level of interest in participation in this study. Schools were selected to provide variation in class size, district location, and socioeconomic status (SES) to ensure diversity among participants. In order to reach a broad spectrum of students, the workshop was presented during Physical Education (PE) class. Student participation was voluntary. Students were sent home with an information sheet for their parents or guardians to review; any parent who did not want their child to participate could withdraw their student from the HSYS Program.
In total, participants consisted of 858 male and female sixth-through eighth-grade students, ranging from ages 11 to 16. Subjects (n = 29) were excluded if they did not complete their preworkshop or post-workshop test.

Demographics
A demographics survey was administered to all students prior to the HSYS workshop. This voluntary survey asked for age, gender, and types of sports played. The results of this survey are shown in Table 1. Of the number of participants who responded to age (n = 807), the average age of students was 13.2 ± 0.85.
The SES (poverty level) of all schools was analyzed by zip code using the US Census Bureau Website (American FactFinder: Community facts, 2014). The population distribution with regard to ethnicity was evaluated using each of the school's district websites (Community: District fact sheet, 2015; Facts and figures, 2015; Facts and figures: SAISD at a glance, 2015). In Texas, 17.6% of residents live below poverty level. All schools except MS3 are located in areas that have a higher level of poverty than the average level of poverty in areas throughout Texas.
Demographic data did not significantly vary. All five schools have a significant Hispanic/Latino population, which is representative of the geographic area that we were evaluating.

HSYS program
Under the guidance of a board-certified child neurologist (LDL), the HSYS Program was developed by a group of medical and dental students. After thorough literature review relating to concussions and traumatic brain injuries associated with sports and youth athletes, the HSYS Program was modified to ensure that both content and language were age-appropriate. Through acknowledgment of health and academic literacy barriers, the content and language were adjusted to a fifth-grade comprehension level in order for all participants to benefit. The program is designed to be a 35-45 min workshop to be given during the participants' scheduled physical education class.
The workshop targeted multiple learning styles and includes a PowerPoint presentation with pictures and video for visual and auditory learners, respectively; synchronous audience response to emphasize key objectives; videos of personal accounts of professional and youth athletes to allow students to connect learning on an emotional level; as well as example scenarios that enable teachback method of learning. A skull and brain model was also available for the students to see and touch. Content of the presentation includes a basic overview of brain and spinal cord anatomy and function, the definition of a concussion and common signs and symptoms, tools to recognize and prevent a head injury, the appropriate actions for a student to take if he or she suspects a concussion, and the consequences of ignoring a concussion. Examples of slides located in Appendix A.

Data collection
Students were handed a stapled packet as they entered the gymnasium. The packets included a raffle ticket, a sports demographic survey as stated above, and a pre-workshop test labeled with the last three digits of the raffle ticket in order to compare de-identified data with the post-workshop test. Students were asked to detach and keep their raffle tickets, then given five minutes to complete the pre-test, after which the pre-test and surveys were collected by school teachers as well as by HSYS medical and dental student volunteers.
The pre-and post-workshop tests use True-False format, multiple choice, and free response. The same questions were utilized in both pre-and post-workshop tests. The test assesses knowledge of the definition of a concussion, associated signs and symptoms, basic skull and brain anatomy, safety preventative measures regarding head injuries, and how to respond in the event of a head injury.
The HSYS workshop was presented by a group of four medical students. Following the presentation, students were asked to label their post-workshop test with the last three digits of their raffle ticket number. The post-workshop tests were administered for 5 min and then collected. A raffle was held in which a number of students received prizes.
Tests were scored by five individuals using a standardized answer key. There were 10 questions total; the maximum score possible is 13 points. The pre-and post-workshop mean test scores were converted as a percentage out of 100%.

Statistical analysis
Comparison of pre-workshop and post-workshop test score averages per school were conducted using the paired t-test for significance in difference of mean test scores to evaluate for HSYS Program effectiveness. Analysis of variance (ANOVA) was utilized to test for differences in performance by gender as well as pre-workshop test mean baselines between schools. Long-term retention of HSYS Program was determined by unpaired t-test, utilizing pre-workshop test data of MS3 students who had previously participated in the program, during its inaugural session 11 months prior, (Y) against pre-workshop test data of those who had not previously participated in HSYS Program (N, Figure 3).

Results
The Head Safety in Youth Sports (HSYS) Program was administered to 858 male and female middle school students (sixth, seventh, and eighth grades), ranging from ages 11 to 16, throughout five different middle schools (MS1-5) within Texas, USA ( Table 2).
Out of all five schools, MS4 demonstrated both the highest pre-workshop as well as highest postworkshop scores with 71 ± 16% and 88 ± 14%, respectively ( Figure 1). Initially, MS2 performed the lowest on the pre-workshop test with a mean score of 58 ± 14%; however, after the workshop, students from MS2 demonstrated the largest improvement in test score of all schools (Figure 1). Postworkshop, MS2 students scored 85 ± 10%, which is comparable to the mean test scores for all participants in the HSYS Program ( Figure 1). As evidenced by Figure 2, participants in all five schools demonstrated significant improvement in test score after participating in the HSYS Program (paired t-test, p < 0.0001). Regarding long-term retention, students from MS3 that had participated in the HSYS Program, 11 months prior, overall scored 7% higher on their pre-workshop tests than those who had not participated in the program before (unpaired t test, p < 0.05, Figure 3).
After participation in the HSYS Program, there was no difference found in either pre-workshop test score or test score improvement by gender.
Additionally, the demographic surveys filled out by students from MS1-5 demonstrated that a majority (66%) middle school aged children are currently participating in sports in which head injuries are very common (Figure 4) (American Association of Neurological Surgeons, 2014).

Figure 2. Improvement in test score after exposure to HSYS Program by all five schools.
Notes: Median is represented by the thickest line. The horizontal lines represent the number of students whose test score improved a specific amount after HSYS program exposure, which is quantified by the y-axis. Longer lines indicate higher frequency. The interquartile range is represented by the box and vertical lines. The dots represent any outliers from the interquartile range.

Figure 1. Average pre-workshop and post-workshop test scores by all five schools (M1-M5).
Notes: The mean ± SD error bars are reported. The legend indicates pre-workshop and post-workshop scores.

Discussion
Head safety education is an important facet of public health that addresses the preventable nature of head injuries. The lack of evidence concerning the effectiveness of national and state sponsored concussion education programs presents a formidable barrier to improvement of existing initiatives.
Regardless of this challenge, knowledge of the risk factors, symptoms, and post-injury protocols can be integral in reducing the occurrence or severity of brain injuries. The education of middle Notes: Demographic survey information was quantified and percentage of students participating in specific sports is reported. Participants identified volleyball, tennis, and swimming in the "Other" category.

Figure 3. Long-term retention of HSYS Program.
Notes: The pre-workshop test scores of students from MS3 that had previously participated in the HSYS Program were compared against students from MS3 against data of those who had not previously participated in HSYS Program, indicated by Y and N on the x-axis, respectively. Median is represented by the thickest line. The horizontal lines represent the number of students whose test score improved a specific amount after HSYS program exposure, which is quantified by the y-axis. Notably, longer lines indicate higher frequency. The interquartile range is represented by the box and vertical lines. The dots represent any outliers from the interquartile range. school-aged youth is an opportunity to increase awareness in young people who may be more amenable to instituting safer behaviors. The majority of students who participated in the HSYS Program are involved in sports in which brain injuries are common (Figure 4) (American Association of Neurological Surgeons, 2014). The effectiveness of the HSYS Program as an educational head safety curriculum is best reflected in the improvement in scores across all schools after participating in the workshop (Figure 2).
The HSYS Program has the benefit of being in its second semester of implementation at local middle schools, allowing for evaluation of long-term retention. The pre-workshop test data from MS3 students who participated in the workshop 11 months prior (n = 78) represent their baseline knowledge, or what knowledge they retained after their first exposure to the HSYS Program, which could be compared to the pre-workshop average from MS3 students who had not previously participated in the program (n = 145, Figure 3). Although this evaluation of long-term retention does not take into account changes in student population due to relocations or the acquisition of knowledge about head safety from outside sources, the limited data based on one middle school are promising and could be further explored in the following semester's program. Limitations in this study that may have affected the comparison of data from year 1 to year 2 are the effect of maturation and increased overall knowledge base, the environmental distractions during testing, and prior exposure to the test. An unstudied variable is the application of improved knowledge to behavior modification.
During our study, we determined that the impact of gender on both pre-workshop test score and improvement after participation in the HSYS Program, yielded insignificant results. This result is inconsistent with a recent study by Bagley et al. (2012). We acknowledge the possibility that restricting the age range in our study to focus on youth athletes in middle schools may have affected the impact of age and gender on pre-workshop test score or improvement.
Demographic analysis of the areas served by the middle schools revealed appreciable differences in socioeconomic characteristics. The community surrounding MS2 had the most individuals considered below poverty level; however MS2 demonstrated the largest improvement in test score after initially performing the lowest on the pre-workshop test (Figures 1 and 2). The United States Census Bureau reports the poverty status of MS2's community as 29.2%, while the other middle schools ranged between 15.4-28.7% (American FactFinder: Community facts, 2014). MS4 scored highest on the preworkshop test, while MS2 scored the lowest. The poverty status of MS4 is 8.7% lower than MS2, suggesting a possible relationship between SES and baseline knowledge on sports-related head injury. It is important to consider level of socioeconomic disadvantage and its impact on health literacy when comparing the quiz scores among the middle schools participating in the HSYS Program. The World Health Organization (WHO) defines health literacy as "degree to which individuals have the capacity to obtain, process and understand basic health information and services needed to make appropriate health decisions" (World Health Organization, 1998). Limited health literacy is strongly associated with socioeconomic indicators such as educational attainment, race/ethnicity, and age. Additionally, limited health literacy may contribute to under-recognition of problems and a delay in seeking appropriate care (Paasche-Orlow & Wolf, 2007). Many of the middle schools participating in the HSYS Program are comprised of students living in areas considered below the average poverty level. Though middle schools from higher SES districts were initially contacted, due to scheduling constraints, these middle schools were unable to participate in the HSYS Program. Future research plans include exploration of the relationships between SES, baseline knowledge of head safety, and improvement in knowledge after conclusion of HSYS Program.
The limitations in the study are important to recognize and form the basis for future improvements in the HSYS Program. In addition, this program was implemented without knowledge if students had previous exposure to formal, mandated head safety instruction. Having an understanding of the fundamentals of head injury before involvement in the HSYS Program may have an impact on individual test scores. With regard to questionnaire development, the same questions were utilized for both pre-and post-workshop tests. Participants did not receive answers, were unaware that they were to receive a post-workshop test, and there were no prompts during the workshop to emphasize postworkshop test subject matter. In the future, we can consider adjusting the post-workshop questionnaire to address the same concepts without utilizing identical questions to avoid re-testing bias.
Several changes can be implemented in order to improve the effectiveness of the HSYS Program. Students from MS2 demonstrated the largest improvement in test scores following the workshop. We believe that the contribution of SES and other underlying factors affected this trend. Specifically, MS2, with the lowest SES, may have had reduced pre-test exposure to concepts regarding head safety knowledge. The significant improvement observed in their test scores could be a result of the utilization of smaller groups and the increased overall average age of the participants, relative to the other schools. In order to determine the validity of this claim, further demographic analysis of the students should focus on better quantification of their baseline knowledge and overall health literacy. The impact of workshop size on test scores should be further examined. Workshops conducted at MS2 and MS5 had, on average, a smaller student to instructor ratio. Students at both of these schools showed the greatest improvement, with mean post-workshop test scores of 85% (Table 2). Lastly, given the workshops were implemented in a gymnasium, participants may have been exposed to distractions or disruptions while pre-and post-workshop test were administered. Though the majority of participants had higher scores on the post-workshop test, a minority showed decline. We believe that these data points represent outliers in participant performance, which may have been affected by a number of variables including: incompatible instructor-to-student ratio, environmental distractions, fatigue, non-conducive educational environment, language barriers, or health literacy barriers (Figure 2). This information may allow for adjustment of class size and subsequent enhancement of the educational experience of participants. Despite the improvement from pre-to postworkshop test scores, it is difficult to assess whether this translates into a meaningful change in behavior, which would reduce the incidence of head injuries among youth athletes. In its third semester of existence, our program has been contacted by multiple local middle schools requesting our workshop for their athletes. Thus, the significant impact of this program may not only be in its ability to improve students' knowledge of head injury, but also in its ability to raise awareness and promote head safety education among youth, coaches, and adult caregivers within the community. Future research may attempt to examine the relationship between caregiver receptiveness to head safety education and athlete performance during head safety education.
Education about the awareness, recognition, and management of sport-related concussion is integral in helping to improve the safety and health of youth athletes (Halstead & Walter, 2010). As the number of student athletes participating in organized sports increases, the need for well-developed educational programs is evident (Halstead & Walter, 2010). Given the paucity of data supporting the use of formal education on sports-related head injury, initiatives like the HSYS Program are imperative (Bagley et al., 2012). With its individualized and multifactorial approach, HSYS Program engages youth athletes in understanding the significance of head safety. It provides an opportunity to initiate this discussion early on and potentially influence youth athletic behavior to reflect head safety awareness.