Incorporation of Edutainment Into Intervention and Evaluation: The Jump With Jill (JWJ) Program

Objective: To incorporate media strategies (e.g., edutainment) into a pilot nutrition education intervention, Jump with Jill (JWJ), to (1) change student knowledge and intentions; (2) stimulate enthusiasm; and (3) enhance evaluation. Methods: Third graders (n = 194) completed a pre-survey, attended a 60-min nutrition education show presented as a school assembly, and then completed a post-survey. Data was collected interactively, where students lined up behind an emoji that best represented their answers. Results: Statistically significant improvements were reported for drink preferences and enjoyment of nutrition education as well as the experience of taking a survey. All aggregate responses for knowledge, attitudes and intention became significantly more positive (p = 0.05). Furthermore, ~95% reported positive ratings to participating in the survey (P < 0.0001). Discussion: Use of edutainment may serve to stimulate change in nutrition knowledge and intention in 3rd grade students. Perfecting this teaching strategy into evaluations of health promotion programming may serve the field by increasing accuracy of student responses when language, reading level, or survey inexperience are barriers.


INTRODUCTION
Through the use of entertaining strategies, mass media has been successful in impacting food intake in children (1,2). Marketing techniques such as repetition, music, characters and excitement are well-thought-out, well-funded and incorporate theoretical frameworks from the fields of behavior change, education and social marketing (3). Use of entertainment as an educational tool, termed edutainment (4), has established guidelines (5). In brief, research-based messages are woven into the audiences' lived experiences (6) using compelling plots, songs, and characters. Storytelling, identification and attachment to strong characters, parasocial interaction, social modeling, competition and liking are incorporated to enhance attitudinal and behavioral change (3,4).
To be effective, edutainment messages should be sustained as part of a comprehensive campaign that connects learners to characters and stories showing behavior change while establishing a new social norm. Because health skills develop through practice, the positive experiences derived from learning through play help develop deep intrinsic motivations that positively influence health outcomes (7) and lead to high levels of participant program satisfaction (6). Edutainment has been a successful component of previous nutrition interventions, improving knowledge, attitude and intention (8)(9)(10) and developing resistance to persuasive messages (11).
Although amenable to change, habits and food preferences formed during childhood typically track into adulthood and may lead to development of chronic disease (12)(13)(14). Hence, it is important to utilize strategies directed at elementaryaged children that influence targeted food and beverage choices known to impact health, such as increasing intake of fruits, vegetables, and low-fat dairy foods as well as decreasing intake of sugar-sweetened beverages (15). However, successful nutrition education aimed at children cannot simply be didactic but should be interactive, enjoyable and developmentally appropriate (7,16,17). Moreover, programming should have a theoretical framework (7,16,17) such as those used in behavior change, education and social marketing (18). Finally, unique evaluation processes must be considered because children are mastering written language and may be inexperienced with surveys. Edutainment strategies integrated into evaluation protocol might be a tool to addresses this challenge.
Although schools are an obvious venue for educating children about healthy eating (16,19,20), many students receive inadequate nutrition education despite its success (21). While teachers consider nutrition education valuable for students (22), Common Core State Standards and high-stakes standardized testing make nutrition and physical education expendable (23). Limited resources (22) and lack of subject expertise make external nutrition education programs attractive (16). Employing media strategies can entice classroom teachers into including nutrition education, thereby overriding cost and time barriers (11).
Furthermore, reliable evaluation strategies are essential parts of any intervention. Assessment strategies for children are generally informal (using some type of written format) and focus on measuring changes in knowledge, attitude, and behavior. Such changes are especially difficult to assess in young children given their cognition levels (24). Moreover, with required standardized testing in elementary schools, questions have arisen regarding student stress in relation to any testing, (23,25) generating interest in investigating alternative evaluation strategies.
Employing media strategies, Jump with Jill (JWJ) is a music-based, nutrition education program. JWJ creates a unique learning experience using original music, lighting, props, and live characters to inspire both children and teachers (Figure 1). During the 60-min school assembly, students dance and sing to behaviorally-focus songs that address increasing consumption of fruits, vegetables, dairy and water. Following the show, these characters and their messages can be used by classroom teachers via supplemental materials (e.g., music downloads, activity sheets, danceable music videos).
This study was undertaken to determine the impact of incorporating edutainment into both a nutrition education campaign and its evaluation. The hypotheses were such incorporation would (1) change student knowledge and intentions; (2) stimulate enthusiasm toward nutrition education; and (3) enhance the evaluation process.

Recruitment
An independent booking agency recruited schools for the 2015-2016 school year to receive JWJ at no cost. Eligible schools (n = 50) were within the 10 New Jersey counties that ranked lowest in the 2014 County Health Rankings (Atlantic, Camden, Cape May, Cumberland, Essex, Gloucester, Hudson, Passaic, Salem, and Union). One performance school per county was randomly chosen for evaluation. While the entire school attended the assembly, only one randomly-selected 3rd grade class completed surveys. Parental, student and teacher consent were handled at the district level. As all schools viewed JWJ as an adjunct to their education, the project was considered exempt from written consent. The University of Pittsburgh's Institutional Review Board approved this study as only deidentified evaluation data was used for the evaluation. No demographic information was collected.

Survey Development
A survey was designed incorporating edutainment strategies to assess student knowledge, attitudes and intentions (e.g., questions repeated phrasing used in the show). For the first question, actual products (milk, water, energy drinks, fruit drinks, and soda) were displayed. Milk and Water were classified as a positive response (coded as 1) and Energy Drink/Fruit Drink/Soda (coded as 0) were classified as a negative response.
For questions two through seven, instead of adapting more traditional "yummy/yucky faces" (26) or indication of "yes" or "no, " customized emojis were used to determine responses (Figure 2). To overcome obstacles associated with using a Likert Scale with children (27), emoji faces varied in relationship to the strength of response and were accompanied by words rather than numbers. In the JWJ Likert Scale Survey Technique for Children, responses were recorded such that each emoji represented a point on a 1 to 5 Likert scale with 1 reflecting "ABSOLUTELY NOT" and 5 reflecting "YES!." In addition, at pre-test students were asked to raise their hands to determine previous experience with survey taking; at post-test, they were asked to stand in front of the emoji that reflected their survey enjoyment.
Prior to administration, cognitive testing with a small, convenience sample (n = 5; grades K-4) was conducted to determine how the expressions would be interpreted. Except for "No, " students indicated agreement with each face label. Originally a face with a tear depicted "No" but students thought that face represented "Sad." The emoji was changed to a face with a line for a mouth.

Survey Administration
Stations containing either a beverage (energy drink, fruit drink, soda, milk or water) (Question 1) or an emoji (Questions 2-7) were arranged. Before starting the survey, students were told by JWJ characters that there was no right answer and that the survey was not a test. To capitalize on the power of peer influence, students made a "Promise to be Honest" as a commitment to being thoughtful "research subjects" where their authentic opinions were valued. To minimize groupthink, students were asked to select their answers before moving to a station. Students then were asked to fully commit to their answer by lining up in front of the emoji that matched their response (Figure 3). The JWJ cast recorded the number of children at each station for each question. The students then joined the assembly. After the show, students reaffirmed their honesty pledge and repeated the pre-survey process.

Data Analysis
Frequencies were calculated from recorded responses. Two types of outcomes were determined using aggregated data from all schools and time points (pre-test, post-test): the proportion of responses (children lining up behind an emoji) and an overall score. The proportion of responses was calculated as: sum of children lining up behind an emoji divided by total of children multiplied by 100. Pairwise differences among dependent proportions were tested using simultaneous 95% confidence intervals among dependent proportions (assuming a multinomial distribution) (28,29). The overall score was calculated (for all but question 1) by multiplying the emoji's response choice (1)(2)(3)(4)(5) by the number of children lining up behind it, then summing all multiples for each time point and school. Higher overall scores indicated more positive responses. Differences in scores between pre-and post-tests were assessed. Distribution of overall scores and differences were determined to be non-normal; hence, the Wilcoxon Signed-Ranks test was conducted using Statistical Analysis Software (version 9.3, 2011, SAS Institute Inc, Cary, NC).

RESULTS
JWJ performed 50 shows reaching nearly 14,700 elementary school children. Over 75% of the 194 3rd graders participating in the evaluation reported no previous experience with taking a prior survey. Following the show, statistically significant differences in individual scores were noted for questions 1, 3, 4, and 5 ( Table 1). Moreover, all aggregate responses became significantly more positive ( Table 2). Furthermore, ∼85% reported the highest positive rating (YES!) to survey participation, and 95% reported positive ratings (YES! or Yeah), which differed significantly from those reporting negative ratings (ABSOLUTELY NOT and No) (P < 0.0001).

DISCUSSION
Overall, using media strategies associated with edutainment to change student knowledge and intention as well as to detect these changes was confirmed. Use of enthusiastic endorsement of the expected behaviors (a Social Learning Theory component) by JWJ characters (a marketing strategy) appeared to stimulate change in nutrition knowledge and intention in these 3rd grade students. These findings confirm previous observations that theatrical interventions in schools can be successful (8-10, 30, 31).
JWJ creates positive, realistic impressions of healthy foods, moderates negative responses, and creates commitment for those that start out as "not sure." Jump with Jill deconstructs the framework that kids have built with their dislikes and breaks ground with newfound aspirational attitudes, as seen by the decrease in the number of extreme negative responses for all questions. The positive experience derived from JWJ helps audiences develop deep intrinsic motivations that lead to high levels of program satisfaction among participants. Moreover, through the many touch points of the program, the students and their teachers attach to the characters. Their relationship deepens as they go from interacting with the show to viewing videos to working in small groups with the characters to complete the survey where their opinion is valued and documented. While the positive results of the intervention are exciting, a major focus of this pilot study was developing an evaluation tool that aligned with the educational approach of the program. Unique to this study was the integration of media strategies into the evaluation process. Edutainment techniques successfully captured changes in 3rd grade students' knowledge and intention. Making the evaluation process interactive and fun (game-like) resulted in over 95% of students enjoying participation in a study, in contrast to reports of student stress in relation to testing (32,33). A positive experience can have meaningful measurements for likelihood of healthy behaviors. Researchers have noted high intensity emotional experiences has an increased influence on behavior, so strong it can even overwhelm cognitive processing (34). Nutrition education in the form of a rock show is a strategic choice to empower audiences to action. Additionally, children in 3rd grade are in Piaget's concrete operational phase of cognitive development, where they begin to use inductive logic and become less egocentric (35).
Using the "Promise to be Honest" pledge built upon their developing senses of having unique thoughts of fairness (35) and decreased social desirability. Students were proud to be forthright with their opinions even though they were aired in front of their peers. Setting clear expectations for behavior (aka defining the rules of the game) is crucial. It is suggested that only a small, familiar group (e.g., one class) be used for this data collection method as the researchers experienced a diminished effect of accountability as group sizes grew. For example, as this methodology is scaled to other JWJ projects and sample size is increased, we are still evaluating classrooms one at a time rather than combining classes. Adjustments in the show and travel schedule are made to allow for these multiple classroom visits before and after the show as part of the intervention design.
Perfecting edutainment within nutrition education evaluations in elementary schools may serve the field by increasing accuracy of student responses when language, reading level, or survey inexperience are barriers. These study participants saw the evaluation as part of the intervention, hence creating a positive experience throughout appears to enhance satisfaction. Furthermore, this was an intentionally simple evaluation study design that could fit into the parameters of a school day, requiring no additional staffing from the school or JWJ. Several limitations must be acknowledged. This current study lacks follow-up to demonstrate students' knowledge retention or behavior change. Other limitations include lack of a control group and unpaired pre-and post-survey data, both of which restrict generalizability. Furthermore, statistically significant proportions that change by a fraction of a decimal to a few points must be interpreted with caution.

CONCLUSIONS AND IMPLICATIONS
The mass media has proven successful at impacting food intake in children (1,2). If "celebrities" can shape social norms about unhealthy behaviors, then nutrition educators might be able to use these same strategies to shift youth toward healthy behaviors. If children demand high quality entertainment, then healthrelated media can compete. The media could be the best guide in developing solutions to address youth health. Suggestions for best practices in incorporating this novel evaluation strategy into other programs include: working in small groups, for example, by classroom; pausing to let students decide answers before reporting to the group; decreasing test anxiety and reducing test fatigue by making the evaluation into a game; and decreasing social desirability bias by using a "promise to be honest" pledge.
To provide additional validation for using media strategies in nutrition education, along with the impact of JWJ on food choice behavior, future studies including a comparison group, more follow-up data points, demographic characteristics of children, and other confounding variables are warranted.

DATA AVAILABILITY
The datasets generated for this study are available on request to the corresponding author.

ETHICS STATEMENT
This study was carried out in accordance with the recommendations of the school districts involved in the study. Parental, student and teacher consent were handled at the district level. As all schools viewed JWJ as an adjunct to their education, the project was considered exempt from written consent. The University of Pittsburgh's Institutional Review Board approved this analytical portion of the study as only deidentified evaluation data was used. No demographic information was collected. The University of Pittsburgh Institutional Review Board approved this study.

AUTHOR CONTRIBUTIONS
The author confirms being the sole contributor of this work and has approved it for publication.

FUNDING
This study was supported by the Young Audiences of New Jersey and Eastern Pennsylvania that received a grant through the Robert Wood Johnson Foundation.

ACKNOWLEDGMENTS
The University of Pittsburgh Institutional Review Board approved this study. Statistical analysis of deidentified data was performed by Salma Musaad MD Ph.D at the University of Illinois at Urbana-Champaign. Madeleine Sigman-Grant provided pivotal guidance. I wish to thank all the administration, teachers and students from all the participating schools for their enthusiastic involvement with this study.