Implementation of a school-based physical activity intervention for Brazilian adolescents: a mixed-methods evaluation

Summary The effectiveness of physical activity interventions can be improved through examining the aspects related to their implementation. However, little such evidence has been collected, particularly in low-and middle-income countries. This study aimed to evaluate the implementation of a school-based physical activity intervention with qualitative and quantitative data from different actors (students, teachers and parents) involved in developing the program. The program was conducted in 2017 with three main components: (i) teacher training, (ii) environmental changes and (iii) educational actions. Mixed-method evaluation was performed by an independent evaluation team using a validated indi-cator matrix for the implementation process, including the self-reported information of students, teachers and parents, as well as interviews with teachers. In the 3 eligible schools, 350 adolescents (51% girls) answered the implementation questionnaire, as did 45 parents (84% mothers), and 47 teachers (70% female). In the qualitative analysis, 18 teachers participated. Categorical analysis found that the intervention was considered feasible by teachers. In general, teachers had a more positive perception of the implementation than did students. The lack of engagement from the school community and parents and the busy schedule of teachers were indicated to be the main difﬁculties. In conclusion, despite the teachers’ motivation, some barriers prevented the successful implementation of the program.


INTRODUCTION
There is a global call for action focused on promoting physical activity (PA) and reducing sedentary behavior (SB) among children and adolescents (World Health Organization, 2018). School-based interventions with multiple components (e.g. educational strategies, environmental changes and personal training) have been reported to address the multidimensional factors of PArelated behavior (Russ et al., 2015;Barbosa Filho et al., 2016a). However, even for multicomponent interventions, the results have been diverse and have shown inconsistent effect sizes (Maniccia et al., 2011;Barbosa Filho et al., 2016b;Jones et al., 2020), and still fewer studies have focused on elucidating how the actions are implemented (i.e. the fidelity, feasibility, and quality of the proposed action for the target audience) (Naylor et al., 2015).
Evaluation of the implementation of an intervention can shed light on the factors that influence the effectiveness of the program (Craig et al., 2013), because it links the theoretical models and practical experience of the intervention (Naylor et al., 2015), helping bridge the gap between planning and adopting strategies (Durlak and DuPre, 2008). Additionally, it can identify which strategies are feasible and favorable to adaptation, or that may be not applicable, improving the cost-benefit balance of interventions (Durlak and DuPre, 2008). Despite the broad acknowledgment of the importance of performing evaluations of implementation (Moore et al., 2015) and the publication of both process and program assessments of PA and SB interventions (Kennedy et al., 2021), the need for more pragmatic uses of the RE-AIM has been highlighted, rather than trying to comprehensively assess all framework dimensions (Glasgow et al., 2019). Therefore, more evidence is needed on evaluations of the implementation of PA interventions (Daly-Smith et al., 2020).
Evidence has highlighted the relevance of using a mixed-methods evaluation design combining quantitative data (e.g. students, teachers and parents) with qualitative data from purposively selected samples (Moore et al., 2015). This design may help develop better intervention strategies based on the perception of different actors in the context of interest (Daly-Smith et al., 2020), as the quantitative and qualitative analyses build upon each other, as, e.g. by using qualitative data to help explain the quantitative findings (Moore et al., 2015;Creswell and Clark, 2017). Therefore, there have been recent calls for mixed-method approaches to identify explanatory processes across RE-AIM dimensions (Glasgow et al., 2019).
Regarding low-and middle-income countries (LMICs), the promotion of PA interventions for adolescents in this context is a research priority, including the development, testing, and implementation and delivery of school-based interventions (Morton et al., 2016;Nagata, Ferguson and Ross, 2016;Fair et al., 2018). Amid an increased call for implementation research in LMICs, the body of evidence is scarce. Therefore, this study aimed to evaluate the implementation of a schoolbased PA intervention in Brazil, considering both qualitative and quantitative data from the different actors (students, teachers and parents) inherent to the implementation of the program.

Design
In this mixed-method study (Creswell and Clark, 2017), we evaluated the implementation of a clusterrandomized controlled intervention, registered in the Clinical Trials (NCT02944318), and approved by the National Research Ethics System (protocol number: 1.259.910; CAAE: 49462015.0.0000.0121; date: 23 November 2015). Methodological and theoretical information regarding the proposed intervention actions was previously published (Silva et al., 2020).
In summary, the inclusion criteria of the schools were (i) having secondary level grades (n ¼ 27); (ii) having at least two classes per grade from the seventh to ninth grades (n ¼ 21 schools remaining); and (iii) were not under environment reform/repair during the intervention period (n ¼ 18 schools remaining). Of the 18 eligible schools, 7 agreed to participate in the intervention. Thus, the school with the lowest number of students was selected to participate in the pilot study, and the other six schools were randomly allocated to the intervention (n ¼ 3) and control (n ¼ 3) groups. All students enrolled from the seventh to ninth grades who were at school at the beginning of the school year were eligible participants (n ¼ 1427). Students were excluded who: (i) were mentally and/or physically disabled; or (ii) missed classes during the first 3 weeks of the school year (period of the collecting data). Therefore, 999 (70% of the total) were authorized by their legal guardians and agreed to participate in the study.
For the present study, only intervention schools were included, and the method of determining the final sample size was based on the opportunistic pragmatic nature of this implementation study (Emmel, 2013).

Description of the intervention
Multicomponent actions were proposed to increase PA and reduce SB based on different theoretical models (Silva et al., 2020). In particular, the Health Promoting Schools framework was used to guide the main pillars of actions used in the intervention: inclusion of health education content into the school curriculum; provision of health opportunities at school through social and/or physical environment; and family engagement (Langford et al., 2014).
The strategies were developed during a school year (March to November 2017) and had the following components of actions: (i) teacher training; (ii) active opportunities in the school environment; and (iii) health education for the school community. The working group for the development of the program involved professionals and master's/doctorate students from the Federal University of Santa Catarina, Brazil. Detailed description of the intervention program can be obtained online (www.movimente.ufsc.br) and in a previous publication (Silva et al., 2020).

Teacher training
Training with certification and a support material (book) with proposed activities on health topics for teachers of all disciplines were prepared. Interactive media (Facebook and WhatsApp) were used for teachers to disclose and discuss their activities in relation to the health topics. Additionally, support materials (three books) specific for each grade were delivered for physical education (PE) teachers.

Active opportunities in the school environment
This action included the creation and revitalization of some spaces of the school for the practice of PA, and making PA equipment (e.g. balls, jump ropes, rackets) available to students during their free time in school.
Health education for the school community Four posters on PA and health, SB and health, PA and academic performance, and eating habits were prepared. Four pamphlets on PA and health, SB and health, eating habits, and PA and SB were disseminated (information shared among teachers, students, parents).

Participants
In this mixed-method study, students who answered the follow-up measures (n ¼ 463) and all teachers (n ¼ 63) in the intervention group were invited to complete the evaluation questionnaire regarding the implementation of the actions. Parents (n ¼ 150) were randomly selected from a prior list of all participating students to complete the questionnaire. For the qualitative evaluation, a purposive sampling approach was used (Emmel, 2013). All teachers from the intervention schools were recruited in person and/or via e-mail or telephone, and interviews were scheduled for teachers who replied to our contact. We included teachers who had not been scheduled but who had time available when the researchers were at school.

Evaluation process
The main objective of the evaluation was to assist decision making by providing information on whether the program was executed as planned (Bauman and Nutbeam, 2014). A team of four members who had not participated in the design and implementation process of the intervention structured and validated an evaluative matrix and defined the variables and their specific objectives according to the logical model of the program based on the dimensions of Reach, Effectiveness, Adoption, Implementation and Maintenance (RE-AIM framework) (Glasgow et al., 2019). Considering the complexity of analyzing all dimensions of the RE-AIM framework (Glasgow et al., 2019), we decided to analyze the implementation dimension to address with details those components most appropriate for our research problem, setting, and stage of research in Brazil. Moreover, as a lesson learned from using this framework for two decades, Glasgow et al. recently recognized the need for more pragmatic uses of the RE-AIM, rather than trying to comprehensively assess all its dimensions (Glasgow et al., 2019). This dimension was evaluated quantitatively and qualitatively at the end of the intervention period (carried out November/ December 2017) regarding the strategies adopted during the scholar year.
Quantitative data included 10 dimensions organized according to each intervention strategy. The strategy of teacher training was structured in six dimensions: (i) teacher training; (ii) discussion on health in regular classes; (iii) themes taught in class; (iv) modifications of the classes on general subjects; (v) difficulties in working on health contents; and (vi) modification of PE classes. The educational strategy was organized into two dimensions: (vii) distribution of intervention pamphlets and (viii) distribution of intervention posters, and the strategy of environmental improvements was divided into two other dimensions: (ix) creation and revitalization of spaces for PA practice and (x) availability of materials for PA practices.
The Dimensions 1 (teacher training) and 5 (difficulties in working on health contents) applied only to teachers, with the first dimension being answered only by teachers who participated in the training (n ¼ 9). Items for the other dimensions were answered by all students and teachers. The parents participated only in Dimension 7 (more details are presented in Supplementary Material 1). The questionnaires for students and teachers were distributed in November and December 2017. The parents answered the questionnaire by telephone during the same period.
Regarding the qualitative data, teachers participated in semi-structured individual interviews focused on the evaluation of the following themes: (i) teacher training, (ii) educational strategies, (iii) environmental actions and (iv) overall evaluation of the intervention program. The overall evaluation included the following: (a) importance of developing the intervention program at school; (b) feasibility for implementing the program; (c) difficulties in implementing the program; (d) suggestions for changing the program's development format and (e) the importance of the program in thinking about interdisciplinarity. After the participants signed the consent form, the interviews were conducted by the staff at a previously scheduled time. The collected data were recorded in audio format and later transcribed.

Data processing
All quantitative data were entered by one researcher and verified by another researcher. Categorical measures were analyzed using relative frequencies.
The transcribed interviews were treated using the content analysis technique proposed by Bardin (Bardin, 2011) to organize the data for better interpretation. This study applied the units of meaning and context. We chose categorical analysis, among the various possibilities of categorization, to discover the nuclei of meaning that make up communication and whose presence or frequency holds meaning for the analytical objective chosen (Bardin, 2011). All the processes were developed simultaneously by two researchers, and disagreements were discussed without the participation of another researcher.

RESULTS
In the 3 eligible schools, out of 463 students who participated in the follow-up measures, 350 adolescents (51% girls) answered the implementation questionnaire, as did 45 out of 150 parents (84% mothers) and 47 out of 63 teachers (70% female). For the qualitative analysis, 18 teachers gave their consent to participate.

Teacher training
The quantitative and qualitative results regarding teacher training are presented in Tables 1 and 2, respectively. Seven out of the 9 teachers who participated in the training considered the content very useful ( Table 1). Most of the teachers reported that the school coordinators encouraged participation in training (f ¼ 7). They also identified a need for better disclosure of the training's existence by e-mail or pamphlets (f ¼ 4), or even at the initial pedagogical meeting of the school (f ¼ 4). The theme was considered relevant for teacher training and/or practice (f ¼ 18), but the unavailability of schedules was pointed out as the main difficulty (f ¼ 6). The positive point most mentioned was the quality of the support material (f ¼ 8), and the negative was the poor functioning of the online platform (f ¼ 5; Table 2).
Most of the teachers (68%) reported holding classroom health discussions, but 52% of the students did not notice these discussions. A total of 48% of the students and 73% of the teachers reported not using the educational materials in these discussions.
According to the teachers, face-to-face training contributed to the updating of the themes (f ¼ 5) and development of practical classes outdoors (f ¼ 4) ( Table 2). A total of 43% of the students and 44% of the teachers confirmed the development of active breaks in the classroom (Table 1).
Almost half of the teachers reported difficulty in addressing health-related topics (46%) ( Table 1). The qualitative results (Table 2) identified the need for the development of outdoor activities in the face-to-face training (f ¼ 6).
Most students (80%) and all PE teachers (100%) stated that more active PE classes were developed and 57% of the students reported that the PE teacher addressed the health content in the classroom (Table 1).

Educational strategies
Most of the teachers and students reported receiving the pamphlets (81 and 55%, respectively) and seeing the posters (79 and 54%, respectively), whereas only onethird of the parents did (Table 1). The teachers stated receiving the materials through the school coordinators (f ¼ 4) or at the teachers' room and at pedagogical meetings (f ¼ 4) (Table 3).
According to 79, 42 and 22% of the teachers, students and parents, respectively, the messages delivered through the pamphlets were able to positively alter the lifestyle of the students (Table 1).
Regarding the main reasons for using the materials (Table 3), the teachers reported the relevance of the theme (f ¼ 9). However, the lack of teacher planning (f ¼ 4), non-participation in face-to-face training (f ¼ 3), and the researchers' failure to disclose the utilization strategy (f ¼ 3) were reasons for non-use. Implementation of a school-based physical activity intervention for Brazilian adolescents

Environmental improvements
A total of 31% of the students and 76% of the teachers perceived the spaces revitalized by the staff, but only one-third of the students utilized these spaces to practice PA. A total of 84% of the teachers noticed the materials supplied by the program; nonetheless, almost one-third of the students did not know about them (Table 1). The teachers suggested the development of play during recess (f ¼ 3) and PE classes (f ¼ 3) to encourage the use of PA equipment (Table 4).

Overall evaluation of the intervention
The teachers reported that the main role of this program was to make students aware of a healthier lifestyle (f ¼ 6), and they reported that the intervention was feasible owing to the importance of the theme (f ¼ 3) and the involvement of the students (f ¼ 2). The non-involvement of all segments of the school was the main difficulty (f ¼ 2). For improving the intervention, the most frequent suggestion was to plan theme-based activity days with parental involvement, inside or outside the school (f ¼ 5) (Table 5).

General findings
Our results indicate that the interdisciplinary and multicomponent approach of this intervention accommodated different points of view on PA and health beyond PE classes. However, mixed results were observed when the indicators were reported by different actors in the intervention implementation (teachers, students and parents).  Training hours were sufficient (þ) 2 Provided the exchange of experiences through digital media (þ) 1 Experience report among teachers (þ) 1 Important in being interdisciplinary (þ) 1 Poor functioning of the online platform (À) 5 Need for diversification of supporting texts and activities (À) 3 Difficulty sharing photos/videos of developed activities (À) 2 Little involvement of researchers in the process (À) 2 Need to create new strategies for the online stage (À) 1 Need for participation of all teachers (À) 1 Extensive training hours (À) 1 Contribution of face-to-face training to teaching practice Updating of themes, contributing to the teaching performance 5 Assistance in the development of practical classes in outdoor environments 4 Adoption of breaks in the classroom 3 Assistance in the elaboration of physical activity dynamics and games 3 Improved teachers' confidence and motivation 2 Facilitated the articulation of the theme with the school component 1 Improvement suggestions for teacher training Need for the development of outdoor activities in the face-to-face training 6 Researchers should participate more actively during the school year 4 Need for another meeting at the end of the school year 1 Need for more specific content on sports 1 Training could be carried out at another time outside of school 1 Note: qualitative data obtained from content analysis technique proposed by Bardin (Bardin, 2011); ƒ: frequency of reports (i.e. how many times the subcategory was cited); þ: positive aspects; À: negative aspects.
Meanwhile, less than half of the students were receptive to the classroom health discussions, and most of them pointed out that messages were insufficient to improve their lifestyle. These results may reflect two main factors: the low number of teachers who participated in face-to-face training and the lack of involvement of all segments of the school community (e.g. school administration). In the first, teachers reported above all their unavailability in terms of scheduling. In the second, although the teachers were appropriately invited for the training, they were not authorized to be released from their class schedules. These findings are in line with previous studies that suggested a lack of time and an unsupportive school climate as important barriers to the implementation of PA strategies at school (Naylor et al., 2015;van den Berg et al., 2017).
Moreover, the lack of knowledge on introducing active strategies in the classroom has also been recognized as an important barrier that should be considered in future PA interventions (Daly-Smith et al., 2020). These findings highlight the importance of considering activities that are easy to implement or require little or no preparation time, the implementation of which must involve key stakeholders (e.g. principals) (van den Berg et al., 2017;Lander et al., 2019).
In the present study, 5 out of 10 teachers adopted active breaks in the classroom, as perceived by the students. Another study found that the number of activities to introduce breaks to students' sitting time Discussion with families about health contents 2 Writing, debates and tasks on healthy behaviors in the classroom 2 Improvement suggestions for pamphlets and posters strategies Deliver directly to students, in the classroom, and/or through digital media groups (via class leaders) 3 Affix the posters in the gym or in front of the PE room 2 Deliver pamphlets directly to teachers 1 The pamphlets' content should be linked to the city's curriculum proposal.
1 Create a thematic place for the project 1 Pamphlets should have had fewer photos and more content 1 Establish how, when, and where pamphlets will be used 1 Posters could be bigger and more expressive 1 Add content on mental health 1 Materials should be developed together with the students 1 Note: qualitative data obtained from content analysis technique proposed by Bardin (Bardin, 2011); ƒ: frequency of reports (i.e. how many times the subcategory was cited).
increased after the end of the intervention (Kö ykkä et al., 2019). However, a concern previously highlighted was the lack of time to plan and apply strategies for reducing SB in the classroom (Routen et al., 2018). The authors pointed out that although teachers are aware of the importance of breaks, they could not easily include breaks in their routine owing to time constraints (Routen et al., 2018), as also reported in our study.
In the present study, most students and all teachers reported that more active PE classes were developed and that students' participation in classes improved. A systematic review noted that factors like self-efficacy, engagement and motivation of teachers can directly affect the implementation of interventions (Naylor et al., 2015). A recent intervention assessed the implementation fidelity of PE teachers' training to improve the opportunities for PA among adolescents during lessons, finding that the intervention had large positive effects on teachers' behavior, with the maximization of opportunities for movement and development of skills, and consequently better support for students (Lonsdale et al., 2019).
In our findings, half of the PE teachers perceived that the development of more active classes enabled positive changes in adolescents' lifestyles. Previous evidence has shown that internet-supported training has positive effects on students' moderate to vigorous PA during PE lessons (Lonsdale et al., 2019). Contrary to our findings, the authors assessed both the students' behavior and the implementation of the strategies objectively. Future research could combine the evaluation of implementation fidelity and student behaviors.

Educational strategies
In this study, 8 out of 10 teachers, half of the students and one-third of the parents received the pamphlets. All posters were made available in schools, but only half of the students noticed them. Pamphlets and posters were rarely used in the activities by the teachers. Among the reasons for not using the educational materials, the teachers reported lack of planning, non-participation in face-to-face training and the researchers' failure to share the strategies for using the materials in the classroom. These results could also indicate the perceived complexity of implementing strategies, reinforcing the idea that activities should be easy to use in the classroom (van den Berg et al., 2017). An important topic that the teachers highlighted is the relevance of the theme and articulation of the intervention pamphlets with respect to the interdisciplinary content. In agreement with previous research (Daly-Smith et al., 2020), although the teachers reported knowing the relevance of PA strategies, they still needed to know how to implement these strategies effectively, indicating the importance of teacher training.
During the interviews, the teachers also suggested that posters should be placed in more visible places, such as at the entrances to courts and sports facilities, given the low number of students who noticed them. Therefore, future interventions should make activities available affording teachers the opportunity to use the educational materials, considering issues such as time constraints (Naylor et al., 2015) and the teacher confidence necessary for delivery (Routen et al., 2018).
Another important point is the involvement of parents in the intervention. Our main strategy was to deliver pamphlets with messages for parents, but these pamphlets seemed to have no impact; few parents noticed this strategy and reported its lack of impact on the behavior of students. Previous studies have similarly identified that ensuring parental involvement is a major challenge for school-based interventions, as parents are seen as important contributors to the success of interventions (Jago et al., 2015;Naylor et al., 2015). Enables different points of view on the same theme 2 Note: qualitative data obtained from content analysis technique proposed by Bardin (Bardin, 2011); ƒ: frequency of reports (i.e. how many times the subcategory was cited).
PA, physical activity.
However, despite being one of the three domains of the Health Promotion Schools framework, the evidence has highlighted them as the most challenging and least successful intervention element (Langford et al., 2015). On the other hand, a previous study highlighted that parents' engagement is important in supporting children and adolescents who are not completely independent in terms of decision making regarding PA-related behavior (Jago et al., 2015). Moreover, school-based interventions that directly engage parents (e.g. educational meetings) have shown more beneficial results for PA and SB in adolescents than indirect involvement (Verjans-Janssen et al., 2018). These findings are in line with the recommendation of the teachers for future interventions: parents need to be directly involved in strategies together with students, e.g. in planning a PA-related theme-based day inside or outside the school.

Environmental improvements
Most of the teachers perceived the implementation of environmental improvements, whereas the students had different perceptions. Most of the students also reported not having used the spaces or materials to practice PA. Despite requiring more investment in materials and time, our process of implementing these actions might not have been the most appropriate, as the target audience had not been reached. Another point that may justify these findings is that the students did not have many free hours during the school period to use the materials or to perceive spaces for PA practice. In the Brazilian school system, the regular class schedule lasts 4 h a day, including only a 15-min recess period. Strategies related to environmental changes are relevant to the promotion of PA in adolescents (Morton et al., 2016), with the organizational structure of schools constituting a major barrier (Fair et al., 2018;Guldager et al., 2018). As alternatives, the teachers highlighted the need of the development of play in the recess period and in PE classes, and the possibility of alternative games (e.g. bowling with recyclables).

Strengths and limitations
The main strength of this study is its focus on implementation evaluation, including data collected from multiple actors to, understand different perspectives on the same intervention program. Furthermore, the mixed-methods approach allowed us to describe a broad picture of program implementation and evaluation by combining qualitative and quantitative sources, which helped to answer complex questions that could not be answered by quantitative or qualitative approaches alone (Creswell and Clark, 2017;Glasgow et al., 2019). Finally, the originality of the study is highlighted because of the scarcity of evidence focused on the implementation of PA interventions in LMICs (Naylor et al., 2015). However, the present study also had some limitations: first, the process evaluation was carried out at a single time, which makes it impossible to determine the continuity or discontinuity of the strategies, and our findings were derived from opinions and self-reported measures rather than observation of the strategies.

Implications for research and practice
Teacher training could be better implemented, particularly regarding whether to consider strategies to involve key school stakeholders (e.g. principals). The training of PE teachers seemed to be an important strategy for the development of more active classes and greater participation of adolescents.
The breaks proved to be a promising strategy because of its good implementation and its feasibility for interventions aimed at promoting PA in the school context. Future studies may focus on large-scale implementation and its effectiveness in the school context.
The educational strategies had low reach to the groups of interest, particularly the students and parents.
Future research should likewise investigate how environmental changes in the school context must occur to stimulate changes in students' attitudes and behaviors.

Conclusion
Our findings suggested that the intervention was acceptable to teachers, who considered the theme, classroom discussions, and development of breaks relevant to raising students' awareness about a healthy lifestyle. However, we found a gap between the strategies planned and the reach among students and parents, most of whom did not perceive the implementation of the intervention.

IMPACT STATEMENT
This study provides evidence that teacher training is an important strategy to promote PA, but it may be better implemented if we include principals' involvement. Moreover, PE teacher training and breaks activities seem to be promising strategies for the development of more active classes. On the other hand, educational strategies had low reach to the students and parents. Furthermore, new studies should explore how to implement environmental changes in schools in the context of LMICs.

SUPPLEMENTARY MATERIAL
Supplementary material is available at Health Promotion International online.