It is not enough to ﬂip your classroom. A case study in the course of Pavements in Civil Engineering

The flipped classroom, as an active learning model, has given remarkable results in several areas of university teaching; however, its execution can still be improved. This research shows the implementation and improvement of the flipped classroom model in the Pavements course. It evaluates their influence on the students’ final grades and on their learning experience. Three groups of students participated in this study, who enrolled in the course of Pavements of the Civil Engineering program. Group A took the course with the traditional model, while Group B took it with a flipped classroom, and Group C experienced it with a reinforced flipped model. Groups took the course in 2017, 2018 and 2019, respectively. Results show that despite using the flipped classroom models, the finals grades did not increase compared to those of the traditional model; however, they improved their learning experience. The students were more satisfied with the method; they even asked for fewer modifications than they did in the traditional model. This research shows that adding little academic tasks to the course would greatly influence the students’ opinion.


Introduction
For decades, educators have questioned teaching methods entirely based on lectures (Barr and Tagg, 1995). In long lectures, students can get distracted and lose some key concepts or ideas in the topic they are learning. In addition, students only retain some knowledge with this method, and sometimes they do not understand it. Learners are passively absorbing the knowledge organized by the teacher, who does most of the work (Le and Do, 2019). Moreover, nowadays, students have started to be less tolerant of lecture-style presentations (Roehl et al., 2013) and, due to technological advances, students prefer innovative methods than traditional ones (Subramanian and Kelly, 2019). Based on these limitations of this method, new learning approaches have emerged.
Active learning is one of those new concepts that engage the students in their own learning process (Prince, 2004).
Students have a more active role, since most of their results depend on them. Instructors are no longer the main actor; they become a guide, who plan, elaborate, and control the activities that students must perform. With these activities, students think about what they are doing, and this promotes the constructive process of knowledge (Bergmann and Sams, 2012;Bonwell, C., and Eison, 1991). Due to its remarkable results, several models have adopted this active learning.
The flipped classroom is one of those models. Its name was given because students have to explore the topic before attending the face-to-face class. They should study from video lectures, screencasts or vodcasts, thus learning concepts out of the classroom (Milman, 2012). In class, there is more time for other engaging activities, such as problem-solving sessions, case studies, collaborative work, observing real situations, discussions, experiments, projects, among others. These academic activities increase their knowledge retention (Blair, 2012;Tucker, 2012). Besides, learning skills are developed when discovering by themselves what they do not know (Le and Do, 2019). Given its promising results, its use is extended to all the educational levels, especially at university level, except in the course of Pavements of the Civil Engineering program.
In this context, this study shows the implementation and improvement of the flipped classroom model in the course of Pavements. It evaluates how the model influences on both the students' final grades and on their learning experience. For this purpose, students who enrolled in the course of Pavements participated in this study. The study considered three groups: group A had the traditional model based on lectures, group B the flipped classroom model, and group C a reinforced flipped classroom model.
To show these results, the rest of this paper is structured as follows. Section 2 shows the literature review about the flipped classroom used in universities, especially in Engineering. Then, Section 3 gives an overview of the experimental development, describing the sample size, model learning description, grading description, course content, data collection, and data processing. Section 4 shows the results considering the students' final course grades and their survey answers. The final part highlights the main conclusions.

Literature review
The flipped model is feasible in universities, since students are used to modern technology (Sohrabi and Iraj, 2016;Subramanian and Kelly, 2019). They also have internet access on most university campuses and homes (Bergmann and Sams, 2012). It allows them to view the web-based instruction at their own pace. They can also watch the digital content as many times as they require. The model favors students who need more time to assimilate knowledge or have attention problems. Another element that promotes the employment of the model is that educators are interested in maximizing their students learning through innovative practices (Hao and Lee, 2016). Based on these elements, the flipped classroom model was used in several areas of knowledge such as Social Sciences, Technology, Engineering, etc.
In Engineering, the teaching-learning approach is represented by technical criteria, within a strong background in Mathematics and Physics (Halbe et al., 2015). Despite this particular background, the flipped model was employed in several Engineering courses with positive results. For example, in Fluid Mechanics, students increased their score in 7,5 points compared with the traditional model group (Webster et al., 2014). In the course of Mechanics of Materials, students in the flipped model performed better than the students in the traditional method (Lee et al., 2015). In an online course of Dynamics, the flipped model improved their learning experience and their problem-solving skills (Fredericks et al., 2013). In Classical Mechanical, most students preferred the flipped classroom model to the traditional one (Bates and Galloway, 2012).
In Civil Engineering, several experiences were conducted employing the flipped classroom model. For example, in the course of Water Resources, students were more involved, and improved their understanding of concepts and problem-solving skills (Li and Daher, 2017). In the course of Statics, learners improved their final grades, became more independent, and they were actively engaged with their learning (García-Ramírez, 2018). Students in the course of Mechanical Engineering, Computing, and Construction Materials admitted that the method had a positive impact on their learning (Gardner et al., 2014). In the course of Transport Engineering, students were satisfied with the flipped model, because they had the opportunity to work at their own pace, interact with their instructor individually, and work collaboratively with their classmates (Karabulut-Ilgu et al., 2016). Despite these positive experiences, in Civil Engineering it has not been employed in the Pavements design field, except for one preliminary study conducted by the author (García-Ramírez, 2019).
Notwithstanding the positive results in Engineering and Civil Engineering, some elements must be considered when applying the flipped classroom model. First, teachers should create quality videos with clear instructions. Second, students cannot ask questions to the teacher during the video (Milman, 2012). Third, students can be distracted by browsing other websites (Roehl et al., 2013). Fourth, in-class and out-class activities must be relevant to the student. Fifth, learners must have devices where they can watch videos. Finally, students should have access to the Internet. The first four elements offer some opportunities to improve the execution of the flipped classroom model.

Sample Size
Three groups participated in this study to answer both research questions. Group A took the course with the traditional model, while Group B took it with a flipped classroom, and Group C experienced it with a reinforced flipped model. Group A had 38 students, Group B had 57 students, and Group C had 70 students. They enrolled in the course in the academic periods of April-August 2017, April-August 2018, and April-August 2019, respectively. All groups were in the fourth year (out of 5) of the Civil Engineering program at the Universidad Técnica Particular de Loja (UTPL). In this course, students learnt how to perform the pavement structural design for highways and airports. The teacher was the same in all groups. Before this course, students had to approve the courses: Highway Geometric Design, Materials, Soils and Rocks, and Physics.

Learning models descriptions
The learning models used for the three groups are shown in Table 1. In the traditional model, students had to study some material shared through the Virtual Learning Environment (VLE) or in books. Then, they attended the face-to-face class, where they sometimes had to solve reading controls at the beginning of the session. Later, they received a lecture about the week topic. Finally, they participated in a problem-solving session individually o in teams, using a solved example from books. The teacher showed some essential parts to solve it and in the end, the answer to the problem. The teacher helped if they had questions to ask. After in-class activities, students had to solve other problems as homework. They could attend a weekly tutoring session if they had questions when solving or trying to solve the homework.
In the flipped model, students have to watch a pre-recorded lecture (in a video) before attending the class. The videos were uploaded on the Youtube platform and shared through the VLE. To check if they watched them, learners had to solve reading controls at the beginning of the class. Then, the instructor answered questions about the weekly topic and explained their main concept. Afterward, students participated in a problem-solving session individually o in teams, using a solved example made by the teacher, who also showed some essential parts to solve it and in the end, the answer to the problem. The teacher helped them if they had questions about it. They also had to solve another problem as homework. A weekly tutoring session was available if students had questions about the problem and /or the weekly topic. In the reinforced flipped classroom model, students had to do the same as in the flipped model, but includingfield visits to a building project related to the course contents. These visits are conducted by the students on their own and without the presence of the teacher. After the visit, they had to make a report about the activities in the construction work. For every activity, photos were mandatory. These photos should include the student, as well as the date and time, to avoid sharing between them or reusing the same photos in their next reports. In this model, the teacher was more involved in their learning, giving feedback in every academic activity, and explaining its aim. So, students knew their mistakes and understood the reason for each activity. Considering that students in group B could skip watching the videos on Youtube, in this case the videos were uploaded on the Edpuzzle platform. In this platform, after students created their users, they had to watch the previously scheduled pre-recorded lecture (start and due date). In this platform, some questions could be added during the video (multiple selections, true/false and short answer) and could set an option to prevent skipping, so students had to watch the whole video. The face-to-face class started with the control reading, and students asked about the field visits or the weekly topic.

Grading description
All three groups had the same grading procedure and grading factors. Reading controls had a weight factor of 20% of the final score grade. The in-class problem-solving session was 15% of the final score, as well as for solving problems as homework. This homework could include up to two more problems related to the week class. The remaining 50% were the midterm exams. Students could get a maximum score of 100 points, and they should have a minimum of 70 points to approve the course.

Course content
The course was 16 weeks long. All three groups had the same course content, as shown in Table 2. This table also included the duration of the pre-recorded lectures. In some weeks, they had no video to watch, because it was the beginning or end of the course, or the midterm exams. Pre-recorded lectures were short according to the literature suggestions (Bonwell, C., and Eison, 1991;Enfield, 2013;García-Ramírez, 2018;Sohrabi and Iraj, 2016). Videos were made using digital slide presentations with voiceovers in Microsoft PowerPoint software.

Data collection
Data collection was performed using two instruments: final score grades and a blind survey at the end of every course. The teacher calculated the student's final scores according to the grading plan. The average grade was estimated in every group, in order to compare it with other groups. A frequency distribution was also made with the individual final score grades to compare the differences between groups. The survey was optional, so 58 students answered it in group A, 55 in group B, and 66 in group C. The questionnaire was composed mostly of several closed-ended questions and a few open questions. Questions were about the course and the applied method.

Survey data processing
The survey had three types of questions: numeric (rates), closed-ended, and open. It calculated the average rate from all answers of the numeric questions. In closed-ended questions, it estimated the percentage where students agreed (strongly agree and agree). This information provided an overview of the opinion of the students, since the other results (neither agree nor disagree, disagree, strongly disagree) are their correlative. Finally, in open questions, similar answers were clustered in several categories. When a response had no match with others, it was classified and grouped in the category "Others".

Final course grades
The final average score grades were: 72,82 (group A), 69,68 (group B), and 71,58 (group C). In order to find a statistically significant difference between those values, an analysis of variance (ANOVA) was performed using the statistical software Minitab 14.2 (2005). A 95% level of confidence was used as a parameter of this analysis. Scores from group A did not differ significantly from group B (p-value = 0,066), which means that traditional model had similar scores to the flipped model. Scores from group A also did not differ from scores from group C (p-value = 0,380), which means the flipped method did not increase the grades significantly when compared to the traditional one.
The individual final scores were drawn in a histogram to analyze their distribution in all study groups (Figure 1). Group A had 68% of students with equal or more than 70%, while group B had 49%, and group C had 51%. Most students had between 70 and 79 points in all groups. Students (2%) with the highest score belong to group B. In the range of 80-89 points, there were 13%, 4%, and 30% of students, from group A, B, and C, respectively. Group A had 19% of learners that obtained between 60 and 69 points, that is, close to passing the course. However, it is interesting that the flipped models had more percentage in this range, 33% (group B) and 30% (group C). Even though, more students approved the course under the traditional model, the flipped models had more distributed scores. In addition, they had some interesting trends and behaviors in some ranges that traditional models did not have. In order to see if groups were academically homogeneous, the difference between the grades was analyzed in the first part of the semester (up to the midterm exam) and compared to the second one (up to the final exam). Most students in group A (76,47%) got the same or a higher grade in the second midterm than in the first one. Students from group B performed a little lower (63,16%), and group C was the lowest (21,74%). These differences could mean that students in group A are better students, in terms of grades, than those in other groups. This could also mean that learners in group A were more motivated than other students in group B and C. These assumptions should be analyzed in future research.

Survey answers
The survey had several parts: rating, selecting options, and responding to open questions. In the rating section, students were asked to rate the course and their self-learning throughout the course. Table 3 shows their average rating values for both questions. The traditional model got a higher average rate than the flipped classroom, and this one got a smaller value compared to the reinforced flipped classroom model. ANOVA analysis was conducted to see if there was a statistical difference between groups of answers. The average rate between group A-B and A-C did not differ statistically (p-value = 0,318 and p-value = 0,126, respectively). However, the average rate between group B-C differs (p-value =0,003), which suggests that the students' opinion about the method is more positive in the reinforced flipped classroom than in the simple flipped model. 1 How many points would you give to the Pavements course?
2 How much self-learning did you do in the Pavements course?

Source: Authors
As expected, concerning self-learning, both flipped classroom models got higher average rates than the traditional one.
In addition, an ANOVA analysis was performed with those answers. An average rate differs statistically between groups A-B (p-value = 0,037), but not in groups B-C (p-value = 0,367). This implies that in the flipped models, students are more aware of their self-learning than in the traditional one and, consequently, they consider that they work more than the students of group A.
Students also responded about the methods executed with closed-ended questions. These items had a five-point Likert scale: strongly agree, agree, neither agree nor disagree, disagree, strongly disagree. Table 4 included the percentages of the agreement answers (agree and strongly agree). 3 The topics of the subject were interesting. 4 In class, the number of problems performed was enough. 5 The problem-solving carried out in class was representative of each topic. 6 The other activities carried out in class (tests, teamwork) were enough. 7 The teacher mastered the topics of the course. 8 The grading of works, lessons, and exams was fair. 9 The learning method was well implemented by the instructor. 10 This methodcan be employed in other courses of Civil Engineering. 11 The field visits helped your learning. 12 The flipped classroom model is better than the traditional one.
-This question was not asked to this group.

Source: Authors
In general terms, topics were interesting for all groups.
Most students estimated that in-class and out-class academic activities were good enough and in accordance with their expectations for all three groups. Additionally, most of them thought that the teacher mastered the topics of the subject. Grading had a lower score in the flipped model than in the others, which means that learners were unsatisfied with the grading scores. Interestingly, the reinforced flipped model got the highest score when students answered about the implementation of the method. However, the flipped model had the lowest score, which means that the students evaluate other aspects, independently of the learning method.
The reinforced flipped group has the highest percentage between groups in all of these questions. Besides, when they perceived that the method was well implemented, they suggested that it should be used in other courses of Civil Engineering. Most students under the reinforced flipped model admitted that the visit fields helped in their learning and complemented the other academic activities. Most of them also considered that the reinforced flipped model was better than the traditional one.
Regarding the few open questions that students answered, one question focused on what changes they suggest for the next course of Pavements. Table 5 shows their answers to those questions after processing them. It is remarkable that they suggested fewer modifications in the flipped models than in the traditional one. This means that in the flipped models, students are more satisfied than in the traditional one.

Source: Authors
Some students from the traditional model and few from the flipped model suggested that field visits should be executed to learn more. For that reason, in the reinforced flipped model, field visits were implemented. In the traditional model, some students required that, at the beginning of the class, the teacher should solve and explain a problem, before students solve their problem example. Therefote, in group B, instead of solving the homework problem, the teacher solved a different problem as guiding document, with similar input parameters to their problem example. However, in group B, few students made the same request again. So, in group C, in addition to the guiding document, feedback was implemented, and very few students complained about it. The teacher did not want to show them how to solve the problem because he/she considered that students could learn more and more permanently if they solved it on their own. Furthermore, some students in group A and C asked for asphalt mixtures theory or including laboratory practices as part of the course teaching plan. Maybe, they made this requirement because previous teachers included those topics. Finally, some students had other requests that ended up in the "other" category.
Students in the flipped classroom models also answered if the videos helped them in their learning process. Most students (98,18%) in group B and group C (98,48%) said that pre-recorded lectures helped them in the course. These answers were clustered in 6 categories, as shown in Table 6.
Two additional open questions were asked in group C in relation to the feedback process and field visits. Most students (83,82%) recognized that they agreed with the feedback given by the teacher. Some of them (10,29%) considered that feedback should include more detailed observations or include videos, documents, or the solved problem. A few students (5,88%) asked for clearer feedback. Concerning the field visits, most students (39,17%) agreed with the frequency and report that they should be done. Other students (16,18%) considered that the teacher should be with them and solve their doubts in-situ. Some students (11,76%) answered that field visits should be arranged in more significant building works. Few students (8,82%) demanded that selfies should not be included in the reports, and others (16,18%) had minor requests that were not categorized.

Conclusions
This article aimed to show the implementation and improvement of the flipped classroom model in the course of Pavements. It evaluates their influence on the students' final grades and on their learning experience. After analyzing the results, the following conclusions were obtained: The flipped classroom did not increase the student's final grades compared with the traditional model. Other studies reported improvements in grades with this method. For example, in the Statics course students got 6,5 points more than with the traditional model (García-Ramírez, 2018), or in the Fluid Mechanics course, learners got 7,25 points more than with the traditional model (Webster et al., 2014).
Despite of the unaffected final scores, the student's opinion about the flipped classroom model was positive. This was also observed by previous researchers (Gardner et al., 2014;Karabulut-Ilgu et al., 2016;Li and Daher, 2017). Even though the flipped models obtained lower values in the survey with a five-point Likert scale compared with the traditional approach, most students in the flipped classrooms did not want to change academic issues in the executed method, which means they were satisfied with the model. This was also found in other studies (Bates and Galloway, 2012;Gardner et al., 2014). Their answers in the open questions show wider support to this aspect.
Students have a favorable opinion about the flipped classroom model. First, the model promoted their self-learning, more than the traditional model. This was also found in a previous work by García-Ramírez (2018). Self-learning is a competence that is valuable in the labor market. Besides, the prerecorded lectures helped their learning process. It must be considered that students learn differently and having online available videos is very helpful for them. This method showed better results than the traditional one in terms of academic experience and not in terms of increasing grades.
The contribution of the paper is basically methodological. First, previous studies only compared two groups (traditional and flipped), while this one compared three, from which the last group made improvements (in the learning methodology and academic activities performed) based on the student requests from the previous groups. Second, academic activities such as field visits (for placing the students in real situations) and timely feedback have not been performed in previous literature related to the flipped model in Engineering. Third, the study employed pedagogical approaches to prevent skipping academic tasks (questions in videos on Edpuzzle or selfies in field visits). Fourth, the methodology used in this study (data collection, processing, and interpretation) can be employed in other course of Civil Engineering. Finally, this research shows that the flipped classroom model is not only about eliminating the lecture method; but it is a process of continuous improvement and commitment between instructors and students.
This study has several limitations. First, groups were not homogenous, since they got different grades in the second part of the semester. Besides, the study was also conducted in one university with a small sample size. Furthermore, it involves just one course that belongs to the Civil Engineering program. In other course from the same area or from other careers, this could have different results.
Despite these limitations, this study extends the understanding of the implications of the flipped classroom model. It showed that the flipped method significantly improves the learning experience compared to the traditional one. It also showed that the flipped model is accepted positively by students. In addition, the model promoted self-learning and helped students learn at their own pace. Besides, this study had better results on educational activities than on the grading. Finally, it showed that it is not enough to flip the classroom, since teachers have to be more involved in the learning students.