A simple flip of an obstetrics clerkship lecture focusing on interactive learning

Objective: To determine the value of interactive learning after a low-tech flip of a traditional lecture during an obstetrics and gynecology clerkship. Design: All third-year medical students completed a flipped learning experience between May 2014 and April 2016. Central to the change was replacement of a mid-clerkship lecture ("late term and prolonged gestation") with interactive learning at seven stations by student pairs (one each on separate obstetrics and gynecology services). Before class, students electronically received a handout that described learning objectives, subject background, and interactive stations. The stations featured manipulative models, instruments, data, and images involving prenatal care, fetal growth and testing, and labor and delivery decision-making. Results: The flipped model was easily executed with proper preparation. The 178 consecutive students completed the two mandated surveys. The median score given by students about the same instructor’s effectiveness increased from 4.0 (previous two years) to 4.4 (on a 5-point scale). Compared with traditional lectures by other clerkship faculty, the flipped classroom was judged by students to be easier for understanding and more interactive. Students perceived being more responsible for learning with better recall and application to practice. Conclusions: A low-tech approach to the flipped classroom was easily executed with favorable responses from students about interactive learning.


Introduction
The flipped classroom instructional strategy is notable for references in the popular media and an abundance of internet resources, but has received very little scholarly scrutiny until recently. Although variously defined, we follow Abeysekera & Dawson (2015) to consider the flipped classroom as a suite of pedagogical approaches that preserve scheduled class time for interactive learning among students that focus on knowledge application and construction of new knowledge (Chi & Wylie, 2014), by requiring completion of pre-class assignments that build the necessary information foundation for the in-class activity. The flipped classroom approach has been described recently in basic-science and clinical education of medical students (Belfi, Bartoletta, Giambrone, Davis, & Min, 2015;Hernandez, Brown, & Foster, 2016;Morgan et al., 2015;Street, Gilliland, McNeil, & Royal, 2015), residents (Tainter, Wong, Cudemus-Deseda, & Bittner, in press;Urban, Swensen, Schulkin, & Schiff, 2016), and general hospital house staff (Tan, Brainard, & Larking, 2015). (2015) and DeLozier and Rhodes (in press) reveal nearly universal preference by learners for the flipped approach compared to traditional lecture but less clear results for the relative benefits to student learning. Evaluating the learning benefits of the flipped classroom versus didactic is hampered by a lack of standard execution of either lectures or flipped instruction and the relative dearth of published studies that rigorously compare student learning in the two modalities. Nonetheless, existing data show no losses in learning in a transition from didactic to flipped along with a mixture of learning gains that range from slight to substantial (DeLozier & Rhodes in press;O'Flaherty & Phillips, 2015).

Recent reviews and meta-analyses by O'Flaherty & Philips
A likely limitation of these meta-analyses is the authors' restriction to reviewing literature that includes variations of the word "flipped" in the title or keywords. Although the "flipped" term is relatively new, the offloading of content knowledge to pre-class preparation in order to replace lecture with active learning has been common in higher education for several decades (Ent 2016;Svinicki 2013) and the benefits to student learning achievement of replacing lecture with active learning are well known (Freeman et al., 2014). Abeysekera & Dawson (2015) postulate that the enhanced learning of the flipped classroom is rooted in the reduction in cognitive load afforded by self-paced learning of dense content, along with autonomy and competence development provided by interactive learning with peers.
The recent popularity of the flipped classroom model is driven by the availability of technology tools for delivering pre-class content and the perception that the current students prefer learning from videos and interactive online tools rather than reading. Some authors (e.g., Prober & Khan 2013;Sharma, Lau, Doherty, & Harbutt, 2015) focused on the preparation of online materials, although the cost and time commitment to generate pre-class videos is also a barrier to adopting the flipped classroom (Hernandez et al. 2016;Moraros, Islam, Yu, Banow, & Schindelka, 2015;O'Flaherty & Phillips 2015). In addition, students commonly complain that videos are too long or boring (Guo, Kim, & Rubin 2014;Khanova, Roth, Rodgers, & Mclaughlin, 2015;Moraros et al. 2015) or prefer to read text because of greater ease of self-pacing, annotation, and efficient relocation of information for study (e.g., Persky, 2015, Long, Logan, & Waugh, 2016. Although students recognize the learning benefits of active learning in the flipped classroom, they commonly resist the added responsibility and time dedication for class-time preparation (O'Flaherty et al. 2015). To help alleviate this pressure, some instructors diminish classroom-contact time to account for learners' self-paced learning prior to class (Morgan et al. 2015, Street et al. 2015, Ryan & Reid 2016. DeLozier & Rhodes (in press), Foldnes (2016), and Jensen, Kummer, & Foster (2015), conclude that the student interactivity during class, and not the pre-class, internet-delivered content, is most critical for flipped-classroom learning . We argue that placing instructional design emphasis on the in-class pedagogy, rather than the technology- delivered pre-class pedagogy, is an especially important consideration in medical education where traditionally, in most countries, courses are instructed by many different teachers, some of whom are responsible for delivering only a single class session. Expecting uniformity of approach and utilization of online tools from multiple teachers can inhibit flipping of an entire course and, therefore, discourage changes by progressive faculty. Focusing on the interactive in-class component of the flipped classroom encourages implementation of the pedagogy by individual teachers.
We describe the flipping of a single clerkship didactic session and the students' evaluative response in comparing the experience to their dominant lecture experiences. The session design focused on interactivity during scheduled classroom time with minimal preparation by students without the necessity to engage with complex technology tools.

Methods
The study was conducted within the third-year clerkship at the University of New Mexico School of Medicine (USA). The clerkship lasts eight weeks, with Fridays being dedicated to an average of five faculty-presented lectures. A total of 178 students participated during nine consecutive clerkship rotations between May 2014 and April 2016. Approximately 14 students are assigned to the clerkship at any time, with a four-week period dedicated to obstetrics training and the other four weeks for gynecology training. No examinations were administered or grades assigned after the lectures, but attendance was taken and protected time was set aside for all Friday classroom sessions. This evaluative educational project was exempt from review by our institutional review board, because no medical student identifiers were collected.
The evaluated classroom session dealt with the subject "Late Term and Prolonged Gestation." The same instructor (WFR) has been responsible for the learning session and objectives since 2005. The lecture was consistently presented at the midway point of the clerkship, when half the students completed their commitment on either the obstetric or gynecologic service. While the passive lecture was previously well-rated by the students, the instructor wished to re-invigorate the session. The goals of the lecture redesign were twofold: (1) to promote interactive learning among students and between students and the instructor throughout the learning session, and (2) to improve the students' learning and develop students as critical thinkers, problem solvers, and team members.
The overall design of the flipped classroom came from an earlier synthesis by Moffett, (2015). Activities undertaken in the pre-class, classroom, and post-class periods are listed in Figure 1. Pre-class activities were used to support cognitive levels of Bloom's taxonomy (Anderson & Kratwohl, 2001) (e.g. remember and understand) and classroom activities were intended to facilitate higher levels (e.g. apply and analyze).

Pre-class Activities
Four days prior to the Friday session, each student received an email message from the clerkship office with a twopage attachment. The message provided a brief explanation of the session format and a link to a 1-minute video describing flipped classroom approach ("What is a flipped classroom?" https://www.youtube.com/watch? v=r2b7GeuqkPc) The two-page attachment included a one page, instructor-authored, required reading with critical background content about prolonged gestation. The second page described the learning objectives andprovided a list of interactive-station topics for the upcoming class session as well as references to 20 pages of optional reading in obstetrics textbooks and a peer-reviewed journal.

Classroom Activities
Rayburn W, Smith G, McGuire B, Skipper B MedEdPublish https://doi.org/10.15694/mep.2017.000024 Page | 4 The 75-minute classroom session began with a stage-setting micro-lecture (5 minutes) by the instructor and assignment of the approximate 14 students in pairs to each of seven stations. Each pair consisted of one student completing the obstetrics service and the other student rotating off the gynecology service. This deliberate pairing was intended to leverage peer-assisted learning (Ross & Cameron, 2007). The student with obstetrics experience would be better prepared to teach the student with only gynecology clinical experience. Each student received a worksheet that included, for each station, a topic description, case scenario, questions to answer, and a list of applicable obstetric terms. The worksheet directed the interactivity between each student as they rotated through the seven stations.
Each station consisted of a brief (25 words or less) case scenario that provided a context for authentic data (e.g., electronic health records, fetal heart rate tracings, fetal sonograms, uterine activity monitor tracings) or manipulatives (e.g., instruments, pelvic model) and three questions related to the topic. The seven stations focused on the following topics: gestational dating; fetal growth patterns; late prenatal care; antepartum fetal surveillance; labor curves; intrapartum instruments/procedures; and intrapartum fetal heart rate monitoring.
A total of 35 minutes were used for the student pairs to rotate to all stations (5 minutes per station). After completing the final station, the student in the pair who just rotated off of the gynecology service was responsible for presenting answers to the entire group about questions at her or his last station (25 minutes total). The instructor added comments that further qualified answers to all questions raised by the presenter and other students and ensured that all of the terms (and many principles) elicited or utilized in the prompts at each station were mentioned and used correctly. The brief remainder of the session was dedicated to feedback as to how to improve this learning activity.

Post-class Evaluation
As in the past, each student was asked to evaluate the effectiveness (using a 1-5 Likert scale from least to most effective) of the instructor in presenting the class material. Written comments about the experience were encouraged in a separate space on the evaluation form. The clerkship coordinator, who was not involved in this study, retrieved all evaluation forms immediately after class.
In addition, students' attitudes were sought about the flipped classroom educational experience and their perceived integration of knowledge from the session. Each student completed a 6-question survey administered electronically at the end of the clerkship (average of four weeks after the lecture). The student was asked to score (Likert scale 1-6; 1: strongly disagree; 6: strongly agree) the value of the flipped classroom session in relation to lecture-based teaching by other faculty during this same rotation.
The two written evaluations, immediately after the lecture and about four weeks later, were mandatory before completing the clerkship. All students were informed that their evaluation would be de-identified and their final grade unaffected by their responses. We chose the evaluative period to cover two full academic years, so that sufficient information would be collected. Data about the instructor's effectiveness in presenting the traditional lecture on the same subject were gathered from the previous two years. All responses were entered into a Microsoft Excel spreadsheet and converted to a SAS dataset. The SAS version 9.3 was used to calculate mean scores for student attitudes toward the experience and about their integration of knowledge.

Results
Rayburn W, Smith G, McGuire B, Skipper B MedEdPublish https://doi.org/10.15694/mep.2017.000024 Page | 5 Complete information was available from 178 (85.6%) of all students who rotated on the clerkship. The primary reason for students being excluded was absence from the classroom activity. This failure to attend the class was uniform during the two-year period. All students reported that pre-class preparation required 20 minutes or less. All pairs of students were engaged in the one-hour 15-minute learning session.
The mean score of the instructor's effectiveness during traditional lecture was 4.0 (range 3.7 to 4.4) between May 2012 and April 2014. Scores for each clerkship block during the subsequent flipped learning session period ranged from 4.2 to 4.9 (mean 4.4).There was no relation between the effectiveness score of the flipped learning session and time of year or order of the rotation during the clerkship year.
Students rated the flipped learning model to be a more positive learning experience than the other traditional lectures given by faculty during the same clerkship rotation. The class led to an improvement in self-perceived knowledge. Table 1 displays the students' responses to the question "To what extent did the flipped classroom compare with traditional lecture setting?" Students were inclined to agree or strongly agree that the flipped classroom was more interactive with better application to practice. The students reported that this experience prompted them to be more responsible toward learning with improved retention, greater recall of principles one month later, and better application to practice. Most students tbelieved this session to be the same or easier for learning compared with the didactic format.
Written comments by the students (Table 2) were overall favorable about the flipped classroom activity. The interactive nature was especially appealing to both students and instructor for discerning what students knew and did not know. Participants stated that priori preparation for the sessioncould be emphasized more, and those who had Rayburn W, Smith G, McGuire B, Skipper B MedEdPublish https://doi.org/10.15694/mep.2017.000024 Page | 6 no obstetrics experience felt less comfortable in being expected to know the material.

Discussion
This evaluative education study aimed to determine the value of flipping a lecture during an obstetrics and gynecology clerkship by replacing a single didactic with an interactive-learning session. There is currently no standard or recommended way to design and implement a flipped learning session during a clerkship. Flipped approaches typically incorporate technology through videos, text, and online interactive learning to deliver course content (DeLozier & Rhodes, in press;O'Flaherty & Phillips, 2015,). However, our approach did not require creation of elaborate online media. Rather, the replacement of lecture in this example focused on providing handson learning using medical devices and screening tests to help students integrate knowledge for assessing and managing prolonged pregnancies. Notably, this example demonstrated the appropriateness of the flipped model for a single class session within an otherwise multi-instructor course; therefore, it was relatively easy to accommodate flipped sessions without flipping an entire course or clerkship.
Our results can be compared to other studies of flipped-classroom pedagogy with health care students. Work by McLaughlin et al. (2014) with pharmacy students involved self-paced online videos as a pre-class activity and class time for interactive exercises. They found that class attendance, students' learning and perceived classroom value increased following participation in a flipped classroom environment?. Another study conducted with nutrition students found that they preferred the flipped classroom approach to traditional lectures with case-based learning (Gilroy, Heinerichs, & Pazzaglia, 2015). Similar to our study, this research demonstrated that a lecture with a well-Rayburn W, Smith G, McGuire B, Skipper B MedEdPublish https://doi.org/10.15694/mep.2017.000024 Page | 7 chosen pre-class activity followed by an interactive, problem-based session increased integration of knowledge and was more valued by learners. Urban et al. (2016) report the flipping of an entire ob-gyn residency rotation, using a traditional approach of requiring only reading to precede interactive didactic sessions. Resident performance on board exams notably increased only for the content of the rotation that was flipped.
This study adds to a small number of published studies of flipped clerkship didactics that can help medical educators design more widespread changes for the clinical component of medical student training. Belfi et al. (2015) found that radiology clerkship students preferred the blended online and in-class experience of a flipped curriculum to either lecture alone or independent learning alone. Comparison of pre-and post-tests showed very significant and slightly significant gains for the flipped format compared to lecture and independent learning, respectively. Morgan et al. (2015) flipped an entire obstetrics-gynecology clerkship by substituting two-thirds of class time with online videos and using the one remaining weekly class hour for interactive learning. National Board of Medical Examiners scores in the obstetrics and gynecology subject area were the same with the flip as previously with the lecture, showing no loss of learning when students engaged with the content in self-paced, interactive approaches rather than a pedagogical model emphasizing more passive transmission of content from experts to learners. Students surveyed by Morgan et al. (2015) showed strong satisfaction with the flipped format while attributing more learning value to the interactive in-class activity than to the online videos.
A major strength of this research was its prospective nature. We designed this study to assess the satisfaction of a cohort of learners, following exposure to a flipped classroom approach. This same cohort was then followed four weeks later to determine further effects on a more interactive approach to learning. Another strength of this research was that students were mandated to give feedback anonymously following the experience which ensured high numbers of responses and prevented response bias. Another major advantage was that the flipped classroom was a good vehicle for students to actively use medical terminology, allowing them to better understand the vocabulary surrounding the topic of prolonged gestation. Lastly, students on the obstetrics service were learning as "teachers" while those on the gynecology service were learning as "students" and "presenters." It was also important that the instructor act as a participant to stimulate discussion and answer questions, requiring a large class space to create a more interactive and collaborative learning environment.
A limitation to our research was that modifications were made to instruction in response to student feedback as we progressed through the study. Some minor clarifications were made to stations with the addition of clinical scenarios to ease in student understanding of concepts. Therefore, students at the beginning of our study were not exposed to the exact same set of conditions as those who went through the clerkship at a later date. However, mean satisfaction scores did not significantly differ between these groups suggesting that the value learners placed on the flipped classroom approach did not depend on these minor clarifications. Another limitation was the comparison with a historical, rather than a contemporary control group, in evaluating the instructor's effectiveness. However, the lecturer and material presented remained the same and the mean satisfaction score was recorded from two years before a trial of the flipped classroom. Because student responses were anonymous,comparison with subsequent individual performances on the final oral and shelf examinations was impossible. Therefore, the increase in retention and improved recall can only be assessed subjectively. Finally, a limitation to the study design was that we failed to differentiate between impressions of students who rotated on either the obstetric or gynecology services before the classroom. Lower satisfaction scores could have come from students on the gynecology service who were less familiar with the material.

Conclusion
Integration of the flipped classroom model was an effective innovation in educating medical students during a busy core clinical clerkship. We consider the students' favorable overall responses to immediate and delayed anonymous evaluations to be encouraging. Here we have created a non-threatening, interactive, case-oriented and problemfocused learning environment. Ultimately, with increasing demands on both instructor time and educational costs, there is a need to study and implement more efficient and creative strategies for medical student education. Greater utilization of interactive face-to-face learning with simple, low-cost, self-paced learning materials warrants broader application during clinical clerkship rotations and long-term effectiveness research.

Take Home Messages
We consider the students' favorable overall responses to immediate and delayed anonymous evaluations to be encouraging. Here we have created a non-threatening, interactive, case-oriented and problem-focused learning environment. Ultimately, with increasing demands on both instructor time and educational costs, there is a need to study and implement more efficient and creative strategies for medical student education. Greater utilization of interactive face-to-face learning with simple, low-cost, self-paced learning materials warrants broader application during clinical clerkship rotations and long-term effectiveness research.