Tips for a Flipped Classroom: What We Wished We Knew Before We Flipped

The ﬂipped classroom approach has been used in health profession education to encourage active learning, but predominantly pre-learning ﬂipped activities are passive knowledge input, such as watching recorded lectures. To encourage active learning, online learning activities focusing on knowledge acquisition and application were developed as pre-work for the ﬂipped classroom in a pathology course in an undergraduate medical programme. These included labeling macroscopic pathology and histology images, case scenarios and multiple-choice questions. Student responses indicated that some of these were helpful, while some interfered with learning. In addition to the pedagogical issues associated with designing a ﬂipped classroom, instructors also need to consider the technical elements of designing pre-course material for independent study. This paper examines the components of this ﬂipped classroom’s pre-work and highlights successful and unsuccessful e-learning components.


Introduction
The flipped classroom instructional model has emerged as an alternative pedagogical approach to traditional lecturebased teaching (1,2). Course materials are delivered outside the classroom, while "active and collaborative application of content with the support of classmates and instructors" takes place inside the classroom. Thus the flipped classroom approach uses blended learning, that combines passive learning and active learning (3). engaged in the flipped classroom because "some students considered the flipped classroom activities to be too simple to require higher-order thinking or discussion with others…" (7). Since optimisation of the of pre-class activities is integral for students' acceptance and engagement in the flipped learning activities (7), students' enthusiasm might have been dampened by the passive video watching, which has been the dominant pre-class learning activities in flipped instructional modules (3).
In this Pathology course, active learning activities encouraging higher-order thinking were used as pre-class preparation. Students watched mini-lectures or read for only ten minutes, then completed learning activities that emphasised knowledge acquisition and knowledge application. These included labeling exercises, written questions requiring text answers and multiple-choice questions, all to further consolidate the knowledge acquisition. Learning materials were delivered via an online learning platform, kuraCloud.
Four flipped instructional modules covering cardiovascular (CVS), central nervous system (CNS), respiratory (R) and gastrointestinal (G) anatomic pathology were constructed and delivered to 104 fourth year students in Department of Pathology and Molecular Medicine, University of Otago, Wellington, New Zealand in 2015. Students completed a survey administered in class time after the four flipped modules were completed. This survey requested students provide open-ended responses regarding their usage and reflections of online-learning using kuraCloud, including if learning activities were completed before or after tutorials, how often a video clip was watched thoroughly and whether the correct answer of a question was checked before or after they attempted the question. Seventy-five out of 92 students participated the evaluation. Eight student volunteers attended a focus group interview conducted to further investigate usability of each learning activity involved in the pre-work.

Rationale
Each video covered one topic at the beginning of each flipped module. These provided information delivery via verbal/listening for learners who prefer this learning style over reading text.

Content design and Production
Most of the video clips were delivered as slides with voice over. The duration of each video clip was four to eight minutes. The videos were a mix of pre-made "pathology mini-tutorials" from University of Nottingham available on itunes-u (https://itunes.apple.com/nz/itunes-u/pathology-mini-tutorials/id396418138?mt=10) and where a relevant topic was not available, videos made by the course instructor (DK) in the same style.

Completion Process
The video clips could be paused and replayed, so students could stop to take notes or rewatch a specific section. The kuraCloud software collects data on duration of video watched for each student and self-reported video usage was collected in the questionnaire.

Results
Most students did not watch the entire video clip. Only 39% of students reported that they watched the whole clip of Kenwright D, Dai W, Osborne E, Grainger R MedEdPublish https://doi.org/10.15694/mep.2016.000079 Page | 3 each module at least once while 39% of students terminated watching the video clip before reaching the end. 22% of students reported that they never watched any video clip on kuraCloud. Students who terminated the videos typically did so around the 5-minute mark.
Student survey free text answers indicated a preference for videos made by the course instructor: "I really like DK's videos. The British ones -not so much." A subsequent focus group interview confirmed that students were generally skeptical of external resources "whereas if it's from your teachers you trust it".

Lessons Learned
Students preferred reading over video watching of voice over lectures because it is more time efficient. The lectures by the instructor (DK) were more popular than external videos, because students regarded them as more authentic and authoritative. Video should be kept short and used sparingly in order to maximize student viewing.

Rationale
By actively labeling image features, students can demonstrate that they know and can apply the key features of an anatomic pathology specimen, allowing them to check knowledge acquisition before attending a classroom session.

Content design and Production
Pictures were selected from the photograph collections of specimens of University of Otago, Wellington Pathology Museum and microphotographs of pathology cases from the University of Otago Wellington. Each macroscopic or microscopic image had resolution of 1280 × 800 pixels.

Completion Process
Students learn the features of macroscopic and microscopic anatomic pathology by viewing PowerPoint slide sets, watching video clips and reading texts. They were then presented with a similar photograph to annotate. By correctly labeling a different image, students demonstrate they can recognize key features.

Results
Sixty-one students (81.3%) perceived labeling exercises as useful, among which 33 students (54.1%) reported that they found labelling pictures very useful in consolidating the acquired pathological knowledge.Students appreciated the instant feedback and the visual rather than written task.

Lessons Learned
Students found labeling tasks useful. Short, non-written tasks such as these provide variety and students responded very positively to being able to check their learning with feedback on the labeling tasks. Testing for ease-of-use could reduce students' frustration and allow students to direct their cognition to the content rather than the technical format of the task.

Rationale
When given a clinical stem, students were asked to answer written questions using a combination of the information within the learning platform and external sites, which they were directed to throughout the module. They are able to check their knowledge acquisition by comparing their answer to a model answer.

Completion Process
Students type their answers in a text box beneath the stem. Clickable hints as to the correct answer are offered to assist students before they attempt the question to address perceived knowledge gaps. The model answer pops up when students click on the "check answer" button.Content design and ProductionAll written questions were generated by a lecturer (DK) using fictitious patient scenarios. Students' answers were saved in the learning platform as a revision resource, for students to access a later date.

Results
Written questions were not popular with students, with students perceiving them as difficult and time-consuming. One student commented "I really hated the having to… write with the thing and it took ages and it was just like -flag, I don't care." Another student expressed similar views: "It was good detail that was in the answers but I just wanted to get it in and go back to it at the end of the year." Thirty-one students (41.3%) reported that they attempted the question first and then used the model answer to correct or improve their own thoughts. Twenty-eight students (37.3%) reported they checked the answer both before and after attempting written questions. Sixteen students (21.3%) reported copying and pasting the answers without attempting an answer, because "I know I'll go back to it but at the time I just needed to hand something in and I wasn't learning much to begin with, like before the lectures. I know I'll go back to it but at the time just copy and paste." Students expected that questions would test what they already knew rather than prompting them to search further for additional information in order to answer them.
"The questions didn't reflect taught material, they jumped straight into application & only catered to people who were already reasonably knowledgeable on topics." They expected to encounter activities which required them to engage in more complex thinking only after they had been given a structured overview of the content: Kenwright D, Dai W, Osborne E, Grainger R MedEdPublish https://doi.org/10.15694/mep.2016.000079 Page | 5 "I prefer lecture first then the kuraCloud quizzes."

"I think my ideal would be give us the objectives, we read the readings, we go to a tutorial and they ask us what we don't understand and we talk about it and then we have a quiz."
Students were reluctant to read widely in order to complete the scenario exercises. They preferred to use the material only within the learning platform.
"I just get frustrated constantly fishing for information -you spend an hour to learn one thing where you could spend an hour to learn lots of things."

Lessons Learned
Student engagement might be improved if all information necessary for problem-solving was provided within kuraCloud. Meanwhile, the magnitude of cognitive load involved in pre-class assignments should be lower than we had designed, and test simple knowledge acquisition rather than more complicated synthesis. Otherwise, students might transform the active learning process into passive learning process to save time and avoid frustration.

Rationale
Simple knowledge recall Multiple Choice Questions (MCQs) were used as the final section of each flipped module. They were designed to focus on knowledge recall rather than knowledge application.Content design and ProductionEach MCQ comprises a stem, followed by one correct answer and four distractors. For each choice, explanations of why it is the correct answer or a distractor were provided.

Completion Process
Students perform the MCQ task by selecting one of the choices and then click on the "check answer" button, which then presented if the selected answer is correct or not will be presented.

Results
Students showed high acceptance of the MCQs, with all students performed the MCQ task in each flipped module. They found the testing of knowledge recall and the instant feedback useful and efficient to reinforce knowledge and find gaps in understanding.

Lessons Learned
MCQs at the end of each flipped module are popular with students, especially if they give instant feedback and specific explanations. Without higher-order thinking involved, they are an efficient way to confirm knowledge acquisition to students.

Conclusions and Implications
Our data shows higher acceptance of MCQs and labelling exercises as pre-work of the flipped modules than for written questions. These responses from medical students suggest that in pre-learning for the flipped classroom, active learning is efficient when flipped learning activities reinforce knowledge acquisition, and do not try to access higher-order thinking or clinical integration. Students preferred videos made by their lecturer to videos from other universities, with learning materials from unfamiliar sources regarded as less authoritative and unconvincing. This suggests that the source of learning materials has a crucial effect on students' perceptions towards them. Learning materials from sources that are convincing and familiar to students in pre-work of flipped modules is a motivator for students.
With respect to the design of pre-work of flipped modules, students showed preference for simple operations, such as drag and drop, to typing or painting. Easy operations are time efficient and not technically challenging to students, which focuses students on knowledge recall rather than being distracted by the technical equipment.
Similarly, students preferred single-sourced information to multiple-sourced information, with searching information from external websites during completing written questions unpopular. Information necessary for problem-solving should be provided within the same platform to prevent students from being distracted by searching information from multiple sources.
Future attempts in designing pre-work for the flipped classroom should focus on testing knowledge acquisition through using active flipped learning activities involving lower-order thinking, leaving highly cognitive-demanding activities to the face-to-face sessions. Single-sourced information and learning materials from convincing sources should be provided to students. Learning activities should be completed using easy operations, with typing and other technically challenging operations avoided.