Drivers of openness to online and flipped learning in medical education

One online teaching approach that has gained popularity in health science education is the ‘flipped classroom’ where students undertake content learning prior to face-to-face class time, often via watching pre-recorded videos. This frees up contact time for activities which stimulate higher-order thinking. While previous studies have generally reported neutral to positive student feedback and effects on academic attainment, the quality of pre-class resources and time required for pre-class work have been identified as potential problems with the approach. A further issue identified in the literature is students’ openness to adopting new forms of learning. This paper examines students’ responses to the flipped classroom in two undergraduate medical education courses, in which students were assigned pre-work comprising a mix of short video lectures and brief online activities. Following the unanticipated finding that students in the fourth and fifth year courses formed very different views of a similar flipped learning intervention, this study explored: • How do students use online pre-class resources in a flipped classroom? • What factors influence students’ openness to using online resources in a flipped classroom? This study used a course evaluation survey followed by focus group interviews to understand students’ use of and attitudes towards the flipped classroom. Many students in groups reversed or wanted to reverse the intended ‘flipped’ order of classes to use the online material as revision aids rather than as pre-class work. Students in both groups identified that time pressure influenced the way they used the pre-class material. The fifth year cohort held a largely positive attitude towards the flipped classroom whereas the fourth year group held a largely negative view. Differences students’ willingness to tolerate uncertainty, connections with clinical learning and metacognitive view of ‘learning’ itself appear to account Kenwright D, Osborne E, Dai W, Grainger R MedEdPublish https://doi.org/10.15694/mep.2016.000080 Page | 2 for this difference in outcome.


Introduction
Flipped learning inverts the traditional pattern of lectures delivered by experts followed by independent study after class by moving didactic lecture content before class and using contact time with instructors for students to clarify and apply the material they have learned before class (Abeysekera & Dawson, 2015;Kurup & Hersey, 2013;Little, 2015;O'Flaherty & Phillips, 2015).Pre-class work engages students largely in passive, lower-order thinking through watching video lectures (Abeysekera & Dawson, 2015;Khanova, Roth, Rodgers, & McLaughlin, 2015;Kurup & Hersey, 2013;Morgan et al., 2015;O'Flaherty & Phillips, 2015;Tune, Sturek, & Basile, 2013) but may include brief activities such as low stakes assessment (Gilboy, Heinerichs, & Pazzaglia, 2015).Face-to-face teaching time focuses on active learning to engage students in higher order thinking, with the guidance of teaching staff (Kurup & Hersey, 2013).Pedagogically, the flipped classroom is well-placed to realise the constructivist principles of designing learning with a focus on engaging all students in activities that "what the student does" by actively processing and restructuring knowledge (Austin, Sharma, Moore, & Newell, 2013), scaffolded engagement with increasingly complex material (King, 1993) and allowing the learner to learn at their own pace (Abeysekera & Dawson, 2015).Furthermore, good flipped classroom design combats the drop in attention that students typically experience in lectures (Risko, Anderson, Sarwal, Engelhardt, & Kingstone, 2012;Young, Robinson, & Alberts, 2009), by using a combination of shorter pre-class tasks (Morgan et al., 2015) and in-class activities which require a higher level of student participation (Khanova et al., 2015;Kurup & Hersey, 2013;Moffett, 2015).
In the context of pathology teaching, online learning can give students greater control over the way they view specimens (Ruiz, Cook, & Levinson, 2009) and expose students to a wider range of cases than they may encounter during clinical placements.Online resources provide a practical solution to managing increasing student numbers and limited access to older, deteriorating pathological specimens (Giroud et al., 2012).These considerations are especially relevant where a university is training more students in the same physical space.

Course design
A flipped classroom approach was implemented in two courses at the University of Otago, Wellington.The first was a year-long fourth year pathology course, in which the flipped classroom was used to teach four of seven modules in anatomic pathology.The second was a fifth year Obstetrics and Gynaecology run, where the flipped classroom was used to teach the pathology of Obstetrics and Gynaecology concurrently with clinical placements over five weeks.
This project used the kuraCloud platform (KuraCloud Ltd, 2014) to deliver flipped pre-class work consisting of: short video segments delivered either by the instructor (DK) or sourced from other pre-existing pathology education videos; written summaries of conditions with links to external sites giving more information about diseases and disease processes; and exercises (labelling slides and images, and short answer questions).Feedback on these exercises via automated answers was integrated to overcome the lack of access to guidance in the e-learning modules.In both courses, the same instructor (DK) oriented students to the role of the flipped classroom and delivered the face-to-face components of the flipped teaching sessions.

Evaluation
Fourth year students completed an evaluation of the flipped classroom immediately after the final anatomic pathology class for the semester.These data indicated students were dissatisfied and disengaged with the flipped classroom.This was at odds with anecdotal reports from the fifth year group, who reported a high level of satisfaction with this approach.Therefore, the study was expanded to investigate the different responses of two cohorts of students who had received a similar form of teaching.
Quantitative survey evaluation of the flipped classroom included questions on students' self-efficacy, taken from the Motivated Strategies for Learning Questionnaire (Pintrich, Smith, Garcia, & McKeachie, 1993), perceived usefulness, satisfaction with instruction and ease-of-use.These variables were measured on a seven-point Likertstyle scale.All quantitative data analysis was conducted in SPSS 22. Since survey data were ordinal variables, nonparametric tests were used.The Mann-Whitney U test was used to compare metacognition, meta-cognitive selfregulation, perceived usefulness of the flipped classroom, perceived ease to use of the kuraCloud and perceived satisfaction with the interaction with the instructors between fourth and fifth year students.
The qualitative component of the survey included questions on the way students used specific components of the pre-work, questions about students' study routines and free text questions on students' overall experience of the flipped classroom.
Focus group questions were developed based on an inductive thematic analysis of the responses to this survey in order to understand the gaps between the aims of the flipped classroom and students' experience of it (see Results section).The focus group interviews were audio-recorded and transcribed then analysed thematically first using open coding, then sorting these codes into themes, beginning with the themes developed from analysing the surveys (Braun & Clarke, 2006;Burnard, Gill, Stewart, Treasure, & Chadwick, 2008).This paper covers the aspects of the evaluation that pertained to students' use of the online resources and their study habits.It excludes data from the evaluation on the face-to-face teaching component in order to focus specifically on students' usage of the online materials.A broader report of the complete flipped classroom trial is available in our forthcoming paper (Kenwright et al., Forthcoming).

Participants
Eighty-one fourth year students (81/95, 85.3%) and 43 fifth year students completed the post-course evaluation survey.The fourth year students completed the evaluation immediately following their final flipped class.The fifth year group were subsequently invited to complete the evaluation, which for some students fell immediately after their participation in the flipped classroom and for some students occurred some months after the completion of their Obstetrics and Gynaecology run.Six fourth year students and four fifth year students participated in two separated focus groups.Participants gave their informed consent to participate in the survey evaluations and focus groups.

Quantitative survey results
The fifth year students held a markedly more positive view of the flipped classroom overall, despite the design for both courses being very similar in content type and organisation.Survey results indicated that fifth year students (M= 5.25, SD = 0.81) perceived the flipped classroom as more useful than fourth year students (M= 3.11, SD = 1.22),U= 7.363, p < 0.01.Fifth year students (M=5.37,SD = 1.06) were also more satisfied with the interaction with instructors compared with fourth year students (M= 3.32, SD= 1.33), U = 6.47, p <0.01.Fifth year students (M = 5.85, SD = 0.74) also perceived kuraCloud as easier to use compared with fourth year students (M= 3.89, SD= 1.24), U= 7.43, p <0.01.Fifth year students (M = 4.55, SD = 1.03) reported a higher level of self-efficacy compared with fourth year students (M= 3.9, SD = 1.17),U= 2.811, p =0.005.

Qualitative survey results
Themes arising from the open questions in the survey administered to the fourth year cohort were used to inform the development of the focus group questions.These results are briefly summarised below.

Focus groups
Both groups identified that busy schedules limited time for study and led to a preference for a single source of content knowledge.Both groups also found the online material useful for revision when they 'un-flipped' the intended order of the course.
Areas of difference between the two groups were in the groups' willingness to tolerate uncertainty in using a new mode of delivery, their concept of learning and the way they perceived the integration between the flipped classroom and their clinical learning.These factors appear to have contributed to the overall negative view of the online content in the fourth year group and positive view in the fifth, as reflected in the whole-class evaluation surveys.

Time constraints
All students reported that having limited time meant that their study was less comprehensive than what they saw as ideal.The students in both groups stated that they began the semester making use of a wider range of resources (lectures, summaries, text books and kuraCloud modules) than what they continued with.In order to manage the lack of time, students prioritised what they saw as both the most urgent and the most important course content to study, and used resources they considered an efficient means of studying.
We're all at the mercy of our runs.(4th year) Stuff gets pushed to the last week and you're like, oh this would have been good to know earlier in the run.(5th year) Prioritising urgent material meant studying material that was likely to be relevant to clinical placement and studying just before tests.The fourth year students found online learning time-consuming.They estimated that online learning sessions took them more than the suggested hour to complete.The time commitment required for flipped pre-work was a deterrent, especially when they were directed to external websites.In contrast, the 5th fifth year students all reported that they completed the online modules in about half an hour.This suited them well because it was easy to fit around other commitments and was a manageable length of time to maintain concentration.

Sources of information
Students in both year groups preferred to use only one source of information as their primary study resource because of their high workload and limited study time.They wanted to study from material that was easy to navigate, consistently organised across topics and facilitated memorisation.In their own study and when selecting reference aids, they used resources that they had identified themselves or had recommended to them by other students.
You hear from other students and junior doctors.If it works for you go for it but there's other resources that people use.(5th year) While students selected and evaluated 'staple' sources of information, they did not want to engage in searching and evaluating multiple sources:

Multiple sources are distracting (4th year)
Just pick one source and stick to it because there's so much information (5th year).
They wanted resources that were aimed at medical students, and were unwilling to use information from consumer information sites or aimed at medical specialists because this required the student to assess what was relevant to their level of study.They also regarded video lectures delivered by teaching staff from the programme as more authoritative and engaging than embedded video content from other universities.
Both groups regarded information that was already summarised and structured as an ideal study resource.They were resistant to undertaking this synthesis themselves as part of the flipped classroom tasks:

Let's throw you all this information and you can put it together. No, that's inefficient. (4th year)
The fourth year group saw tasks which directed them outside the kuraCloud platform as a design failure rather than seeing the task of synthesising and evaluating different sources as a component of learning.
Why can't we use one website if it's that good -why do we need multiple sources.(4th year) Otherwise it's like you go and look at ten websites and you don't know which one is better.(4th year) For example, fourth year students disliked tasks that required them to look at multiple sources of information about the prevalence of diseases because they considered this should have a straightforward answer; minor differences between sources was confusing.Looking at multiple sources took more time than the students were willing to spend and ran counter to the fourth year students' view that learning and applying content knowledge should be separate tasks.They did however indicate greater willingness to consult multiple sources where they saw value in different perspectives, such as looking at treatment options.Although the fifth year group were not perturbed by not knowing enough to complete the activities, they were still disinclined to spend time searching for additional information.

Un-flipping
Students in both groups universally rejected the 'flipped' order in their weekly study routines.Students preferred to use the intended pre-work after face-to-face teaching as a revision tool.Fourth year students reported completing pre-reading for class using their textbook, or using videos of their own choosing but not using the flipped resources because they found these too time-consuming.This group articulated a willingness to use the online modules as an optional tool for revision but resented it being a compulsory part of the course.They saw themselves as experienced, successful learners and wanted to be trusted to continue in their established study routines: Yes, we should choose at this stage and know how we learn.We shouldn't have to do kuraCloud if it's not beneficial to us (4th year).
While this group indicated that they had found the flipped classroom unhelpful during the semester, some participants nonetheless indicated their intention to go back and redo the online modules for exam revision.Likewise, although the fifth year students reported far more enthusiasm for the online content, they also completed the activities after class or later in the course for revision.

Clinical focus
Students in both groups reported that they strove to learn pathology to become better clinicians.Thinking about what they were likely to encounter in clinical situations influenced students' view of what was useful learning.The fourth year group had mixed views content that was less likely to arise in a clinical setting, as evidenced in their views of histology:

Being able to recognise it [abnormalities on a slide] off your own bat without prompting is quite important. (4th year)
Knowing what a histology section looks like is not relevant for us yet.(4th year) Whereas not seeing pathology in clinical situations diminished its importance for the fourth year focus group, the fifth year students saw the online material as an opportunity to fill gaps in what they were exposed to in clinical situations.The found learning pathology "the hardest because it's not like you'll see it" in clinical placement (5th year student).For these students, online learning was a good means of applied learning in pathology and for learning about conditions that were too rare to be able to count on seeing them on the ward.
Clinical is the best but obviously we have to learn about the rare stuff, which you can't see so either questions [online question bank] or kuraCloud is good.(5th year)

Learning versus applying
Students in the 4th year group conceptualised learning and applying knowledge as separate activities.They described 'learning' as acquiring new factual knowledge, distinct from applying this knowledge in flipped classroom activities or quizzes:

Unanimously we do not use kuraCloud to learn. We use it to apply knowledge. That's what it should be for. (4th year)
This perceived dichotomy was incompatible with the flipped pre-work in which students were expected to attempt problems with a short introduction to the content and guidance on where to find additional information, rather than a comprehensive lecture.These students experienced anxiety when they were required to apply knowledge before feeling confident of knowing the correct answer: I found that there were lots of incidences -and I emailed to say -this is the wrong way around.You made me try and answer this question and then you told me how to answer it.(4th year)

If I'm going to learn something, I like being taught what it is and then being tested -I don't so much like the go away and answer it for yourself because you can have it wrong. (4th year)
In contrast, in the fifth year group, students viewed learning through a mixture of content acquisition and problemsolving much more positively: I find applying it, I learn it better.(5th year) Although they found the flipped classroom problems challenging, this challenge did not provoke the anxiety that it did for the fourth year students.

Uncertainty and ambiguity
Students' openness to uncertainty affected their perceptions of flipped learning.The constructivist course design pushed students to problem-solve using the content that they had just learned.The fourth year group responded very negatively to this approach, while the fifth year group responded positively.For the fourth year students, pathology played a special role as the most structured part of their curriculum:

path[ology] has quite clear objectives about what they expect from your learning. In everything else half the time they don't even tell you what you're meant to be learning. (4th year)
When faced with the "unstructured world of fourth year" (4th year), students wanted to retain a highly predictable approach to studying pathology.Students explained that they used their pathology notes to arrange their learning from other areas, for example adding in notes they had made in clinical placements into their pathology notes.
In contrast, the fifth year students were more open to learning through problem solving.Although both groups reported feeling they sometimes lacked sufficient knowledge to complete the online problems, this knowledge gap did not provoke anxiety for the fifth year students; they were willing to use workaround strategies, such as searching for unfamiliar terms or returning to the problem later in the course.They framed the level of uncertainty that they experienced in the flipped classroom as comparable to other components of the run as a whole: O&G [obstetrics and gynaecology] is quite a different run -I feel like it's the run where I have to teach myself the most because it's so out of order.(5th year) Participants accordingly expressed confidence in their ability to manage unfamiliar material that they encountered in the flipped classroom without immediate guidance from teaching staff: It's not really that new.We teach ourselves a lot.(5th year) In addition, the groups benchmarked the relative level of uncertainty they encountered in the flipped classroom differently.The fourth year group contrasted the flipped classroom against the more structured teaching in their previous years.On the other hand, the fifth year group compared the flipped classroom favourably against seminarstyle classes in the rest of the run presented by either other students or specialists, which were less likely to be organised well and pitched at their level.

Discussion
Students in both year groups either un-flipped or wanted to un-flip by using the online modules as revision exercises to consolidate their in-class learning or to revise for assessments.This student-led adaption of the course design supports recommendations to limit to restrict pre-class work to material that draws on lower-order thinking (Kurup & Hersey, 2013) and with the cognitive loading principle of gradually increasing the complexity of material that students encounter (Abeysekera & Dawson, 2015;Van Merriënboer & Sweller, 2010).This study adds support to the recommendations in the existing literature by providing an example of the way students respond to e-learning which exceeds these recommended restrictions.Students' concerns about the time pre-class work took and the quality of the online resources are consistent with qualitative findings from other flipped classroom trials (Khanova et al., 2015;White et al., 2015).
When presented with flipped learning, both groups of students identified similar challenges and used similar strategies to manage those challenges.What differed was their emotional response to using these strategies.A possible explanation for this difference is where in the medical degree structure the students encountered the flipped classroom.Students begin clinical placements in fourth year, and these students used the flipped classroom in the first half of this year.They expressed feeling overwhelmed by the less structured nature of learning on the ward, and looked to pathology, which was taught in the classroom, to provide them with structured learning and a point of continuity with their earlier study.The fifth year students in the focus group completed the flipped classroom in the second half of the year while on the Obstetrics and Gynaecology rotation.These students expressed a much greater degree of confidence in their ability to handle learning in an unstructured environment and learning without direct instruction.These views expressed in the focus groups concord with the higher levels of self-efficacy reported by the fifth year students.This aligns with research that suggests students' tolerance for uncertainty in medical education increases throughout their course of study (Hancock, Roberts, Monrouxe, & Mattick, 2015;Nevalainen, Mantyranta, & Pitkala, 2010).These students' greater exposure to less structured learning environments appears to have prepared them to adjust to the unknowns undertaking pre-work without direct guidance of teaching staff.Abeysekera and Dawson (2015) postulate that if flipped learning increases student motivation, it does so by increasing students' autonomy, competence and relatedness through way they learn, self-paced learning which allows students to master content before continuing, and greater interaction with peers and instructors.Further, they argue the self-paced nature of the flipped classroom may reduce cognitive load.This theoretical model would seem to be a good fit for students' experience of the flipped classroom in this trial.Students reported that they found flipped learning demotivating when it was compulsory, suggesting a lack of autonomy, and when pre-class activities required them to extend rather than consolidate their knowledge, suggesting that the flipped classroom did not enhance students' sense of competence in this situation.Conversely, students modifying the way they used the online modules to suit their needs may have increased their sense of autonomy and competence and lowered cognitive demand on students and thus had a positive effect on students' motivation.

Limitations
This small-scale initial study was limited to two undergraduate medical courses at the same institution and therefore the generalisability of these findings may be limited.Recruitment in the fifth year group was low for the survey and some students completed the survey some time after participating in the flipped classroom, which may have affected their recall of their experience.The numbers of students participating in focus groups was small and self-selected and may not be representative of the entire student body, although the views expressed by students in the focus groups did appear to align with the survey results from their respective year groups.

Conclusion
Students in this study preferred to un-flip their use of online resources.This study supports existing literature by confirming that if a flipped approach is used, passive learning may be more suitable for pre-work and that there is the risk of cognitive overload in active flipped learning activities.Alternatively, this study does provide support for utilising active online learning in pathology teaching.It indicates the potential for online learning to meet a need for revision resources and to fill in gaps in exposing students to problems they may not encounter clinically.The finding that students in these two groups viewed a similar approach very differently was unexpected and appears to stem from students' perceptions of learning in medical school and the manageability of an approach that they were not familiar with.To better engage students with online learning, instructors may need to clearly communicate the gap that online learning is filling and to consider how students view applied learning and managing uncertainty in learning.Dr. Rebecca Grainger is a consultant rheumatologist.She teaches chemical pathology and is the course convenor for the undergraduate pathology teaching programme at University of Otago Wellington.

Take Home Messages
It depends on who is presenting.Lots of them are student presenters and they haven't really got their head round it.You just have to go into it again.(5th year) They go into quite a lot of detail.Like if they're a specialist, that's what they've been doing for 30 years.That's why Diane [flipped classroom instructor] is quite good -she does very much what you need to know.(5th year) Students welcome active online learning exercises as revision aids.Students are reluctant to engage in active learning before face-to-face teaching occurs.Medical students are more open to using online learning when they see a clear complementary relationship with their clinical learning.Instructors implementing new approaches to learning should be mindful of other new challenges students are encountering in other areas of the curriculum.Notes On Contributors Dr. Diane Kenwright is the Head of Department of Pathology and Molecular Medicine, University of Otago, Wellington.She investigates student engagement with elearning.Emma Osborne is the Student Learning Advisor of at the University of Otago Wellington.She has a Masters of Education in adult education, with a focus on how people learn in the context of community development projects.Emma is interested in active learning and embedding transferable academic skills into students' coursework.Wei Dai is a research assistant of Department of Pathology and Molecular Medicine, University of Otago, Wellington.She is currently doing her Ph.D. in Educational Psychology in Victoria University of Wellington.Her research interest lies in the area of involvement of cognitive resources in the learning process.