Abstract
Context
Medical students are faced with the challenge of synthesizing large volumes of information quickly. With the increasing accessibility of technology, a “flipped classroom” allows students to learn foundational material independently. Class time is instead devoted to in-depth skill building with instructors, promoting an active learning environment. This method of content delivery is also relevant given the current COVID-19 pandemic.
Objectives
To comprehensively evaluate the benefit of adopting a flipped classroom approach in teaching physical exam skills (PES) to first-year osteopathic medical students.
Methods
A cohort study was conducted comparing first-year osteopathic students in the class of 2022 (n=201), who had taken the PES course traditionally, with the class of 2023 (n=203), who experienced the flipped classroom approach. Objective metrics such as cumulative grades, objective structural clinical examination performance (OSCE), and practical exam performance were compared using nonparametric Mann–Whitney U rank sum tests. Subjective measures such as student course evaluations were used to analyze course perceptions using independent sample t-tests assuming unequal variances. A faculty survey was administered to faculty who taught both cohorts to assess instructor attitudes toward the flipped classroom approach. Due to the COVID-19 pandemic, Spring 2020 quarter data was not included, given the transition of all classes to an online learning environment.
Results
The flipped classroom approach significantly improved objective student performance for the Fall (p=0.009) and Winter (p<0.001) student cumulative grades and the History-Taking OSCE (p=0.010). Performance on Fall and Winter practical exam scores had no significant association with the flipped classroom. General student perceptions of the course remained unchanged between both cohorts. Faculty survey results from 10 faculty members showed that six (60%) faculty members preferred the traditional classroom, while four (40%) preferred the flipped classroom.
Conclusions
The flipped classroom approach showed some statistically significant improvement in student PES. Further studies are needed to evaluate the benefits of a flipped classroom approach using skills-based assessments styles to measure student performance, with a focus on standardization of in-classroom groupwork.
In a ‘flipped classroom’ approach, pre-coursework is completed by students at home to encourage foundational instruction [1], and class time is then spent skill building to further apply concepts in greater depth. Flipped classroom models have traditionally been associated with interventions in K-12 grade levels [2]. However, with increasing costs of medical education and the growing need for medical students to synthesize large volumes of information quickly, the flipped classroom could allow medical students to learn at their own pace and utilize class time to work through complex concepts [3]. Flipped classroom models also maximize instructors’ time, allowing them to provide enriched learning experiences rather than passive lectures [4], [5], [6], which have been shown to be ineffective for learning outcomes; active participation in class increases student performance [7], [, 8].
For a hands on course such as Physical Exam Skills (PES), where students learn to conduct a comprehensive history and physical exam with osteopathic components, having additional class time to actively practice skills in groups with physician faculty could increase student competence in performing these skills with standardized patients on exams. The knowledge achieved in this first-year course supports students throughout their medical careers, increasing their comfort in performing physical exams during rotations [9].
Previous research has assessed the impact of flipped classroom approaches in medical school settings. For example, a systematic review of 118 research studies [10] showed subjectively positive perceptions of flipped classrooms from students, yet the objective measures were inconsistent. Of the studies covered in that review, only nine had controlled methodology [10]. However, another research study [11] compared performance in a medical certificate writing course for fourth year undergraduate medical students who received instruction in a traditional model (n=141) with performance from students who participated in the following year’s flipped classroom approach (n=142). That study [11] found that students in the flipped classroom scored significantly higher than their traditionally taught peers on a medical certificate writing assessment (mean score [SD], 10.97 [1.25] vs. 8.87 [1.70]), respectively. Using a similar model to the latter study, our prospective cohort study was designed to help address existing discrepancies in the literature. By incorporating multiple quantitative measures – such as cumulative grades, objective structural clinical examination (OSCE) scores, and practical exam scores – to objectively analyze the impact of flipped classroom initiatives, we sought to evaluate student performance between both cohorts in several contexts. We also assessed subjective data, such as course evaluations and faculty survey responses. We hypothesized that the flipped classroom format would yield improvements in both quantitative grade outcomes and faculty and student perceptions of the course. Through comprehensive evaluations like ours, support for the flipped classroom approach can potentially be expanded in medical school settings.
Methods
This study was reviewed by the Midwestern University Institutional Review Board, which deemed it exempt. Data collection began after review and exemption.
Course descriptions
PES is a year-long weekly laboratory course that introduces first-year osteopathic medical students to skills for taking a comprehensive history and performing a full physical exam. This course parallels the Patient Symptom Presentation (PSP) course, which provides a traditional lecture-based overview of topics aligned with the PES workshop.
The class of 2022 attended PES in a traditional method, which included a total of 31 sessions of 55 min each: a 15–20 min introduction by the course director demonstrating weekly physical examination skills, followed by small group teaching with clinical faculty for the remaining 35–40 min. Session introductions from the class of 2022 were recorded and available for the class of 2023 students to view in the 2019–2020 year on Canvas (Instructure, Inc.), a web-based learning management system used for all courses at Midwestern, before attending PES. The class of 2023 attended PES in a flipped classroom format. Students started each of the 31 workshops with a five-question quiz based upon the corresponding class of 2022 recorded session introduction video and a weekly outline. The remainder of the 55 min session was spent in small groups.
Clinical faculty members rotated through the small groups during each quarter for both cohorts (2022 traditional and 2023 flipped). There were eight small groups, each with one clinical faculty member. The faculty corresponding to each small group did change, at times based on physician availability, but generally faculty rotated their small groups on a quarter-to-quarter basis. Faculty were assigned to their small group students at random. The faculty for the class of 2023 was not specifically trained in flipped classroom delivery but was made aware of the new curriculum plan of transferring to a flipped classroom education. With the flipped classroom approach, the faculty was given more time to expand on teaching PES to the class of 2023 students, but did not receive any unique or specific training that differed from the class of 2022. Faculty for both years participated in a preworkshop faculty “huddle” to ensure consistency in the teaching of PES. The length and content of these preworkshop faculty “huddles” did not change between the cohorts. One author (B.P.) served as faculty for both the class of 2022 and the class of 2023.
Sample
Demographic information for both cohorts was recorded by Midwestern University’s Dean’s Office upon Fall enrollment. Data was used for every student who completed a given assessment.
Objective measures: examinations
Items such as Fall and Winter cumulative grades, history-taking OSCE scores, and Fall and Winter practical exam scores were used to compare student performance between the class of 2022 and class of 2023 cohorts. The curriculum was kept standard for both cohorts, with the same topics and skills being covered in the same weeks of each quarter. The methods of assessment and grading rubrics for the practical exams and OSCEs were mostly consistent between both cohorts, with the addition of the weekly 5-point quiz for the class of 2023. Full physical OSCE scores, Spring quarter practical exams, and Spring cumulative grades were removed from the analysis due to the class of 2023 experiencing instruction off-campus due to coronavirus 2019 (COVID-19) online learning modifications.
Subjective measures: students and faculty
Each student cohort received a PES course evaluation from Midwestern University after course completion. Surveys were accessible on each student’s Midwestern University online portal and were anonymous. Student evaluations opened the week after the final exam for the respective quarter. Spring quarter course evaluations from the class of 2022 were compared to Winter quarter course evaluations from the class of 2023, due to COVID-19 restrictions during Spring quarter. Students were asked to agree or disagree with the following items on a 5-point Likert-type scale, where 1 represented strongly disagree and 5 represented strongly agree: “The course objectives were clear.”; “The course content was well organized.”; “The evaluation methods adequately sampled information I was expected to learn.”; “Overall, I rate this course highly.”
The course evaluation focused on general perceptions of the course. In this research design, we refrained from asking students direct questions about the flipped classroom. To our knowledge, students had not experienced a PES-like course as first-year medical students, thus students may have lacked an accurate baseline comparison for judging traditional vs. flipped classroom teaching in this course. Students could also bring potential biases from previous experiences with flipped classrooms not in the context of PES.
Faculty who taught both cohorts were asked to complete a five-item survey (Supplemental Material) sent via Midwestern University e-mail and data was recorded anonymously using the REDCap (Vanderbilt University) electronic data capture tools, one week after the end of courses for the class of 2023 cohort [12], [, 13]. This survey focused on the instructor perception of the traditional vs. flipped classroom and which method instructors felt was better for promoting student learning. The first four items were multiple choice questions directly comparing the traditional vs. flipped classroom instruction. The last item on the survey provided an open-ended comments section.
Data collection
Individual student OSCE scores, practical exam scores, and cumulative grades were collected by the Department of Clinical Integration and available to the research team through the principal investigator’s (B.P.) role as the Course Director for PES. The research team (S.B., B.P.) performed statistical analysis and worked with deidentified student data. The Course Director (B.P.) had access to the deidentified student data on Excel, which was then updated to include only raw scores arranged according to grade from low to high, which were then sent to the other author (S.B.) for student and statistical analysis. Data was not sent in alphabetical order to ensure confidentiality. Aggregate data such as participant number, mean scores, and standard deviations from student course evaluations was provided by the Department of Clinical Integration. The faculty survey was administered to core faculty via Midwestern e-mail and survey response data were collected using Midwestern University’s REDCap electronic data capture tools to ensure confidentiality [12], [, 13].
Data analysis
To compare objective measures such as cumulative grades, OSCE scores, and practical exam scores, mean and median scores for both cohorts were examined. Nonparametric Mann–Whitney U rank sum tests were used at a 95% confidence level to determine statistically significant differences, as objective data lacked normal distribution. Mean scores were examined to compare subjective feedback from the Spring course evaluations for the class of 2022 and the Winter course evaluations for the class of 2023. Independent sample t-tests assuming unequal variances were used to detect statistically significant differences between the two cohorts at a 95% confidence level. For analyzing the subjective measures from the faculty survey, the number and percent of faculty who responded to each answer choice were calculated. All statistical analyses for the objective measures was performed using Microsoft Excel for Office 365 (2016). Analysis for the subjective student course evaluations was performed using the OpenEpi website (Version 3.01) due to the research team not having access to individual student data points. Thus, aggregate data was used instead to calculate the independent sample t-tests using the OpenEpi free software [14].
Results
Student enrollment in the class of 2022 was 201 for Fall quarter, 200 for Winter quarter, and 199 for Spring quarter. Student enrollment in the class of 2023 was 204 for Fall quarter and 203 for Winter and Spring quarters. Data was included in this study for every student who completed a given assessment, yielding a sample size of 201 for the class of 2022 and 204 for the class of 2023. Student demographic data is shown in Table 1.
Sample demographic information from the incoming classes of 2022 and 2023.
| Measure | Class of 2022 n=201 | Class of 2023 n=204 |
|---|---|---|
| Women, n (%) | 107 (53.2%) | 108 (52.9%) |
| Men, n (%) | 94 (46.7%) | 96 (47.1%) |
| Mean age, years | 23.5 | 24.0 |
| Mean total GPA | 3.67 | 3.60 |
| Mean science GPA | 3.61 | 3.50 |
| Mean MCAT score (scale, 472–528) | 507 | 508 |
GPA, grade point average; MCAT, medical college admission test.
Impact on student performance
Mean and median cumulative course grades by semester and type along with mean and median examination scores for the traditional cohort and the flipped classroom cohort are shown in Table 2. The mean (SD) percentage Fall cumulative grade for the class of 2023 was 92.80 (4.4) vs. 91.69 (5.5) for the class of 2022 (p=0.009) on a scale of 0–100. Winter cumulative grade were also higher for the class of 2023: mean, 93.81 (3.2) vs. 92.40 (3.3) in the class of 2022 (p<0.001). On a scale of 1–50, the class of 2023 achieved a mean (SD) score of 45 (2.8) on the history-taking OSCE, which was higher than the mean score of the class of 2022 at 44.18 (3.2) (p=0.010).
Objective examination measures comparing student performance from the classes of 2022 and class of 2023 cohorts.
| Measure | Group | Number of students | Mean | Median | Standard deviation | p-value |
|---|---|---|---|---|---|---|
| Fall cumulative grade | Class of 2022 | 201 | 91.69 | 92.00 | 5.49 | 0.009 |
| Class of 2023 | 204 | 92.80 | 94.00 | 4.44 | ||
| Winter cumulative grade | Class of 2022 | 200 | 92.40 | 93.00 | 3.29 | <0.001 |
| Class of 2023 | 203 | 93.81 | 94.00 | 3.17 | ||
| History-taking OSCE scoresa | Class of 2022 | 200 | 44.18 | 45.00 | 3.16 | 0.010 |
| Class of 2023 | 203 | 45.00 | 45.00 | 2.78 | ||
| Fall practical exam scoresa | Class of 2022 | 201 | 46.99 | 48.00 | 3.53 | 0.892 |
| Class of 2023 | 204 | 46.85 | 48.00 | 3.82 | ||
| Winter practical exam scoresa | Class of 2022 | 200 | 47.46 | 48.00 | 2.75 | 0.258 |
| Class of 2023 | 203 | 47.73 | 49.00 | 2.95 |
OSCE, objective structural clinical examination. aAssessment was out of 50 total points.
The class of 2022, educated with the traditional model, scored higher on two measures. On a scale of 1–50, the class of 2022 achieved a higher mean performance on the Fall practical exam, with a mean (SD) score of 46.99 (3.5) compared with 46.85 (3.8) in the class of 2023 (p=0.892). On the Winter Practical Exam, out of a total of 50 possible points, the class of 2022 also had a higher mean score of 47.46 (2.8) compared with 47.73 (3.0) in the class of 2023 (p=0.258). Neither exam yielded a statistically significant difference between the two cohorts.
Impact on student perception
Student course evaluations from both cohorts were compared to understand whether general course perceptions were impacted by the flipped classroom curriculum. Twenty-seven students (13.6%) from the class of 2022 and 40 students (19.7%) from the class of 2023 participated in the course evaluations. Mean response scores to the four survey questions administered to students (on a 5-point Likert-type scale) and p values are shown in Table 3; there were no statistically significant differences in response values between the two cohorts on the student evaluation items.
Subjective course evaluation measures comparing student classroom perception from class of 2022 (n=199) and class of 2023 (n=203) cohorts.a
| Item | Group | Number of students, n (%) | Meanb (SD) | p-value |
|---|---|---|---|---|
| The course objectives were clear. | Class of 2022 | 27 (13.6%) | 4.00 (0.96) | 0.576 |
| Class of 2023 | 40 (19.7%) | 4.13 (0.88) | ||
| The course content was well organized. | Class of 2022 | 27 (13.6%) | 3.78 (1.01) | 0.570 |
| Class of 2023 | 41 (20.2%) | 3.63 (1.13) | ||
| The evaluation methods adequately sampled information I was expected to learn. | Class of 2022 | 27 (13.6%) | 3.89 (0.85) | 0.868 |
| Class of 2023 | 40 (19.7%) | 3.93 (1.10) | ||
| Overall, I rate this course highly. | Class of 2022 | 27 (13.6%) | 3.37 (0.97) | 0.158 |
| Class of 2023 | 40 (19.7%) | 3.70 (0.85) |
SD, standard deviation. aSubjective Student Course Evaluations median data was not available to research team due to only receiving aggregate data from the Department of Clinical Integration. bThese scores are based on a Likert-type scale ranging from strongly disagree (1) to strongly agree (5) .
Impact on faculty perception
Of 16 core faculty, 10 (62.5%) participated in the survey (Supplemental Material). Of the 10 respondents, four (40%) preferred video instruction (flipped classroom) while six (60%) preferred in-person instruction (traditional approach). Four (40%) believed students were more comfortable and prepared in class because of prior video instruction. Nine (90%) felt their time as an instructor was used more effectively with having the full time dedicated to group sessions with the flipped classroom. Four faculty members (40%) responded that they would prefer the flipped classroom for the future class of 2024, while six (60%) would prefer the traditional approach. Six (60%) wrote comments in response to the final open-ended question, with mixed perceptions reported. In support of the flipped classroom, one faculty member wrote, “With video, student knew what to expect,” and another commented “I believe that all time in lab should be dedicated to in-person instruction without any interruption.” A primary concern raised against the flipped classroom by three faculty members was that students came to class without watching the videos entirely. Another faculty member wrote “… I prefer flipped classroom approach, but I believe we would need to find a way to better standardize sessions between instructors,” emphasizing an additional consideration with the flipped classroom implementation.
Discussion
The results from our study showed that objective student performance slightly improved with the flipped classroom approach. Fall and Winter cumulative grades, along with history-taking OSCE scores, were significantly higher for the class of 2023 compared with the class of 2022. The addition of the 5-point weekly quizzes in the class of 2023 may have impacted the statistical significance of Fall and Winter cumulative grades, but the practical and OSCE grading criteria were unchanged in both cohorts. While differences in other exam outcomes were not statistically significant, in every instance where there was significant difference between both cohorts, the flipped classroom cohort outperformed the traditional cohort.
The results of our study are inconsistent with previous literature, which is not surprising given the wide range of results from previous studies. A meta-analysis [15] using 28 studies with similar methodology of prerecorded videos in health professions settings showed that student perception and performance were both higher in flipped classroom cohorts. Another meta-analysis of 11 studies [16] comparing traditional vs. flipped classroom education in pharmacy school settings found no significant difference between final examination scores. In flipped classroom research by Moffett and Mill [5] in a veterinary professional skills course, student performance declined with the flipped classroom. In Heitz et al.’s flipped classroom study [17] in an emergency medicine clerkship, there were no statistically significant improvements with the flipped classroom. However, in Belfi et al.’s flipped classroom research [18] in a radiology introductory clerkship, students participating in the flipped classroom (“blended” classroom) had significant improvements in posttest results compared with control groups. In each of these previous studies, however, student performance was evaluated through written examinations only. The evaluation methods for PES in this study are based on both written exams and graded evaluation of PES performance with standardized patients.
Regarding subjective student course perceptions, less than 20% of students from both cohorts completed the course evaluations. Previous medical education flipped classroom studies have shown that students generally have positive perceptions of the flipped classroom [10]. However, those studies gauged positive perception by asking students direct questions about the flipped classroom, a method avoided in our current study to limit bias [19], [, 20]. Future researchers of this topic are encouraged to find a way to assess student course perceptions with participation from most students without directly asking students about flipped vs. traditional classroom approaches. Furthermore, studies to establish the efficacy a “hybrid” flipped classroom model would be beneficial, with some content delivered outside the classroom and some content delivered traditionally.
In our study, faculty members in both cohorts participated in a “huddle” before PES, where the roles of the faculty were clarified before each session to ensure consistency in PES teaching. An additional improvement to this structure could be to have faculty members watch the instructional videos provided to students, to ensure all faculty are in alignment with the weekly session content. However, the fact that 90% of faculty reported feeling their time was used more efficiently with the flipped classroom showcases how this approach has potential to improve the faculty teaching experience in PES with some modifications.
Osteopathic considerations
Implementation of the flipped classroom in osteopathic medical education is important for modeling lifelong learning and critical for enhancing professionalism in students. Future research could replicate the current study and accumulate more support for implementation of the flipped classroom by comparing skills-based assessments with standardized patients rather than solely using written examinations as an objective means of comparison between cohorts. Increasing evidence supporting the flipped classroom using skill-based assessment performance as a measure can benefit the osteopathic medical student experience in multiple areas. OMM coursework in osteopathic medical schools also uses hands on performance during practical examinations as a means of student evaluation [21]. Expanding research around the flipped classroom model using skills-based assessments as an objective measure is essential to expand the osteopathic medical student curriculum not just in PES, but in other preclinical coursework.
Limitations
A potent limitation of this study was the shift to off-campus learning during Spring quarter for the class of 2023 due to COVID-19 online learning modifications. Thus, data from the Spring cumulative grades, Spring practical exam scores, and full physical OSCE exam scores were not compared between the two cohorts. However, despite these limitations, this study had five strong objective measures where student performance from both cohorts were compared to evaluate the traditional vs. flipped classroom approach.
Another limitation of our study was the assumption that all students in the class of 2023 completely participated in the flipped classroom protocol. We attempted to control for this concern by administering a 5-point quiz on content covered in the prerecorded videos and outline to start each PES session. Despite this control, some faculty members believed students did not watch the full instructional videos, driving the preference of 60% of faculty for the traditional classroom. Future studies should consider developing further means of measuring student participation and adherence to the flipped classroom protocol. This could be accomplished through administration of a student survey, similar to the one described Tune et al.’s research [22], where students would be able to anonymously report whether they adhered to watching the instructional videos.
Another limitation of our study design was that two cohorts from two separate academic years were analyzed. An alternative research design would be to have the same cohort participate in both the traditional method of teaching and the flipped classroom curriculum within the same academic year to potentially limit any confounding variables. However, we refrained from adopting this design because providing each student with the same PES experience is important and we wanted to avoid any negative impact on student education.
Conclusions
A flipped classroom approach yielded some statistically significant improvement in student performance of PES. Data from student and faculty perceptions were both limited in response rate. Given the COVID-19 pandemic’s effect on medical education, including a need to develop innovative methods to deliver osteopathic physical examination content, the flipped classroom provides an opportunity to balance virtual with in-person learning. Further studies are needed to expand the support for evaluating the benefits of a flipped classroom approach using skills-based assessments styles to measure student performance with a focus on standardization of in-class groupwork.
Funding source: Midwestern University Kenneth Suarez Research Grant
Acknowledgments
The authors thank Amy Stein, PhD, for her assistance in data analysis for this manuscript. The authors also thank the PES faculty and students for their participation in this study.
Research funding: Student Doctor Bhai was awarded the Midwestern University Kenneth Suarez Research Grant to pursue summer research on this project.
Author contributions: All authors provided substantial contributions to conception and design, acquisition of data, or analysis and interpretation of data; drafted the article or revised it critically for important intellectual content; gave final approval of the version of the article to be published; and agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
Competing interests: Authors state no conflict of interest.
Ethical approval: This study was deemed exempt by the Institutional Review Board at Midwestern University.
References
1. Roehl, A, Reddy, SL, Shannon, GJ. The flipped classroom: an opportunity to engage millennial students through active learning strategies. J Fam Consum Sci 2013;105:44–9. https://doi.org/10.14307/JFCS105.2.12.Search in Google Scholar
2. Galway, LP, Corbett, KK, Takaro, TK, Tairyan, K, Frank, E. A novel integration of online and flipped classroom instructional models in public health higher education. BMC Med Educ 2014;14:181. https://doi.org/10.1186/1472-6920-14-181.Search in Google Scholar
3. Vogel, L. Educators propose “flipping” medical training. CMAJ (Can Med Assoc J) 2012;184:E625–6. https://doi.org/10.1503/cmaj.109-4212.Search in Google Scholar
4. Kurup, V, Hersey, D. The changing landscape of anesthesia education: is Flipped Classroom the answer? Curr Opin Anaesthesiol 2013;26:726–31. https://doi.org/10.1097/ACO.0000000000000004.Search in Google Scholar
5. Moffett, J, Mill, AC. Evaluation of the flipped classroom approach in a veterinary professional skills course. Adv Med Educ Pract 2014;5:415–25. https://doi.org/10.2147/AMEP.S70160.Search in Google Scholar
6. Boucher, B, Robertson, E, Wainner, R, Sanders, B. “Flipping” Texas State University’s physical therapist musculoskeletal curriculum: implementation of a hybrid learning model. J Phys Ther Educ 2013;27:72–7. https://doi.org/10.1097/00001416-201307000-00010.Search in Google Scholar
7. Bajak, A. Lectures aren’t just boring, they’re ineffective, too, study finds. Available from: https://www.sciencemag.org/news/2014/05/lectures-arent-just-boring-theyre-ineffective-too-study-finds [Accessed 24 Jul 2020].Search in Google Scholar
8. Jensen, JL, Kummer, TA, Godoy, PD. Improvements from a flipped classroom may simply be the fruits of active learning. CBE-Life Sci Educ 2015;14:ar5. https://doi.org/10.1187/cbe.14-08-0129.Search in Google Scholar
9. Whipple, ME, Barlow, CB, Smith, S, Goldstein, EA. Early introduction of clinical skills improves medical student comfort at the start of third-year clerkships. Acad Med 2006;81:S40–3. https://doi.org/10.1097/00001888-200610001-00011.Search in Google Scholar
10. Chen, F, Lui, AM, Martinelli, SM. A systematic review of the effectiveness of flipped classrooms in medical education. Med Educ 2017;51:585–97. https://doi.org/10.1111/medu.13272.Search in Google Scholar
11. Tusa, N, Sointu, E, Kastarinen, H, et al.. Medical certificate education: controlled study between lectures and flipped classroom. BMC Med Educ 2018;18:243. https://doi.org/10.1186/s12909-018-1351-7.Search in Google Scholar
12. Harris, PA, Taylor, R, Thielke, R, Payne, J, Gonzalez, N, Conde, JG. Research electronic data capture (REDCap)–a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inf 2009;42:377–81. https://doi.org/10.1016/j.jbi.2008.08.010.Search in Google Scholar
13. Harris, PA, Taylor, R, Minor, BL, et al.. The REDCap consortium: building an international community of software platform partners. J Biomed Inf 2019;95:103208. https://doi.org/10.1016/j.jbi.2019.103208.Search in Google Scholar
14. Sullivan, KM, Dean, A, Soe, MM. OpenEpi: a web-based epidemiologic and statistical calculator for public health. Publ Health Rep 2009;124:471–4. https://doi.org/10.1177/003335490912400320.Search in Google Scholar
15. Hew, KF, Lo, CK. Flipped classroom improves student learning in health professions education: a meta-analysis. BMC Med Educ 2018;18:38. https://doi.org/10.1186/s12909-018-1144-z.Search in Google Scholar
16. Gillette, C, Rudolph, M, Kimble, C, Rockich-Winston, N, Smith, L, Broedel-Zaugg, K. A meta-analysis of outcomes comparing flipped classroom and lecture. Am J Pharmaceut Educ 2018;82:6898. https://doi.org/10.5688/ajpe6898.Search in Google Scholar
17. Heitz, C, Prusakowski, M, Willis, G, Franck, C. Does the concept of the “flipped classroom” extend to the emergency medicine clinical clerkship? West J Emerg Med 2015;16:851–5. https://doi.org/10.5811/westjem.2015.9.27256.Search in Google Scholar
18. Belfi, LM, Bartolotta, RJ, Giambrone, AE, Davi, C, Min, RJ. “Flipping” the introductory clerkship in radiology: impact on medical student performance and perceptions. Acad Radiol 2015;22:794–801. https://doi.org/10.1016/j.acra.2014.11.003.Search in Google Scholar
19. Boet, S, Sharma, S, Goldman, J, Reeves, S. Review article: medical education research: an overview of methods. Can J Anaesth 2012;59:159–70. https://doi.org/10.1007/s12630-011-9635-y.Search in Google Scholar
20. Cheng, X, Ka Ho Lee, K, Chang, EY, Yang, X. The “flipped classroom” approach: stimulating positive learning attitudes and improving mastery of histology among medical students. Anat Sci Educ 2017;10:317–27. https://doi.org/10.1002/ase.1664.Search in Google Scholar
21. Snider, KT, Johnson, JC. Professionalism score and academic performance in osteopathic medical students. J Am Osteopath Assoc 2014;114:850–9. https://doi.org/10.7556/jaoa.2014.171.Search in Google Scholar
22. Tune, JD, Sturek, M, Basile, DP. Flipped classroom model improves graduate student performance in cardiovascular, respiratory, and renal physiology. Adv Physiol Educ 2013;37:316–20. https://doi.org/10.1152/advan.00091.2013.Search in Google Scholar
Supplementary Material
The online version of this article offers supplementary material (https://doi.org/10.1515/jom-2020-0198).
© 2021 Sahar Amin Bhai and Brian Poustinchian, published by De Gruyter, Berlin/Boston
This work is licensed under the Creative Commons Attribution 4.0 International License.
