Lecture Evaluations by Medical Students: Concepts That Correlate With Scores

doi:10.1016/j.jacr.2015.06.025

Journal of the American College of Radiology

Volume 13, Issue 1, January 2016, Pages 72-76

https://doi.org/10.1016/j.jacr.2015.06.025 Get rights and content

Abstract

Purpose

The didactic lecture remains one of the most popular teaching formats in medical education; yet, factors that most influence lecturing success in radiology education are unknown. The purpose of this study is to identify patterns of narrative student feedback that are associated with relatively higher and lower evaluation scores.

Methods

All student evaluations from our core radiology elective during 1 year were compiled. All evaluation comments were tagged, to identify discrete descriptive concepts. Correlation coefficients were calculated, for each tag with mean evaluation scores. Tags that were the most strongly associated with the highest- versus lowest-rated (> or < 1 SD) lectures were identified.

Results

A total of 3,262 comments, on 273 lectures, rated by 77 senior medical students, were analyzed. The mean lecture score was 8.96 ± 0.62. Three tags were significantly positively correlated with lecture score: “interactive”; “fun/engaging”; and “practical/important content” (r = 0.39, r = 0.34, and r = 0.32, respectively; all P < .001). More tags (n = 12) were significantly negatively correlated with score; the three tags with the strongest such correlation were: “not interactive”; “poorly structured or unevenly paced”; and “content too detailed or abundant” (r = –0.44, r = –0.39, and r = –0.36, respectively; all P < .001). Analysis of only the highest- and lowest-rated lectures yielded similar results.

Conclusions

Several factors were identified that were strongly associated with lecture score. Among the actionable characteristics, interactive lectures with appropriately targeted content (ie, practical/useful) were the most highly rated.

Introduction

The didactic lecture is one of the most time-tested and popular teaching methods used today. As a simple method of transferring knowledge from instructor to student, its benefits are many. The format is scalable, allowing for instruction of tens of students up to thousands, particularly when information is disseminated via the Internet [1]. In addition, the format is very familiar, and can be time efficient for preparing and delivering content.

The traditional lecture format, however, is prone to several pitfalls [2]. Students’ attention spans for passive learning are extremely limited, often to just 20 minutes [3]. Lecturer styles of presentation are unique, and sometimes ineffective, requiring learners to adapt to each presenter. Additionally, lecturers choose the level of information and the pace of presentation, which may not be ideally suited to all learners. Finally, lectures are limited in what they can teach; they often focus on facts and concepts, rather than on skills or creativity [4].

Due to these challenges, some educators have argued that alternative teaching formats should have a greater role in education. Problem-based and team-based learning are implemented widely in current medical school curricula 5, 6, 7. Blended and “flipped” learning models are popular 8, 9, 10. Particularly now, in the digital age, online interactive modules increasingly are being promoted.

Each method offers its own unique advantages and disadvantages, yet none has completely replaced traditional lecturing. For that reason, educators should continue to improve the effectiveness of lecture delivery. Efforts to improve lecturing have come far in recent years. Several authors have recently shared insightful techniques to promote active learning, including incomplete outlines, break activities, relatable examples, and teachable skills 4, 11, 12, 13, 14.

Kessler et al [15] compiled presentation techniques from highly rated faculty at the American College of Emergency Physicians, focusing on key objectives, increased audience participation, and manageable slide content. Multimedia design principles have been applied to lecture creation. Issa et al [16], for example, found that audiences preferred a lecture design focused on visual representation, and without bullet points. To our knowledge, very little of the radiology literature has examined what characteristics are most desired by learners, particularly medical students.

In this study, we analyzed medical student feedback on lectures in our core senior radiology elective. Specifically, we analyzed all narrative comments collected during 1 year, and determined the comment phrasing that was most associated with various average numeric evaluation scores. Our hypothesis was that high- versus low-scoring lectures would be associated with a definitive set of unique adjectives.

Section snippets

Methods

This study is exempt from institutional review board requirements. Only anonymized lecture comments and scores, extracted from an existing course-evaluation database, were reviewed.

Results

The feedback from 317 lectures given by 54 different lecturers was available in the course-evaluation database. Forty-four sessions were excluded from analysis, owing to nontraditional lecture formats (eg, orientations, tours, and hands-on modules). These formats differed fundamentally from the traditional lecture model, and could, theoretically, have been evaluated differently by the students. The final sample included 273 lectures given by 48 lecturers. These lectures were evaluated by a

Discussion

Via a systematic review of 3,262 medical student comments, we examined the association between specific phrasing in evaluation comments and the overall numeric rating for a lecture. We found that students described high-scoring lectures as interactive, engaging, and practical. We found that a longer list of descriptors was associated with low-scoring lectures, including lack of interactivity, poor structure, and an overabundance of content.

The descriptors most associated with positively rated

Take-Home Points

■
Teaching evaluations often contain free-text narrative comments, which can offer a wealth of information to the educator.
■
By analyzing more than 3,000 narrative comments regarding lectures in our main radiology course, we identified certain phrases that are most associated with well received and poorly received lectures.
■
“Interactive,” “engaging,” “useful,” and “organized” lectures tend to receive the highest scores.
■
Educators can be mentored to create lectures with these characteristics in mind;

References (24)

E.M. Webb et al.
Learning objectives in radiology education: why you need them and how to write them
Acad Radiol
(2013)
L.M. Belfi et al.
“Flipping” the introductory clerkship in radiology: impact on medical student performance and perceptions
Acad Radiol
(2015)
C.S. Kessler et al.
Qualitative analysis of effective lecture strategies in emergency medicine
Ann Emerg Med
(2011)
K. Moss et al.
Effective learning in science: the use of personal response systems with a wide range of audiences
Comput Educ
(2011)
T. Chapman et al.
Introductory lecture series for first-year radiology residents: implementation, investment and assessment
Acad Radiol
(2013)
F.G. Martin
Will massive open online courses change how we teach?
Commun ACM
(2012)
Flexner A. Medical education in the United States and Canada: a report to the Carnegie Foundation for the Advancement...
T. Mustafa et al.
Lectures in medical educaton: what students think?
J Ayub Med Coll Abbottabad
(2014)
G.C.H. Koh et al.
The effects of problem-based learning during medical school on physician competency: a systematic review
CMAJ
(2008)
A.J. Neville
Problem-based learning and medical education forty years on. A review of its effects on knowledge and clinical performance
Med Princ Pract
(2009)

D.P. Hunt et al.

The effect of using team learning in an evidence-based medicine course for medical students

Teach Learn Med

(2003)

R. Sadeghi et al.

Comparison of the effect of lecture and blended teaching methods on students’ learning and satisfaction

J Adv Med Educ Prof

(2014)

Cited by (14)

A Review of COVID-19's Impact on Radiology Residents’ Academics and Wellness in North America
2023, Journal of the American College of Radiology
The COVID-19 pandemic caused major disruptions in radiology departments throughout North America. Radiology residency programs were forced to make dramatic changes to their training programs, which had major impacts on resident academics and wellness. The goal of this review is to evaluate the impact of COVID-19 on radiology residents’ academics and wellness in North America, while also identifying effective measures taken by programs to mitigate the effects of the pandemic.
The search strategy involved database search via PubMed, Embase, and Web of Science with specific key words related to COVID-19, radiology residents, education, wellness, and virtual learning. Studies discussing the education and wellness of radiology residents in North America published after 2020 were included. The data were analyzed using a narrative synthesis approach.
The three main domains affected by the pandemic include the residency curriculum, research, and resident wellness. The decline in case volume and diversity of cases had negative overall impact on education of radiology residents, but simulated cases and virtual learning proved its value during the pandemic and may have lasting implications for the postpandemic world. Research initiatives transitioned to a remote format with greater emphasis on quality improvement and COVID-19–related studies. Reduced face-to-face interaction opportunities made it difficult to establish strong and meaningful interpersonal connections and had a negative impact on resident wellness, mentorship, and professional development. Implementing mentorship programs and virtual “town hall meetings” were effective measures to maintain connections during times of social distancing. Finally, the COVID-19 pandemic introduced unprecedented stressors and challenges for radiology residents that negatively impacted their mental health and wellness. Incorporating wellness initiatives such as wellness hours and team-building activities and using social media were helpful in promoting wellness and mental health for radiology residents.
The COVID-19 pandemic has had a significant impact on the academics and wellness of radiology residents across North America but has taught us many lessons that can help us navigate the ongoing challenges of the pandemic, the postpandemic world, and future pandemics.
The Best of Both Worlds: Hybrid Radiology Clerkships
2023, Academic Radiology
A practical guide to undergraduate radiology education
2022, Clinical Radiology
Citation Excerpt :
During the pandemic, lectures moved online, either as pre-recorded or synchronous (live) sessions. Previous research shows that students rate radiology lectures more favourably when they include “practical content” and are “interactive”.9 In a virtual learning environment, the facilitator must use digital solutions to ensure interaction.
The new Royal College of Radiologists (RCR) undergraduate curriculum and the impending introduction of a universal General Medical Council (GMC) medical licensing assessment (MLA) for all undergraduates in the UK heralds a new era of undergraduate radiology education. This is a practical guide to both implementing and delivering undergraduate radiology education using our experience and the available literature. It aims to provide ideas so that more universities and radiologists can integrate radiology into everyday learning and are ready to embrace the new RCR curriculum and GMC MLA. Allied to this, strategies are provided to show how to encourage early undergraduate interest in a career in radiology.
CASE-based and Guidelines-based Lectures are the Most Preferred Form of Online Webinar Education: Results from the Urology Collaborative Online Video Didactics Series (COViD)
2021, Urology
Citation Excerpt :
In our analysis, practice update lectures scored better than surgical technique in terms of subject relevance, but in terms of lecturer effectiveness and usefulness to learning it did not score any higher. Previous studies have noted that lectures where the information is too abundant or advanced were more difficult for learners to appreciate.12 Practice update lectures may have fallen into this pitfall, where the assumption of basic understanding of the viewer was overestimated.
Objective To evaluate the most preferred style of online didactic lectures. The COVID-19 pandemic has had a significant impact on surgical resident education, instigating a major shift towards online webinar didactics as a major of resident teaching. We hypothesize that a case-based format of online didactics are the most preferred format for this style of lecture.
Study Design We analyzed viewer evaluations following 82 online hour-long lectures in the Urology Collaborative Online Video Didactics Lecture Series. We categorized each lecture as case-based, guidelines-based, practice updates, or surgical technique-based and assessed viewer responses to survey questions regarding subject area relevance, lecturer knowledgeability, lecturer effectiveness, and usefulness to learning. We performed logistic regression to control for viewer level, instructor level, and lecture topic, and using surgical technique-based lectures as the baseline variable.
Results 2176 evaluations were analyzed. Case-based, guidelines-based and practice updates were all scored significantly higher than surgical technique for subject area relevance. Case-based and guideline-based lectures scored significantly higher for usefulness to learning. Case-based lectures scored significantly higher for lecturer effectiveness. There was no significant difference in scoring between any lecture style when rated on lecturer knowledgeability.
Conclusion When preparing online webinar based didactics for surgical resident education, case-based lecturers appear to be the most preferred and well received lecture style, followed closely by guidelines-based lectures. Practice updates and surgical technique-based lectures are less preferred formats for this teaching modality.
Resident Experiences With Virtual Radiology Learning During the COVID-19 Pandemic
2021, Academic Radiology
Citation Excerpt :
For example, residents were highly satisfied with the delivery of online morning teaching conferences and this is a curricular strength. Our institution has encouraged lecturers to use polling software to promote audience engagement for virtual lectures, as the incorporation of audience participation has been correlated with higher lecture evaluations (15). Audience participation in lectures has similarly been advocated in virtual radiology curriculums at other institutions (2,4,7).
COVID-19 has disrupted radiology education and forced a transition from traditional in-person learning to a virtual platform. As a result of hospital and state mandates, our radiology residency program quickly transitioned to a virtual learning platform to continue dissemination of knowledge, maintain resident engagement, and ensure professional development. The goal of this study is to assess the strengths and weaknesses of the virtual learning platform at our institution using resident ratings.
This institutional IRB-exempt study involved a survey of 17 questions which was electronically distributed to 45 radiology residents using SurveyMonkey. Questions encompassed resident satisfaction with teaching and professional development, scheduling changes, and engagement with the virtual platform. Answers to most questions were submitted on a Likert scale.
A total of 31 of 45 respondents completed the survey (response rate = 69%). Most residents were satisfied with the virtual platform with teaching activities identified as a strength and the incorporation of professional development as a weakness. The most frequent barriers to attending the virtual curriculum were technical difficulties (43%) and childcare (36%). Residents who reported experiencing barriers were less likely to adhere to the virtual curriculum (p = 0.004). Most respondents (81%) reported a desire to maintain elements of the virtual learning practice postpandemic.
The majority of residents reported high satisfaction with virtual learning during the COVID-19 pandemic. Teaching activities are a curricular strength. Weaknesses identified include the incorporation of professional development and extrinsic barriers, such as technical difficulties and family obligations, which require further support for trainees.
Medical Student Education Roadblock Due to COVID-19: Virtual Radiology Core Clerkship to the Rescue
2020, Academic Radiology
Medical schools were upended by the COVID-19 pandemic, resulting in suspension of all in-person educational activities, and leaving clinical clerkships on hold indefinitely. A virtual curriculum and novel teaching methods were needed to fulfill curricular requirements. We developed a comprehensive virtual radiology clerkship and evaluated the efficacy of this novel method of teaching.
A 4-week virtual radiology clerkship was designed to accommodate medical students who had not yet completed the required clerkship. The design included online flipped classroom modules, large group didactic lectures, and small group homeroom activities. Student performance was assessed via a standardized online final exam. Feedback from students was collected using online surveys. Student performance was compared to the in-person radiology clerkship.
One hundred and eleven medical students were enrolled in the virtual radiology clerkship. Final exam scores were similar to the in-person clerkship. Students indicated that small group homeroom activities had the highest overall satisfaction. Students recognized enthusiastic teachers regardless of class format. Exceptional course content and organization were also noted. Course weaknesses included didactic lecture content which was repetitive or too advanced, the limited opportunity to build personal connections with faculty, and scheduling conflicts with other competing school activities.
A completely virtual radiology core clerkship can be a successful educational experience for medical students during a time when remote learning is required. A small group learning environment is most successful for student engagement. Personal connections between faculty and students can be challenging in a virtual course.

View all citing articles on Scopus

: The authors have no conflicts of interest related to the material discussed in this article.

View full text

Original articleTraining and educationLecture Evaluations by Medical Students: Concepts That Correlate With Scores

Abstract

Purpose

Methods

Results

Conclusions

Introduction

Section snippets

Methods

Results

Discussion

Take-Home Points

Acad Radiol

Acad Radiol

Ann Emerg Med

Comput Educ

Acad Radiol

Will massive open online courses change how we teach?

Commun ACM

Lectures in medical educaton: what students think?

J Ayub Med Coll Abbottabad

The effects of problem-based learning during medical school on physician competency: a systematic review

CMAJ

Problem-based learning and medical education forty years on. A review of its effects on knowledge and clinical performance

Med Princ Pract

The effect of using team learning in an evidence-based medicine course for medical students

Teach Learn Med

Comparison of the effect of lecture and blended teaching methods on students’ learning and satisfaction

J Adv Med Educ Prof

Original article
Training and education
Lecture Evaluations by Medical Students: Concepts That Correlate With Scores