An Evaluation of Emergency Medicine Core Content Covered by Podcasts

Introduction Podcasts are used broadly for learning in emergency medicine (EM); however, there is concern about the potential for knowledge gaps for learners who rely on podcasts for their learning. The extent to which EM podcasts cover the core curriculum of EM is not known; thus, we sought to quantify the extent to which podcasts represent the core content of our specialty. Methods We performed a retrospective review of all EM podcast episodes published in 2019. All podcasts were given credit for the content they covered as it related to the 2016 American Board of Emergency Medicine (ABEM) Model of Clinical Practice in Emergency Medicine (EM Model). The primary outcome was a description of how podcasts represented the ABEM EM Model content topics compared to the topic representation of the ABEM Qualifying Exam. Results We included 54 unique EM podcast programs and 1,193 podcast episodes. They covered 2,965 total EM Model core content topics. The topics most covered were “other” (which includes interpersonal skills and professionalism), procedures, and signs and symptoms. Musculoskeletal, hematology, and environmental each accounted for less than 1% of all topics covered. Almost three-quarters of podcast episodes covered other core competencies of the practice of EM. Conclusion Podcasts had a broad yet imbalanced coverage of the ABEM EM Model core content subtopics in 2019, with a notable coverage of other core competencies of the practice of EM. Learners, educators, and scholars should be mindful of these gaps and focus future work on exploring how podcasts should best be used in EM education.


INTRODUCTION
Medical students, residents, and practicing physicians increasingly use podcasts for their education, with some preferring podcasts to textbooks and journal articles. [1][2][3][4][5][6] Podcasts are an easy-to-use and engaging medium that provide learners with broad exposure to educational content and targeted learning opportunities. 7,8 While educational podcasts are used in many medical specialties, [9][10][11][12][13] podcasting is particularly popular in emergency medicine (EM). To date, EM has the largest number of active podcasts, podcast episodes, and hours of content. 14 While some hail the rise of podcasts as the "Netflix" of medical education, 15 others sound notes of caution. In addition to concerns over quality, 16,17 some argue that online educational resources lack the comprehensiveness of textbooks and other commonly used curricula designed to impart the breadth of core-content knowledge dictated by the American Board of Emergency Medicine (ABEM) Model of the Clinical Practice of Emergency Medicine (EM Model). 18,19 Thus, learners who rely primarily on these resources may have knowledge gaps due to over-or under-representation of topics. 20 This begs the question, however, of whether the amalgam of EM podcasts should be comprehensive. If podcasts are poised to "fundamentally reimagine medical education" in the next decade, 21 then we must wrestle with their role and purpose. Although some podcasts have been designed specifically to cover EM core content, 22-24 should we ask podcasts to cover the breadth of an EM education? And if so, can they? It is unclear the extent to which even the core content-focused podcasts cover the breadth of the specialty. Further, within EM core-content podcasts, are they disproportionately covering topics that are probably best taught in a visual (eg, dermatology) or kinesthetic (eg, procedures) format? Or should we ask instead of podcasts that they bring multi-specialty expertise to controversial topics, 25 model clinical reasoning and diagnosis, 26 analyze recent clinical literature, 27 focus on mindset and mental performance, 28 tackle gender equity in medicine, 29 or carry forward conversations about racism in medicine? [30][31][32] Given trainees' current dependence on these resources, and the ways in which many learners listen by absorbing whatever topic is pushed to them in the queue, 7 our responses to these debates should be informed by a comprehensive understanding of what podcasts are currently covering. This knowledge will begin to help educators make thoughtful recommendations to their learners, help podcast creators see the domains they may have been over-or under-emphasizing, and help scholars make educated arguments about the role and purpose of podcasts in EM education. The only existing studies looking at the comprehensiveness of online educational resources have focused on blogs, 19,20 leaving a gap in our understanding of the content podcasts provide. Thus, we sought to quantify the extent to which podcasts represent the core content of our specialty.

Study Design
We performed a retrospective review of all available episodes from 54 EM educational podcasts posted online in 2019. We followed the approach used in a previous study evaluating blog content, and we mapped the content of all included podcast episodes to the 2016 ABEM EM Model content topics. 18,19 Our local institutional review board deemed the study exempt.

Podcast Selection
As there was no updated listing of all EM podcasts, we used a three-step process to identify all EM podcasts that released episodes in 2019. First, we included all accessible podcasts documented in a recent study that relied on a web search to identify podcasts in every medical specialty (including EM). 14 This yielded 32 EM podcasts for inclusion. Next, we searched the Social Media Index (SMI) on March 31, 2020, to find podcasts that we had not yet included. The SMI is a website that lists EM and critical care blogs and podcasts, ranking them based on their impact. 33,34 The SMI yielded 39 additional podcasts. Finally, we emailed a group of seven EM podcast creators for their content expertise. We showed them our existing list and asked them to recommend any podcast we might have missed. They recommended 14 additional podcasts. From a total of 85 podcasts, we then excluded duplicates (those on both the web search study and the SMI), podcasts not focused on EM, and podcasts without any episodes during 2019.
The 54 remaining podcasts yielded 1,270 unique podcast episodes. We excluded individual episodes if they were a video/vodcast, mailbag-response episode, summarized another podcast, largely advertisements, or focused on personal stories. A total of 77 episodes were excluded, leaving 1,193 for our analysis ( Figure 1). The list of included podcasts is available as eAppendix A.

Measurement methods
We used the 2016 ABEM EM Model clinical categories and subtopics to define EM core content. 18 The 2016 model was used because the updated 2019 ABEM EM Model was not publicly available at the start of the study. There are a total of 20 broad categories in the EM Model, which are each divided into subtopics. We defined each category within the EM Model as a core content topic.
One author (GP) evaluated each podcast episode for EM Model, core content coverage by reviewing the podcast episode title and show notes. If show notes were unavailable, the author listened to the podcast. If a podcast covered any EM Model core content topic in an educational manner, we documented that podcast as having covered its corresponding EM Model core content category. We gave podcasts credit for as many EM Model core content topics as were mentioned. In other words, a single podcast could have covered multiple EM Model core content topics.

Riddell et al.
An Evaluation of EM Core Content Covered by Podcasts If a topic was merely mentioned incidentally but not expanded upon in an educational manner, the podcast episode was not credited with covering that topic. For example, if a podcast was discussing a cardiovascular diagnosis and then mentioned that the differential diagnosis included a gastrointestinal topic but did not describe it in any detail, then the podcast would only receive credit for the cardiovascular topic. However, if the gastrointestinal topic was described in further detail, then the podcast episode would be credited with covering that topic as well.
Throughout the coding process a second author (JR) reviewed and discussed any unclear podcast topic coverage until consensus was achieved. A third author (SK) then abstracted data from a random sample of 5% of podcasts and coded each podcast in a similar manner to the primary abstractor. Their inter-rater agreement was excellent (pooled k = 0.84). The inter-rater agreement broken down by each EM Model topic is available in eAppendix B.

Outcomes
For the purpose of comparison given our primary focus on resident education, and in keeping with previous blogbased studies, 19,20 we used the ABEM Qualifying Exam (QE) content specifications as the standard weighting of core content. 35 The primary outcome was a description of how ABEM EM Model content topics were represented by EM podcasts from January 1-December 31, 2019 compared to the topic representation of the ABEM QE. Additional outcomes included the number of unique EM podcast episodes that covered each EM Model category.

Data Analysis
We calculated proportional representation of ABEM categories covered by podcasts with associated 95% confidence intervals (CI). We also calculated differences between categorical representation in podcasts and the ABEM QE.
The percentage of all podcast episodes that covered each ABEM topic are shown in Table 2. Almost three-quarters (72.4%, 95% CI 70.8-74.0) of podcast episodes covered other core competencies of the practice of EM, which was 70.4% more than the topic's relative weight on the ABEM QE. Signs and symptoms and procedures were also covered on more than 32% and 39% of podcast episodes, respectively. While many topics were over-represented relative to the ABEM QE, relatively few were under-represented as a proportion of all podcast episodes, including abdominal/GI (-2.4%); head, eyes, ears, nose, and throat exam (-1.3%); musculoskeletal (-1.2%); and hematology (-0.9%).

DISCUSSION
We found notable differences in how ABEM EM Model content topics were represented by EM podcasts compared to the topic representation of the ABEM Qualifying Exam. Overall, our data suggests imbalanced yet broad coverage of the core content of EM. The finding that, as a percentage of all podcast episodes, very few topics were proportionally underrepresented relative to their ABEM QE weight speaks to the thorough coverage of core content that podcasts provided. This is in line with a previous study of blogs that also found five topics under-represented compared with the QE model. 20 When considering all topics covered, we found a range of proportional representation relative to the QE from +27% to -5%. The imbalance we found with podcasts is not strikingly dissimilar to previous studies of blog posts. 19,20 The imbalance in podcasts, however, appears different from the imbalance found in the blog studies. While these studies also reported high proportions of signs and symptoms, procedures, and other competencies, 19 we found a strikingly higher level of discussion of other core competencies of the practice of EM on podcasts. The subtopics for other core competencies of the practice of EM are shown in Figure 2.
The finding that other competencies accounted for 29% of all topics covered and was covered on 72% of all podcasts reveals a lot about the role podcasts may be filling in the EM educational milieu. Whereas a leading EM textbook has now relegated chapters on multiculturalism, bioethics, medicolegal issues, wellness, stress, and the impaired physician to its "Bonus Online Content," 36 it appears podcasts are elevating, or at least making more frequent, these discussions. There are several factors likely contributing to this discrepancy. First, it may be a function of time. As the first study highlighting imbalanced coverage of core content studied the landscape in 2013-2014, 19 and the other between 2015-2017, 20 it is possible that podcast creators have reacted to these studies and/or current events in creating content that addresses current pressing issues around subjects such as professionalism or systems-based practice.
Similarly, while some topics like "other" may be overrepresented because they are covered by recently published, high-impact, primary literature articles, the topics that were under-represented might not have been featured in recent primary literature publications. For example, the EM Model subtopic "status epilepticus" made up 8.5% of the coverage for the nervous system category (data not shown). This is largely due to multiple podcast episodes in 2019 that discussed the ConSEPT and EcLiPSE trials following their publication in Lancet in May 2019. 37, 38 We did not, however, find similarly impactful trials published about "inter-departmental and medical staff relations" or "clinical decision support," two of the most covered topics in the category "other." To address the questions raised in the introduction, our data suggests that learners who rely primarily on podcasts may have knowledge gaps due to over-or under-representation of topics. Given that most podcast listeners (including EM residents, attendings, and other healthcare professionals) indicate that they listen to podcasts to learn EM core content, 2,3,5,39 it is possible that some learners may be unaware of the gaps in EM core content coverage. If they do not continue to use a broad range of resources for their education, they could miss out on the full breadth of knowledge necessary to pass the ABEM QE and be prepared for every patient who enters the ED. 40 Therefore, listeners, educators, and podcast creators should be aware of these gaps. Several podcasts have, however, been created to specifically cover the breadth of EM core content. Most notably, there is a podcast that covers every chapter of Rosen's Emergency Medicine textbook. 22,41 The question of whether the amalgam of EM podcasts should be comprehensive is a thornier one. While our data does not definitively answer that question, it does suggest that podcast creators are overwhelmingly bringing what they think is important into their audio conversations. It is likely that   resident learners also listen to podcasts, and podcast producers create podcasts for reasons other than learning EM core content. Podcast users have various reasons for listening including to be entertained, to connect with the EM community, to be inspired, and to engage in targeted personalized learning. 5,7 These important reasons for using podcasts do not rely on them to broadly cover all of EM core content. Rather, the benefit of podcasts in the EM educational space may have more to do with the depth that the format enables. Podcasts offer long-form "deep dives" infused with immersive storytelling that can provide more gravitas than their written counterparts. 42 The personal nature of audio allows the creator to stimulate the imagination of the listener using richly textured narratives and self-reflexive gestures, thereby influencing both motivation and cognition. 43,44 In contrast to blogs and textbooks, the heightened intonations and subtle adjustments in volume, inflexion, and phrasing allowed by the spoken word can communicate emotions, create a sense of intimacy, and captivate audiences for extended periods. 8,42,45 The format allows for conversations about workplace-based violence, compliance, litigation, reimbursement issues, burnout, and diversity in ways that just aren't possible with other educational media (blogs, textbooks, journal articles, etc).
Therefore, future work might focus less on ensuring that every core content subtopic gets covered by a podcast and more on understanding the content that is included in podcasts. It would be fruitful to explore why topics are chosen, how they stimulate emotions and imagination, how to make podcast learning more effective, 43 how podcasts impact professionalism and systems-based practice and, ultimately, which parts of the core curriculum of our specialty are best suited to audio learning.

LIMITATIONS
This study has many limitations. First, we evaluated only podcasts released in 2019, and it is possible that including podcast episodes released in prior years would reduce the gaps in coverage of EM core content. Further, impactful studies during the year of release could also have skewed the data. Second, podcasts were evaluated based mostly on the episode title and show notes. We were not able to listen to every podcast episode due to logistical time constraints. It is possible that EM Model core content could have been covered in actual audio podcast recordings without being outlined in the show notes, which could have led us to underestimate the extent of EM core content coverage by podcasts and perhaps points to the need for a standardized template for show notes.
Third, determining what constitutes a meaningful educational discussion of a subtopic was difficult. We attempted to remedy this by discussing all questionable educational points and achieving group consensus, including a large number of podcast episodes, and having a second abstractor review a random percentage of podcasts with excellent inter-rater reliability. Finally, our study was designed to simply evaluate how podcasts covered ABEM EM Model core content. We did not evaluate the depth or quality of coverage, nor did we explore podcast creators' qualifications or their use of primary literature references. Future studies should continue to explore the issues affecting the quality of EM educational podcasts. 33,46

CONCLUSION
Podcasts had an imbalanced yet broad coverage of ABEM EM Model core content subtopics in 2019. We found imbalances in content representation compared to the weight given in the ABEM Qualifying Exam, with a strikingly higher level of discussion of other core competencies of the practice of emergency medicine on podcasts. Learners, educators, and scholars should be mindful of these gaps and focus future work on exploring how podcasts should best be used in EM education.