Digital Education for Health Professionals: An Evidence Map, Conceptual Framework, and Research Agenda

Background: Health professions education has undergone major changes with the advent and adoption of digital technologies worldwide. Objective: This study aims to map the existing evidence and identify gaps and research priorities to enable robust and relevant research in digital health professions education. Methods: We searched for systematic reviews on the digital education of practicing and student health care professionals. We searched MEDLINE, Embase, Cochrane Library, Educational Research Information Center, CINAHL, and gray literature sources from January 2014 to July 2020. A total of 2 authors independently screened the studies, extracted the data, and synthesized the findings. We outlined the key characteristics of the included reviews, the quality of the evidence they synthesized, and recommendations for future research. We mapped the empirical findings and research recommendations against the newly developed conceptual framework. Results: We identified 77 eligible systematic reviews. All of them included experimental studies and evaluated the effectiveness of digital education interventions in different health care disciplines or different digital education modalities. Most reviews included studies on various digital education modalities (22/77, 29%), virtual reality (19/77, 25%), and online education (10/77,


Introduction
The world is faced with a shortage and an unequal distribution of health workforces across low-income, middle-income, and high-income countries [1]. The shortfalls and inequitable distributions affect the likelihood of reaching the United Nations' third Sustainable Development Goal-health and well-being for all by 2030 [1][2][3][4][5]. To enable an increase in and a more equitable distribution of competent health workforce, there is a need for more effective and accessible health professions education.
The use of digital technology in health professions education can help in overcoming some of the health workforce-related challenges by providing more accessible, standardized, relevant, timely, and affordable medical education and training [6,7]. Until recently, digital education was perceived as primarily supporting in-person health professions education [8]. The social distancing measures introduced to control the COVID-19 pandemic have dramatically changed the delivery of health professions education worldwide. Many medical schools and health professions education institutions had to pivot to digital education [9,10]. With this sudden shift, research and evidence in digital health professions education have become even more important.
The evidence on digital education has grown substantially in recent years and has been the subject of many systematic reviews. Existing reviews seem to mostly focus on the effectiveness of different digital education modalities [11][12][13][14][15][16]. However, the adoption of digital education is complex and includes other research questions, in addition to its effectiveness. It is important to identify evidence that already exists and evidence gaps across the full scope of relevant questions to inform and guide future research and reduce research waste. To address this need, we seek to (1) create a map of existing research, (2) develop a conceptual framework outlining key components of digital education, and (3) highlight specific research questions across a comprehensive research framework. We do this by systematically identifying and analyzing previous systematic reviews on digital health education.

Methods
We used an evidence map methodology to identify and summarize systematic reviews on digital health professions education [17]. We also developed a novel conceptual framework using an established methodological approach [18] and identified specific research questions in alignment with this conceptual framework.
includes modalities ranging from the basic conversion of content into a digital format (eg, a book converted into a PDF or HTML format) to more complex applications such as mobile education, digital games, virtual patients, and virtual reality (VR). Systematic reviews were included if they focused on ≥1 modality of digital education (as defined in Textbox 1) delivered in a stand-alone or blended format [20]. We defined blended education as education that incorporates aspects of traditional and digital education. Traditional education was defined as education that encompasses the use of nondigital educational materials (eg, textbooks or models) or in-person human interactions. We included systematic reviews of all the types of studies. We excluded older reviews because of the rapid evolution of the field, with the assumption that most of the active research questions from the reviews published >5 years ago would be collated in more recent reviews. Textbox 1. Digital education technologies and modalities and working definitions and descriptions.

Offline digital education
• Education delivery requires no internet or local area network connection and can be delivered through external media, including CD-ROM, external hard disk, and USB stick [21].

Online digital education
• Computer-assisted instruction using the internet or a local intranet as the means of delivery, also referred to as online, internet-based, or networked [22,23], includes multiple media formats (eg, text, videos, and images and online discussion (eg, via email, chat, or videoconferencing) and is designed to be primarily delivered on PCs.

Massive open online course
• A (free) online course available over the internet to a large number of geographically dispersed participants [24] Mobile education (m-Learning) • Flexible and accessible learning delivered via personal mobile devices, such as smartphones and tablets [25] Serious gaming and gamification • Knowledge and training activities are set within a competitive activity. Games are intended to promote the development of knowledge, cognitive skills, or psychomotor skills in a virtual environment [26].

Virtual reality
• Interactive exploration of a digital (3D) multimedia environment can reflect a real-world environment (eg, clinic) or an artificial or surreal context (eg, positioning the learner within the human body) [16,27].

Virtual patient
• A computer program that simulates real-life clinical scenarios where students take on the role of a health professional and obtain a patient's history, conduct a physical examination, and make diagnostic and therapeutic decisions [28] High-fidelity manikins • Realistic, computerized mannequins that mimic elements of human physiology (eg, breathing and heart rhythm) and are used to simulate a real-life clinical scenario [29].

Blended education
• The use of digital education modalities in combination with traditional education methods

Traditional education
• Education that uses nondigital educational material (eg, textbook or model) or in-person human interaction

Study Selection
The search results from different databases were combined in a single EndNote library, and duplicate citations were removed. A total of 2 review authors (SP and BMK) screened all titles and abstracts for inclusion independently and in duplicate. Disagreements during the title and abstract screening were resolved by consensus. Full texts of articles considered eligible or uncertain based on the title and abstract screening were retrieved and screened independently and in duplicate by the same 2 authors.

Data Extraction
From the included systematic reviews, 2 authors (SP and BMK) used a standardized form to independently extract information on the review aim; the study design, participants, interventions, comparators, and outcomes of the original research studies included; the method used to appraise the quality of the included studies (eg, risk of bias) or overall evidence (eg, the Grading of Recommendations Assessment, Development and Evaluation assessment); and recommendations for future research. We classified the outcomes according to the definitions presented in Multimedia Appendix 2. We also extracted information on all additional outcomes reported in the included reviews that did not correspond to our predefined outcome-related definitions. We classified the systematic review in terms of the single digital modality they most focused on, according to our framework (Textbox 1). In most instances, it was clear that a given review focused predominantly on 1 modality. Less often, a review encompassed multiple modalities equally, in which case we classified it as digital education; that is, the use of digital technology in health professions education in general. Finally, we identified recommendations for future research by extracting exact quotes from each review that articulated such recommendations. At every stage, disagreements between the review authors were resolved through discussion and input from the third author (LTC).

Analysis
Authors SP and BMK rephrased the quoted research recommendations into research questions and then refined these by applying consistent terminology, removing duplicates, and merging or subdividing themes. Questions that focused on specific digital modalities (eg, online modules) were rewritten to make them relevant to the broader research agenda for digital health education. The final list of research questions was refined by LTC.
We developed a conceptual framework outlining various digital health professions' education components according to the methodology described by Jabareen [18]. We consulted and built on our previous conceptual work in this area and existing frameworks for the implementation or adoption of digital education generally [30][31][32][33][34][35][36][37][38][39]. We identified these frameworks through a focused literature search on PubMed, Google, and Google Scholar. On the basis of the discussions and consensus among the review authors, key domains and subdomains were finalized. The framework and its components were represented diagrammatically. Recommendations for future research were classified according to the proposed framework in parallel by 2 authors (SP and BMK). Discrepancies were resolved through consensus and with the guidance of the third author (LTC). On the basis of the analysis of the included reviews and the observed gaps in the literature, we outline a research agenda for digital health professions education and present it in the Results section.

Search Results
Of 7294 systematic reviews from our initial search, we identified 73 (1%) eligible systematic reviews ( Figure 1). Another 4 systematic reviews were identified through Google, Google Scholar, ResearchGate, and OpenGrey. In total, 77 systematic reviews were included for data extraction.  A breakdown of the digital modalities being investigated in the included systematic reviews is shown in Figure 2. Most systematic reviews focused on digital education in general (22/77, 29%), VR (19/77, 15%), and online education (10/77, 13%). Of the 19 reviews on VR, 17 (89%) were on VR complemented with physical objects or devices such as probes or handles and focused on psychomotor, procedural, or technical skill development. There were fewer reviews published on m-Learning (6/77, 8%), digital game-based learning (3/77, 4%), and virtual patients (2/77, 3%).
Studies, n (%) Characteristics of the systematic reviews and the evidence they include c Defined in the included systematic reviews as a combination of skills and behavioral changes as a result of the intervention. d Includes quasi-randomized controlled trials, nonrandomized controlled trials, before-and-after studies, and interrupted time series designs. e Includes study designs not described above or a combination of different study designs.

Conceptual Framework of Digital Health Professions Education
To outline different aspects of digital health professions education and identify gaps in the literature, we developed a novel conceptual framework ( Figure 3) grounded in key findings of these systematic reviews together with 7 existing frameworks for digital education general [35][36][37][38][39][113][114][115][116][117] and a framework we developed previously for health professions education [33].
Broadly, the fundamental domains include an enabling and supportive context, sound infrastructure, and the optimal use of education tools and processes. The context is a combination of institutional norms, sociocultural norms, and settings in which the learner resides, as well as the level of education the learner is at. Subdomains of the context have a direct impact on the infrastructure components required and available for the delivery of digital education-digital and physical spaces, policies and regulatory standards, and human resources. Both context and infrastructure components are important in consideration of health professions education. Learners, individually and as part of a larger group, are at the core of digital health education, and their needs, preferences, prior expertise, and competencies should shape how education is delivered. The interaction among components within and across each layer is dynamic, with different parts being interconnected, as reflected using dotted lines to separate context, infrastructure, education, and learners. Studying and identifying optimal relationships between the components are handled by the research and quality assurance blocks, which are connected to the rest of the framework. Table  2 provides the detailed operational definitions for each domain of the framework.

Context
The acceptability and adoption of digital education as a form and norm of education within the society Sociocultural norms The acceptability, impact, considerations, and processes concerning the adoption of digital education at the institutional level

Institutional norms
The setting in which digital health education is conducted or implemented, including clinical or classroom environments; low-, middle-, and high-income countries; and rural or urban environments

Settings
The impact and integration of digital education with other forms of education (eg, inter-and intraprofessional training opportunities) and clinical work in which participants are engaged

Infrastructure
The physical learning space within which the in-person component of blended digital health education is taking place Physical The information and communication technology devices (both hardware and software) to support and create learning environments (virtual environments, digital networks, technological modifications) or media for digital health education, as well as the speed and capacity of internet access Digital Policies and regulatory standards for health professionals' licensing and accreditation, as well as those relating to the design and delivery of digital health professions education

Regulatory
The human resources required for digital health education to be maintained and sustained, including educators, administrators, and information technology staff Human resources

Education
The choice and configuration of digital education modality (eg, online learning and m-Learning) and its potential blending with in-person education

Modality
The method and practice of digital health professions education encompassing teaching strategies, learning principles, learning outcomes, and the assessment approach

Classifying Research Questions Addressed by Existing Systematic Reviews
We also classified the included systematic reviews based on their research questions and using our conceptual framework (Table 3; Multimedia Appendix 3 [11][12][13][14][15][16]32,34,109]). The research questions addressed by existing systematic reviews mostly revolved around digital education modality (ie, the effectiveness of various digital education modalities delivered as stand-alone or blended interventions) and content (ie, the effectiveness of digital education within a particular health care area or discipline). Some reviews assessed interactivity (engagement), various aspects of instructional design in digital education, the impact of digital education on institutional outcomes (context-institutional norms), and the impact of setting (eg, low-income and middle-income countries) on learning outcomes. Table 3. Research questions identified from the included systematic reviews on health professions digital education.

Principal Findings
We present an evidence map of 77 systematic reviews on digital education for health professionals published between 2014 and July 2020. The reviews mostly focused on the effectiveness of various digital education modalities in surgery, health professions education in general, and nursing. Most reviews have focused on online and offline learning. Only a few reviews focused on other digital education modalities such as m-Learning, VR, digital game-based learning, and virtual patients. We developed a novel conceptual framework outlining key components of digital health professions education, namely context, infrastructure, education, learner, research, and quality assurance. Within these reviews, we identified 61 unique recommendations (questions) for future research, focusing primarily on digital education modality, instructional design, and assessment.

Limitations and Strengths
Our study has some limitations. First, to cover the most recent evidence in the field of digital education for health care professionals, we excluded studies before 2014; earlier reviews might have identified important research questions that remain unanswered. Our focus on systematic reviews also excluded other article types, such as editorials or viewpoints, which might have identified additional research questions. Second, although extraction and classification of research questions were done in duplicate and using a standardized approach, other classifications could be justified in some instances, which implies a degree of imprecision in the reported frequencies of specific questions. Moreover, our method did not allow us to prioritize the numerous research questions; such prioritization would require input from a representative group of experts and could be the focus of a future study. Third, there are overlaps among different concepts specified within this conceptual framework, which could be delineated and presented differently depending on potential chosen emphasis or entry points. Fourth, reviews classified as online education varied substantially in their inclusion of other modalities (eg, some expressly excluded modalities such as virtual patients, digital games, or massive open online courses, whereas others included these and other modalities). Finally, our novel conceptual framework may require revision as our understanding of this field matures and evolves, additional evidence accumulates, and new technologies emerge.
Our study also has several strengths, such as a thorough literature search for relevant studies, encompassing several indexed and gray literature databases without restrictions. We followed an established evidence map methodology and performed the steps in duplicate and independently [17]. In the development of our conceptual framework, we drew from the existing frameworks, our previous work, and discussions with experts.

Integration With Prior Work
We did not find other frameworks presenting a high-level overview of the use and implementation of digital education for health professionals. Therefore, we drew from the general digital education literature and found several relevant frameworks.(Multimedia Appendix 5). The included frameworks, while providing an overview of digital education, often had additional objectives such as exploring the role of specific stakeholders (eg, the private sector or the ministries), identifying barriers to adoption, or analyzing a particular digital education aspect, setting, or configuration [33,[35][36][37][113][114][115][116][117]. Our framework complements other frameworks by pulling together domains previously presented only in isolation and by adding novel subdomains such as the impact of training levels, the role of regulations and accreditations, and the importance of physical infrastructure (Multimedia Appendix 4 [11][12][13][14][15][16]32,34,[98][99][100][101][102][103][104][105][106][107]).
Several viewpoint articles have offered research agendas for digital health professions education [28,[118][119][120]. They focus primarily on the design of interventions and research studies in this field, which correspond to the domains of education (modality, instruction design, assessment, and engagement), research (quality of methods and reporting), and context (setting and level of education) in our framework. The agendas espoused in these viewpoints include questions that probe more narrowly and deeply on specific issues relevant to the design and focus of future studies (eg, the choice of comparison intervention and avoidance of confounding, integration of digital education across different institutions, and the need for interdisciplinary collaboration). Our framework was intentionally broad and comprehensive and enabled us to accommodate a variety of additional questions on previously neglected topics.

Implications for Research and Practice
Most reviews in our evidence map focused on the effectiveness of digital education interventions and rarely addressed issues around their implementation and adoption. These reviews also mostly compared the effectiveness of digital interventions with that of nondigital education. Findings from studies comparing digital and nondigital education have limited generalizability as these studies cannot account for variance within and between these 2 educational formats [118]. Future research should compare different digital education modalities as such studies are more likely to generate meaningful, generalizable findings. It should also aim to explore potential challenges related to the implementation and adoption of digital education interventions in different settings.
There is also a need for more methodologically robust research and clearer terminology in this field. The quality of the evidence, as reported in the included reviews, was relatively low, with a limited number of studies measuring skills and knowledge retention. Furthermore, it was at times difficult to determine which modality (or modalities) the included reviews focused on because of poorly explained inclusion criteria. Such ambiguity was particularly common in reviews on e-learning and blended, online, and internet-based education.
We also express concerns about the paucity of studies from lowand middle-income countries. Such countries could greatly benefit from digital education, especially by using free or low-cost education (eg, massive open online courses). Although some research findings have a universal application (eg, fundamental principles of effective learning), others (such as implementation, infrastructure, and learners) are more context specific. Given the presence of unique needs of low-and middle-income countries (eg, distinct content priorities, learner demographics, and infrastructure), we urge more research in these contexts.
Our conceptual framework will benefit researchers, funding agencies, and educators, among others. The specific questions identified and classified according to this framework provide a map for future research and can help prioritize original research studies and guide the planning of new or updated systematic reviews. We encourage investigators to broadly consider the questions we identified in this evidence map, especially those specific to areas previously less studied, such as infrastructure, learners, or quality assurance in digital education. Our framework can also be used by funding agencies to better understand the limitations of the existing research and identify areas with limited evidence with the aim of informing their funding calls in this field. Finally, this framework can encourage those developing new courses to anticipate and plan for issues that are important but might be inadvertently overlooked, such as the digital education context, infrastructure, and learners.