Enhancing Eﬀectiveness of Residents' Virtual Medical Education During COVID-19 Pandemic

Introduction: The COVID-19 pandemic inherently led to the transfer of face-to-face learning to virtual learning on a global level. Virtual learning was never a popular learning method in Saudi Arabia. Due to the preconceived notions about virtual education, our residents' learning outcomes in the early stages were aﬀected negatively. Methods: We compared the knowledge acquisition from diﬀerent teaching methods, face-to-face, virtual didactic, and virtual interactive lectures, by comparing the residents' exam scores for MCQ tests, which assessed memorization of some information discussed during these diﬀerent teaching sessions. Results: There was a statistically signiﬁcant drop in the residents' exam marks when we switched from face-to-face to virtual didactic learning (p-value 0.0469). But we found no diﬀerence when we compared their exam marks for tests done after the interactive virtual learning and that after the previous face-to-face learning (p-value 0.0547). Conclusion: Virtual learning is an eﬀective way of learning in medical education, even with those who are new to this teaching method, on the condition we do it using the appropriate andragogical approach. In this paper, we also discussed various ways that can be used to enhance learning acquisition from online teaching sessions.


Introduction
Online learning, or e-learning, is the learning that takes place over the internet, where the learners can access lectures, read some materials, and complete homework at their own pace and on their schedule (Moore, Dickson-Deane and Galyen, 2011). On the other hand, virtual learning is the learning that happens through live lessons (WizIQ, 03/05/2021). In contrast, blended learning involves a combination of traditional face-to-face learning in addition to online or virtual learning.
Both online and virtual learning has been used in medical education in many countries for many years. Some studies have shown that most students prefer face-to-face as there is more interaction (Kemp and Grieve, 2014). Simultaneously, some other studies have found that both are a very effective way of learning (Yuwono and Sujono, 2018). Nevertheless, in many other countries, such as Saudi Arabia, virtual education was never a popular learning method among learners and educators before the COVID-19 pandemic (Tashkandi, 2020). From the beginning of the pandemic, many medical residency programs worldwide, including those in Saudi Arabia, quickly changed their teaching methods from traditional face-to-face to virtual lectures (Tashkandi, 2020).
At King Faisal Specialist Hospital in Riyadh, our residents and trainers, who were not used to this type of education, faced many challenges which negatively affected the residents' learning outcome. Among these challenges was the absence of direct face-to-face interaction, which caused poor residents' motivation and the high chance of learners' distraction. This paper compares the learning outcome from the face-to-face and that from virtual teaching and discusses various methods to improve virtual education effectiveness.

Methods
This paper presents an analytic cross-sectional study comparing the residents' marks from MCQ exams that we used to test the learning outcome from traditional face-to-face teaching and virtual teaching, before and then after introducing some crucial changes that encouraged more residents interaction and made them more active learners. We also asked our twenty residents and eight trainers about their opinion regarding virtual education and investigated their understanding of adult learning (andragogy) to help us understand the problem in a better way and find the appropriate solution.
All the statistical analyses of the collected data were analyzed using the software package SPSS, version 26.0 (IBM Corp., Armonk, NY). We reported descriptive statistics for the continuous variables as mean and summarised categorical variables as frequencies and percentages. We did hypothesis testing for continuous variables using the Two-tailed t-test and one-way ANOVA. The level of statistical significance is set at p < 0.05.

Results/Analysis
When we started delivering our residents' educational activities virtually, some residents and some trainers felt that these virtual sessions were not effective. Therefore, we decided to assess the learning outcome after virtual learning and compare it to that after face-to-face learning. We used the last four MCQ exams that we did after the face-toface teaching sessions before the COVID 19 pandemic and four MCQ exams that we did after the first four virtual teaching sessions after the beginning of the pandemic. We compared the marks of the residents they scored when they set these exams to see any difference in the knowledge acquisition (see Table 1). We used a two-tailed t-test for Elabd K MedEdPublish https://doi.org/10.15694/mep.2021.000125.1 Page | 3 this comparison. We found that our residents' marks had dropped after we started the virtual learning (p-value 0.0469) (See Figure 1). Therefore, we investigated the matter by asking our residents and trainers to find out what went wrong to try to fix it. Sixteen of our residents (80%) and five of the trainers (62.5%) enjoyed virtual learning as they found it convenient. But half of the residents (50%) felt that the learning outcome from virtual learning is much less because of the lack of face-to-face interaction and the distraction that might happen at home. Twelve of our residents (60%) did not know much about andragogy theory or problem-based learning (BPL). On the other hand, although most of our trainers (87.5%) knew the importance of active adult learning and problem-based learning, they did not know how to follow these principles in a virtual environment.
Consequently, we implemented some changes to make the residents take a more active role in the learning process. Later on, we reassessed the learning outcome from these virtual active learning sessions (see Table 1). We found out the residents' marks have improved. There was no statistical difference between the marks they scored after face-toface learning and the virtual sessions, which involved more active learning (p-value 0.3261, 95% confidence interval -6.27 to 4.77) (See Figure 1).

Discussion
Virtual medical education is a convenient teaching method for both residents and medical educators, but we have to adopt active learning principles to make it effective. Switching from face-to-face to virtual teaching requires some adjustments for both residents and trainers. Delivering the same lecture, which is meant to be for a face-to-face session in a virtual session, will not help our residents learn efficiently.
As part of our continuous efforts to improve our residents' learning outcomes, we introduced MCQ tests after each lecture we gave to our residents two years ago. We were hoping that these tests might increase the residents' attention during these lectures and improve their learning (Yang, Razo and Persky, 2019). Each MCQ test was made up of 20 questions: testing memorization of some information mentioned and discussed during these teaching sessions, i.e., K1 level of Bloom's taxonomy (Armstrong, 2010). We also continued using these tests after using virtual teaching during the COVID-19 pandemic, which gave us the chance to compare knowledge acquisition from face-to-face and virtual learning.
Many studies have looked at virtual learning and found it beneficial and not less than face-to-face learning (Pei and Wu, 2019;Wilcha, 2020). But we have to keep in mind that any lecture, whether virtual or face-to-face, has to be interactive and following the adult learning principles "Andragogy" to improve its learning outcome (Chametzky, 2014;Datta, Datta and Venkatesh, 2015). It might be even more important to stick to these principles during virtual sessions as there are many challenges that we face during these sessions more than the face-to-face ones (Kebritchi, Lipschuetz and Santiague, 2017; Janse van Rensburg, 2018). Unfortunately, the lack of knowledge and experience in adult learning and problem-based learning among our residents and trainers, especially in virtual teaching sessions, had badly affected these teaching sessions' quality. As a result, there was a drop in the residents' exam marks after virtual teaching compared to those done after the face-toface teaching (p-value 0.0107). The difficulty in initiating or maintaining the residents' active participation during the learning sessions reduced their learning outcome and consequently their exam marks.
Hence, it was essential for us to investigate this problem thoroughly. Our investigations showed that although virtual teaching was convenient for both residents and trainers, these sessions had become didactic with less peer interaction. As a result, our residents had become passive receivers of information.
To improve the residents' learning from virtual education, we introduced few changes to how we deliver their lectures to make our residents active learners and not just passive receivers of information. We continued following every teaching session with the MCQ exam as we know that testing enhances learning. But we made these tests online as it is more suitable for virtual learning. We delivered some training sessions for our trainers to help them improve their online teaching skills. We agreed with our trainers to encourage residents to be more active in their learning through, for example, giving them some material and resources about the lectures' topic to study individually or in groups before its date. Also, to foster more active peer interaction during the lectures, through, for example, initiating group activities and discussions among the residents to solve various problems to make the sessions more interactive and enhance collaborative learning. This group discussion can be done using breakout rooms in any online learning platform, such as Microsoft Teams or Zoom.
Besides, we encouraged trainers to use different real-life case scenarios during their online teaching to make lectures more relevant to the residents and develop their problem-solving and critical thinking skills. We encouraged the lecturers to use more audio-visuals to enhance memorization and residents' attention span and avoid long lectures to prevent losing their learners' attention. We also started using polls to generate more engagement through active responses to various quizzes during the teaching sessions. Moreover, we introduced the flipped classroom technique, where one or more of the residents prepare and deliver the teaching session while the trainer supervises and guides the discussion.
After we implemented these changes, we compared the knowledge acquisition (memorization) that happened after these active virtual learning sessions to that which occurred after the face-to-face teaching sessions. We did this in the same way by comparing the residents' exam marks. We did not find any statistical difference (p-value 0.3261, 95% confidence interval -6.27 to 4.77).
Interaction and active discussion that happen during teaching sessions are essential for deep learning and understanding. They help the residents be active learners; they increase their attention span and eventually improve their learning outcome (Smith et al., 2011). According to Malcolm Knowles, adults learn better when actively seeking knowledge and not just passive receivers of information. They also learn better when they feel they are learning something relevant to them, and they are in control of when they learn it and how they learn it. Furthermore, Knowles pointed out that adults have previous experience and knowledge, and they are continuously reformatting their prior knowledge when they learn something new (Knowles, 1977). Educators have to keep this in mind when they teach adults and seek to know their learning needs to make lectures beneficial.
There are two approaches to adults' learning in medical education, pedagogy and andragogy. We can see a pure form of andragogy when medical students or residents are independently and actively seeking to learn theoretical knowledge and practical experience from their clinical practice and from reading the theory. This independent learning is an important way of learning that we should always encourage our students and residents to follow throughout their life. On the other hand, a pure form of pedagogy is when educators try to "spoon-feed" information to their learners, who become passive receivers of information. The educators who use this approach will usually try to deliver as much information as possible without paying much attention to their learners' previous knowledge and not allowing much interaction with them during the teaching session (Holmes and Abington-Cooper, 2000). Unfortunately, this teaching method might only help the residents memorize some information and for a short time. Spoon feeding information to residents will not allow them to think for themselves; therefore, they will not develop any higherorder thinking skills such as problem-solving and critical thinking, which are essential thinking skills they need to build to manage patients safely and independently (Asok et al., 2016).
Despite what is mentioned earlier about pedagogy, it is still an essential way for teaching in medical education. There will always be a need for experienced and senior doctors to pass on their knowledge and experience to others. With some modifications, a pedagogical approach can be used with a better learning outcome, which can be achieved by keeping in mind the learners' previous knowledge and experience and incorporating their learning needs into the teaching. It is also vital to make learners feel what they are being taught is relevant to them, which can be done by starting the lectures with real-life scenarios or problem-based learning. Lecturers should avoid long teaching sessions to avoid losing learners' concentration. They should also try to use different audio-visuals in their presentations, such as relevant pictures and videos, to help their learners retain information. Moreover, learners must try to read about the topic before the teaching sessions, which will help them engage in an interactive discussion about the topic during the teaching sessions. The trainers should encourage active residents' discussion during the teaching session to clarify any misunderstanding and share their knowledge and experience about the topic. This pedagogical teaching approach can also be called "teaching-centered pedagogy." Another modification for the pedagogical approach that can improve the learning outcome is by shifting instruction from the teacher to the learners, "Flipped classroom teaching" (Hoque, 2018). This type of pedagogy has also been called "learner-centered pedagogy." In this approach, one or more residents go and study the topic well and maybe prepare a presentation, and then they discuss what they have learned with the other residents. At the same time, the trainer will only guide and facilitate the discussion. The trainers in this type of teaching are only supervisors who make sure the discussed information is correct and an effective peer discussion.
Medical educators and trainers should aim to help their learners memorize and understand information and apply their knowledge in real life. Memorization and understanding are the lower-order thinking skills of Bloom's taxonomy. They are keystones for learning and developing higher-order thinking skills (Armstrong, 2010;Vidergor and Krupnik-Gottlieb, 2015). Despite that, they are not enough for doctors to manage patients safely and independently in real life. These higher-order thinking skills include important skills such as applying the knowledge to solve patients' problems, thinking critically to come up with an appropriate differential diagnosis, chose the proper investigations, and formulate the right management plan. To develop these higher-order thinking skills, we need to encourage our residents to be active learners (Asok et al., 2016). It is also important to enable them to practice problem-solving, analyzing, and thinking critically when managing patients in real life or discussing real-life scenarios in lectures and problem-based learning sessions (Hoque, 2017).
Learning theoretical knowledge alone is not enough for our medical students and residents. To manage patients safely, residents need to acquire the other two learning domains: skills and attitude. According to Benjamin Bloom and his colleagues, learning happens when the learners develop one or more of the three learning domains: knowledge, skills, and attitude (Sugarman, 1987). Knowledge is the cognitive ability of the learner to process information. In contrast, skills are the physical abilities to perform activities and tasks. While attitude is the way of thinking or feelings, the learner might develop about someone after the learning process. Online and virtual learning might help learners acquire some theoretical knowledge, but learning different skills and attitudes cannot be learned without actual hands-on experience and feedback from experienced supervisors (Sugarman, 1987). In the medical field, a lot of our learning happens through experience, which means that we learn from seeing and treating patients and doing various medical procedures. According to David Kolb, learning from experience, or "experiential learning," occurs in a cycle. To achieve the best learning outcome, learners have to go through every step of this learning cycle (Sugarman, 1987). According to the experiential learning theory, learners go through concrete experience, reflective observation, abstract conceptualization, and active experimentation. In other words, when a resident, for example, goes through this cycle, he will practice seeing and managing patients in real life, then he will reflect or get feedback on his performance from his trainer. This reflection or feedback will help him identify his learning needs, which he can fulfill by reading and updating his knowledge. Later on, he will then go and start trying what he learned and keep going through the cycle to improve and reformat his knowledge and skills (See Figure 3).  Therefore, teaching residents through online or virtual methods alone is not enough to develop competent, skilled doctors. But we can combine virtual learning with face-to-face training "blended learning" to enable hands-on experience with different clinical skills.
This paper discusses some key points regarding virtual teaching and how to make it more effective and improve our resident learning outcome from virtual learning. It also discusses the adult learning "Andragogy" and experiential learning theories in medical education. It showed statistical evidence that virtual learning can be as effective as faceto-face learning in helping residents acquiring knowledge if we did it using the adult learning principles even with trainers and the residents who are new to virtual learning.

Conclusion
Virtual education might be convenient for trainers and residents; however, it will not be an effective learning method unless it follows adult learning principles. In this paper, we present a statistical comparison that proved knowledge acquisition from virtual teaching done in a didactic way was less than face-to-face teaching. The paper also showed no statistical difference between the knowledge acquired from face-to-face teaching and virtual teaching when it was done using andragogy principles. We also discussed a few helpful tips that can help trainers and residents make online teaching sessions more interactive, enjoyable, and effective.
It is crucial to teach and train our trainers and our residents on the adult learning "andragogy" principles and make sure they are familiar with it and apply it in their teaching and learning, whether face-to-face or virtual.

Take Home Messages
Virtual learning is an effective way of teaching and learning if done appropriately Learning outcome from virtual learning is as good as the face-to-face learning Residents and medical educators should adopt andragogical principle in virtual learning to make it effective Few important tips have been discussed in this article that can be used to make virtual teaching session more interactive

Notes On Contributors
Dr. Kossay Elabd is a consultant in Family Medicine working at King Faisal Specialist Hospital and Research center in Riyadh. He is the deputy director for the family medicine residency program and the director for the family medicine clerkship program in the hospital. He has a diploma in medical education from Manchester University, FRCGP, and MRCP from the UK. He worked as GP in the UK for 10 years and has been working in Saudi Arabia since 2012.