Prevalence of medical students’ satisfaction with online education during COVID- 19 pandemic: A systematic review and meta-analysis [version 2; peer review: 3 approved with reservations]

Background: Electronic (E)-learning is defined as the use of electronic tools for education, training, and communication.(ERT) is defined as use of alternative methods of teaching such as remote learning during a crisis circumstances Education, among many other sectors, has been profoundly affected by the spread of the coronavirus disease 2019 (COVID-19). More than 90% of the world’s students are unable to attend teaching sessions due to the COVID-19 pandemic. Methods: This study was conducted in accordance with the published guidelines for meta-analysis and reviews (PRISMA) reporting guidelines. A database and electronic search was performed on September 21st, 2021 using PubMed, Medline and Embase through the OVID platform, and ScienceDirect. We removed duplicates, and screened the title, abstract, and full texts of included papers. We included studies published only in English and excluded studies without sufficient data, case reports, editorials, and protocols. The quality of included articles was examined using the AXIS tool for cross-sectional studies, and the Newcastle–Ottawa scale for observational case-control studies. From the included studies, demographic and satisfaction with online education (OE) prevalence data were extracted and analyzed. We calculated the pooled prevalence of medical students’ satisfaction. in pooled medical students’ with online Publication bias was assessed and reported. Conclusions: The pooled prevalence of medical students’ satisfaction with online education was 57 %. Online learning satisfaction was associated with students’ prior experience with OE. The greatest benefit of OE is overcoming obstacles faced with learning Major challenges for implementing OE were technical and infrastructural resources. This article is a meta-analysis of medical student satisfaction with online learning during the COVID-19 pandemic. This is a systematic literature review covering medical students’ satisfaction with online learning during the COVID-19 pandemic.


Introduction
According to United Nations Educational, Scientific and Cultural Organization (UNESCO) estimates, more than 90% of students globally are not able to attend teaching sessions due to the COVID-19 pandemic 1 . Education, among many other sectors, has been profoundly affected by the spread of the coronavirus disease 2019 (COVID-19) which upset many aspects of our lives 2 . Widespread closure of schools and educational facilities has resulted into the implementation of prevention policies, including social distancing policies, to tackle the pandemic 3 . Nevertheless, in such exceptional circumstances, an increase in academic activities which utilize Electronic (E)-Learning is a common trend. Such a trend represents a rapid changeover from on-campus classes to virtual sessions 4 .
As an adaptation to overcome the lockdowns and social restrictions, social media platforms and many others medical education tools have been largely and creatively utilized, however, not without difficulties: these methods have faced setbacks in replicating in-person clinical experiences for training those who are expected to join the healthcare workforce soon 5 . Other concerns include the difficulties to maintain dynamicity over video meetings and other platforms, as balancing home and work life, and handling the raised workload due to providing these activities has been known to lead to burnout among facilitators 6 . (ERT) is defined as use of alternative methods of teaching such as remote learning during a crisis circumstances.
E-learning is defined as the use of electronic means for education, training, and communication 7 . Studies have suggested a number of advantages of E-learning, including better communication and interaction between the teacher and students, higher adaptability and accessibility 8 . Irrespective of their location or time, E-learning sessions can reach a large audience 9, 10 . This advantage, however, comes with possible limitations and prerequisites, as all enrolling students need to own a minimum range of equipment; also in addition, E-learning presents limitations relating to effectiveness, and difficulties in keeping academic decency 8,11,12 .
We aimed to systematically review the literature and evidence on the level of medical students' satisfaction with online education (OE) during the COVID-19 outbreak.

Methods
This study was conducted in adherence to the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) reporting guidelines 13,14 . Institutional review board approval was not required.
Medline and Embase search strategies are attached in the appendix. We removed duplicate references, and screened the title and abstract of all included papers, followed by full-text screening for all eligible studies investigating medical student satisfaction with online education during COVID-19.

Eligibility criteria
We only included studies published in English, and only medical students was involved, all published studies at time of search were included. We excluded all case reports, editorials, and protocols, because of the weak evidence they reported. Studies with insufficient data, which did not involve the main items from our PICO(Participants, Intervention, Comparison, and Outcomes), i.e., had either no data about satisfaction, had not been carried out during COVID19 pandemic, did not focus on medical students, or did not mention OE, were excluded. Of all included studies, two independent reviewers (HA, OM) screened the titles and abstracts. A third reviewer helped resolve any discrepancy between the reviewers by consensus (LM).
Data extraction and quality assessment Two independent reviewers (HA, OM), used Microsoft Excel to examined the quality of each article using the Newcastle-Ottawa scale for observational case-control studies, and the AXIS tool for cross-sectional studies 18,19 . Using an arbitrary cut-off, articles scoring 50% or more were considered of good quality.

Amendments from Version 1
In this version, we correct some typing mistakes in abstract, and the text; we such as prevalence of satisfaction were converted to percentage. We add the term Emergency Remote Learning (ERL) to distinguish it from online education, and the definition of it.
We correct some references errors, and we add references to all included papers in Table 1, and make some correction to study periods.
All typing errors are corrected. .

REVISED
Two independent reviewers (HA, LM) extracted information from each included study as follows: summary of involved studies (first author, country, study design, including study period, sample size, previous experience with online education, medical students' satisfaction with online education result, and method of satisfaction assessment, advantages and disadvantages/ challenges of online education) and the data for meta-analysis (total number of students and number of satisfy with online education). Data was gathered using Microsoft Excel 2019 (Microsoft Corporation, Redmond, WA). The met-analysis was conducted using MARVIS app (R-based web application for meta-analysis), using the DerSimonian model and inverse variant method 20 .

Study characteristics
The search led to the selection of 96, 630, and 14 records in PubMed, ScienceDirect, Medline, and Embase through the OVID platform, respectively. After removing duplicates, we included 708 studies for title and abstract screening, and 664 were excluded due to irrelevant data. We excluded 10 records after full-text screening with an agreement kappa of 0.9 21 . One study was included through the snowball method, and the remaining 34 studies were assessed for quality. A total of 18 studies, from Asia and the United States of America (US), were foundto be of good quality and were used for qualitative synthesis (Table 1). Selection process details are showed and summarized in Figure 1.

Demographic characteristic
Most studies were from Asia and the US, and only one was from Europe. The total number of assessed students was 7,907.

Satisfaction with online education
In some studies, students reported OE to be positively influencing their perception of their online learning experience and therefore was one of the main factors that affected their satisfaction. These benefits included saving time 22-26 and physical effort relative to commuting to attend lessons in-person 22 . In other studies, students reported that one of the key advantages of OE was good, focused interaction with instructors 4,22,23,25-29 , and with peers 22,23 . Moreover, OE provided a convenient and flexible learning experience 22,23,25,27,30,31 , accompanied by an improved, stress-free environment 22 . Students reported the experience as a whole to be more enjoyable 22,30 . They also mentioned good access to resources 27,32,33 , high image resolution, and the ability to record as important technological benefits 22 . Other reported benefits of online learning included better knowledge gain and understanding 22,34 , improved integration and linking the knowledge with clinical skills, as well as help building interest in clinical medicine 34 , better self-efficacy in the fields of learning, technology, time management 24 , concentration 22 and satisfaction with academic performance 35 .

Challenges
Among negative impacts on their experience with online learning, students mentioned that OE was not particularly useful in developing the required clinical and practical skills needed for practice 4,23,27,30 . In some studies, students reported low levels of confidence in taking clinical examinations 4 and managing patients without hands-on sessions 4,27,29 . Technical issues and limited access were some of the most common issues mentioned in the literature 4,22,[30][31][32]34 . Several studies reported issues such as low levels of interaction with instructors 23,29,30 and peers 23,26 and diminished level of interest and focus 4,22,30 . Another major concern was the stress brought on by keeping up with schedules and tasks 4,25,30,31 . This was further explored when OE was reported to be boring and lengthy 30 , leading to long screen time and long waits for downloading learning materials 31 . Other complaints were about the insufficiency of collaborative activity 31 and feedback on assignments 4 .

Meta-analysis
The pooled prevalence of medical student satisfaction with OE during the COVID-19 pandemic was 57% (95% CI: 47 -67%) ( Figure 2). An I 2 test revealed significant study heterogeneity (I 2 = 99%, p < 0.01). We examined the risk of publication bias using an Egger test (t-value = -0.85, p-value=0.933) and carried out a visual examination using a funnel plot ( Figure 3). Drapery plot representing the P-value of each individual study ( Figure 4).

Discussion
The COVID-19 outbreak forced educational systems to migrate towards OE. This study aimed to evaluate the prevalence of medical students' satisfaction with OE globally during the COVID-19 pandemic, and provides insights into major advantages and challenges that faced students using OE. The overall prevalence of medical students' satisfaction was 57%.
OE is widely used in educational programs worldwide. In 2013, more than 7.1 million students were enrolled in OE according to a survey conducted by the Babson Survey Research Group 36 .
Advantages of OE are time saving, good time management and increased time for self-study, advanced technology like virtual microscopes, high resolution on radiology imagesSimilar findings were reported by Gilbert et al. and Subhashni et al. 37,38 .
We found major disadvantages were that medical student were not confident in treating patients relying only on OE and not real-life cases, as well as a lack of skill, sense of isolation and isolation from friends. A similar finding was presented in a study by Maria et al. 39 .
One of the important advantages frequently mentioned in the literature was the convenience and flexibility of OE, and being mainly conducted in a relatively stress-free environment 40 .
The satisfaction was low in developing countries in comparison with developed countries. A review by Than et al. concluded that many obstacles stand in the way of OE in developing countries, such as cost of information and communications technology infrastructure, the need to develop a new curriculum adapted for OE, absence of motivation, insufficient experience of the tutors and lack of computer competency to use OE platforms, and electricity supply problems 41 . On the other hand, developed countries do not suffer from these obstacles due to availability of adequate resources 42 .     Medical schools that have clinical or practical aspect are less likely to be satisfied than collages that mainly have theoretical aspect 46 . The advanced technology in OE increases the satisfaction of students e.g., they were found to be deeply impressed by virtual microscope 47 .

Conclusions
Online education is a novel and effective method of teaching and assessing the performance of students. The prevalence of medical students' satisfaction with online education was 57 %. The major benefit of OE is to overcome obstacles faced with learning, and to adapt for any challenges at any circumstances in order to continue the educational process, especially during pandemics. Efforts to facilitate the running of OE, in terms of connectivity, training, and platforms must be considered.

Strengths and limitations
To the best of our knowledge following our search, this is the first comprehensive meta-analysis to assess medical students' satisfaction with online education. Students from schools spanning all medical disciplines were involved. However, we were met with some limitations: First, we assimilated only studies in English, which may have led to language bias. Secondly, we searched only for published articles, which can also lead to some degree of publication bias.

Recommendations
We need to think of electronic learning not just as an alternative method to be used as required, but also as an integrated part of the curriculum after analysis and evaluation of available resources. Moreover, blended or hybrid approach of education which involve both online methods for theory and hands-on sessions for clinical skills, may help bridge the gap in clinical skill learning with OE.
A qualified training of both tutors and students must be conducted as preparatory sessions to overcome the technical difficulties and to familiarize them with the platform. Governmental and institutional should support developers and technology companies to make and update the appropriate, unified platforms, as well as providing accessible and cheap or free internet access, especially in developing countries.
More research must be conducted to analyze and quantify the benefits against challenges faced by educational institutions, and to design appropriate solutions for effective and easy delivery of educational content.

Results:
The sections "Satisfaction with Online Education" and "Challenges" contain useful information for consideration of student satisfaction in medical education moving forward.

Discussion and Conclusions:
Contains interesting discussion points related to advantages and challenges of online learning in medical education, and specifically in clinical education.

○
While the transformation of in-person learning to online learning during the COVID-19 ○ pandemic was a dramatic shift, online education in general should not be referred to as "novel", as it has used for decades.
Minor errors: Several minor errors and mis-keys in typing are contained in the article.

Are sufficient details of the methods and analysis provided to allow replication by others? Partly
Is the statistical analysis and its interpretation appropriate? I cannot comment. A qualified statistician is required.

Richard B. Hays
College of Medicine and Dentistry, James Cook University, Townsville, Qld, Australia As the world moves out of the pandemic to a 'more normal' situation, many educators and institutions are re-thinking learning and assessment strategies in response to the apparent 'success' of the emergency remote teaching arrangements that had to be used. This is therefore a timely contribution, but I am not sure that it will be much of a guide for education providers to manage the transition. Should we go 'back to normal' or increase the use of remote learning methods? It would appear that the spectrum of responses is wide. The review method is sound but the small numbers, variation in dates (back to 2005?) and narrow geographic sampling raise questions about extrapolation.
I am not best placed to comment on the statistical modeling.
An important issue to resolve is a definition of 'on-line education'. This is more of a format than a particular method, because many methods can be moved on-line. Many institutions have used online methods for many years to provide learning resources around topics to guide reading, formative assessment, discussions etc. Similarly, many institutions have made lectures into podcasts for either initial learning or revision. The unanswered question is just how much on-line learning could increase without reducing learner satisfaction? Emerging reports suggest that most medical students like to attend to mix with and learn with and from their peers.
Further, what does a satisfaction rate of 57% mean? Is that good or bad? I wonder if the message from this article is that caution must be exercised in increasing the use of on-line learning. While such methods will always be important and valuable in providing resources and facilitating some forms of discussion and communication, program design should start with purpose and content and then be eclectic with regard to methods and formats best suited to purpose and content. The post-pandemic period provides an opportunity to review programs and their delivery. The article would be strengthened by commenting on this broader education debate.

Are sufficient details of the methods and analysis provided to allow replication by others? Yes
Is the statistical analysis and its interpretation appropriate? I cannot comment. A qualified statistician is required.

Medical Education and Informatics, Sultan Qaboos University, Al Khoudh, Oman
This is a systematic literature review covering medical students' satisfaction with online learning during the COVID-19 pandemic.
Although the study itself appears to have been well performed, there are several issues that need to be addressed in the paper. Most of these are really important and do need to be corrected. However, because it is possible to correct them without re-doing the study, I have rated the paper as "Approved with Reservations" in the hope that the authors will be able to apply the corrections.

Online Learning and ERT:
The single most important point is that I think it is crucial for the authors to emphasise that the online learning introduced as a result of the COVID-19 pandemic was not typical online learning, but is best described as Emergency Remote Teaching (ERT) (Hodges et al., 2020). Medical Education has used online teaching and learning for decades, and it differs quite substantially from ERT. Masters and Ellaway, 2008;Stojan et al., 2021).
The main reasons for identifying this distinction are to: In the Introduction: demonstrate to the readers that the authors are well-versed in the concepts and history of online medical education. ○ All through the paper: ensure that there is no implication that online medical education is new or as a result of COVID-19 (And so, avoid such potentially embarrassing statements like "Online education is a novel and effective method of teaching and assessing the performance of students". It is not novel at all, and has been practised for at least two decades). The statement in the conclusion that follows is similarly of concern. ○ In the Introduction: acknowledge that the expectations, especially student satisfaction, are different from properly organised and formalised online education, and one would expect a low satisfaction rate with ERT.

○
In the Discussion: view the outcomes (again, especially student satisfaction) in light of the expectations.

○
In the Discussion and Conclusion: ensure that any commentary on these outcomes takes this distinction into account, and it should not be seen as a commentary on online education.

○
It is crucial to the quality of the paper that all of these points are made explicitly.
Disclaimer: I am a co-author on two of the publications I have cited above (See list of references below), and I am not suggesting that the authors cite those publications: they are mentioned to provide guidance to the authors, to demonstrate that online medical education has a wellestablished record, and also to reinforce the argument that ERT differs substantially from established online learning.

Other issues:
Although PICO is a well-known abbreviation, it needs to be first written out in full. Also, I am ○ not entirely sure that PICO applies here, so the authors might wish to consider how to refer to this or to re-phrase.
"The search led to the selection of 96, 630, and 14 records in PubMed, ScienceDirect, Medline, and Embase through the OVID platform, respectively." Three figures but four databases are listed. I suggest re-phrasing so that it is clearer that the last figure applies to two databases.

○
In the exclusion of the 664 articles, the authors use the term "irrelevant data". I'm afraid that this is extremely broad, and needs to be explained in more detail, and the sub-categories need to be shown in Figure 1.
○ From there on, there is quite a bit of discrepancy between the description in the text and the description in the diagram (Figure 1). E.g. the text speaks of 10 records excluded, but Figure 1 shows 11. The text says they were rejected after full-text screening, but Figure 1 says they were rejected because the Reports were not retrieved. The reader would ask: if the reports were not retrieved, then how could there be any full-text screening? ○ Table 1: There are many problems with Table 1: The Items appear to be listed in an arbitrary order, sometimes with surnames, sometimes with first names. This makes finding an item extremely difficult. The items should be listed in Alphabetical order of first author's surname. (I have seen some reviews list them in chronological order, but this is generally not preferred).

○
Next to the Authors' surname, the reference citation number needs to be given. This is absolutely necessary so that readers can relate the citations listed in the Discussion to the items found in the Review; otherwise there is no connection between the two.

○
In fact (see my comment on the Discussion below) there appear to be other problems with the referencing and citing, so that will have to be corrected first before Table 1 can be corrected.
○ What is the column headed "Events"? This is not explained or referred to in the paper, and the figures appear to be identical to the column headed "Number of students who were Satisfied". (This "Events" heading is then used again in Figure 2).

○
In would be useful to the reader if the number of satisfied students was also given in Table 1 as a percentage of the sample. The total of the entire group should then also be stated with the percentage at the end of the table (or in the Results referring to the Table).
○ In many cases, the study period does not indicate the year. This should be stated for each study (unless they were all 2020, in which case this can be stated once and then removed from the Table). If the year is not stated in the study, and cannot be inferred, then this should be noted with an asterisk and footnote in the Table. ○ ○ In the Discussion, there appears to be a disjunction between the citing and the referencing.
○ For example, in the Discussion, the paper says "A similar finding was presented in a study by Maria et al. 38  In fact, the more I look at it, the more I struggle to find any Table 1 items in the list of references, so it appears that something has gone terribly wrong with the list of references.

○
The "arbitrary cut-off, articles scoring 50% or more were considered of good quality" should be mentioned in the limitations.
○ "queried forthe following strategy" A space is required between the "for" and the "the" ○ There many other small language, typing and punctuation errors in the paper, too numerous for me to raise here, so I would strongly recommend that the authors conduct a very careful re-read of their paper to correct these. ○ So, the paper currently has potential, but still requires quite a bit of work. I look forward to Version 2 in which these issues are addressed.