Impact of simulation-based education on the performance assessment, knowledge retention and mentality of nursing students: A systematic reviews and meta-analysis

Background: Simulation-based education is a new type of teaching method that is suitable for clinic-related majors. This article aims to determine its impact on clinical skill assessment, performance maintenance, and mentality of nursing students. Methods: After searching PubMed, Embase, Web of Science, The Cochrane Library, MEDLINE, and EBSCO database, we conducted a systematic collection of randomized controlled trials (RCTs) on the impact of simulation-based education on performance evaluation and mentality of nursing students. The retrieval time limit was from the establishment of the database to March 12, 2021. Meta-analysis was performed using RevMan5.4 software, and two researchers used ROB2.0 software to evaluate the risk of bias in the included literature. The quality evaluation of the outcome indicators was performed using GRADEpro. A standardized mean difference (SMD) with a 95% condence interval (CI) was used in estimating the pooled effects of RCTs. Results: A total of 21 RCTs were performed, including 1683 nursing students. The results of meta-analysis showed that simulation-based education signicantly improved the assessment scores of nursing students (SMD = 1.46, 95% CI = 1.02 to 1.90, P < 0.00001), self-condence (SMD = 1.19, 95% CI = 0.48 to 1.90, P = 0.001), satisfaction (SMD = 0.86, 95% CI = 0.13 to 1.60, P = 0.02), and knowledge retention (SMD = 1.89, 95% CI = 0.76 to 2.87, P = 0.0008), and the difference was statistically signicant. The results of subgroup analysis showed that long-term intervention (MD = 1.44, 95% CI = 0.62, 2.26, P = 0.0006) and short-term intervention (MD = 1.46, 95% CI = 0.95, 1.97, P < 0.00001) improved the performance assessment of nursing students, and the difference was statistically signicant. Conclusions: Simulation-based education can signicantly improve the assessment scores, self-condence, and satisfaction of the nursing students, and the scores can be effectively retained for a period of time.

the reports independently to determine the eligibility of the reports according to the following inclusion criteria: The experimental group adopts simulated teaching or simulator teaching; The control group adopts traditional teaching methods; Other intervention measures are required to be consistent between the experimental group and the control group; Intervention objects are nursing students, and the students' grade and gender, and education level are not limited; and outcomes assessment is focused on individual performance. The following exclusion criteria were used: The control group uses a simulator to teach, even though the simulation level of the simulator is lower than that of the experimental group; The intervention objects are nursing staff who receive training in the society after graduation or the intervention objects include other medical majors in addition to nursing students; Non-Chinese English literature; The original research data are missing or the data cannot be extracted; and The full text cannot be obtained. Any disagreement was resolved through discussion or negotiation with a third party (QW).

Data extraction
Two researchers (YX and YX) independently screened the literature and extracted and cross-checked the data. Disagreements were resolved through discussion or negotiation with a third party (QW). Information that is not yet determined but is important to this research was obtained by contacting the original research author via email or phone. The extracted data included rst author, publication year, country, study design, comparison, sample size, sample source (age and gender), type of simulation, duration, outcome measures, and results. The key elements of bias risk evaluation, outcome indicators, and outcome measurement data of concern were also obtained. Outcome indicators comprised the main outcome indicators test scores and knowledge retention (that is, the score obtained by measuring the assessment results again at a certain interval after the intervention) and secondary outcome indicators self-con dence, self-e cacy, satisfaction, and anxiety.

Quality evaluation
Two investigators independently evaluated the risk of bias in the included studies and cross-checked the results. Disagreements were resolved through discussion or negotiation with a third party. The RCT bias risk assessment tool ROB2.0, which was recommended by Cochrane manual, was used in risk of bias assessment, which had six parts: randomization process, deviations from intended interventions, missing outcome data, measurement of the outcome, selection of the reported result, and overall bias.

Statistical analysis
RevMan5.4 software was used for statistical analysis. A standardized mean difference (SMD) with a 95% con dence interval (CI) was used in the estimation of the pooled effects of the RCTs. Heterogeneity among the results of the included studies was analyzed with χ2 test (α = 0.1), and the degree of heterogeneity was quantitatively evaluated by combining with I 2 . When no statistical heterogeneity was observed among the results of each study, the xed effects model was used for analysis. When statistical heterogeneity was observed between the results of each study, the source of the heterogeneity was analyzed, and after the in uence of obvious clinical heterogeneity was excluded, the randomeffects model was used for analysis. The level of meta-analysis was set at 0.05. Obvious clinical heterogeneity was treated through subgroup analysis.

Study characteristics
A total of 2053 related articles were obtained during the initial inspection, and after a layer-by-layer screening, 21 RCTs were nally included , which included 1683 nursing students. The literature screening process and results are shown in Figure 1. The publication years of the included literature ranges from 2006 to 2020, and ve of the studies are conducted in Turkey, ve in Jordan, two in the UK, two in China, and two in Singapore. The United States, Brazil, France, Portugal, and Japan have only one article each. The research objects are mainly nursing students from different grades, mainly students in the third grade whose ages range from 10.29 years to 33.00 years. The sample sizes of the studies range from 31 to 146. The detailed information of the 21 studies is shown in Table 1. teaching, which creates hospital-like treatment environments for students. One of the articles [28] uses simulator training, that is, high-level simulators, such as a robotic arm, is used in helping nurse students learn blood pressure measurement techniques. Seven of the articles [11,12,15,20,27,29,30] use a combination of simulators and simulation scenarios for education. The intervention times of most studies are short, mostly 1-2 hours, and only six studies have an intervention period of more than 1 week. The PICO information of 21 studies is shown in Table 2. Abbreviations: Test scores Knowledge retention Self-con dence Self-e cacy Satisfaction Anxiety.
Quality of the studies The proportion of items assessed for bias risk in the RCTs is shown in Figure 2, and the tra c light chart is shown in Figure 3. The RCT bias risk assessment tool ROB2.0 was used in risk assessment. The risk of bias is shown in Figures 2 and 3. Seven trials (33.33%) have a low risk of bias for the randomization process. All trials have low risk of bias for deviations from intended interventions, missing outcome data, measurement of the outcome, and selection of reported result. Seven trials (33.33%) have low risk of bias for overall bias. All the studies included in this article describe the random process in detail, and the main reason for the high risk of bias in the randomization process is that most of these studies do not mention the process of allocation concealment. A small number of studies use single-blind or double-blind studies, and the research subjects and data analysts are blinded. However, the intervention measures of the research mainly focus on courses and training. The blinding method is unsuitable for most research subjects and interveners. Therefore, we believe that the lack of blinding will not affect the measurement of nal outcome indicators.

Meta-analysis for outcome measures
In the included studies, a meta-analysis was conducted on the performance and mentality of nursing students after intervention. The data evaluated in each area is summarized below.

Knowledge retention
Seven RCTs [20, 22-25, 27, 30] consisting of 507 nursing students were included. The analysis results of the random-effects model show that the degree of knowledge retention of the simulation-based education group is higher than that of the control group (SMD = 1.89, 95% CI = 0.76-2.87, P = 0.0008; Figure 5).

Self-e cacy
Three RCTs [21,28,30] consisting of 233 nursing students were included. The analysis results of the random-effects model show that the self-e cacy of the simulation-based education group is higher than that of the control group (SMD = 0.24, 95% CI = −0.27-0.74, P = 0.36; Figure 7).

Satisfaction
Two RCTs [29,30] consisting of 188 nursing students were included. The analysis results of the random-effects model show that the satisfaction of the simulation-based education group is higher than that of the control group (SMD = 0.86, 95% CI = 0.13-1.60, P = 0.02; Figure 8).

Anxiety
Two RCTs [12,28] consisting of 131 nursing students were included. The analysis results of the random-effects model show that the anxiety of the simulation-based education group is lower than that of the control group (SMD = −0.36, 95% CI = −1.69-0.96, P = 0.59; Figure 9).

Subgroup analysis
The meta-analysis results show that I 2 >50% of many results indicates high heterogeneity between the studies. To determine the source of heterogeneity, we conducted a subgroup analysis based on the length of the intervention. The short-term intervention included six studies [12,14,21,25,26,30], and the main intervention time was within a few hours or 2 days. The long-term intervention included 15 studies [11, 13, 15-20, 22-24, 27-29, 31], and the intervention time ranged from 1 week to 3 months. Given that few related studies involve knowledge retention, self-con dence, self-e cacy, satisfaction, and anxiety, we conducted a subgroup analysis on the index of assessment performance. The results are as follows.

Test scores
After the studies were grouped according to time, the heterogeneity of the meta-analysis results was signi cantly reduced. Thus, a random-  Figure 10.

GRADEpro evidence assessment
In the evaluation of GRADEpro's level of evidence, the meta-analysis results have three pieces of intermediate evidence: one piece of low-level evidence and two pieces of very low-level evidence. The overall level of evidence is low. The main reasons for degradation are inconsistency and imprecision. In terms of inconsistency, most indicators are statistically heterogeneous, that is, I 2 > 50%, and they are all downgraded to one level. Two main problems were determined in terms of imprecision. On the one hand, the sample sizes of some indicators do not meet the optimal sample size, that is, the total sample size of the total continuous variable is less than 400. On the other hand, the effect sizes of some indicators cross the invalid line, that is, P ≥ 0.05. Through downgrade treatment, both types of indicators have been downgraded two times. No downgrade treatment was performed on the other three aspects of grade evaluation. In terms of limitations, some studies do not use blinding, and the allocation concealment report is insu cient, although they are not downgraded because blinding and allocation concealment have little effect on the experimental results. In terms of indirectness, although the intervention measures are not completely consistent (different types of simulation scenarios), no signi cant difference was found between PICO and the research question, and thus it is not downgraded. In terms of publication bias, the literature search in this study is comprehensive, the included studies do not involve commercial interests, and no clear evidence of a risk of bias is found. Hence, publication bias will not be downgraded. See Table 3 for details. ; Imprecision: continuous variable, the total sample size of the two groups of research subjects is less than 400 cases or the con dence interval crosses the invalid line.

Discussion
Through GRADE evaluation, different types of evidence was obtained, which showed that simulation education has a certain contribution to the improvement of the assessment performance, self-con dence, and satisfaction of nursing students. Moderate-quality evidence shows that compared with traditional teaching methods, simulated teaching is more conducive to improving assessment results, self-con dence, and knowledge retention among nursing students. Low-quality evidence shows that students' satisfaction with the curriculum after simulated education is signi cantly higher than that of students who underwent traditional teaching programs. Evidence with extremely low quality shows that the impact of simulated education on the self-e cacy and anxiety of nursing students is not different from that of traditional education.
The acquisition and application of knowledge is the ultimate goal of teaching. An e cient and real teaching environment can promote students' clinical skills [32]. A number of studies have shown that [33,34] rich simulation teaching can improve the assessment results of nursing students, enhance their clinical skill levels, and even replace clinical practice. The results of this research showed that compared with traditional teaching methods, simulated education can signi cantly improve the assessment results of nursing students. Students' scores were retested from 40 days to half a year after the intervention, and the degree of knowledge retention of nursing students in the simulation teaching group was signi cantly higher than those of the traditional teaching method group. This nding may be related to the use of simulated scenes and simulated instruments in simulation teaching, which enabled students to visualize abstract knowledge and convert them to concrete operations and left a deep impression on them. A study [35] con rmed that scene simulation can help students in memorizing information. Therefore, simulation teaching can be incorporated into teaching because it enables students to apply knowledge in advance, thereby enhancing the mastery rate of knowledge.
Self-con dence is a key factor affecting students' assessment results and clinical skill application. Having a high degree of self-con dence can reduce students' sense of tension in the clinic and enable them to display knowledge that they have learned [36]. The results of this study showed that compared with traditional education methods, clinical simulation education can signi cantly enhance the self-con dence of nursing students. This may reduce the fear of the unknown environment when actually contacting the patient instead of having been in contact with the simulation scenarios. However, no difference in self-e cacy was observed between the simulated and traditional teaching groups. This result is different from the results of some studies [37], and this difference may be related to the small number of included studies, small sample size, and large heterogeneity. A study [38] showed that students are prone to nervousness, low con dence, anger, and other emotions during the assessment process. These emotions affected their performance. Simulation education can alleviate this psychology. Therefore, strengthening the simulation teaching of trial practice in advance is necessary because this approach allows students to play their actual roles in their respective clinics.
Student satisfaction with the course is an important indicator for evaluating the course and which can re ect the effectiveness and interest of a teaching method. Some studies [39,40] showed that learning methods and learning environment directly affect learners' learning interest, learning continuity, and satisfaction. The present study showed that students' satisfaction with simulation education was higher than that of traditional medical education possibly because simulation teaching is more vivid, interesting, and authentic. Increase in learning satisfaction has a signi cant effect on willingness to continue learning [41]. A real teaching environment can strengthen people's learning enthusiasm [35], thereby enhancing their sense of satisfaction. Scenes, teaching aids, and auxiliary teaching methods can increase students' skills in recognizing possibilities.
Anxiety is a common emotion in the examination process of students. A certain degree of anxiety can promote students' performance, but excessive anxiety has a considerable adverse impact on student performance [42]. In the present study, no difference among the effects of different education methods on nursing students' anxiety was found, and this result was different from the results of previous studies [43]. The possible reason was the small number of included studies and the low level of evidence. Thus, sample size should be increased, and the experimental design should be optimized.
To determine the source of heterogeneity, we divided the included literature into two subgroups for in-depth analysis according to the intervention time of simulated education. The intervention time of the long-term intervention group was more than 1 week, whereas the intervention time of the short-term intervention group was within 1 day. The results of the subgroup analysis suggested that regardless of the length of the intervention, the impact of simulation education on the assessment results of nursing students is better than that of traditional teaching. After subgroup analysis, the heterogeneity of the meta-analysis was reduced. These results indicated that intervention time is an important confounding factor in the meta-analysis of the present study.

Strengths and limitations
GRADEpro and RoB2.0 bias risk assessment methods were used in evaluating the quality of the evidence, making the conclusions objective and scienti c. For the rst time, a meta-analysis was performed on the retention of nursing students' knowledge through simulation education. Positive results were obtained, which provided support for the effectiveness of simulation research. However, some shortcomings in this research were observed. Knowledge acquisition and retention are closely related to intervention time and test time, but the measurement of knowledge retention in this study cannot exclude other interfering factors. For example, the measurement of the subjects' performance after a period of time cannot exclude the period. The subjects themselves also consolidated and reviewed their knowledge. The meta-analysis results of this study were mostly heterogeneous possibly because of the different simulation scenarios. Given that most of the interventions had short durations, our subgroup analysis only focused on the index of assessment performance. Observing the heterogeneity of other indicators through subgroup analysis is impossible. The included studies did not focus on the indicators of nursing students' mood. Satisfaction, self-e cacy, and anxiety are important factors that affect nursing students' clinical skills [8], and data collection in this area should be strengthened in the follow-up. Owing to the particularity of the interventions in this study, the included studies seldom used blinding, and many studies did not mention the allocation concealment process. Although not downgraded in the present study, randomization and allocation should be further strengthened. Moreover, blinding should be performed to improve the quality of evidence.

Conclusion
Clinical simulation education not only signi cantly improves the performance of nursing students but also affects their self-con dence in operation and satisfaction with the curriculum. It has a signi cant effect on the long-term retention of nursing students' performance. This study analyzed the impacts of simulated education programs with different intervention times on the assessment results of nursing students.
The results of the subgroup analysis showed that regardless of the length of intervention time, the scores of nursing students after simulated education were better than those of students who underwent traditional education programs. However, postgraduate simulation education had little effect on the levels of anxiety and self-e cacy of nursing students.

Declarations
Authors' contributions YX and YX conceived and designed the study. YX wrote the paper. YX and YX contributed to the critical review and revised the manuscript. YX, YX, QW SZD, XJ and GHX selected the available articles, extracted the data, and analyze the data. All authors approved the nal manuscript as submitted and agree to be accountable for all aspects of the work. Risk of bias summary.

Figure 4
Forest plot: effectiveness of simulation-based education interventions on test scores.

Figure 5
Forest plot: effectiveness of simulation-based education interventions on knowledge retention.

Figure 6
Forest plot: effectiveness of simulation-based education interventions on self-con dence.

Figure 7
Forest plot: effectiveness of simulation-based education interventions on self-e cacy.

Figure 8
Forest plot: effectiveness of simulation-based education interventions on satisfaction.

Figure 9
Forest plot: effectiveness of simulation-based education interventions on anxiety.