Medical students’ satisfaction with the Applied Basic Clinical Seminar with Scenarios for Students, a novel simulation-based learning method in Greece

Purpose: The integration of simulation-based learning (SBL) methods holds promise for improving the medical education system in Greece. The Applied Basic Clinical Seminar with Scenarios for Students (ABCS3) is a novel two-day SBL course that was designed by the Scientific Society of Hellenic Medical Students. The ABCS3 targeted undergraduate medical students and consisted of three core components: the case-based lectures, the ABCDE hands-on station, and the simulation-based clinical scenarios. The purpose of this study was to evaluate the general educational environment of the course, as well as the skills and knowledge acquired by the participants. Methods: Two sets of questions were distributed to the participants: the Dundee Ready Educational Environment Measure (DREEM) questionnaire and an internally designed feedback questionnaire (InEv). A multiple-choice examination was also distributed prior to the course and following its completion. A total of 176 participants answered the DREEM questionnaire, 56 the InEv, and 60 the MCQs. Results: The overall DREEM score was 144.61 (±28.05) out of 200. Delegates who participated in both the case-based lectures and the interactive scenarios core components scored higher than those who only completed the case-based lecture session (P=0.038). The mean overall feedback score was 4.12 (±0.56) out of 5. Students scored significantly higher on the post-test than on the pre-test (P<0.001). Conclusion: The ABCS3 was found to be an effective SBL program, as medical students reported positive opinions about their experiences and exhibited improvements in their clinical knowledge and skills.


Introduction
Simulation-based learning (SBL) has been proven to be a cost-effective, easily accessible, and promising educational method in modern education [1]. Numerous studies have proposed it as an effective and safe educational tool for many educational goals, including practice in emergency medicine [1,2]. SBL is a safe environment to practice skills, as it allows participants to repeatedly perform clinical tasks and processes and to debrief about their performance and possible mistakes, without any risk of harming or disturbing a patient. Large meta-analyses and reviews have demonstrated the positive impact of SBL on educational goals and patient-related out-

The basic idea of the seminar
The ABCS3 was designed in order to create a simulationbased educational environment that involved the medical students with a variety of common clinical scenarios and familiarized them with fundamental skills and clinical competencies. The seminar lasted for two days and it consisted of three core components: the case-based lectures (Le), the first-aid and resuscitation course (named "ABCDE hands-on station"), and the simulation-based clinical Scenarios (Sc).
A total of 230 medical students from medical schools all over Greece attended the seminar from April 3 to April 4, 2015. Interested students applied online and a scientific committee selected participants based on information from their curricula vitae (year of study, grades in medical school, previously attended courses, and any publications, presentations, or conference abstracts). The attendees who were selected were then split into two groups: the full category, consisting of 60 students (in their fourth, fifth, and sixth years of study) who attended all the modules of the course, and the observer (OB) category, consisting of 170 medical students of any year of study. The OB group included delegates who attended Le, as in the full group, although they observed the Sc in a large lecture hall, via video live-streaming, without having the chance to interact.
The Le core consisted of 15 case-based lectures and was attended before the Sc section, as an introduction that would provide relevant theoretical knowledge and introduce participants to the actual procedures involved in SBL.
The ABCDE hands-on station was held between the Le and the Sc sections, aiming to teach and/or provide a refresher regarding the ABCDE approach to the attendees, as they would be asked to apply it in the subsequent scenarios.
The Sc core was the main part of the seminar and consisted of moulage scenarios and skill stations. Each scenario lasted for thirty minutes, and the 60 participants (from the full category) were divided into 10 groups (I-X) of six people. Each group participated in one scenario (I in Sc1, II in Sc2, etc.). After 30 minutes, all 10 scenarios were completed, and then, with the sound of an electronic alarm, all 10 groups moved forward to the next scenario respectively (I to Sc2, II to Sc3, and so on). Consequently, after 10 cycles of rotation (and 10 repeats of each scenario by different groups), all groups (I to X) had attended all the scenarios (Sc1 to Sc10) ( Table 1). Each scenario took place in an appropriately designed room, with all the necessary medical equipment. Mannequins were used in Sc3, Sc5, and Sc9, while in Sc4, Sc6, Sc7, Sc8, and Sc10 trained actors (standardized patients) who were dressed appropriately presented their medical histories and simulated the appropriate symptoms and reactions, according to the specific requirements of each scenario. In the Sc1 station, the delegates practiced suturing, while in Sc2 the participants practiced peripheral and central venous catheter insertion. A laptop was available to visualize lab and imaging test results.
In each repetition, the six participants and two instructors were present in the room. The role of the instructors was to direct the process of the scenarios; to enforce time limits; to help them function as a team, with distinct roles for each team member; and to follow the most appropriate diagnostic and therapeutic strategy to save the patient, within narrow time limits. Table 1 contains details about the topics of the Le and Sc components.

Evaluation
Numerous methods and instruments for assessing undergraduate medical learning environments have been published [5,6]. One of the most widely-used learning environment assessment tools is the Dundee Ready Educational Environment Measure (DREEM). The DREEM and two other instruments (the internal evaluation inventory [InEv] and the MCQs), were used to assess the students' opinion of the course (DREEM, InEv) and the contribution of the course to their clinical knowledge (MCQs) (Fig. 1). The data were processed using SPSS version 20.0 (IBM Corp., Armonk, NY, USA).

The DREEM inventory
DREEM has been translated into and validated in many languages and has been used for many evaluation purposes such as diagnosing deficiencies in a current educational environment, comparing different groups' experiences with a given educational environment, and comparing actual experiences of an educational environment with an ideal or expectations within a single group [7,8,9,10,11].
The DREEM inventory consists of 50 five-point Likert questions, with a scale ranging from 0 to 4: 4, strongly agree; 3, agree; 2, uncertain; 1, disagree; and 0, strongly disagree. Questions 4, 8,9,17,25,35,39,48, and 50 are negative statements with inverse scales. The 50-item DREEM has a maximum score of 200, which indicates the ideal educational environment, and a minimum score of 0, which would indicate a very worrying result. The interpretation of the overall score is: 0-50, very poor; 51-100, plenty of problems; 101-150, more positive than negative; and 151-200,excellent. The DREEM has also five subscales, each consisting of a set of question items and with its own interpretation guide: (a) students' perceptions of learning (SPL), with 12 items and a maximum score of 48; (b) students' perceptions of teachers, with 11 items and a maximum score of 44; (c) students' academic self-perceptions (SASP), with eight items and a maximum score of 32; (d) students' perceptions of atmosphere (SPA), with 12 items and a maximum score of 48; and (e) students' social self-perceptions (SSSP), with seven items and a maximum score of 28.
All 230 participants were asked to fill in the validated DREEM questionnaire online, with two extra questions (Q1: "Do you think that the course will prove to be beneficial to your clinical skills?" and Q2: "Would you suggest the course to another student?") following the completion of the course. The reliability of the inventory was tested using Cronbach's alpha. A descriptive analysis of the data (DREEM, Q1, and Q2) was performed. Moreover, the Mann-Whitney U test was perform- InEv c) 56 full participants Score = 4.12 ± 0.56 60 full participants Pre-score = 70.08 ± 16.23% & Post-score = 85.53 ± 13.40% Table 1. Topics of the lecture (Le) and scenario (Sc) sessions of the seminar Session Topic

Le1
Approach to the emergency room in a nutshell Le2 Burns Le3 Emergencies in cardiology Le4 Chest trauma Le5 Serious pelvic fractures Le6 Pre-ABCDE lecture Le7 Abdominal trauma and gastrointestinal emergencies Le8 Emergencies in vascular surgery Le9 Emergencies in neurosurgery Le10 Emergencies in neurology Le11 Hyperkalemia Le12 Metabolic acidosis Le13 Shock and sepsis Le14 Diabetic ketoacidosis and hyperosmolar nonketotic state Le15 Drug poisoning Sc1 Trauma and suturing Sc2 Peripheral and central venous catheterization Sc3 Skull and spine trauma Sc4 Diabetic ketoacidosis with hyperkalemia Sc5 Acute myocardial infarction Sc6 Pulmonary embolism Sc7 Abdominal Trauma Sc8 Differential diagnosis of abdominal pain Sc9 Allergic shock: anaphylaxis Sc10 Orthopedic

Internal evaluation (InEv) inventory
The InEv inventory was designed as an evaluation tool by the SSHMS and applied specifically to the scenarios of the ABCS3. It consisted of four subsets of 10 questions each and one general question, involving the self-assessment of the respondent's improvement in clinical skills (ICl). The four subsets referred to four different parameters of the 10 scenarios: adequacy of trainers (ATr), adequacy of facilities and equipment (AFa), adequacy of time (ATi), and general satisfaction (Sat) of each scenario. When applied to all 10 scenarios, this resulted in 40 questions (ATr1, ATr2,..., ATr10, AFa1, etc.) The answers were on a five-point Likert scale, from 1 (minimum) to 5 (maximum). Five more scores were obtained: the trainers' score (TrS, mean of ATr1 through ATr10, maximum score of 5), the facilities/equipment score (FaS, mean of AFa1 through AFa10, maximum score of 5); the time score (TiS, mean of ATi1 through ATi10, maximum score of 5); the general satisfaction score (SatS, mean of Sat1 through Sat10, maximum score of 5), and the overall score (OvS, = (TrS+FaS+TiS+SatS+ 4 × ICl)/8, maximum score of 5) ( Table 2). The TrS, TiS, FaS, and SatS represent the overall score for each of the above four parameters, while the OvS represents the overall score of the seminar, with a special emphasis on ICl, as the improvement of the participants' clinical skills was the primary goal of the seminar. The 60 full participants were asked to fill out the questionnaire immediately after the completion of the seminar.
The reliability of the InEv inventory was tested using Cronbach's alpha test. A descriptive analysis of the data and the results was performed. Furthermore, the independent-samples t-test, analysis of variance, the Mann-Whitney U test, and the Kruskal-Wallis test were used to determine whether gender, year of study, or school affected the ICl, TrS, FaS, TiS, SatS and OvS results. Finally, differences in the distributions of Sat1 to Sat10 were tested with the Kruskal-Wallis test, in order to identify which scenarios "generally satisfied" the participants to a significantly greater extent.

MCQs
Tests with MCQs have been broadly used as assessment tools. A norm-referenced test was designed by the scientific committee in order to assess the participants. The 60 students in the full category were asked to answer a set of 22 complex case-based questions in one hour and 20 minutes. For each question, one answer was correct and three were wrong. The same test was distributed twice, once prior to the seminar (pretest), and again after its completion (post-test). The descriptive analysis evaluated the characteristics of the examinees, as well as the results of the pre-and post-tests. The related-samples Wilcoxon signed rank test was used to detect improvement or deterioration in the performance between the pre-test and the post-test, and the Mann-Whitney U test and Kruskal-Wallis test were used to examine whether gender, year of study, or school significantly affected performance in the pre-and post-test.
In order to ensure the privacy and confidentiality of the delegates' personal details, we obtained signed, written consent from them prior to the seminar, which stated that the individuals agreed to fill out these three questionnaires anonymously, without revealing any private information.

DREEM
The response rate was 176/230 (76.5%), with 53 responses (30.1%) from the full group and 123 responses (69.9%) from participants in the OB category. Cronbach's alpha was ≥ 0.8 for the overall DREEM score and all subscales (except SSSP), indicating acceptable reliability of the inventory. The descriptive analysis presents the results for each question, as well as for the subscales and the overall score. The overall DREEM score was 144.61 ( ± 28.05), which may be interpreted as more positive than negative. All subscale scores were in the upper range of the second highest level in each category. The mean scores of the extra questions were 3.66 (± 0.91) for Q1 and 3.59 ( ± 0.63) for Q2 (Table 3). Gender did not affect the DREEM scores to a significant extent (overall and subscales, P > 0.05). School only influenced the SSSP score to a significant extent. However, the category of participation and year of study significantly affected almost all scores, with full participants scoring higher than OB participants on the overall DREEM score (P = 0.038), in the SPL subscale (P = 0.041), in the SASP subscale (P = 0.033), and on Q1, Q2 (P = 0.026 and P = 0.022, respectively). Among different years of study, fourth-year students scored significantly higher on the DREEM scale than sixth-year students, which was the only significant difference found in overall scores among different years of study, while students in their first, fourth, fifth, and sixth years of study were more willing to suggest the seminar to someone else (Q2). The year of study did not affect Q1 or SPL (Table 4). Finally, Q1 and Q2 responses correlated with the overall DREEM score (Kendall's tau test) with correlation coefficients of rτ = 0.401 (P < 0.001) and rτ = 0.287 (P < 0.001), respectively.

InEv
A total of 56 out of 60 participants (31 males and 25 females) in the full category answered the questionnaire. The reliability of the InEv inventory was tested with Cronbach's alpha test and found to be acceptable ( > 0.8 for the entire inventory and the TrS, FaS, and TiS subsets, but not for SatS[0.781]). With 5 as the maximum score, the results were: TrS = 4.24 ( ± 0.55), FaS = 4.21 ( ± 0.70), TiS = 3.94 ( ± 0.82), SatS = 4.08 ( ± 0.53), ICl = 4.16 ( ± 0.73) and OvS = 4.12 ( ± 0.56) ( Table 2). No statistically significant differences were found according to gender, years of study, and school (P > 0.05). The independentsamples Kruskal-Wallis test revealed statistically significant differences (P< 0.001) in general satisfaction among Sc1 through Sc10: Sc2 scored significantly lower than all the other scenarios, except Sc3 and Sc4 (not significantly lower). Sc10 had the highest score, and differed significantly from all the other scenarios, except Sc1, Sc5, and Sc9. Finally, Sc5 had the second highest score, scoring significantly higher than Sc2, Sc3, and Sc5 (Fig. 2).

MCQs
A total of 60 full participants took the MCQs. The pre-test score was 70.08 ( ± 16.23), while the post-test score was 85.53 ( ± 13.40). The independent-samples Wilcoxon signed rank test showed that significant improvement took place (P< 0.001). This improvement in performance was also confirmed among subsamples according to gender, year of study, and school (P < 0.05 for all tests comparing the pre-test and post-test results). Furthermore, gender and year of study did not affect performance on either the pre-test or post-test, while the students of Aristotle University of Thessaloniki scored significantly higher than those of other universities on the pre-test (P = 0.049) and post-test (P = 0.016) ( Table 5).

Discussion
The ABCS3 seminar included the use of mannequin-based simulators, partial task trainers, and standardized patients. Throughout the entire process, the participants had the opportunity to experience the pressure of the emergency room and to try to develop a diagnostic and therapeutic strategy under these challenging conditions. In this seminar, students improved their clinical knowledge and their ability to take action by diagnosing and intervening. As demonstrated by the pre-and post-MCQs tests, students performed better after the completion of the seminar. This improvement was present among different subsamples.
The participants were attracted to this form of learning and concluded that the course would be substantially beneficial to their clinical skills in the future. The DREEM and InEv ques-   Refers to the target (the 230 participants in the seminar). b) Refers to the sample that answered (176 students). A.U.Th., Aristotle University of Thessaloniki; SPL, students' perceptions of learning; SPT, students' perceptions of teachers; SASP, students' academic self-perceptions; SPA, students' perceptions of atmosphere; SSSP, students' social self-perceptions; OB, observer. tionnaires presented an analytic and comprehensive perspective regarding the strong and weak points of the seminar. The DREEM score was 144.61/200, which may be interpreted as more positive than negative. Students also ranked the first ABCS3 at the upper limit of the second category of each one of the five subscales. First, fourth, fifth, and sixth-year students would recommend the seminar to their colleagues. This may imply that, on the one hand, junior students are attracted by this clinical environment that differed from the strictly theoretical framework of the preclinical first year, while, in contrast, the senior students recognized the value of the seminar, as it offered a different educational experience with the novel aspect of simulation and the benefits thereof, which are not present in the school curriculum. The recognition of the simulation element becomes more obvious when the difference of the DREEM score between the full and OB categories is considered. If we consider the OB category as the control group (having participated in Le and watched the livestreamed Sc http://jeehp.org  The score (%) was calculated as (c/22) × 100%, with c referring to the number of correct answers. The maximum score was 100%, while the minimum was 0%. The related-samples Wilcoxon signed-rank test was used to compare pre-and post-test results. All tests were conducted after the appropriate assumptions were confirmed. a)