A small group learning for evidence-based medicine in pre-clinical medical students

Background Evidence-based medicine (EBM) is the application of current, best-available clinical evidence to health care decisions for individual patients. Many medical schools put EBM courses in their curriculum as they considering it is important. However, to teach the EBM course in pre-clinical year medical students is challenging owning to their clinical inexperience. Methods A prospective study of the third-year medical students of Walailak University located in the southern part of Thailand. They participated in a two-week course of evidence-based medicine. The eﬀectiveness of the course organization was assessed by percentage of students whose scores reached the minimal passing level and using pre-study and post-study self-reported evaluation. Results The percentage of students whose scores reached the minimum pass level (70%) was 100%. The scores are normally distributed with a mean of 88.59 (SD 3.33). Self-reported evaluation of knowledge and skills increased 4.28 scores (SD 2.06, p-value <0.001) and 4.24 scores (SD 2.08, p-value<0.001), respectively. Conclusion Using small group learning for evidence-based medicine in pre-clinical medical students achieved remarkable learning outcome regardless of clinical experiences. However, the role of the facilitator was of crucial importance as student learning depended on the facilitator's proper guidance and evaluation in the small group sessions. Our paper presented a novel view of learning evidence-based medicine in pre-clinical students who had no experience of real clinical cases. The ﬁnding was quite surprising and suggested that teaching evidence-based medicine by a small-group learning model was eﬀective in achieving required learning objectives. Self-reported evaluation of knowledge and skills increased 4.28 scores (SD 2.06, p-value <0.001) and 4.24 scores (SD 2.08, p-value<0.001), respectively. All students’ total scores passed a minimum pass level with a high mean total score of 88.59 (SD 3.33). The results implied that using small group learning for evidence-based medicine in pre-clinical medical students achieved good learning outcomes regardless of clinical experience. Moreover, despite the absence of clinical experience, most of them (95.7%) preferred studying using evidence-based medicine in their pre-clinical year and VAS in order to evaluate the happiness level revealed the strength of agreement of 8.51 (SD 1.27). The main limitation of results was the combination of two learning components which consisted of learning days and learning model experience. the analysis did not enable us to determine the eﬀectiveness in learning with a pure small-group learning model. Future studies should use an experimental design to compare a lecture-based learning model and small-group learning model in order to determine the best learning method in evidence-based medicine teaching in pre-clinical students.


Background
Phenwan T, Tawanwongsri W MedEdPublish https://doi.org/10. 15694/mep.2017.000098 Page | 2 Evidence-based medicine (EBM) is the application of current, best-available clinical evidence to health care decisions for individual patients. EBM consists of five major steps: asking a clinical question, systematic retrieval of the best-available evidence, critical appraisal of the evidence, application of results, and evaluation of performance (Sharma, Boeckmann, & Wong, 2016). This adds value to health systems in order to achieve gaining on all there aims at once: care, health, and cost. For instance, applying the knowledge gained from large clinical trials to patient care promotes consistency of treatment and optimal outcomes, helps establish national standards of patient care, and sets criteria to measure and reward performance-based medical practice. (Haughom;Lewis & Orland, 2004). Many medical schools put EBM course in their curriculum because of its importance and do so using small group learning [SGL]. SGL has been used in secondary and higher education for many decades to promote student engagement. It has been successfully adopted in medical education to enhance knowledge and skills (Biswas, Jain, Agrawal, & Bindra, 2015;Chou, Masters, Chang, Kruidering, & Hauer, 2013;Lewin & Lanken, 2004;Wilkinson & Rudland, 2004;Willett, Rosevear, & Kim, 2011). Two previous studies done in Thailand revealed that most students had a good attitude towards EBM. Teaching methods promoting evidence-based practice would be useful. Most students were fairly satisfied with their small-group learning experience (Boonluksiri, 2005(Boonluksiri, , 2006. Recently, one previous study revealed a brief small-group interactive workshop in EBM at the start of residency was effective in developing fundamental EBM skills (Al Achkar & Davies, 2016). However, teaching an EBM course with preclinical year medical students is somewhat challenging because of their lack of direct patient experience. There is no available study evaluating the effectiveness of using a small-group learning model with pre-clinical medical students to achieve the EBM skills.

Methods
This was a prospective study of third-year medical students at Walailak University which is located in the southern part of Thailand. The students participated a two-week course of evidence-based medicine in academic year 2016. Walailak ethical committee of the studied institute has approved the study protocol (WU-EC-MD-0-029-60). The study complied with the International Conference on Harmonization Good Clinical Practice and principles of the Declaration of Helsinki.

The course organization
The course description covered principles of literature review in basic medical science, and the application of principles of critical thinking in literature analysis. Before the start of class, there was a meeting of facilitators in order to agree the way students would be evaluated using the student evaluation form (as shown in Figure1).
A class was divided into six groups of eight students each. This two-week course for learning evidence-based medicine was divided into two parts. One part which took two days included mini-lectures about clinical question setting, using a computer for searching related articles, critical appraisal, using a software tool for managing bibliographies, and literature review principles. The other was designed as small group learning under the guidance of facilitators. The course director allocated three facilitators to each group. The small-group activities included clinical question setting, searching related articles, assessing the quality of chosen articles, and oral presentation about the evidence-based answer. A final examination was given to the students.
Learning evaluation was divided into two part. One was process evaluation with weight of 40% and the other was output evaluation with weight of 60%. These activities consisted of clinical question setting, evidence-based answering, group oral presentation and group process evaluation. Thirty percent of the total score was evaluated by the facilitator individually. Fifteen percent of the total scores came from final examination scores. And the remaining 5 percent of the total scores was evaluated by peer evaluation. The minimum passing level (70 % of a Phenwan T, Tawanwongsri W MedEdPublish https://doi.org/10.15694/mep.2017.000098 Page | 3 total score) was fixed before the course started.
To achieve the learning outcomes, facilitators played an important role in this small-group learning model. A facilitator is a guide rather than a teacher, one who creates a safe learning environment in which every member of the group is encouraged to participate. The effective facilitator intervenes appropriately to promote problem solving and metacognition, convey vital information, and potentially offer real-life experiences, but does not lecture, interrupt with their own agenda, or dominate the group (Rao, 2017;Susan Hawkins, 2016).

Study design
The effectiveness of the course organization was assessed by using pre-study and post-study self-reported evaluation, and the percentage of students whose scores reached the minimal passing level. The students filled out pre-study self-reported knowledge and EBM skills at the beginning of the course. The group process was evaluated by their facilitator using the rubrics which categorized student's learning characteristics in given scores. After searching and choosing an original article, each student was required to discuss his or her critical appraisal result with their facilitators. And at the end of the course there was post-study self-reported knowledge, and EBM skill evaluation and there was also a final examination. Self-reported scores were evaluated by using a visual analog scale (VAS) in which a respondent selects a whole number (0-10 integers) that reflects the strength of agreement. We felt confident in using this evaluation method partly because self-assessment evaluation correlated with examiners' subjective assessment as well as the comprehensive final examination (Inayah et al., 2017;Stephens et al., 2012).

Statistical analysis
Mean and standard deviation (SD) or median and range were used to describe continuous data. Frequency and percentage were used for categorical data. A self-reported efficacy score was evaluated on visual analog scale (VAS) in which a respondent selects a whole number (0-10 integers) that reflects the strength of agreement. We used Student's t-test to compare before and after scores. We used regression analysis to find the correlation between the self-reported post-study scores and final examination scores. A p value of < 0.05 by two-tailed tests was considered statistically significance. The statistical analysis was performed by SPSS software version 17 (SPSS Inc., Chicago, IL, USA).

Results
A total of 46 third-year medical student out of registered 47 (97.9%) completed their pre-post questionnaires. Students' characteristics, self-reported evaluation scores and their opinions of course organization were shown in Table1. The students' mean age was 21.2 ± 0.5 years. There were 30 (96.8%) female students. Thirteen (28.3%) students reported that they had never read a scientific paper. Most of them preferred group study (45, 97.8%) than studying alone (1. 2.2%). And they strongly needed facilitators in learning processes including clinical question setting and question answering. Their perceived importance of evidence-based medicine for every physician was 9.04 of 10.00 agreement score (SD 1.05). Moreover, studying with happiness VAS mean was 8.51 (SD 1.27).
The short-answer final examination was analyzed for difficulty index, power of discrimination, and delta. The test had a mean difficulty index of 0.58 (95%CI 0.51-0.65) with mean delta of 12.23 and power of discrimination of 0.34 (95%CI 0.22-0.47) was considered as good difficulty and discrimination indices respectively. Page | 5 * Self-reported scores was evaluated by visual analog scale (VAS) in which a respondent selects a whole number (0-10 integers) that reflects the strength of agreement.
The effectiveness of the course organization can be seen form Figure. 2, a percentage of students whose scores reached the minimal passing level (70% of a total score) was 100%. The scores are normally distributed with a mean of 88.59 and a standard deviation of 3.33 (Shapiro-Wilk test 0.10).

Figure 2
A bar graph shows students' total score given as the percentage (ranged between 81.59 -94.12 points) with minimal passing level of 70% (vertical dotted line). Owning to self-reported evaluation (shown in Table 2), knowledge in the literature review principles increased 4.28 scores significantly (SD 2.06, p-value <0.001). Additionally, skills in critical appraisal and result interpretation of original articles increased 4.24 scores significantly (SD 2.08, p-value<0.001). However, there was no evident relationship between post-study self-reported evaluation in knowledge and final examination score. In the same way, we found no correlation between post-study self-reported evaluation in EBM skills and EBM skill competency scores evaluated by facilitators. Table 2 The self-reported evaluation of knowledge in the literature review principles and skills in critical appraisal and result interpretation of original articles.

Phenwan T, Tawanwongsri W
*Self-reported evaluation by visual analog scale (VAS) in which a respondent selects a whole number (0-10 integers) that reflects the strength of agreement. Mean scores in each category of evaluation were compared across the six groups. There is no statistically significant difference in grade point averages (GPAX) between groups. Table 3 shows there is a statistically significant difference in both percentage of group work and process evaluation among groups (p-value<0.001). But there is no statistically significant difference in the percentage of individual work and final examination scores among groups. In addition, we found a moderate correlation between GPAX and the percentage of individual work and final examination with correlation coefficient of 0.47 (p-value 0.001). Table 3 The scores by group were given by output evaluation and process evaluation.
To investigate whether scores evaluated by facilitators were consistent with other evaluation categories, correlation tests were done as shown in Table 4. It has been found that there was only correlation between scores evaluated by facilitators and group work scores (p-value 0.03) and between scores evaluated by facilitators and individual work and final examination (p-value 0.03). Table 4 Correlation coefficient (p-value) measured the linear correlation between a percentage of scores evaluated by facilitators and others evaluation types.
The correlation between scores from process evaluation and scores from product evaluation t was tested. There is a significant linear relationship between scores from process evaluation part and scores from product evaluation part with the correlation coefficient of 0.37 (p-value=0.01) as shown in Figure. 3.

Figure 3
The correlation between scores from process evaluation part and scores from product evaluation part. A logistic regression with block-wise selection was performed. It revealed that factors was able to predict the total score with 99.5% including all evaluations except question setting evaluation and peer evaluation.
At the end of the EBM course, the final grade for course was calculated. The mean of total scores was of 88.59 with SD 3.33. All students' scores were above the minimum passing level. Additionally, twenty-three students got grade A, nineteen students got grade B+, and remaining five students got grade B.

Discussion
The objective of this study is to evaluate the effectiveness of small-group learning for evidence-based medicine in pre-clinical medical students who had never experienced real clinical cases. Key factors for achieving learning objectives are clinical setting, critical appraisal, and application to their clinical vignettes. The effectiveness of a small-group learning for evidence-based medicine in pre-clinical medical students has not been well studied.
Small group learning is one of the cornerstones of problem-based learning. By implication then, the role of the /facilitator is of pivotal importance, as student learning would depend on the facilitator's understanding and appreciation of his/her responsibilities in small group sessions (Dolmans, Wolfhagen, Schmidt, & van der Vleuten, 1994). It has been successfully adopted in medical education to enhance knowledge and skills (Biswas, et al., 2015;Chou, et al., 2013;Lewin & Lanken, 2004;Wilkinson & Rudland, 2004;Willett, et al., 2011) . Phenwan T, Tawanwongsri W MedEdPublish https://doi.org/10.15694/mep.2017.000098 Page | 9 Our paper presented a novel view of learning evidence-based medicine in pre-clinical students who had no experience of real clinical cases. The finding was quite surprising and suggested that teaching evidence-based medicine by a small-group learning model was effective in achieving required learning objectives. Self-reported evaluation of knowledge and skills increased 4.28 scores (SD 2.06, p-value <0.001) and 4.24 scores (SD 2.08, p-value<0.001), respectively. All students' total scores passed a minimum pass level with a high mean total score of 88.59 (SD 3.33). The results implied that using small group learning for evidence-based medicine in pre-clinical medical students achieved good learning outcomes regardless of clinical experience. Moreover, despite the absence of clinical experience, most of them (95.7%) preferred studying using evidence-based medicine in their pre-clinical year and VAS in order to evaluate the happiness level revealed the strength of agreement of 8.51 (SD 1.27).
The main limitation of the results was the combination of two learning components which consisted of lecture-based learning in the first two days and a small-group learning model for the majority of the learning experience. Thus, the analysis did not enable us to determine the effectiveness in learning with a pure small-group learning model. Future studies should use an experimental design to compare a lecture-based learning model and a small-group learning model in order to determine the best learning method in evidence-based medicine teaching in pre-clinical students.

Conclusion
This prospective study of the third-year medical students who were in their pre-clinical year. A worrisome problem was the lack of students' clinical experiences. The effectiveness of the course was assessed by using pre-study and post-study self-reported evaluation, and percentage of students whose scores reached the minimal passing level. Selfreported evaluation of knowledge in the literature review principles increased significantly, 4.28 scores (SD 2.06, pvalue <0.001) and skills in critical appraisal and result interpretation of original articles increased significantly, 4.24 scores (SD 2.08, p-value<0.001). All students' total scores were higher than the minimum pass level of 70%. The results implied that using small group learning for evidence-based medicine in pre-clinical medical students achieved good learning outcome regardless of clinical experiences. However, the role of the facilitator was of crucial importance as student learning would depend on the facilitator's proper guidance and evaluation in the small group sessions.

Notes On Contributors
Weeratian Tawanwongsri is a medical lecturer at school of medicine, Walailak University located in Nakhon Si Thammarat, Thailand. He is an internal medicine physician. He is also one of the department members who organizes his school's medical curriculum and re-evaluates learning outcome in terms of the quality control. He is now working on an infectious disease research and course evaluation. Tharin Phenwan is a medical lecturer at school of medicine, Walailak University located in Nakhon Si Thammarat, Thailand. He is a family physician. He is interested in Palliative care and medical education. He is also an internal speaker in fields of Palliative care. Moreover, he is now working on the curriculum organization.