Introducing a course on Evidence Based Medicine for undergraduate medical students – experience from an Indian medical school

Introduction: Although evidence based medicine (EBM) is part of undergraduate training in several countries, Indian medical schools are yet to make a meaningful start. We report our experience with EBM training for undergraduate. Methods: Third year students undergoing clinical clerkship at a teaching hospital underwent an 8 week course on EBM with modules on concepts, formulating questions, searching for evidence and critical appraisal. Feedback was taken and assessment done on knowledge and searching skills. Results: 90% of students had poor awareness of EBM. 92% considered evidence the most important component of EBM; 5% considered clinical expertise and patient values as part of EBM. At baseline majority cited Google the most reliable search tool. Only 27% felt EBM relevant to their curriculum. 66% rated the course good, very good or excellent. Hands-on training enhanced learning lack of background knowledge of diseases hindered it. Conclusion: Awareness of EBM is low among undergraduates and most ﬁnd it irrelevant to their curriculum. Our experience suggests that it is feasible to integrate EBM training into undergraduate curriculum. More studies reporting knowledge and skills before and after EBM courses would strengthen the cause for making it part of undergraduate medical curriculum in India.


Introduction
The paradigm shift in the curriculum of medical education worldwide has also started to make inroads into the Indian medical education system with the Medical Council of India bringing out its draft Regulations on Graduate medical education (2012). One major highlight of this report is the stress on competency based learning to achieve the goal of producing an 'Indian medical graduate' who can function effectively as the physician of first contact. Recognizing the increasing importance of evidence based medicine (EBM) in clinical practice, this report lists lifelong learning as a core competency and the ability to search and critically evaluate medical literature to apply it in the care of patients as a specific objective. It is therefore quite evident that EBM is no longer a domain restricted to practising clinicians or postgraduates. Several universities have either started integrating EBM into their curriculum or have experimented doing so (Wanvarie S, et al 2006; Ghali WA, et al 2000). We report the experience and findings of our attempt to introduce EBM among undergraduate medical students at a tertiary care teaching institution.

Methods
We developed a draft course using inputs from experts who have undergone formal training in evidence based medicine and clinical epidemiology, faculty who have undergone short term certified workshops in EBM, literature review of similar courses or workshops experimented at different universities and from authentic books on how to impart training in evidence based medicine. The tentative course was modified based on further discussions within the medical education unit of the institute. The entire course was designed for a period of eight weeks from March 2016 to May 2016, and divided into four modules. The target participants were students in their third year of undergraduate training and undergoing clinical clerkship in six disciplines (internal medicine, general surgery, anesthesiology, critical care, casualty and orthopedics) by rotation over an eight week period. Feedback from the participants was obtained at the end of each stage, based on which further modifications were made throughout the course, thus making it dynamic and responsive. The anonymised feedback included questions on perceived baseline knowledge, skills and attitude to EBM before and after these sessions as well as active comments on ways to improve and make the course more participant friendly. Finally a comprehensive feedback was obtained on completion of the course.
We framed specific learning objectives for the entire course such that at the end of the course the student would be able to 1.
List the principles and components of EBM  10. List the various sources for searching for the current best evidence 11. Demonstrate sufficient skill in using the web to search for evidence to a given foreground question 12. Demonstrate skill in identifying the types of evidence to be searched (therapy, diagnosis, prognosis, harm) to appropriately answer a given patient problem

Teaching learning methods
The course was completed over a period of eight weeks in the form of four modules. The modules were designed to cover various concepts and components in the understanding and practice of EBM while retaining reinforcement and continuity between them.

Module 1: Introduction to EBM
This module was spread over the initial two weeks in the form of a.
Interactive This part of the course, spread over the next two weeks was in the form of group discussions where students discussed questions that emerged from their interactions and experiences with patients they had been seeing during their respective clerkship postings. Each group presented one or more clinical problems they encountered during their posting, identified appropriate PICO components and then attempted framing answerable clinical questions. The strengths and weaknesses of the entire process was discussed among the different groups leading to refinement of the foreground questions.

Module 3: Searching for current best evidence.
This module was organized in the form of Assignments to search for evidence on specific problems given by the facilitator on topics of clinical relevance.

Module 4: Appraising evidence
This module too was spread over two weeks, and comprised of a.
Interactive lectures on basic study designs b. Group discussions on randomized controlled trials and systematic reviews c. Problem based learning sessions on Likelihood ratios and Numbers needed to treat d. Hands on sessions on using online tools for appraising evidence for internal validity and generalizability.
Assessment was done at the end of eight weeks using a two component exercise. The first component assessed the cognitive domain. We employed a dichotomous test (score of either 5 points or no points, based on complete knowledge or incomplete knowledge respectively) to assess knowledge related to the hierarchy of evidence. This was followed by assessment of application of EBM using a real time computer based search for current best evidence on a hypothetical clinical problem given to the students at the assessment area. The students were instructed to frame one appropriate background question, find the answer to this using summaries of evidence (UpToDate); frame a foreground question based on relevant PICO, and search for the best evidence using PubMed. The weightage of marks was based on the relevance of the PICO, the search strategy as detailed by the student and the quality of evidence obtained after search. A student who provided a relevant systematic review or meta-analysis was scored higher than one who provided a randomized controlled trial and other levels of evidence were not accepted as the clinical scenarios given to them dealt only with therapy. The assessors had predefined the weightage and acceptable background and foreground questions. They also searched PubMed at the time of setting the scenarios as well as at the time of scoring the responses for the current evidence on the scenarios. Whenever there was conflict between the final evidence presented and the search strategy, the latter was considered for genuineness of the process and award of scores. Final scores were obtained by adding the individual scores of the two components (25% for cognitive part and 75% for application). Marks were entered in Excel and analyzed. An arbitrary cut off of 50% was chosen to determine success.

Ethics:
The Institute Ethics Committee (IEC) of the Pondicherry Institute of Medical Sciences waived off need for review and consent for the study in view of educational research.

Results
Ninety six students underwent the course over the entire 8 week period. 86 out of the 96 (90%) participants rated their baseline awareness of evidence based medicine low before the course and admitted a self-perceived improvement at the end of the course. The feedback on the first module which predominantly concerned The second module on how to frame answerable clinical questions (foreground questions) was predominantly based on real life scenarios picked up by students during their clerkship. All students acknowledged that they were unaware of the concept of background and foreground questions before this course. By the end of this module, we noticed that almost all of them were able to identify relevant PICO components with respect to their clinical problems and frame questions based on these components. 81 out of the 96 (85%) students mentioned that identifying PICO enabled them to formulate the foreground question quickly and be more specific. 18% of them felt that although PICO was important, the framing of a good foreground question might be, to a large extent influenced by the knowledge and fluency in English language. The third module on search strategies was initiated by an interactive lecture on the various methods available, the need to search effectively so that clinicians find the right articles within a short time frame to make decisions and finally how to use PubMed search to find evidence. This was followed by daily hands on sessions to find evidence for foreground questions based on real life case scenarios identified by the students. This included both small group sessions as well as individual tasks using hand held devices connected to wireless internet facility. Multiple facilitators were present during these sessions to provide help to the groups and individuals. Although PubMed search was the predominant mode, the participants were also introduced to use of summaries of evidence like UpToDate and BMJ Best Practice (both available for free to students and faculty through institutional membership). Feedback on search strategy module showed that students recognized the importance of PICO better when these were used to search for evidence. 75 out of 96 (79%) participants felt that the sessions dealing with PubMed search were useful. 40% rated the sessions on search strategy in general as very easy to understand while 12% felt it was very difficult to understand. 50% of students agreed that the discussions at EBM sessions were relevant to clinical practice and that the search module significantly enhanced their ability to find answers to clinical problems; however only 27% (26 of the 96) felt that it was relevant to their current undergraduate curriculum. They rated UpToDate search much easier and less time consuming as compared to PubMed search (69%). 72% of the participants believed that the introductory sessions on the components of EBM, hierarchy of evidence and components of foreground questions enhanced their understanding and application of search strategy significantly. Finally we asked participants to rate the entire EBM course; 7% rated the course excellent, 21% rated it very good and 38% rated the course good. The course was rated poor by 5% of participants. Among factors that possibly enhanced learning, majority of students believed that inputs from multiple faculty during sessions and discussion of problems based on real cases were of great help (Table no.1). Among changes perceived by students after the course as compared to before the course, 70% identified a positive change in attitude to patient care and their perception of EBM as a whole ( Table 2). Lack of adequate background knowledge of diseases was cited as a factor hindering the effective learning, by a significant proportion of students ( Table 3). 73% of participants recommended continuation of the course in a modified form for subsequent batches of students, while 15% felt that there was no need to modify the existing format and design of the course.Assessment was conducted at the end of the course as described in the methods. 90 of the 96 students participated in the assessment. Alternate students were provided different case scenarios and asked to frame one background question and one foreground question relevant to their scenario. An example of a problem used in assessment is depicted in figure 1. Subsequently they were expected to identify the answer to the background question using summaries of evidence (UpToDate in this case) and one relevant article to answer their foreground question (using PubMed) with detailing of the search strategy they used. Award of credits was based on pre-specified points including whether background questions matched any of those predetermined by assessors; if any other background question was provided by a student, marks were awarded based on consensus among assessors. The median score of the participants was 13 (65%) on a maximum score of 20 (range: 2.5 to 20). 5% of participants obtained full scores while 70% of them passed the assessment satisfactorily crossing a pre-specified arbitrary cut off of 50% of total score.   Figure 1A. Example of the clinical structured exercise used for EBM assessment Figure 1B. Example of the acceptable search responses used for assessment of EBM (for the sample clinical exercise used in 1A).

Discussion
Evidence based medicine, merely a concept several decades ago has steadily become a way of life in many western medical schools. Since its popularisation in the 1980s by Dr. David Sackett and colleagues the acceptance and application of EBM has widened exponentially. However in many of the South East Asian countries including India EBM still continues to be a novel entity. There may be several reasons for this. In a cross sectional survey among dental and medical students in the Middle East, lack of time and difficulty in understanding the subject were reported as the greatest barriers to practice of evidence based medicine (Bahammam MA & Linjawi AI 2014). The same study found that the knowledge and attitudes related to EBM was poor among medical students. Other hurdles to practice of EBM by clinicians include restricted access to search tools, absence of evidence based practice culture in the work place leading to poor motivation and lack of skills needed to find and appraise evidence (Van Dijk, et al N 2010). We have introduced a course on evidence based medicine for the undergraduate medical students, for the first time in this part of India, to the best of our knowledge. The course targeted third year medical students undergoing clinical clerkship in six disciplines on rotation. Our experience suggests that it is feasible to introduce . Nevertheless the study found that students who attended the workshop were more confident in transforming a clinical problem into an answerable question. In contrast to most of these EBM teaching modules, we designed a more comprehensive and extended course that spanned eight weeks. Wide variations also exist in the settings where EBM training interventions have been conducted. A recent review of twenty studies on EBM training found that training was imparted in most cases in classrooms followed by clinics (Maggio LA, et al 2013). Few interventions took place in either of these settings along with online facilities. This course however utilized all these settings for EBM training by introducing theoretical concepts in the classroom, identifying real life patient problems in clinics and converting them into answerable questions, discussing them again in the classroom to formulate search strategy and applying online resources hands on to find the evidence.In a cross sectional study from India which focused predominantly on the patterns of internet usage by medical students, it was found that postgraduate students had much higher knowledge regarding the principles of evidence based medicine than undergraduate medical students (Banerjee I, et al 2011). This can easily be explained by the near-total absence of evidence based medicine in the undergraduate medical curriculum in India. The medical students in our program also admitted that they had very poor or no knowledge regarding evidence based medicine; self-perceived improvement in the knowledge and skills required for practice of evidence based medicine was also noted.Literature describing interventions on EBM for students have focused mostly on clinical students (60%). Only 10% of the studies targeted students in both clinical and pre-clinical students (Maggio LA, et al 2013). We chose students in their third year of undergraduate training during their clinical clerkship. This ensured that the target group had been exposed to relatively sufficient background knowledge of preclinical and clinical disciplines. In turn, this enabled them to understand the differences between background and foreground questions and the clinical implications of research. Again in line with the findings from a review of interventions on EBM, instructors in our course were predominantly physicians (Maggio LA, et al 2013). Assessment of the students at the end of course was using a modified objectively structured examination where a clinical scenario was provided and different steps of EBM practice were assessed (except critical appraisal). Similar methods have been used by other investigators in the past (Tudiver F, et al 2009). The other method using EBM case report was not used as we felt it was better suited for formative assessment (Ivan J Perry 1998).We had several limitations. We did not objectively assess the baseline knowledge, skills and attitude of students in EBM; hence we cannot draw solid and objective conclusions on the effectiveness of the course. Neither did we randomize students to receive or not receive the course to detect any differences. Another limitation related to the relatively insufficient training provided in critical appraisal skills compared to other steps in practice of EBM. We felt that understanding critical appraisal skills would require that students be first convinced of the relevance of EBM both within and outside of their curriculum. Consequently we did not include critical appraisal in the final assessment. Further we also did not assess other components of EBM like clinical expertise and patient values and preferences.

Take Home Messages
This is the first attempt to integrate EBM as a course into undergraduate curriculum in India to the best of our knowledge. Our experience suggest that it is feasible and the feedback indicates that it is likely to be accepted by students and improve their knowledge of and attitudes towards EBM. However future programs should attempt to convey the relevance of EBM in the context of undergraduate training with possible longitudinal integration into the curriculum. Concurrently institutions must also find ways and means to improve access to current best evidence. Major changes in perception of faculty regarding EBM would have to take place before attitudinal transformation begins among students. More studies describing baseline knowledge and assessing change after EBM course would strengthen the cause for introduction of similar courses into undergraduate training by medical schools in India.

Notes On Contributors
AB designed and co-ordinated the course; drafted the manuscript. NI revised the course contents and designed the assessment; TA contributed to the intellectual content, design and co-ordination of the course, revised the manuscript; SP analysed data and revised the technical content; VYV and SG revised the manuscript, designed feedback and modified the course based on feedback analysis.