Objective Structured Clinical Examination (OSCE) Seven Years Experience From Department Of Surgery, Faculty Of Medicine,

: Aims and objectives: In this article we aim to sharing and exchanging experiences at all levels of applying objective structured clinical examination (OSCE), to reduce trial and errors cost inefficiencies and to help and accelerate the acceptance the new comers to this assessment tool. Setting and Methods: The department of surgery, faculty of medicine, Umm-Alqura University Makkah adopting OSCE as an essential tool of assessment in fourth, fifth and sixth years, from the first batch of the students (1420H/1999/2000). It was first held in may 2000 (batch I- 21 students) and the numbers gradually increasing, till it reached 146 students in 2007 (batch 7). During these seven years experience of running the examinations we have run till now more than twenty five consecutive OSCE examinations, and sometime quadruplicated, which have been continuously evaluated and reviewed with the help of our external examiners. We are always refining the process and content with some development of new and innovative implementation, the department has improved the process and contents of OSCE. Lessons learned and conclusions: A well-organized OSCE requires significant investment in terms of planning, resources and enthusiasm, but can have potentially beneficial impacts on the assessments tools especially in big number of candidates. However OSCE is a very good adjunct tool of assessment, and can be used in multicentric ways for big number of candidates, but also has its limitations and should be combined with other forms of assessment, which may be more valid and comprehensive to test competencies not easily tested within the OSCE format.

bjective structured clinical examinations (OSCE) are increasingly used and becoming the favored method of assessing the range of clinical skills at both undergraduate and postgraduate level. OSCE was first described in the mid seventies 1, 2 and has been increasingly recommended by medical educationists ever since, as an objective method of assessing clinical competencies for under and post graduate. Since 2000 (1420H), the Department of Surgery, Umm Alqura University-KSA, launched the OSCE as a tool of examination for clinical clerkships in surgery. The OSCE was based on real and simulated patient; data show projections and data interpretation. It emphasized content core areas in curriculum and areas difficult to be assessed in classic clinical examination, such as surgical procedures, attitude and communication skill sets targeted at an examination level appropriate for future intern. Students were assessed by means of content checklists for knowledge and global rating scales for process variables. Equal weight was given to both content and process in student evaluation. Our experience in OSCE proved successful and hoping to be of help for those O already using the OSCE as a tool of assessment, and may provide new ideas or be the impetus for an exchange of ideas. Also the development and implementation of innovative methods of OSCE stations (conjoined stations, model and simulated patient, mini OSCE with MCQs and video clips), proved to be very useful and helpful in saving time and broaden the aspects as well as improving the reliability across our seven years experience. For those planning to introduce OSCE, this article may provide basic guidance in development, structure, and the personnel required. For those already using the OSCE assessment, our article may provide new ideas or invitation for an exchange of ideas. For those who are facing such formal assessment as candidates, we hope this article shows the efforts that are made to achieve the necessary structure and objectivity in this type of examination. Although the OSCE is very demanding tool of assessment and needs time and efforts for preparation, it deserves these efforts and proved successful, and very helpful especially in big number of students. Our department experience has produced a body of evidence regarding the feasibility, reliability, and validity of the OSCE for evaluating clinical clerkships in surgery, and this has been reported elsewhere in the medical education literature [3][4][5] .

WHAT IS OSCE?
OSCE is an acronym for Objective Structured Clinical Examination, an assessment method that is based on objective testing and direct observation of student performance during planned clinical encounters (also called interactions or test stations 2 . Originally described by Harden (1975) 6 , the OSCE includes several "stations" in which examinees are expected to perform specific clinical tasks within a specified time period (as brief as 5 minutes to 10 minutes). To complete the examination, students rotate through a series of stations (as few as 2 or as many as 20) 7,8 , in which the components of clinical competence such as history taking, physical examination, simple procedures, interpretation of laboratory results, patient management problems, communication, attitude etc. are tested using observers with agreed check lists.

Why we adopt OSCE?
The traditional clinical examination has been shown to have serious limitations in terms of its validity and reliability 3,4 , in addition to the time consuming. The OSCE was introduced into some medical schools in the 1970s and a lot of information is now available regarding its reliability, validity and effectiveness as an assessment tool 8 . It was created to enable better assessment and quantification of clinical skills acquisition by medical students. The OSCE as a valid and reliable method and process of evaluation provides some answers to these limitations in traditional examination and has become very popular 9 . Studies also have demonstrated that the OSCE is an effective tool for evaluating areas most critical to performance of health care professionals. The reliability and objectivity of the OSCE -which had become well documented-led to its becoming the standard for performance-based assessment in many medical specialties 10 . It is clear that OSCE has numerous strengths which led to the department of surgery at Um-Alqura School of Medicine adopting it as a tool of assessment. It allows multiple samples of insight into a candidate's abilities, limits the effect of chance and examiner bias and stations can be designed and structured with a great deal of flexibility to fulfill the objectives of the curriculum.

SETTING AND PROCESS:
The department of surgery Umm-Alqura school of Medicine has adopted the OSCE format as tool of assessment since May 2000. We now have seven years experience of running the OSCE examination and we are continuously refining the process and content. This format of clinical examination assesses different areas of clinical knowledge, skills and attitudes. Each clinical competence that is assessed is broken down into its components and tested separately at different stations. The candidate rotates through these stations in turn, spending a specified time at each station, (usually 5 minutes). The sounding of a signal indicates the end of the time period at a given station. The number of stations is usually 20 and the time allocated is the same for every station. At each station is displayed material on which the clinical competence is to be tested. The variety of material may vary from live stations at which real patients, simulated patients, artificial models or manikins are utilized with an examiner being present, or to stations where clinical photographs, investigation reports, radiographs, post mortem specimens etc are displayed and recently we introduced data show projections and video clips have been introduced. At the stations where charts, reports or other diagnostic equipment are kept no examiner is present. The candidate answers the questions asked or performs a task which is scored by the examiner present at the station, according to a predetermined checklist of activities the candidate is expected to perform. These tasks may vary from the assessment of procedural skills such as inserting chest tube, catheterization to the communication skills and counseling (consent / breaking bad news). The candidate takes around a standard answer sheet on which his/her responses are recorded according to the numbering system displayed at each of the stations. Rest stations placed between some examination stations gives the candidate extra time to think or write. Rest stations vary in number depending on the examination. The OSCE is held in one setting, usually clinical skill laboratory, a side-room or in more than one setting especially when the number of candidates to be accommodated is large. In our department we passed through the experience of setting of the examination to be duplicated, triplicate and quadruplicated in different places.

WHY OF OSCE:
The OSCE is a more objective examination since it bypasses many of the inconsistencies that were mentioned under the conduct of the conventional clinical examination 11 . The competencies that are to be tested can be planned out and organized before hand with a checklist to be adhered to. This will make the assessment of the candidates uniform, structured and hence more valid. The consistency in the patients or material tested on, makes the examination results more reliable and valid, especially if it is of a competitive nature e.g. for ranking purposes. A wide range of competencies can be assessed. History taking, ability to impart knowledge, communication skills etc., which are essential characteristics of a clinician can be assessed in a planned manner at an OSCE. As an evaluation tool it eliminates the luck of the draw, reduces variation in marking standards from examiner to examiner, and can accurately reflect the real-life tasks of the doctor 11,12 . The more important point of introducing the OSCE as an assessment tool, that the content of the examination covers a much greater area of the subject or curriculum being assessed since about 20 different stations are included. Assessment of procedural skills is almost negligible in the conventional examination, while in an OSCE a wide sample of skills can be tested and scored objectively with a checklist. The efficiency of an OSCE to test a large number of candidates at a specified time is also greater than the `long' and `short' cases, which takes many days or a week to complete for a batch of say 150 students.

PROBLEMS ENCOUNTERED DURING IMPLEMENTATION:
The current set up of this assessment has some problems, one of those, is that OSCE tests skills, attitudes and knowledge in separate compartments and it does not evaluate the comprehensive understanding of the candidate. Although clinical decision making can be incorporated in an OSCE, the ability to look at a patient as a whole is not assessed. In this respect the conventional clinical examination is superior. A series of stations on a given clinical situation flowing into different aspects of the patients problems may partially reduce this compartmentalization but an integrated approach of decision making can not be adequately assessed with an OSCE 12 .
The conduct of an OSCE can be much more exhausting for the examiners who have to repeatedly and continuously, throughout the examination, watch the candidates perform a skill or procedure. Similarly this may be even more demanding to the real patient or simulated, who has to subject themselves to repeated examinations by an entire round of candidates 12,13 . For this reason simulated patients are perhaps better than real patients. Examiner effort is very much more in the days and weeks preceding the examination as well as on the day of the examination, due to the demands of preparing, displaying and staffing the OSCE's. This extra work is not only for the coordinator of the examination but the entire panel of examiners who shares the workload.
Prior to the examination an OSCE involves more preparation and time, than the conventional `long' and `short' cases. The number of invigilators needed is also greater than at the conventional examination.

THE DEPARTMENT EXPERIENCES IN ORGANIZATION AND RUNNING THE OSCE: PLANNING AND PREPARATION:
Advance planning is essential. The combined effort of all staff and appointed examiners is mandatory for the OSCE to be successful and the day of the examination to run smoothly. Planning should begin at least three to four weeks ahead, if not earlier. At the outset the examiners should meet and decide on the scope of the examination. The content of the OSCE should be determined ensuring that the entire ranges of competencies to be assessed are represented. For example history taking, observational skills, interpretation of laboratory reports, radiological, nutritional advice, attitude to patients etc. A marking scheme should also be determined. The content and allocation of marks should be outlined. An example is as follows: history taking stations 20%, physical examination stations 30%, laboratory investigation stations 15%, interpretation of charts 10%, communication skills 15%, and nursing procedure stations 10%. This decision would obviously depend on the objectives of the examination as a whole. This should precede the specific details of each examination station being prepared. Once this broad outline is worked out the stations should be distributed among the examiners or "OSCE station designers"; and a record kept of this distribution. It is very important that each station designer takes full responsibility for the stations he/she prepares. This includes providing the test material-patients, surgical instruments, pathological specimens, charts, reports etc. These should be, in adequate quantities as necessitated by the number of venues the OSCE is to run at. The questions to be asked should be clearly labeled and correctly numbered. Checklists of activities and the marking schemes should be prepared by the respective station designers. The department experience in setting multiple choice question type of stations is attempted in the data show projection in the recent two years and it is found to be very helpful in saving time and efforts especially in marking large number of the students. A coordinator is essential to maintain an overall plan of the stations. The layout plan should include location of rest stations, since it is important that the rests are given in the identical stations if the OSCE is conducted in more than one venue. Briefing examiners, time keepers and other support staff is also an important function of the coordinator, who should also prepare an answer sheet in accordance with the format of answers expected. The examiner could enter the marks awarded at live stations in to the relevant cage in the answer sheet carried around by the candidate instead of maintaining additional mark sheets. For the preparation of the skeleton answer sheet all examiners should hand in their fully prepared questions well in time to enable structuring and typing of the answer script. If any examiner does not attend to the station/s allocated in his/her entirety, and on time, the coordinator will not be able to finalize arrangements; and the examination day is doomed for disaster. Instructions to the candidates should also be prepared and the necessary clinical equipment that should be brought to the examination hall made known. The examination can not begin until all invigilators and examiners arrive and are in place.

PITFALLS / SOLUTIONS TO BE CONSIDERED:
In the department experiences some pitfalls have been observed; sometimes examiners not showing up, in such cases the facilitators may have to be ready and step in. Rest stations positioned so that nearby stations can be overheard, noisy stations being next to quiet ones. Candidates getting lost around the circuit require clear guidance. Others like candidates removing instructions from a station, examiners hoarding and mislabeled mark sheets all these problems should be anticipated and catered for.

POST OSCE:
In our set-up, marking of the OSCE is carried out immediately after the examination. If marking is at a later date, care should be taken when dismantling the displayed material to ensure that the numbering and labeling of material is retained and noted. The candidates' deficiency areas should be brought to the notice of the candidates, the course organizers and trainers. OSCE's once used can be included in an OSCE bank for reuse in the future.

LESSONS LEARNED AND CONCLUSSIONS:
There are many advantages of the OSCE over the traditional clinical examinations 14 . Firstly, the OSCE is fairer than the traditional approach due to the standardization of tasks that have to be performed, an aspect much appreciated by students. Secondly, the wider sampling of competencies and the use of structured marking sheets contribute to improvements in reliability and content validity.
However, the OSCE, had a significant impact on future doctors training and practice 13 . Although OSCE is very good adjunct tool of assessment, but also has its limitations and it should be combined with other tools of assessment which may more validly test competences not easily tested within the OSCE format. For example, attitudinal and behavioral aspects of patient care may be better assessed in practice-based settings rather than in examination settings. In addition, there are many aspects of clinical competence that can be more efficiently tested using a written format 15,16 . Student feedback was invaluable in influencing faculty teaching, curriculum direction and strengthening the development of the OSCE. Finally it should be more possible to utilize the results obtained to fulfill the desired purpose of the examination i.e. for identifying areas of deficiency in training and gaps in curriculum, however, there has been a tendency by many internal and external examiners of replacing or reinforcing this type of clinical examination with the more structured and objective OSCE.

ACKNOLEDGEMENTS:
We are very grateful to all staff members and honorary members of surgical department demonstrators, residents and interns of the Faculty of Medicine Umm-Alqura, who collaborated with us in running OSCE's for the Clinical Stream; which provided the experience on which this article is based.