Objective structured clinical examination in physiotherapy teaching : a systematic review

Introduction: Problems related to the clinical abilities of physiotherapy students are not always identified throughout the educational process and might only be observed when these future professionals have to treat patients. The Objective Structured Clinical Examination (OSCE) includes a problematization approach and can be used in Health Sciences teaching to help this identification before internship practices. However, there are few studies on its use in Physiotherapy. Objective: To gather evidence of the OSCE use to evaluate clinical abilities in Physiotherapy teaching. Method: Articles published from 2000 to 2016 were surveyed in the Biblioteca Virtual em Saúde (BVS) (Virtual Health Library), Centro Latino-Americano e do Caribe de Informação em Ciências da Saúde (BIREME) (Latin-American and Caribbean Health Sciences Information Center), PubMed, Scielo and Web of Science, using the descriptors “educational assessment”, “assessment methods”, “objective structured

Introduction: Problems related to the clinical abilities of physiotherapy students are not always identified throughout the educational process and might only be observed when these future professionals have to treat patients.The Objective Structured Clinical Examination (OSCE) includes a problematization approach and can be used in Health Sciences teaching to help this identification before internship practices.However, there are few studies on its use in Physiotherapy.Objective: To gather evidence of the OSCE use to evaluate clinical abilities in Physiotherapy teaching.Method: Articles published from 2000 to 2016 were surveyed in the Biblioteca Virtual em Saúde (BVS) (Virtual Health Library), Centro Latino-Americano e do Caribe de Informação em Ciências da Saúde (BIREME) (Latin-American and Caribbean Health Sciences Information Center), PubMed, Scielo and Web of Science, using the descriptors "educational assessment", "assessment methods", "objective structured Introducción: En el proceso educativo, ni siempre se identifican problemas relacionados con las habilidades clínicas de los estudiantes.Estas son observadas cuando el futuro profesional está delante del paciente.Para realizar esta identificación antes de las prácticas, se desarrolló la Objective Structured Clinical Examination (OSCE).Incluyendo un abordaje problematizador, OSCE puede usarse en la enseñanza de las ciencias de la salud, pero hay pocos estudios sobre su uso en la Fisioterapia.Objetivos: Sintetizar evidencias acerca del uso del OSCE para evaluación de habilidades clínicas en la enseñanza de la Fisioterapia.Método: Se seleccionaron artículos publicados em el periodo de 2000 a 2016 y consultadas las bases de datos Biblioteca Virtual en Salud (BVS), Centro Latinoamericano y del Caribe de Información en Ciencias de la Salud (BIREME), PubMed, Scielo y Web of Science, utilizando los descriptores educational assessment, assessment methods, objective structured clinical examination, clinical competence, professional competence, clinical skills, student competence, student skills, physiotherapy, y los booleanos OR y AND.Resultados: El número de publicaciones identificadas fue de 3.242.De estos, siete fueron incluidos en esta revisión.Se desarrollaron dos estudios en Brasil, cuatro en Australia y uno en Canadá.Los estudios se calificaron de 7 a 12 con respecto a la calidad metodológica y se clasificaron 1b y 2b en relación con la evidencia científica.Conclusión: Las habilidades clínicas de los estudiantes fueron agrupadas en tres clases, a saber, cognitivo, psicomotor y afectivo, y cuatro estudios describieron el uso de las mismas.Hay una escasez de evidencias sobre el OSCE y el instrumento colabora en la evaluación de habilidades y competencias en la enseñanza en Fisioterapia.

Introduction
The evaluation of university students plays an important role in their initial education, either as a guide of the individual formative trajectory or as an adjustment of formative options.It should focus not only on the knowledge achieved (cognitive skills), but also on the psychomotor and affective areas, which jointly configure the necessary professional skills of working as a physiotherapist.Thus, planning an effective evaluation has to consider specific educational objectives of each phase of the qualification, according to the desired profile of competences and skills [1][2][3], to perform in the different jobs that are included in the definitions of legal norms and pedagogical projects of the course.Competence allows the individual to mobilize knowledge in order to face certain situations promptly and in the necessary manner.Competence is a set of knowledge and abilities, a know-how related to work practice [2,4,5].
The education of health professionals requires that the students master clinical abilities such as communication with patients, physical examination, clinical reasoning and proposition of functional evaluation measures along with the therapeutic approach [5,6].There are several alternatives to evaluate these skills, which frequently approach the practice from questioning.The most reliable alternative, according to Newble [7], is the in loco assessment.However, this is not always possible and problems related to skills and attitudes are often only identified when the future professional has to deal with a patient [8][9][10].Given this context, questioning the evaluation of teaching and learning in the perspective of professional action reinforces the necessary transformation of university education, as pointed out by health education policies.
Aiming at carrying out this identification prior to the start of the mandatory curricular internship, the simulation of attendance evaluation was designed, also known as OSCE -Objective Structured Clinical Examination [11,12].This method provides to the clinical evaluations the advantages equivalent to objective written tests, in which students are evaluated for the accomplishment of tasks related to the collection of clinical background, physical examination, clinical reasoning and procedure [12,13].
Nowadays, OSCE is considered one of the most reliable and effective methods to evaluate skills, mainly in the health area [14][15][16].Most of the studies available in literature refer to the evaluation of students in medical schools [10].In Brazil, its use outstands in two large medical colleges in the state of São Paulo: the Faculdade de Medicina de Ribeirão Preto da Universidade de São Paulo (FMRP-USP) (Ribeirão Preto Medical School -University of São Paulo) and the Faculdade de Medicina de Marília (FAMEMA) (Marilia Medical School).Other few studies have been developed in Nursing [16,17] and Pharmacy [11,18,19] courses, for example.The low adhesion to the use of this instrument in universities might be due to lack of knowledge on the tool, lack of professor training or the controversies regarding its evidence in teaching.
Although OSCE is a useful tool, which can also be used in Physiotherapy teaching since it includes this questioning focus, it is not a systematic practice in Higher Education Institutions.Therefore, evidencing positive and negative effects of a tool to investigate students' clinical skills might interest professors, coordinators and administrators of physiotherapy undergraduate courses to broaden the scope of the student-professor-patient relationship throughout academic education.In this sense, a literature review might provide information about the areas to which it can be applied and on the perspectives of this instrument that might become an effective intervention from its use in higher education.OSCE has not yet received a structured review for implementation in institutions that qualify physiotherapists.Therefore, the objective of this review was to synthesize evidence about the use of OSCE to evaluate clinical skills in Physiotherapy teaching.

Design
Publications were selected from 2000 to 2016 that discussed OSCE and its use in the education of physiotherapists.There were no restrictions regarding the methodology of the studies, however, publications that were editorial, interviews, projects, pilot studies and preliminary or conceptual data were excluded.Likewise, articles whose theme addressed other educational evaluation methods were excluded, as well as those investigating physiotherapeutic treatment techniques or treatments in other health areas.This strategy was important to identify studies that evidenced the use of this tool in a study as a method of evaluation among other investigations available in literature.

Sources of information
The included studies were published in English, Spanish or Portuguese, and were available at the Biblioteca Virtual em Saúde (BVS) (Health Virtual Library), Centro Latino-Americano e do Caribe de Informação em Ciências da Saúde (BIREME) (Latin-American and Caribbean Health Sciences Information Center), PubMed and Web of Science databases.The search period ended in May 2016.Three search themes were combined using the Boolean operators "AND" and "OR".The first search included instruments to evaluate students' clinical skills, combined in the title/abstract from the keywords "educational assessment, assessment methods" or "clinical competence, professional competence, clinical skills, student competence, student skills".Next, studies that analyzed OSCE were identified using "objective structured clinical examination".And, finally, a third search was carried out, in which "physiotherapy" was included.The articles should contain the terms listed above, not necessarily all of them, however, the term "objective structured clinical examination" was the most relevant factor for inclusion.

Selection phases
Publications containing the searched keywords in the title or abstract were tracked.Studies with abstracts available and whose complete text could be accessed for free were selected, then read for data extraction.With the previously defined sample, an exploratory reading was carried out, using a proper form, specifically devised for data analysis and extraction, and the recommendations by the Cochrane Library [20] were adjusted according to the authors' interest.The items considered were: objectives, methodological design, and place where the study was developed; sample size, main results, description of students' clinical skills and performance.

Evaluation of methodological quality and level of scientific evidence of the studies
The assessment of the methodological quality of the studies was carried out by two independent evaluators (AMR and JSCA).This assessment was based on the evaluation instrument by the McMaster Occupational Therapy Evidence-Based Practice Research Group (http://srs-mcmaster. ca) -Table 1 [21][22][23].The studies were scored in each one of the 14 items, scoring one point if the methodological rigor was suitable or zero if the methodology was not suitable.The final score could range from 0 to 14 points.The studies were also categorized regarding levels of evidence according to the Oxford Centre for Evidence-based Medicine -Levels of Evidence [24].

Data analysis
After collection, the data were consolidated in a database for analysis.Due to the methodological heterogeneity and the different outcomes sought with the use of OSCE, a descriptive analysis of the results was carried out.The analysis sought to identify the main reflections, discussions, limitations and potentialities of the use of this tool.Data presentation followed strictly the terminology used by the authors, from a proper form, specifically developed to analyze those data, adjusted according to the authors' interest and following recommendations by the Cochrane Library [20].

Results
2,766 publications were identified in PubMed, 14 in Bireme and BVS and 462 in Web of Science, totaling 3,242 studies.After applying the exclusion criteria, seven studies were selected for review (Figure 1), they were all published between 2011 and 2016.The use of the instrument was reported in detail in four studies (57.1%) and the most prevalent investigation design was the cross-sectional study (n = 3; 42.9%), followed by the randomized clinical test (n = 2; 28,6%).From these, four were developed in Australia, two in Brazil and one in Canada.The characteristics of the studies included regarding objectives and level of evidence are shown in Table 1.
None of the studies was scored for all validity criteria from the evaluation of the methodological quality (Table 1), with scores ranging from 7 to 12. Regarding the level of scientific evidence, the studies showed levels 1B and 2B.The general features of the studies regarding objectives are described in Table 2.  To describe the electronic OSCE (eOSCE) implementation and evaluation using iPad in two universities.

/ 2B
Note: *The self-video was a university study in which the students recorded their performance in a certain task assigned during the study.

Sample and OSCE use in Physiotherapy teaching
The general features of the included articles are listed in Table 2.The total sample size was 632 physiotherapy students (variation 29-206) and 25 examiner tutors.Table 3 shows data of the OSCE used and the main results obtained.

Cacho et al., (2016) 14
Professors met prior to the examination to devise clinical cases.The concepts "yes", "no" and "insufficient" were ascribed to the task accomplishment, with a final score of 10 points.
The second phase referred to the use of OSCE.Students experienced the practice in stations (attendance simulated environment) where two examiners applied the instrument.
Excellent reliability was seen in the stations 1 (CCI=0,89), 2 (CCI=0,99) and 3 (CCI=0,99), and satisfactory (CCI=0,73) in station 4, being shown as a useful method for the evaluation process of professional education.However, it requires pedagogical preparation and examiners' previous experience.It is preferable to use narrow response, e.g.correct or incorrect.  2  Predictive variables (course entrance interview) and outcome variables (students' marks during the course and in the OSCE) were analyzed.There was no description of the way OSCE was carried out.
The course entrance interview scores significant association with performance in three of the six clinical internships of the course.The authors highlighted the role of entrance measurements in the selection of physiotherapy undergraduate students.

Maloney et al., (2013) 28
Students were divided into two groups: one group was assigned the task of learning an ability through the self-video; to the other, the self-video showed an ability less relevant to the clinic case.At the end of the semester, both groups took OSCE based on the following abilities: introduction to the patient, interpretation of the clinical examination result, conduct execution, giving feedback to the patient, reformulating conduct or evaluation technique, proper handling, communication with the patient, etc.
The students got a significantly higher score in the OSCE when the clinical ability had been analyzed with a self-video on how to perform the task (p=0.048).The analysis of students' perception identified that the self-video contributed to the improvement of the clinical performance and self-confidence in future clinical practice.The clinical ability with highest score was reached when the traditional teaching methods were complemented with the self-videos.
(To be continued) Author, year Methodology Main results

Silva et al., (2011) 29
Two types of examinations were evaluated: traditional and OSCE.The traditional test consisted in four theoretical and one practical question.OSCE included five stations (0-2 points/ each).The students had a minute to read a task description and five to accomplish it.The abilities evaluated were divided into classes, cognitive, psychomotor and behavioral skills.
The OSCE mean score oscillated between 4.4 and 9.6, with good internal consistency between stations (0.7).The agreement between examinations determined that they are not comparable.Low agreement was also observed between the two examinations (r=-0.1,p=0.9).OSCE evaluates distinctly different abilities when compared to the traditional examination, which suggests that OSCE complements the evaluation of skills and competences that traditional exams fail in evaluating.

Maloney et al., (2013) 32
Three teaching methods were compared: "traditional", prerecorded lessons (tutorial videos) and students' self-performance videos.The students received a clinical case that required the use of their skills (evaluated by OSCE).Performance was scored according to their communication abilities, clinical reasoning, safety and risk management and patient handling, and the results were classified as "concluded", "partially concluded" or "insufficient".
No significant differences were found in the clinical performance between the three teaching practices.There was significant difference between the methods regarding the education value noticed by the students, with teaching approaches using tutorial videos and students' self-performance videos, in which the tutorial videos were considered superior.Alternative teaching methods might produce equivalent learning results when applied to the development of practical skills.They might be applied instead of live lessons, rather than being a mere complement or a supplementary method to traditional approaches.

Snodgrass et al., (2014) 38
The electronic OSCE (eOSCE) was devised using an iPad to improve students' feedback.The eOSCE was tested in two universities regarding user friendliness, preference (electronic or printed) and performance in the feedback given to the students.The authors did not describe how the OSCE was applied.
Most of the examiners (68%) preferred eOSCE, mainly due to the consistency and speed of the students' feedback.The advantages listed were automation, individual and immediate feedback and time saved in the OSCE applications.The disadvantages were the necessary pre-exam preparation and challenges to the examiners that were not comfortable using the technology.Note: OSCE: Objective Structured Clinical Examination; eOSCE: electronic OSCE.

Clinical skills assessed in the studies
The main clinical skills identified in the studies were grouped in different classes, namely cognitive, psychomotor and affective.Four studies described the evaluation method of such abilities [14,26,28].Table 2 shows that the cognitive skills were defined as the ability to integrate intellectual knowledge and mastering of the content to accomplish a task or devise a suitable therapeutic procedure in a clinical case [28].The execution of evaluation components was mentioned and, whenever necessary, changes were performed in the evaluation technique according to each case; the result interpretation of the clinical evaluation [28], the use of suitable equipment for the therapy and whether the selected procedure achieved the proposed objective [14].
As regards psychomotor abilities, the students' skills in executing a physiotherapeutic technique or a physical examination were tested [28].The authors reported the correct application of the evaluation techniques, proper manual handling of the patient [27]; handling patients' transfers and suitable care with exoskeleton devices [14].
Finally, affective or behavioral skills were evaluated through the students' reactions, their speech, as well as verbal command and intonation when interacting with the patients along with their initiative (proactivity) [28].The considered aspects were: whether students introduced themselves when interacting with patients [14]; whether it was a suitable introduction; whether they adjusted their communication according to the patients' needs; whether they confirmed patients' understanding on the evaluation techniques and treatment using suitable terms; whether they obtained patients' informed consent; whether they used proper non-verbal communication (eye contact) [26]; whether they provided patients with feedback at the right time (language and clarity) [14]; whether they considered patient comfort, dignity and respect, including proper undressing for evaluation or treatment; whether they presented a professional posture [26]; and whether they considered risks and took measures to minimize them [27].

Discussion
This study aimed to synthesize data about OSCE and to investigate its use as an evaluation tool to assess clinical skills in physiotherapy teaching.The results evidenced a gap in literature regarding the instrument and its use in physiotherapy, maybe as a result of lack of knowledge on the tool [11], methodological difficulties when performing scientific investigations, added to the challenge of implementing this evaluation tool in undergraduate courses due to its complex organization and the time spent in the selection/training of patients [15], along with coordinators' and professors' preferences.All these factors determinate the difficulty to develop scientific studies, in addition to demonstrating different strategies of evaluation among the different higher education institutions.
The identified study designs varied and reflected limitations to establish relations between learning and the use of OSCE, with different levels of evidence, a wide variety of investigation purposes, restriction in sample size and low number of publications, which prevent data systematization with robust statistics.The characteristics of the identified studies in this review include the use of the OSCE in research as an evaluation of learning in physiotherapy undergraduate courses [14,28], as a proficiency examination [25] in convenience sampling [28], and the fact that there were only two Brazilian studies [14,28].These results did not allow a conclusion about the use of the instrument to evaluate students' abilities and competences by the professor, due to the difficulty of visualizing results from the evidence, but also for the lack of an accurate method of diagnosis able to reach these elements and because evaluation is an issue that involves the kind of education experienced by the professor [2,29,30].
The included studies revealed that very often the appreciation of students' performance was based on the reference norm, with the use of marks [10,25,26,28,31] while the use of OSCE might reveal formative aspects that go beyond the summative.Literature points out a few recent initiatives that started to use Learning Based on Problems as a strategy to reorganize the curriculum, and students' evaluation became criterion-referenced, using concepts that qualify the students know-how in relation to profiles of professional competence defined for different undergraduate courses [14,30].The traditional examination focuses on cognitive and psychomotor skills, while the OSCE addresses attitude and psychomotor skills that are needed to develop procedures in clinical practice.Students' evaluation based on cognitive skills tends to promote a learning strategy of information memorization, while evaluations based on the ability of integrating knowledge tend to develop an attitudinal learning strategy.
One of the studies included [31] in this systematic review aimed at comparing the efficacy of two different methods of teaching practical skills in relation to traditional teaching.Those authors did not find significant differences in clinical performance among the three modalities, and they emphasized that the allocation of teaching resources in alternative methods to train skills is recommended, however, it is relevant to consider the complementarity and interdependence between them, and one cannot be considered a priori better than the other.In another study, Silva et al. [28] evidenced that the lack of association between the results of student performance in a traditional examination and in OSCE might be explained by the specificity of each exam, which may suggest that both exams are complementary but not mutually excluding.In addition, confirming the studies by Brasford [32], we observed the predominance of the discussion about the objective of a learning evaluation by privileging the development of critical-reflexive thinking, intellectual independence and creativity, built through the development of metacognition ability.
Even if traditional evaluations provide relevant data about students' academic and clinical competence, objectivity and standardization are still incipient.The results obtained regarding students' skills are strongly influenced by the profile of patients available for certain examination, the type of service they are involved with, and the whole evaluation depends on the examiner's subjective concepts.In addition, students are tested in very different conditions, which limits peer comparison.These particularities contribute to the low reliability of these methods [33].
OSCE appears as an opportunity to deal with a particularly suitable instrument to evaluate many, certainly not all, the components of clinical competence.For example, attitudinal and behavioral aspects are probably better overcome with the use of multiple classifications collected along a period of time during internship, in different modalities of in-service teaching and during interaction with different professionals of the team [2].At the other end of the instrument, the relevant knowledge needed to be competent, including aspects of diagnosis, investigation and treatment, can be more efficiently and cheaply tested in written form, differentiating from the traditional clinical examination [34].
The main benefit of using OSCE is the possibility of linking a set of essential competences to the configuration of simulated clinical situations, which allow the measurement of certain structured competences.This implies that OSCE can be considered a "gold standard" to evaluate the ordinary education of health professionals.Some of the OSCE positive points that can be highlighted are the possibility of selecting the type of skill to be evaluated, the level of complexity required and the evaluation circumstances, without depending on the availability of cases, environmental factors or circumstances [33].The use of real patients (not simulated) is usually considered more suitable, since they have realistic problems, needs, demands and conditions with a degree of complexity which is common in the everyday professional practice, particularly in outpatient care.However, direct observation is more detailed and provides a closer and integrated view of the students' clinical skills, making it easier to identify the learning areas to be reinforced and including the interaction that qualifying professionals have with the service in which they are inserted and the teams they belong to.The use of structured tools, such as a check-list, increases the accuracy of failure detection and offers objective elements to provide students with relevant feedback [35][36][37][38].
The current scenario of physiotherapists' initial education requires a certain vision of the profile of competences desired in higher education institutions in our country [39,40].For Reis and Monteiro [41], the teaching-learning process has emphasized the acquisition of technical competences rather than the understanding and analysis of problems and application of knowledge based on evidence, which reveals a certain unsuitability in physiotherapy initial education, as pointed out by Bispo Júnior [42].The institutions still privilege the technical model based on the biological paradigm, stimulation to specialization and emphasis on biological research, without paying much attention to the aspects of students' behavioral education.A fast increase in the number of newly graduate physiotherapists has also been observed, who are usually inserted in the labor market with very little experience in clinical environments [43].In a complementary way, Davies et al. [44] pointed out that students in the rehabilitation area reported that many times they are not really sure of what is expected from them concerning their clinical skills and competences.Regulating agencies such as the Physiotherapy and Occupational Therapy Federal and Regional Councils, in their Ethics and Deontology Codes, provide basic norms, regulations and conducts linked to the fundamental beliefs or values of the profession, which are undertaken through the professional socialization that one student experiences when taking part in an environment of practice [44,45].Thus, the evaluation of skills and competences, both in the academic and clinical environments, should emphasize the estimated value of these behaviors [44].The information provided before and the feedback of the clinical training should allow the students to recognize subjectively their clinical skills and to develop learning plans to improve their knowledge, behaviors and skills step by step [43,45].
However, the lack of uniformity when professors give marks/concepts, failure to detect mistakes made by the students and the high level of stress experienced by them, as well as the short time available to carry out the tasks, reducing the professional-patient interaction, are limitations presented in the use of OSCE according to the studies selected for review [25,27,28].Another inconvenience is the need of a complex organization process, including time, cost and guarantee of confidentiality [36].Snodgrass et al. [37] devised an online version of an electronic OSCE (eOSCE), and, when comparing it to the printed version, they found significant improvement regarding accuracy, financial viability and time-cost relation [38].Despite these advantages, the challenge of implementing the eOSCE is to guarantee that all examiners and course coordinators feel comfortable with this technology.
Systematically reviewing clinical research on the use of OSCE in physiotherapy courses emphasized its relevance in relation to the scenario of improvement of the courses offered by higher education institutions, which might result in changes in the Brazilian curricular guidelines, according to the profile required by the current health system and the education policies in the health area.As a result, the instrumentation of the teaching job, based on the qualification of the students' evaluation becomes relevant and might, in addition, stimulate the teaching-learning process and help in the efficiency of the subjective evaluation.However, the low number of researches included in this systematic review shows the scarcity of studies related to the use of OSCE in physiotherapy, revealing a gap in scientific literature.These data suggest the need for reflection upon how the development of skills and competences in physiotherapy undergraduate students has been evaluated [32], mainly considering the context of changes in the initial education, as pointed out by the current policies of the fields of education and health, expanding the professional attributes considered necessary for their work in teams and different jobs in the health system [39,40].
Regarding future investigations, it seems necessary to analyze the ability of the OSCE examination to predict the students' performance in the practical training of different modalities of learning, in service or in their well-succeeded entrance in the job market.In relation to the professors' practice, some caution seems necessary when making decisions, especially in relation to the study profiles, the methodology adopted and the outcome measurements.Regarding the implications to the research, due to the absence of homogeneity in the methodological approaches and outcomes found, it is not possible to determine systematic positive effects able to justify the effects on the clinical skills of physiotherapy students.The findings regarding the use of OSCE to evaluate students seem to be promising.However, in order to make broader statements on the development of teamwork, services and health systems concrete, adjustments are still needed in future studies.

Conclusion
The results of this systematic review indicate the need for further studies on the use of OSCE, mainly with high methodological quality, in which the main objective is to investigate the role of this instrument in the evaluation of clinical skills in physiotherapy undergraduate courses.The use of this tool, either to analyze students' clinical skills or to reflect upon the learning conditions offered by the higher education institutions seems to be relevant.This study offers an international panorama regarding the use of OSCE in physiotherapy, revealing that its use in Brazil is still limited.
Such scenario also brings about the relevance of great depth to the discussion and practice of evaluating professional skills throughout the undergraduate education, to face the different initiatives of changes in the professional profile required by the current health system and the education policies in the health area, as well as the development of teamwork, services and health systems.

Table 1 -
Evaluation of the methodological quality of the studies Figure 1 -Flowchart with the number of identified studies, included and excluded from the literature review.

Table 2 -
General features of the studies included regarding objectives and level of evidence

Table 3 -
Characteristics of the OSCE used and results evidenced from the selected publications