Evaluating Simulated Consultation Videos in Teaching Patient-Centered General Practice

This article was migrated. The article was marked as recommended. Introduction In the general practice course at Copenhagen University, students are taught patient-centered consultations. The aim of this study is to evaluate the feasibility of a new method for measuring the effect of this teaching, and of adding access to simulated consultation videos to usual teaching. Methods The university assigned 293 final-semester students to three groups: a ‘Control Group’ with usual curriculum, an ‘Access Group’ that watched simulated consultation video clips online and a ‘Teaching Group’ where the video clips were discussed in teaching sessions. The outcome was the change in students’ ability to identify patient-centered elements in a test video consultation, measured with a questionnaire before and after the course. Results An overall teaching effect was observed, which was most apparent in communication items such as “making a contract about the topic for the consultation” and “summarizing”. Changes in clinical items and general issues were small. Conclusion A tool for measuring the effect of teaching general practice consultation skills combining a test video and a questionnaire is presented. Topics needing to be highlighted in teaching could be identified using the tool.


Introduction
In the general practice course at Copenhagen University, students are taught patient-centered consultations.The aim of this study is to evaluate the feasibility of a new method for measuring the effect of this teaching, and of adding access to simulated consultation videos to usual teaching.

Methods
The university assigned 293 final-semester students to three groups: a 'Control Group' with usual curriculum, an 'Access Group' that watched simulated consultation video clips online and a 'Teaching Group' where the video clips were discussed in teaching sessions.The outcome was the change in students' ability to identify patientcentered elements in a test video consultation, measured with a questionnaire before and after the course.

Results
An overall teaching effect was observed, which was most apparent in communication items such as "making a contract about the topic for the consultation" and "summarizing".Changes in clinical items and general issues were small.

Conclusion
A tool for measuring the effect of teaching general practice consultation skills combining a test video and a questionnaire is presented.Topics needing to be highlighted in teaching could be identified using the tool.

Introduction
At the University of Copenhagen, the patient-centered consultation as defined by Levenstein and Brown is taught during the final semester clinical course in general practice (Levenstein et al., 1986) (Brown et al., 1986).Their model describes four general tasks for the doctor: exploring health, disease, and the illness; understanding the whole person; finding common ground and enhancing the patient-clinician relationship.
The teaching is based on the experiential learning model, which several authors have found effective (Abdulwahed and Nagy, 2009) (McLeod, 2010).This model demonstrates a continuous learning process with four basic elements: concrete experience, observation, and reflection, forming abstract concepts, and testing skills in new situations (Kolb and Kolb, 2012).
Students at the course are between 25 and 30 years old and thus adults.According to theories about adult learning, the students should affect the content and process of their learning.The teaching should focus on adding to their pre-existing knowledge and skills.It must be practical and centred on problem-solving instead of memorizing content (Knowles, 1980).The course in general practice is based on these two learning principles.
The effectiveness of teaching patient-centered consultation skills to medical students can be evaluated in several ways.Teachers can rate student performance from audio and video recordings (Burt et al., 2014) (Cömert et al., 2016).The ratings are dependent on the observer's goals, preferences, and the quality of the rating scale.Secondly, self-efficacy questionnaires have been used (Zachariae et al., 2015), but physicians and students are poor assessors of their own performance, as low performers tend to overestimate their own skills (Davis et al., 2006).A third method is interviewing students individually or in focus groups about the teaching (Braverman et al., 2016).The result is dependent on the interviewer, when and where the interview takes place, and the right items (Berk, 2005).
Teachers and curriculum planners must react to new types of students arriving, who have grown up with cell phones, tablets, and constant access to the internet (Prensky, 2001).Flipped classroom (FC) moves homework into the classroom as the students prepare for teaching sessions at home (Abeysekera and Dawson, 2015).Current evidence suggests that the use of flipped classroom in health professions education makes a significant improvement in student learning compared with traditional teaching methods (Hew and Lo, 2018).
The aim of this study is to evaluate the feasibility of a new method for evaluating the effect of teaching patient-centered consultations and evaluate the effect of adding simulated consultation videos to usual teaching in an FC design.

Method
An open prospective cohort study was conducted using a simulated consultation video and a questionnaire before and after the course in general practice at the University of Copenhagen (KU) in 2013.Teaching general practice at KU is taught in the first and final semesters.Kolb experimental model and Knowles' four adult learning theories are used in planning the curriculum in the final semester (see Table 1).
Students were invited by e-mail before the course to watch a ten-minute lecture about patient-centered consultations on the learning platform.The students were familiar with the platform, as all communication between the university, the teachers and the students are handled via this platform.
On the first day of the course, the authors also gave a one hour lecture on patient-centered communication.Final-semester students work eight days in a general practice clinic, where they have consultations with real patients, video-record these consultations and receive feedback from their tutor general practitioner (GP).In small group sessions at the university, they discuss their videos with their peers and a teacher (twenty hours in all).The small group sessions and lectures alternate with days in general practice.
The teachers were 17 trained GPs with a special interest in teaching, eight women and nine men.The university employed six and eleven were associated teachers.The teachers were the same during the study period.
For the oral exam, students select one of their own consultation videos to present for analysis.The students are assessed by their performance in the video and their ability to analyse it according to the patient-centered method.They are presented with a clinical case for discussion and a theoretical question about the role of general practice in the primary healthcare sector.
The questionnaire: The DanSCORE questionnaire (Danish Structured Consultation Observation Registration and Evaluation) was originally developed for research purposes.It was designed by one of the authors (KW) to evaluate a whole consultation in general practice.The questionnaire has 33 items.Fifteen items reflect communication skills (items 1-7, 10, 14-20), five items clinical skills (8-9, 11-13) and thirteen (21-33) are about the consultation in general (see Supplementary material 1).The general items dealt with topics related to the structure and duration of the consultation, doctor's role, use of understandable language, interrupting the patient, and reaction to patient mood; these are observable and not directly related to the original patient-centered method.
The questionnaire was piloted in two studies in 2012.In the first pilot study, 13 final-semester students used the questionnaire to analyse their own videos in a teaching session.They assessed each element and its response categories for comprehensibility.Minor linguistic corrections were made.In the second pilot study, 45 final-semester students watched the test video before and after the general practice course, completed the questionnaire and commented on the questionnaire as a whole.Again, minor text corrections were made.For analysis, responses were collapsed into two categories (correct or incorrect).
A simple framework to help medical students master the patient-centered elements working in general practice was developed (see Table 2).
Test video: The test video of 15 minutes shows a consultation between a general practitioner (GP) and a simulated patient.The GP (university teacher) was informed that the consultation would be video-recorded for teaching purposes but knew nothing about the patient and the symptoms.The simulated patient played a stressed person with a headache and was instructed to act as a real patient.The teachers, all experienced GPs, had difficulty in evaluating the video from the perspective of a medical student, and their replies differed to some extent.To reach consensus on correct answers for the test video, a modified Delphi method was used.There was still disagreement regarding five items, so the authors decided to transcribe the test video.Two authors (MJ, KW) then set the criterion standard, making a final decision on correct replies.For two items, two replies were accepted as equally correct because the response categories overlapped.Simulated consultation clips: A major part of the teaching comprises analysis of video consultations.Therefore, to intensify the teaching 16 brief video clips (0.35-5.36 minutes) with three different simulated patients and two authors (MJ, KW) in the GP role were produced.The teaching material could be accessed anywhere by mobile phone or tablet, so patient anonymity could not be ensured.Therefore, SPs (Simulated Patients) were used.To keep consultations as authentic as possible, scripts were not used, and actors were instructed to present symptoms they were familiar with.The doctors were comfortable in their roles and varied their behaviour to increase learning potential.A professional company used two cameras and the "first takes" of all 16 simulated consultations were successful.The clips were edited on the spot, based on whether the doctor's or the patient's face was to be shown.Apart from that, no editing was done.Each video was followed by questions about elements of the patient-centered consultation.The students received no feedback, as the simulated consultation clips were meant to lead to reflection.The students' use of simulated consultation clips was measured electronically on the learning platform.
Interventions: Final-semester students in 2013 were placed by the University in three teaching groups.The first of these was a 'Control Group', receiving the usual teaching; the next group the 'Access Group' also had access to the simulated

Relation to a work situation
Learning is related to a specific context (GP clinic) Feedback to students on their own consultation videos, discussing patient-centered elements

Problembased learning
The student deals with complex symptom presentation from patients in General Practice In the small group sessions, clinical and communicative problems are discussed with the teacher and peers consultation clips.The students in the third group.the 'Teaching Group' were asked by the teachers to watch four simulated consultation clips before each small group session for discussion.The teachers' main priority was still to give feedback on students' own consultation videos recorded during their work in general practice.After the course, teachers were asked how many video clips they had discussed in their small groups.
Testing: On the first day of the course, all students watched the test video during a lecture and completed the questionnaire.Five weeks later, on the last day of the course, the procedure was repeated.Students were not allowed to talk while completing their questionnaire.Each student's pre-course and post-course questionnaires were linked, without revealing students' identities.
Statistics: The students' answers were evaluated as either correct or incorrect.They were placed in four groups according to changes between the first and last session: IncorrectIncorrect, Incorrect-Correct, Correct-Incorrect and Correct-Correct.A teaching effect was calculated for each item as the percentage of students that improved, e.g.changed from incorrect answer before the course to correct answer after the course.Group differences between the use of video clips and between examination marks were calculated by an X2 -test.

Results
All 293 students in the three groups were included.Pre-course and post-course data were available for 217 students (74 %) who completed both questionnaires (Table 3).
Students who did not participate in the post-course test had similar answers in the pre-course test as those who completed both tests.The "teaching" group watched more clips on average than the "Access" group, but the difference was not significant (p = 0.07).The video clips were to be watched before four teaching sessions (out of five).The teachers reported having discussed eight of the 16 simulated consultation clips on average in the small group sessions (see Table 4).
The teaching effect was clearest in communication items, but mostly lower in clinical and general items (see Supplementary Material 2).The change in students' ability to identify communicative consultation skills (patient-centered elements) is seen in Table 5.
Table 2. Framework for students working in general practice: structure and elements to observe in a consultation In The teaching effect is calculated as the percentage of students changing from an incorrect answer before the course to a correct answer after the course.In almost all questions, a positive teaching effect is seen.The teaching effect can be small if teaching is insufficient or if students know about the topic beforehand, so there is little room for change.
In all three groups, a clear teaching effect was observed, especially in three items: "making a contract about the topic for the consultation" (item 1) and "summarizing" (items 7 and 10).In five items, the change in the ability to identify communicative consultation skills was low: "asking about patient expectations" (item 6), "informing the patient" (items 15 and 16), "safety netting (item 19), and "asking the patient to summarize the instructions" (item 20).The students had only minor problems evaluating the clinical items, with small variations between the groups.They agreed that the doctor's conclusion was likely (item 11) and that the decision about treatment was medically correct (item 13) (see Supplementary Material 2).The change in students' ability to identify general issues was relatively low and varied between the groups (see Supplementary Material 2).
No significant difference in grade distribution is seen (X2 > 0.05) (see Table 6).

Summaries of activities
Students' motivation is related to their need for new skills, and they benefit from applying new knowledge immediately.
Alternating clinical work and feedback in small group sessions contribute to this, and the need comes from taking on a new role as GP (Peters et al., 2017).The study was done in a real teaching environment while developing teaching methods.The introduction of the online material was stepwise, online video use was registered, good follow-up of participants established and stability in the teachers' group, The test video as such could not produce intensified learning since a consultation video has for years been shown the very first day of the course in general practice at the University of Copenhagen (Spreckelsen and Juenger, 2017).
If existing rating scales or observation guides had been used instead of the questionnaire, it might have overwhelmed undergraduate medical students.For example, The Calgary-Cambridge guide to the medical interview has more than 70 items (Kurtz and Silverman, 1996).All three groups gave mostly similar pre-course answers, indicating that students found the items and response categories understandable.The test video consultation was transcribed.and patient-centered elements were identified.
Two items could be answered in several correct ways.Regarding the contract about the agenda (item 1), the doctor says, "So you come with a headache?"and the patient nods.We teach students to ask if the patient has other topics to be addressed, so "no" would be our preferred answer, but the nod makes "partly" in this case also acceptable.For the item on the doctor interrupting the patient (item 28), the response categories were "no", "yes but acceptable", and "yes, unacceptable".The doctor interrupts with half a word, so a "no" and a "yes but acceptable" are equally correct.In the small group sessions, students watch a number of each other's videos, so they have probably forgotten the test video when they see it again.
Results might have been different if we had used another test video at the end, but it could have led to a discussion of which video was easiest to observe.

Discussion of results
The students entering the course already had experience working with patients so taking history and examining patients were well known to them.Many had problems evaluating the information given to the patient in the test video (item 15, 16, and 17).This could be expected, as primary care is a new environment.The epidemiology and handling of diseases are quite different from hospital settings (Braverman et al., 2016) (Boggiano et al., 2017).
The teaching effect was best in communication items, as the patient-centered consultation was new to students.They were not accustomed to asking for patient expectations, as their medical experiences came from hospitals, where the patients not usually are asked about this.The same is the case for safety netting and for letting the patient summarize the instructions.
Very few students went from correct to incorrect answers during the study.
The duration of the short simulated consultation clips of up to five minutes seemed appropriate (Brame, 2016).The overall use of clips, however, was much lower than expected.The teachers were asked, after the course, how many simulated consultations clips they had discussed with their students.A recall bias may exist, or they may simply have tried to please the investigator.On the other hand, introducing a new teaching method requires some time before optimal use is found (Thorell et al., 2015).The students were also somewhat unfamiliar with the flipped classroom concept.
No convincing effect of introducing simulated consultation videos in the teaching was seen, but students' ability to identify communication items was generally improved.

Limitations
The study was controlled but not randomized or blinded.A randomized controlled study would have been ideal, but this design is rare in educational studies (Norman and Eva, 2014).In our study, it would have been impossible to blind the groups, as students share teaching materials across groups.
The teachers were reluctant to use online material optimally.They experienced a lack of time for important structured personal feedback and only used half of the planned videos.
Evaluating students' ability to analyse consultations is a simplified proxy of learning the complexity in General Practice, but learning is a stepwise process and analyzing and evaluating are part of the steps (Adams, 2015).

Conclusion
The students completing the course in general practice at KU have learnt to identify important patient-centered elements in a consultation, but the new teaching method was somewhat difficult to implement for teachers as well as for the students.
the patient's part an agreement about the agenda of the consultation the patient's ideas about the symptoms the impact of illness on daily functions feelings aroused by the symptoms the patient's expectations regarding the consultation In the doctor's part history taking based on hypotheses formed during the patient's narrative clinical problems appropriately considered In the joint part mutual agreement about diagnosis is reached mutual agreement about a plan is reached the doctor informs the patient about the safety net In the general part appropriate use of time in the consultation the doctor informs the patient about what happens next in the consultation (signposting) the doctor's attentiveness to the patient summarizing to obtain a shared understanding consultation structure

Table 1 .
Teaching in general practice.Knowles' four adult learning strategies applied to the course in General Practice at Copenhagen University

Table 3 .
Participants, number, and gender

Table 4 .
Use of video clips.The topics and duration of the simulated consultation clips

Table 5 .
Change in students' ability to identify communicative consultation skills

Table 6 .
Exam gradesOne student in the teaching group did not participate in the exam due to illness.A new and feasible way to evaluate the effect of teaching general practice consultation skills, combining a test video and questionnaire, has been presented.Topics needing to be highlighted in teaching can be identified in this way.MD.GP.Associate Professor at the Research Unit for General Practice and Section of General Practice, Department of Public Health, University of Copenhagen, Copenhagen, Denmark.For fifteen years (2001-2016) course instructor in the course in general practice at the University of Copenhagen.Has for many years taught evidencebased medicine to young doctors in the specialist training.ORCiD: https://orcid.org/0000-0002-2877-8020Klaus Witt, MD.GP.PhD.Senior Researcher at the Research Unit for General Practice and Section of General Practice, Department of Public Health, University of Copenhagen, Copenhagen, Denmark.Course instructor in the general practice course for ten years (1991-2001).Has for many years taught evidence-based medicine to young doctors in specialist training.Marjukka Mäkelä., MD.GP.PhD.Professor at the Research Unit for General Practice and Section of General Practice, Department of Public Health, University of Copenhagen, Copenhagen, Denmark.Research Professor at The International Network of Agencies for Health Technology Assessment, National Institute for Health and Welfare, Helsinki, Finland.Has for many years taught evidence-based medicine to medical students and doctors.ORCiD: https://orcid.org/0000-0001-5111-1358 Bibliography/References Abeysekera, L. and Dawson, P. (2015) Motivation and cognitive load in the flipped classroom: definition, rationale and a call for research.Higher Education Research & Development.Routledge, 34(1), pp.1-14.Reference Source Adams, N. E. (2015) Bloom's taxonomy of cognitive learning objectives.Journal of the Medical Library Association: JMLA.Medical Library Association.103(3), pp.152.Reference Source Berk, R. A. (2005) Survey of 12 Strategies to Measure Teaching Effectiveness.International Journal of Teaching and Learning in Higher Education.17(1), 48-62.Reference Source (Accessed: 17 January 2021).Boggiano, V. L., et al. (2017) The Patient-Centered Care Challenges and Surprises: Through the Clerkship Students' Eyes.Family Medicine.49(1), pp.57-61.Available at: Reference Source (Accessed: 17 January 2021).Brame, C. J. (2016) Effective Educational Videos: Principles and Guidelines for Maximizing Student Learning from Video Content.CBE life sciences education.American Society for Cell Biology.15(4).Reference Source (Accessed: 17 January 2021).Braverman, G., et al. (2016) Finding the words: Medical students' reflections on communication challenges in clinic.Family Medicine.48 (10), pp.775-783.Brown, J., et al. (1986) The patient-centred clinical method 2 definition and application.Family Practice.3(2), pp.75-79.Reference Source Burt, J., et al. (2014) Assessing communication quality of consultations in primary care: Initial reliability of the Global Consultation Rating Scale, based on the Calgary-Cambridge guide to the medical interview.BMJ Open.4(3).Reference Source Burt, J., et al. (2016) Rating Communication in GP Consultations: The Association Between Ratings Made by Patients and Trained Clinical Raters.Medical Care Research and Review.Reference Source Cömert, M., et al. (2016) Assessing communication skills of medical students in Objective Structured Clinical Examinations (OSCE) -A systematic review of rating scales.PLoS ONE.11(3).