Video- assisted reflection: improving OSCE feedback

Background: Objective Structured Clinical Examinations (OSCEs) are commonly used to provide feedback to students on their performance in formative examinations. However, students are often unable to act independently on the feedback they re-ceive. This study explored how the use of video- assisted reflection in OSCEs can enhance students’ ability to reflect and engage in sustainable feedback. Methods: Twenty- one students undertaking a mock- final OSCE consented to have one of their examination stations filmed. Participants completed a series of reflective forms immediately after the OSCE, after verbal feedback from an examiner and finally, after watching the video of their own performance. Students were asked to predict their overall grade as well as list areas for improvement. Pearson r correlations examined the relationship between the examiners’ grades and the candidates’ self- predicted grades. Wilcoxon signed- rank tests were used to compare the length of reflections at each stage. Semi- structured interviews were conducted to explore students’ beliefs on self- efficacy and how the video- assisted reflection altered their ability to act on feedback. Results: The students’ ability to self- assess and gauge their own performance improved significantly after undertaking the video- assisted reflection ( p < 0.01). Furthermore, video- assisted reflection significantly increased the length of the student's reflections. In interviews, participants described multiple ways in which the video- assisted reflection improved their confidence and ability to act on feedback, highlighting a clear enhancement in self- efficacy. Discussion: Video- assisted reflection of recorded OSCE stations represents an ef-fective approach to increase student self- efficacy and subsequently improve engage-ment in sustainable feedback practice. However, evidence suggests that students struggle to act upon the feedback they receive. 2 Ideally, students should be able to make use of feedback independently in order to support their future learning— which is part of a practice known as ‘sustainable feedback’. 2 Numerous processes enable students to use feedback effectively. Self- assessment is a mechanism for one to identify ones’ strengths and weaknesses. 3 Following self- assessment, students need to make appropriate goals for their learning, a process entitled ‘self- regulation’. 2


| INTRODUC TI ON
Objective Structured Clinical Examinations (OSCEs) are widely used to assess medical students' skills in history taking and clinical examination.
Formative OSCEs represent an excellent opportunity to provide feedback to students as they constitute authentic learning experiences but occur within a controlled environment. Clinical skill development occurs more quickly when students are provided with appropriate feedback. 1 Formative OSCEs represent an excellent opportunity to provide feedback to students as they constitute authentic learning experiences.
However, evidence suggests that students struggle to act upon the feedback they receive. 2 Ideally, students should be able to make use of feedback independently in order to support their future learning-which is part of a practice known as 'sustainable feedback'. 2 Numerous processes enable students to use feedback effectively. Self-assessment is a mechanism for one to identify ones' strengths and weaknesses. 3 Following self-assessment, students need to make appropriate goals for their learning, a process entitled 'self-regulation'. 2 Ideally, students should be able to make use of feedback independently in order to support their future learning.
However, setting goals is no guarantee of achieving them.
Self-efficacy-defined as the 'confidence to carry out the courses of action necessary to accomplish desired goals' 4 -is increasingly being recognised as a crucial determinant of success. [4][5][6] Video recordings of OSCE stations can help students engage with these feedback processes by providing students with a further opportunity to self-reflect on their performance. This video-assisted reflection (VAR) has been shown to help students improve their performance in subsequent OSCEs. 7 However, little research exists as to how this process occurs. Our study aimed to establish how VAR can help students develop more sustainable feedback practices by exploring their ability to self-assess and self-regulate, and by exploring their self-efficacy beliefs.

| ME THODS
We conducted a mixed methods evaluation to gauge the impact of VAR on student feedback processes. The impact of VAR on self-assessment and self-regulation was evaluated using quantitative measures, whereas self-efficacy was evaluated using qualitative approaches.
We invited 168 medical students from six different hospitals who were undertaking a mock-final OSCE to participate in the study.
Students were reminded that there was no obligation to take part and that non-participation would not impact upon their involvement in the mock examination. Twenty-one students consented to participate in the study. Figure 1 outlines the number of participants involved in each stage of the study.
During their mock OSCE, participating students had one of their OSCE stations filmed. The mock OSCE consisted of a six station OSCE, with five stations lasting 11 minutes, and one extended station of 22 minutes. Examiners gave each participant an overall score of 1-5 for their performance, as well as written feedback on areas that went well and areas to improve.
The examiners'score was recorded but not revealed to participating students until after the final reflection (Stage 3).
During their mock OSCE, participating students had one of their OSCE stations filmed.
After the OSCE station, participants completed a series of reflective forms; immediately after the OSCE (Stage 1), after verbal feedback from the station examiner (Stage 2) and 1 month later, after watching the video of their recorded station (Stage 3). The reflective forms required participants to predict their overall score for the station (on the same 5-point scale) and to identify areas for improvement.
The correlation between the examiners'score and the students' self-rated scores at each stage of reflection was completed with Pearson r coefficient and Fisher r-to-Z confidence intervals.
Wilcoxon signed-rank test was used to undertake one way, paired testing which allowed for separate comparisons of the word count of student reflections at each stage.
All study participants were invited to take part in a semi-structured interview aimed at exploring the student's perception of the VAR, and how watching the video helped them to prepare for their actual OSCE.
Two students participated in the semi-structured interviews. The interviews were transcribed by the researchers and member checking was performed. Researchers were present as both interviewers and moderators during the interview. Content codes were arranged into categories and themes 8 informed by self-efficacy theory.

| Self-assessment
There was no significant correlation between the participants' ini- There was no significant difference between the student and examiner score correlation at Stage 1 compared to Stage 2, as well as between Stage 2 and Stage 3. However, there was a significant difference between the student and examiner score correlation at Stage 3 compared to Stage 1.

| Self-regulation
The mean word count of student reflections immediately after the OSCE (Stage 1) was 4.05 words per reflection. This increased to 7.38 words per reflection after the examiner feedback (Stage 2) and to 21.1 words per reflection after the VAR (Stage 3). There was a statistically significant difference in word count between each stage of reflection (p < 0.01 for each difference) ( Figure 3).
Broadly, students reflected on similar topics across each stage of reflection; however, there was some evidence of increased content, and a trend towards more specific, measurable improvement points.

| Self-efficacy
Participants described how they found VAR-assisted feedback more objective, specific and honest, and that it helped them to identify more areas for improvement compared to traditional feedback. They described how VAR helped them set their own learning agendas,

Recruitment
168 medical students undertaking a mock-OSCE examinaƟon were invited to take part in the study.
21 students consented to parƟcipate in the study (n=21)

ReflecƟon Stage 2
AŌer the OSCE, students received 6 minutes of verbal feedback from the staƟon examiner and then completed a post-feedback reflecƟve form (n=21).

ReflecƟon Stage 3
One month later, students were invited to watch the video of their recorded OSCE staƟon and complete a final, video-assisted, reflecƟve form (n=12).

Semi-structured interviews
All parƟcipants were invited to take part in virtual, semi-structured interviews to explore thier views on the filming and reflecƟve process (n=2).

F I G U R E 1
The structure and timeline of the participant recruitment and reflective process enabled them to track their own progress and enhanced their ability to accept feedback.
Participants described how they found VAR-assisted feedback more objective, specific and honest.
One participant explained how they had used the VAR to benchmark their improvement after the OSCE and subsequently, how this had given them the confidence they needed to implement any learning goals. This link between a specific domain of action (OSCE performance) and increased confidence in their capability to accomplish a goal demonstrates improved self-efficacy as defined by Bandura. 5

| Self-assessment
The strongest correlation between student self-score and examiner score came after the VAR. However, it is difficult to fully interpret the impact of VAR on a student's ability to self-assess given the cumulative nature of the interventions. Student self-score Examiner score

Stage 3
Post-VAR self-score

| Self-regulation
The significant increase in the word count of the reflections after watching the videos (Stage 3) demonstrates that VAR allowed students to generate more specific, personalised learning goals.
Goals with greater specificity have been shown to be more effective in improving outcomes compared to more general ones 10 which may explain why VAR has been shown to improve performance in subsequent OSCEs. 7 A potential limitation of this analysis might rest on the assumption that a longer reflection is more specific. However, in interviews, the students described how they felt watching the videos allowed them to generate more specific and numerous learning points compared to the 'generic' feedback they received from examiners. In addition, the post-VAR mean word count was significantly higher than the post-feedback word

| Self-efficacy
In the interviews, students described the impact of VAR on student self-efficacy ( Table 2). Research has demonstrated that self-observation of good performance can enhance student selfefficacy, but observation of relative 'failure' can diminish it. 6 If the students who took part in this study were generally higher scoring candidates, this may explain why the VAR appeared to improve selfefficacy in these students.
Feedback from examiners provides students with an external perspective on their performance. We hypothesise that VAR allows students the opportunity and confidence to integrate these external perspectives with their own view of their performance. This idea was reinforced in the semi-structured interviews where students described being more confident and more likely to achieve any learning goals set out by examiners if they were also able to observe it TA B L E 2 Summary of the thematic analysis of the participants' semi-structured interviews

Feedback Components Specific Areas Summary Quotes
Self-Assessment Third-person perspective Students believed that watching the videos gave them a third-person perspective which allowed them to more objectively assess their performance can also involve unanticipated costs to the institution. 12 However, many institutions already have data storage platforms with protected access that can be used for this purpose, which we feel adjusts the cost-benefit ratio favourably. VAR involving patients will need to follow GMC guidelines on filming for secondary purposes, as well as data protection legislation. There may also be additional difficulties finding volunteer patients or actors who consent to OSCE stations being filmed. If a patient or actor is unwilling to be filmed, or if consent and data protection are prohibitively burdensome, there may still be considerable educational benefit in filming the student alone-such as in communication stations or the students' interaction with the examiner. We hope to examine the utility of VAR in other teaching settings such as communication skills teaching, during which we intent to utilise evidence-based feedback forms to standardise the feedback process. 13

| CON CLUS ION
In conclusion, we have demonstrated that VAR can help increase the sustainability, accuracy and actionability of the feedback process by improving student self-efficacy. This process is feasible to implement, requiring minimal equipment, and could be used in both formal educational settings such as clinical skills sessions and formative assessments.
We have demonstrated that VAR can help increase the sustainability, accuracy and actionability of the feedback process by improving student self-efficacy.

FU N D I N G
None.

CO N FLI C TO FI NTE R E S T
None.

E TH I C A LA PPROVA L
This study was given ethical approval by the Medical Education Ethics Committee (MEEC) at Imperial College London on the 19 December 2019 (ref MEEC1920-178).
When advertising the study, we emphasised that participation was entirely voluntary, and that non-participation would not harm them. Students interested in participating were asked to e-mail the investigators who then sent further written information. All participating students completed written consent forms and consented for any data gathered during the study, including from semi-structured interviews, to be anonymised and used for publication.
All participants were free to opt out of the study, or of their data being used, at any point. The study was conducted and data collected from January to April 2020.