Original ReportsA Randomized Controlled Trial to Compare e-Feedback Versus “Standard” Face-to-Face Verbal Feedback to Improve the Acquisition of Procedural Skill
Introduction
Feedback can be defined as “specific information about the comparison between a trainee’s performance and a standard, given with the intent to improve the traineeʼs performance.1” Such feedback is fundamental for reinforcing learning when teaching procedural skills. The intended effect of this feedback is to help students and trainees learn and improve their performance. It is based on the assumption that feedback creates awareness of shortcomings and thereby motivates learners to improve or change.2 In order for the feedback to be effective, it should be specific, objective, documented, and promote a specific learning goal.3 In addition, it should focus on the process as feedback relating to the personal level is rarely effective.4 In the context of procedural skills, feedback is usually based on objective assessment, during or following direct observation, using a structured pro forma in a summative setting such as an Objective Structured Clinical Examination or in a formative setting such as an Objective Structured Assessment of Technical Skill (OSATS).5 Standard feedback is delivered verbally after direct observation of the procedural skill; however, this requires the physical presence of trained faculty, which can be difficult to arrange in a busy clinical environment. A recent student survey6 revealed that across the UK, and in all undergraduate curricula, students are unhappy with the amount of feedback they receive from their respective faculty, yet most demonstrate good insight and empathize at the difficulties teachers encounter in providing effective feedback.7 Both teachers and students recognize that time and resources are limiting factors, which can make individualizing feedback difficult. Technology has been implemented in various applications in training and simulation. Ericsson called for procedures to be video recorded for educational and research purposes which offer a new perspective for medical education, including residency training and continuing education.8 Personal review of a recorded skill has been shown to improve the acquisition of procedural skill.9 However, using video recording to provide remote e-feedback by a trainer has not been investigated before. This has the potential benefit of overcoming time and cost barriers for providing faculty. We present a randomized controlled trial (RCT) that aimed to compare e-feedback with standard verbal feedback during the acquisition of a basic procedural skill by surgical novices. We hypothesized that e-feedback is acceptable and equally effective to face-to-face feedback in improving suturing skills for novices. A validated pro forma was used to standardize the assessment and provide feedback. Minor modifications to the checklist were performed to make the pro forma applicable to the context of novices performing surgical suturing. An integral factor of an OSATS is the use of predefined pro formas against which the performance of a specific surgical skill can be measured and subsequent constructive feedback can be provided.10
Section snippets
Methods
This prospective RCT was conducted over 3 days in February 2015 at the University of Sheffield. Ethical approval for the study was sought and granted via the University of Sheffield Ethics Committee process. The study participants were 38 undergraduate medical students who were assigned to an Integrated Learning Activity relating specifically to surgical skills as part of a Student Selected Component of the undergraduate curriculum. All students were informed that the Integrated Learning
Results
Thirty-eight participants from the final year of the medical undergraduate degree course at the University of Sheffield, UK, were included in the study (Fig. 4). One student in group 2 did not attend the second day and was therefore excluded from the analysis. There was a significant improvement in the overall mean score for all participants on the second performance of the task (first performance mean = 11.59, second performance mean = 15.95; p ≤ 0.0001) (Table 1). From a maximum achievable
Discussion
This RCT compared e-feedback with standard feedback as a means of improving suturing skills among medical students. The results were assessed quantitatively using a modified OSATS assessment tool and revealed no significant difference between the 2 groups, although the performance has improved in both groups compared to baseline. In addition, the study revealed good interrater reliability of video-based assessments, and acceptability of e-feedback among the participants.
Feedback in medical
Conclusion
Our study demonstrated that e-feedback after watching a video recording was acceptable among surgical novices and was not quantitatively different from standard feedback in improving suturing skills. Such an application of technology in surgical education requires further investigation to assess whether it can be projected to other sets of surgical procedures and postgraduate trainees.
Acknowledgment
The authors thank Dr Nour Obeidat for her advice and assistance with the statistical analysis, D. Equeall (Department of Medical Illustration) for his assistance in video recording, and the staff at the Clinical Skills Centre, Northern General Hospital. They also thank the medical undergraduates of the Medical School of the University of Sheffield for their participation in this study.
References (23)
- et al.
Video assessment of basic surgical trainees’ operative skills
Am J Surg
(2008) Setting standards for the assessment of operative competence
Eur J Vasc Endovasc Surg
(2005)The effects of delay of feedback on a delayed concept formation transfer task
Contemp Educ Psychol
(1992)- et al.
What is feedback in clinical education?
Med Educ
(2008) - et al.
Impact of workplace-based assessment on doctors’ education and performance: a systematic review
Br Med J
(2010) Focus on formative feedback
Rev Educ Res
(2008)- et al.
The power of feedback
Rev Edu Res
(2007) - et al.
Objective assessment of technical surgical skills
Br J Surg
(2010) - National Student Survey. Findings and trends 2006-2010 [Internet]....
- et al.
Student perceptions and preferences for feedback
Asian Soc Sci
(2009)
Necessity is the mother of invention: video recording firsthand perspectives of critical medical procedures to make simulated training more effective
Acad Med
Cited by (19)
Virtual Interactive Surgical Skills Classroom: A Parallel-group, Non-inferiority, Adjudicator-blinded, Randomised Controlled Trial (VIRTUAL)
2022, Journal of Surgical EducationCitation Excerpt :We ensured that any resulting differences in intervention efficacy were identified by employing a consistent method of objective proficiency assessment. Al-Jundi et al. previously compared distanced e-feedback via video-communication to face-to-face verbal feedback and concluded that the two mechanisms are similarly effective for basic surgical skills improvement.29 Our sample was recruited from across all years of study at medical school and was demographically representative of the background medical student population with respect to sex and hand dominance.30,31
Changing the Status Quo: Developing a Virtual Sub-Internship in the Era of COVID-19
2021, Journal of Surgical EducationCitation Excerpt :The ability to practice skills such as suturing has been shown to improve medical student comfort in clinical situations, and we felt it a key part of a surgical rotation that students have the opportunity to improve technical skills.21 Although the lack of in-person teaching could be perceived as a challenge, studies have shown that video feedback is effective in the development of basic surgical skills, such as suturing.22,23 Rather than redeveloping our own videos and modules, we decided to incorporate the Surgery Resident Skills Curriculum and the Medical Student Simulation-Based Surgical Skills Curriculum, which were developed and validated by the American College of Surgeon (ACS), the Association of Program Directors in Surgery (APDS), and the Association for Surgical Education (ASE).24-26
Stepwise Training in Laparoscopic Surgery for Complex Ileocolonic Crohn's Disease: Analysis of 127 Training Episodes
2019, Journal of Surgical EducationCitation Excerpt :The time needed for constructive feedback and structured LCAT assessment has not been directly measured as a study end-point, mainly due to the trainer/assessor being scrubbed in theatre for the whole procedure. Median operating time of 152 minutes represents a surrogate of video length for review and assessment; however, further studies are needed to explore the role of e-feedback for procedural skills acquisition in surgery.26 Increased rates of adverse clinical outcomes at the early stage of the learning curve raise ethical questions and highlight the need for mechanisms to reduce complications and conversions during the initial stage of independent practice.