Elsevier

Journal of Surgical Education

Volume 74, Issue 3, May–June 2017, Pages 390-397
Journal of Surgical Education

Original Reports
A Randomized Controlled Trial to Compare e-Feedback Versus “Standard” Face-to-Face Verbal Feedback to Improve the Acquisition of Procedural Skill

https://doi.org/10.1016/j.jsurg.2016.11.011Get rights and content

Background

Constructive feedback plays an important role in learning during surgical training. Standard feedback is usually given verbally following direct observation of the procedure by a trained assessor. However, such feedback requires the physical presence of expert faculty members who are usually busy and time-constrained by clinical commitments. We aim to evaluate electronic feedback (e-feedback) after video observation of surgical suturing in comparison with standard face-to-face verbal feedback.

Methods

A prospective, blinded, randomized controlled trial comparing e-feedback with standard verbal feedback was carried out in February 2015 using a validated pro formas for assessment. The study participants were 38 undergraduate medical students from the University of Sheffield, UK. They were recorded on video performing the procedural skill, completed a self-evaluation form, and received e-feedback on the same day (group 1); observed directly by an assessor, invited to provide verbal self-reflection, and then received standard verbal feedback (group 2). In both groups, the feedback was provided after performing the procedure. The participants returned 2 days later and performed the same skill again. Poststudy questionnaire was used to assess the acceptability of each feedback among the participants.

Results

Overall, 19 students in group 1 and 18 students in group 2 completed the study. Although there was a significant improvement in the overall mean score on the second performance of the task for all participants (first performance mean 11.59, second performance mean 15.95; p ≤ 0.0001), there was no difference in the overall mean improvement score between group 1 and group 2 (4.74 and 3.94, respectively; p = 0.49). The mean overall scores for the e-feedback group at baseline recorded by 2 independent investigators showed good agreement (mean overall scores of 12.84 and 11.89; Cronbach α = 0.86). Poststudy questionnaire demonstrated that both e-feedback and standard verbal feedback achieved high mean Likert grades as recorded by the participants (4.42 [range: 2-5] and 4.71 [range: 4-5], respectively; p = 0.274).

Conclusion

e-Feedback after watching a video recording appears to be acceptable and is not quantitatively different than standard feedback in improving suturing skills among novice trainees. Video assessment of procedural skills is reliable.

Introduction

Feedback can be defined as “specific information about the comparison between a trainee’s performance and a standard, given with the intent to improve the traineeʼs performance.1” Such feedback is fundamental for reinforcing learning when teaching procedural skills. The intended effect of this feedback is to help students and trainees learn and improve their performance. It is based on the assumption that feedback creates awareness of shortcomings and thereby motivates learners to improve or change.2 In order for the feedback to be effective, it should be specific, objective, documented, and promote a specific learning goal.3 In addition, it should focus on the process as feedback relating to the personal level is rarely effective.4 In the context of procedural skills, feedback is usually based on objective assessment, during or following direct observation, using a structured pro forma in a summative setting such as an Objective Structured Clinical Examination or in a formative setting such as an Objective Structured Assessment of Technical Skill (OSATS).5 Standard feedback is delivered verbally after direct observation of the procedural skill; however, this requires the physical presence of trained faculty, which can be difficult to arrange in a busy clinical environment. A recent student survey6 revealed that across the UK, and in all undergraduate curricula, students are unhappy with the amount of feedback they receive from their respective faculty, yet most demonstrate good insight and empathize at the difficulties teachers encounter in providing effective feedback.7 Both teachers and students recognize that time and resources are limiting factors, which can make individualizing feedback difficult. Technology has been implemented in various applications in training and simulation. Ericsson called for procedures to be video recorded for educational and research purposes which offer a new perspective for medical education, including residency training and continuing education.8 Personal review of a recorded skill has been shown to improve the acquisition of procedural skill.9 However, using video recording to provide remote e-feedback by a trainer has not been investigated before. This has the potential benefit of overcoming time and cost barriers for providing faculty. We present a randomized controlled trial (RCT) that aimed to compare e-feedback with standard verbal feedback during the acquisition of a basic procedural skill by surgical novices. We hypothesized that e-feedback is acceptable and equally effective to face-to-face feedback in improving suturing skills for novices. A validated pro forma was used to standardize the assessment and provide feedback. Minor modifications to the checklist were performed to make the pro forma applicable to the context of novices performing surgical suturing. An integral factor of an OSATS is the use of predefined pro formas against which the performance of a specific surgical skill can be measured and subsequent constructive feedback can be provided.10

Section snippets

Methods

This prospective RCT was conducted over 3 days in February 2015 at the University of Sheffield. Ethical approval for the study was sought and granted via the University of Sheffield Ethics Committee process. The study participants were 38 undergraduate medical students who were assigned to an Integrated Learning Activity relating specifically to surgical skills as part of a Student Selected Component of the undergraduate curriculum. All students were informed that the Integrated Learning

Results

Thirty-eight participants from the final year of the medical undergraduate degree course at the University of Sheffield, UK, were included in the study (Fig. 4). One student in group 2 did not attend the second day and was therefore excluded from the analysis. There was a significant improvement in the overall mean score for all participants on the second performance of the task (first performance mean = 11.59, second performance mean = 15.95; p ≤ 0.0001) (Table 1). From a maximum achievable

Discussion

This RCT compared e-feedback with standard feedback as a means of improving suturing skills among medical students. The results were assessed quantitatively using a modified OSATS assessment tool and revealed no significant difference between the 2 groups, although the performance has improved in both groups compared to baseline. In addition, the study revealed good interrater reliability of video-based assessments, and acceptability of e-feedback among the participants.

Feedback in medical

Conclusion

Our study demonstrated that e-feedback after watching a video recording was acceptable among surgical novices and was not quantitatively different from standard feedback in improving suturing skills. Such an application of technology in surgical education requires further investigation to assess whether it can be projected to other sets of surgical procedures and postgraduate trainees.

Acknowledgment

The authors thank Dr Nour Obeidat for her advice and assistance with the statistical analysis, D. Equeall (Department of Medical Illustration) for his assistance in video recording, and the staff at the Clinical Skills Centre, Northern General Hospital. They also thank the medical undergraduates of the Medical School of the University of Sheffield for their participation in this study.

References (23)

  • K.A. Eriscsson

    Necessity is the mother of invention: video recording firsthand perspectives of critical medical procedures to make simulated training more effective

    Acad Med

    (2014)
  • Cited by (19)

    • Virtual Interactive Surgical Skills Classroom: A Parallel-group, Non-inferiority, Adjudicator-blinded, Randomised Controlled Trial (VIRTUAL)

      2022, Journal of Surgical Education
      Citation Excerpt :

      We ensured that any resulting differences in intervention efficacy were identified by employing a consistent method of objective proficiency assessment. Al-Jundi et al. previously compared distanced e-feedback via video-communication to face-to-face verbal feedback and concluded that the two mechanisms are similarly effective for basic surgical skills improvement.29 Our sample was recruited from across all years of study at medical school and was demographically representative of the background medical student population with respect to sex and hand dominance.30,31

    • Changing the Status Quo: Developing a Virtual Sub-Internship in the Era of COVID-19

      2021, Journal of Surgical Education
      Citation Excerpt :

      The ability to practice skills such as suturing has been shown to improve medical student comfort in clinical situations, and we felt it a key part of a surgical rotation that students have the opportunity to improve technical skills.21 Although the lack of in-person teaching could be perceived as a challenge, studies have shown that video feedback is effective in the development of basic surgical skills, such as suturing.22,23 Rather than redeveloping our own videos and modules, we decided to incorporate the Surgery Resident Skills Curriculum and the Medical Student Simulation-Based Surgical Skills Curriculum, which were developed and validated by the American College of Surgeon (ACS), the Association of Program Directors in Surgery (APDS), and the Association for Surgical Education (ASE).24-26

    • Stepwise Training in Laparoscopic Surgery for Complex Ileocolonic Crohn's Disease: Analysis of 127 Training Episodes

      2019, Journal of Surgical Education
      Citation Excerpt :

      The time needed for constructive feedback and structured LCAT assessment has not been directly measured as a study end-point, mainly due to the trainer/assessor being scrubbed in theatre for the whole procedure. Median operating time of 152 minutes represents a surrogate of video length for review and assessment; however, further studies are needed to explore the role of e-feedback for procedural skills acquisition in surgery.26 Increased rates of adverse clinical outcomes at the early stage of the learning curve raise ethical questions and highlight the need for mechanisms to reduce complications and conversions during the initial stage of independent practice.

    View all citing articles on Scopus
    View full text