Association for Surgical Education
Use of decision-based simulations to assess resident readiness for operative independence

https://doi.org/10.1016/j.amjsurg.2014.10.002Get rights and content

Abstract

Background

Recent literature has called into question resident readiness for operative independence at the end of general surgery training.

Methods

We used a simulation-based exit examination to assess resident readiness. Six chief residents performed 3 simulated procedures: bowel anastomosis, laparoscopic ventral hernia (LVH) repair, and pancreaticojejunostomy. Faculty assessed resident performance using task-specific checklists, Objective Structured Assessment of Technical Skills (OSATS), and final product analysis.

Results

Residents' individual task-specific checklist scores ranged from 25% to 100% across all 3 procedures. Mean OSATS scores ranged from 4.06 to 4.23/5.0. Residents scored significantly higher on “instrument knowledge” (mean = 4.78, standard deviation [SD] = 23) than “time and motion” (mean = 3.94, SD = .48, P = .025) and “ability to adapt to individual pathologic circumstances” (mean = 4.06, SD =.12, P = .002). Final product analysis revealed a range of errors, including incorrect technique and poor intraoperative planning.

Conclusions

Despite relatively high OSATS ratings, residents had a wide range of errors and procedure outcomes. Exit assessments using multiple evaluation metrics may improve awareness of residents' learning needs.

Section snippets

Setting and participants

Study participants were graduating chief residents at an academic general surgery training program. Data collection occurred in a single setting approximately 1 month before graduation. This study was reviewed by the University of Wisconsin Health Sciences Institutional Review Board and was approved as a no more than minimal risk study. Written informed consent was obtained from all participants.

Simulated procedures

Chief residents performed 3 simulated surgical procedures. The procedures were selected to assess

Demographics

Six graduating chief residents completed this study. Half of the study participants were men. All the graduating residents planned to complete fellowships. Fellowship specialties included critical care (n = 2), minimally invasive surgery (n = 1), and colorectal surgery (n = 3). The participants logged total major cases ranging from 870 to 1,162 (mean = 974.8; standard deviation [SD] = 103.1). They also logged an average of 15.0 (SD = 2.6) LVH repairs; 20.7 (SD = 6.1) open small bowel

Comments

New requirements from ABS for documenting graduating resident skills in the clinical and operative environments have highlighted the need for standardized assessments of resident operative performance. In this study, 6 graduating chief residents performed 3 simulated operative procedures: LVH repair, bowel anastomosis, and pancreaticojejunostomy. The aim of this study was to assess graduating resident readiness for operative independence on 3 simulated procedures using multiple assessment

Acknowledgements

Funding for this study came from the Department of Defense grant no. W81XWH-13-1-0080 entitled, “Psycho-motor and Error Enabled Simulations: Modeling Vulnerable Skills in the Pre-Mastery Phase.”

References (26)

  • C.M. Pugh et al.

    Intra-operative decision making: more than meets the eye

    J Biomed Inform

    (2011)
  • J.D. Richardson

    ACS transition to practice program offers residents additional opportunities to hone skills

    Bull Am Coll Surgeons

    (2013)
  • B.D. Hoyt

    Executive Director’s report: looking forward—February 2013

    Bull Am Coll Surgeons

    (2013)
  • Cited by (0)

    Funding for this study came from the Department of Defense grant number W81XWH-13-1-0080 entitled, “Psycho-motor and Error Enabled Simulations: Modeling Vulnerable Skills in the Pre-Mastery Phase”.

    View full text