Cut Down, Continue, Commence: Resident and Attending Perceptions of a Standardized, Face-to- Face Feedback System

Introduction: Standardized feedback systems, such as "Stop, Start, and Continue" can improve written feedback quality, but their effect on verbal feedback is not well known. Objective: To evaluate resident and attending perceptions of feedback before and after implementation of a standardized feedback model for inpatient rotations. Methods: N=65 internal medicine residents and N=16 attendings rotating on a general wards teaching service between 9/2018-12/2018 at a single academic center participated in this study. We evaluated the implementation of a standardized feedback system called the 3 C’s ("Continue, Cut-Down, Commence"). Participants were sent surveys regarding feedback preand postintervention. Survey items included perceptions of feedback frequency, quality, and duration. Results: Survey response rates were 52.3% for pre-intervention/post-intervention residents, 93.8% for preintervention attendings, and 81.3% for post-intervention attendings. Only 36% of attendings reported using a formalized system for in-person feedback before the intervention, while 76% of attendings reported using the 3 C’s method after the intervention. Residents reported higher satisfaction with verbal feedback (χ2=4.3, df=1, p=0.03) and feedback quality related to medical knowledge (χ2=4.31, df=1, p=0.037) following the intervention. Residents reported participating in verbal feedback less often than attendings in both the pre-intervention (p=0.003) and postintervention periods (p<0.001). Attendings did not report a perceived improvement in feedback frequency or quality following the intervention. Discussion: Standardized feedback systems may improve resident satisfaction with verbal feedback without significantly impacting attending perceptions of feedback quality. Even after the implementation of standardized feedback, residents may perceive they participate in verbal feedback less frequently than attendings. Liang E, Singhal S, Yang J, Kumar A MedEdPublish https://doi.org/10.15694/mep.2021.000001.1 Page | 2


Introduction
Feedback is critical for effective learning and professional development (Ende, 1983;Ericsson, 2008). Consistent and systematic feedback improves clinical performance across multiple learning environments (Veloski et al., 2006). In recognition of the importance of feedback, the Accreditation Council for Graduate Medical Education (ACGME) requires formative feedback in addition summative evaluations for residency training (Common Program Requirements, no date). Unfortunately, formative feedback may be underutilized in graduate medical education (GME;Sender Liberman et al., 2005;Yarris et al., 2009;Hajar, Wanat and Fett, 2020).
Previous approaches to address formative feedback during residency training have utilized the implementation of standardized feedback systems (Peccoralo et al., 2012;Wilkinson et al., 2013). These systems can include faculty development (Wilkinson et al., 2013), conversation guides (Peccoralo et al., 2012), or standardized templates (Peccoralo et al., 2012;Hoon et al., 2015;Albano et al., 2019). There is growing evidence that these methods may improve trainee performance, but their implementation is not well studied (Veloski et al., 2006). In this brief report, we discuss the implementation of a standardized feedback system called the "3 C's: Cut-Down, Continue, and Commence" and compare its effectiveness to our pre-existing system.

Study Participants & Setting
This was a prospective cohort study that evaluated internal medicine residents' and attendings' perceptions of feedback at a single academic medical center. Residents and attending hospitalists on our inpatient wards teaching service between 9/2018-12/2018 were included. Our Institutional Review Board reviewed and approved the study.
Prior to the intervention, our institution did not utilize a standardized face-to-face feedback system. Resident and attending evaluations were completed online (www.medhub.com) at the end of the four-week rotation. It was at the attendings' discretion to provide additional face-to-face feedback.

Resident Feedback System
This study evaluated the implementation of an standardized feedback system called "Stop, Start, and Continue," (Hoon et al., 2015). This model has been shown to improve the depth and quality of written feedback, but it has been less studied for face-to-face feedback (Hoon et al., 2015). We adapted this model into the "3 C's" system for resident feedback: "Cut-Down, Commence, and Continue" (Figure 1). The term "stop" was adjusted to "cutdown" as this follows the best-practices of feedback by making it non-threatening and more actionable (Schwartz, Tsang and Blair, 2016). To make the system easier to memorize, we used the letter "C" to describe the other feedback components.
At the beginning of the intervention, all attendings received an in-person orientation to this feedback system. Due to scheduling constraints, the residents received an orientation email as opposed to in-person orientation. Both groups were given pocket feedback cards to serve as a reminder ( Figure 1) and were sent weekly email reminders to use this standardized system.  Figure Footnote: This study utilized an abbreviated feedback system akin to the "Stop, Start and Continue" method called the 3 C's (above). Stop was modified to Cut-Down to allow the feedback to be more revisable and nonthreatening.

Measurements
Both groups were sent electronic surveys pre-and post-intervention to assess their perceptions of feedback (Supplementary File 1). These surveys included perceptions of the frequency and quality of feedback as related to the ACGME core competencies. All post-intervention surveys were sent at the conclusion of the intervention.

Statistical Analysis
All surveys utilized a symmetric 5-point Likert scale. Categorical variables were dichotomized by Likert values of 4 & 5 vs. all others and were analyzed via Chi-Square Analysis. Continuous variables (e.g. frequency of feedback) were analyzed using Mann Whitney test. All analyses were performed using R (Vienna, Austria).
Despite feeling more satisfied with face-to-face feedback, the post-intervention residents did not report higher quality of positive feedback (χ2=0.44, df=1, p=0.51) or constructive feedback (χ2=1.06, df=1, p=0.30). Postintervention residents reported significantly improved quality of feedback related to the ACGME Milestone for Medical Knowledge (χ2=4.31; df=1; p=0.037). Post-intervention resident perceptions of feedback quality did not improve for the other ACGME milestones (Table 1). Attendings did not feel they gave higher-quality feedback across any sub-domains following the intervention period (Table 1). Interestingly, only 21% of residents preferred the 3 C's method (46% unsure), while 31% of attendings preferred the 3 C's (62% unsure) to their usual method of feedback.  (p<0.001). The majority of respondents in both groups agreed the average face-to-face feedback session was 5-10 minutes in length before and after the intervention (pre-intervention residents: 54.5%, post-intervention residents: 50.0%, pre-intervention attendings: 71.4%, post-intervention attendings: 76.9%).

Discussion
In this study, we assessed a standardized feedback system called the 3 C's ("Cut Down, Commence and Continue") in terms of resident and attending perceptions of feedback quality, frequency and duration. Residents reported higher satisfaction with face-to-face feedback after this system was implemented, while attendings reported no difference. Despite this improvement, residents did not strongly prefer the 3 C's method. Furthermore, residents and attendings did not report improved quality of feedback across several domains, including the ACGME milestones. Finally, residents and attendings differed significantly in their perception of how often feedback was given.
Previous authors have shown that higher trainee satisfaction with feedback may not relate to the actual quality of feedback (Boehler et al., 2006). Additionally, there are well-reported discrepancies between resident and attending perceptions of feedback in terms of quality, frequency, or duration (Sender Liberman et al., 2005;Carr et al., 2018;Albano et al., 2019). As suggested by this study, the implementation of a standardized feedback system like the 3 C's may not immediately improve these issues. More abbreviated feedback methods like "Stop, Start, and Continue" can improve written feedback quality, but it is unclear if they can improve verbal feedback quality (Hoon et al., 2015). Feedback should be specific, timely, unambiguous, non-threatening, and revisable (Schwartz, Tsang and Blair, 2016), but formal feedback systems may be employed at the end of the rotation where the feedback is no longer timely or immediately revisable. Our intervention encouraged frequent feedback, which may have improved resident satisfaction. However, its abbreviated nature may have made it challenging to improve feedback quality around more complex topics such as ACGME milestones.
Limitations of this study include its small sample size, potential selection bias from surveys, and lack of randomization. The post-intervention surveys were sent at the end of the intervention period, and the results may be subject to recall bias. Finally, trainee or attending perceptions of feedback quality are imperfect measures. Future studies should use standardized methodologies to evaluate feedback quality.

Conclusion
Standardized feedback systems like "Cut-Down, Commence, and Continue" may improve resident satisfaction with feedback without improving the perceived quality of feedback. Residents may have differing perceptions of feedback frequency compared to their attendings. Together, these results suggest potential caveats to more abbreviated feedback systems for verbal feedback and add to the growing literature on the nature of feedback in

Take Home Messages
Residents and attending physicians may have differing perceptions of feedback frequency Standardized verbal feedback systems like may improve resident satisfaction with feedback, but the overall quality of feedback may not improve.