A pilot clinical skills coaching program to reimagine remediation: a cohort study

Background New approaches are needed to improve and destigmatize remediation in undergraduate medical education (UME). The COVID-19 pandemic magnified the need to support struggling learners to ensure competency and readiness for graduate medical education (GME). Clinical skills (CS) coaching is an underutilized approach that may mitigate the stigma of remedial learning. Methods A six-month CS coaching pilot was conducted at Harvard Medical School (HMS) as a destigmatized remedial learning environment for clerkship and post-clerkship students identified as ‘at risk’ based on objective structured clinical examinations (OSCE). The pilot entailed individual and group coaching with five faculty, direct bedside observation of CS, and standardized patient encounters with video review. Strengths-based coaching principles and appreciative inquiry were emphasized. Results Twenty-three students participated in the pilot: 14 clerkship students (cohort 1) and 9 post-clerkship students (cohort 2). All clerkship students (cohort 1) demonstrated sustained improvement in CS across three OSCEs compared to baseline: at pilot close, at 6-months post pilot, and at 21-24 months post-pilot all currently graduating students (10/10, 100%) passed the summative OSCE, an HMS graduation requirement. All post-clerkship students (cohort 2) passed the HMS graduation OSCE (9/9,100%). Feedback survey results included clerkship students (9/14; 64%) and post-clerkship students (7/9; 78%); all respondents unanimously agreed that individual coaching was “impactful to my clinical learning and practice”. Faculty and leadership fully supported the pilot as a destigmatized and effective approach to remediation. Conclusion Remediation has an essential and growing role in medical schools. CS coaching for remedial learning can reduce stigma, foster a growth mindset, and support sustained progress for ‘at risk’ early clerkship through final year students. An “implementation template” with suggested tools and timelines can be locally adapted to guide CS coaching for UME remediation. The CS coaching pilot model is feasible and can be generalized to many UME programs.


Introduction
Remediation programs constitute a vital resource for medical students who are at risk of insufficiently advancing towards clinical competence milestones [1][2][3][4] .Yet remediation is commonly fraught with stigma that both limits its acceptance and impedes effectiveness 5 .Discontinuation of the United States (U.S.) Medical Licensing Examination Clinical Skills (CS) Step 2 examination, which had a 5% failure rate 6 , highlights the urgency for medical schools to identify and remediate 'at risk' students and certify clinical competence and readiness to advance to graduate medical education (GME) 7 .A recent survey of American Medical Association (AMA) Accelerating Change in Medical Education consortium schools found that among 25 medical school coaching programs, only three programs focused on remediation and two programs focused on clinical skills development, the majority of programs focused on academic and professional development 8 .Coaching programs less commonly focus on CS development or remediation in UME, though these are areas of growing research 9,10 .CS coaching offers a promising avenue to transform remediation as an explicitly learner-driven process that can address this gap and provide destigmatized clinical learning support in medical school 8,11,12 .A coach can serve as a non-judgmental advocate who provides a positive environment where a struggling student is prompted to reflect on strengths, co-create meaningful learning, and build clear and achievable steps towards success 11,13 .Advances in remediation are essential, notably as the COVID-19 pandemic has magnified the need to support struggling learners and to ensure competency across the UME to GME continuum 14,15 .The aim of this pilot was to establish the feasibility of CS coaching as normalized remedial support that allowed students to simultaneously engage in medical school classes and clerkships while improving clinical skills.

Ethics
All studies that involve Harvard Medical School (HMS) students and/or teaching faculty as study subjects or analysis of existing data related to HMS students require the approval of the HMS Program in Medical Education (PME) Educational Scholarship Review Committee.The Harvard Longwood Campus IRB has given this committee authority to designate studies it deems quality improvement to be IRB exempt.The clinical skills coaching pilot met the criteria for a quality improvement project and was deemed IRB exempt and participant consent was waived as part of this exemption.
A six-month CS coaching pilot was developed at Harvard Medical School (HMS) to promote a destigmatized learning environment for clerkship and post-clerkship students identified as 'at risk' based on objective structured clinical examination (OSCE) performance.Students were identified as 'at risk' based on low OSCE performance and/or concerns on narrative evaluations.A 'below passing' OSCE performance was the lowest fifteenth percent of scores for the clerkship cohort (cohort 1).Two students in the clerkship cohort 1 with OSCE scores at the mean on pilot entry were identified based on narrative evaluations.[Table 1] The post-clerkship cohort (cohort two) was identified based scores below the passing threshold score of 68% on the HMS summative graduation OSCE.The 68% total passing score for the HMS summative graduation OSCE was calculated using a modified Hofstee method 16 that incorporates historical data of students' performance and a faculty mean score based on expectations for minimum and maximum passing scores and percentages.
The pilot entailed three overlapping phases: faculty development; co-production of student individualized learning plans (ILPs) coupled with group and individualized coaching; and post-pilot transition planning.Faculty development and coaching occurred from July to December 2019, and post-pilot longitudinal OSCE follow-up spanned two years (January 2020 to December 2021).A total of six CS focus areas corresponded to elements of the Association of American Medical Colleges (AAMC) Core Entrustable Professional Activities (EPAs) for Entering Residency: history-taking, physical examination, clinical reasoning, oral presentation, communication, and learning on a clinical team 17 .
A coaching team was identified by HMS leadership for their CS expertise and included five faculty from the core HMS OSCE faculty based on their diverse roles as experienced clinical educators, interest in coaching, and respective clinical training sites and disciplines.Faculty development focused on appreciative inquiry methods and a strengths-based coaching framework to engage each student in goal-based progress and support their development as master adaptive learners (Figure 1) [18][19][20] .These coaching frameworks were chosen over models of behavior change to promote self-reflection, self-directed learning, and co-creation of learning goals.This

Amendments from Version 1
Manuscript Version 2 incorporates feedback from two peer reviewers, Dr. Kalman Winston and Dr. Calvin Chou.Point-by-point responses to each set of queries are included in the right-hand column that accompanies the Version 2 manuscript.The Version 2 manuscript provides more specific information on the following: clarification of the identification of 'at risk' students in both cohort 1 (clerkship students) and cohort 2 (post-clerkship students) based on objective structured clinical examinations (OSCE) performance, rationale for choosing the appreciative inquiry and strengths-based coaching frameworks, and differentiation between this coaching pilot and coaching programs previously described in the literature.Additionally, this updated manuscript includes more information on students in the pilot who are underrepresented in medicine.An updated Figure 2 clarifies that the post-clerkship (cohort 2) students participated in one group coaching session, rather than multiple sessions.More detail is also provided on standardized patient and core educator faculty OSCE checklist scoring with a discussion of the risks for and safeguards against unconscious and implicit bias in the Harvard Medical School OSCE program.The discussion section describes a "prevention strategy" planned for this academic year that will offer optional student workshop sessions to all clerkship students based on the three group sessions for cohort 1 clerkship students described in the pilot.meetings with students and iterative updates to student individualized learning goals made by program leadership and students working together, as well as student feedback surveys (Figure 3 and Figure 4).The pilot closed at sixmonths with student transition planning which included the identification of student goals for clerkship rotations

Results and analysis
A total of twenty-three students participated in the pilot: 14 clerkship students (cohort one) and 9 post-clerkship students (cohort two).There were 10 female and 13 male students across both cohorts.Students who were underrepresented in medicine made up less than 20% of both cohorts in the pilot.While group coaching sessions were required, optional hour-long individual coaching sessions were utilized by 57% (8/14) of clerkship students (24 total sessions, range two-seven sessions/student) and 44% (4/9) of post-clerkship students (six total sessions, range one-two sessions/student) during the pilot.Notably, 86% (12/14) of clerkship students participated in optional individual coaching during the combined pilot and post-pilot phases.Optional individual clerkship student (cohort 1) check-ins occurred in person, over email, and by phone and were offered monthly, with a range of 0-4 check-in meetings per clerkship student.
Pilot outcomes are summarized in Table 1.Five main outcomes include: OSCE assessment, student feedback on group and individual coaching, student feedback on coaching to normalize remediation, faculty satisfaction and feedback, and demonstrated feasibility as a program.
All clerkship students (cohort one) demonstrated improvement in CS from the baseline OSCE across two subsequent formative OSCE assessments (Figure 5): at the close of the six-month pilot (corresponding to clerkship month three) and at six months post pilot (corresponding to clerkship month 9).Scores that were at or above the mean increased from 14% (2/14) on the pilot entry (pre-clerkship) OSCE, to 43% (6/14) at the close of the six-month pilot (clerkship month three), and 50% (7/14) six-months post pilot (clerkship month nine).Two years post pilot, all eligible clerkship (cohort one) students passed the summative HMS graduation OSCE (10/10, 100%).The remaining students (4/14, 29%) will be eligible for this OSCE prior to graduation after planned research time and/or concurrent degree programs.All post-clerkship students in cohort two (9/9, 100%) passed the summative OSCE retake.
A brief student feedback survey (cohorts one and two, Figure 3 and Figure 4 respectively) highlighted strengths of group and individual coaching (Table 1).All clerkship students who responded to survey questions targeted at normalizing remediation (9/14; 64%) were "likely" to "very likely" to recommend CS coaching to peers, and all responding post-clerkship students (7/9; 78%) rated the value of CS coaching as "helpful" to "very helpful".History-taking practice and standardized patient (SP) encounter with coaching feedback were identified as most impactful for the clerkship (cohort one) group sessions.OSCE-type SP practice with coaching were most framework aligned with the pilot's CS coaching goals which aimed to engage students and promote lifelong learning strategies.Monthly faculty team conferences debriefed the coaching sessions' successes and challenges to ensure all coaches engaged in productive and growth-empowered remediation.
Coaches were compensated at the school's usual teaching compensation rate.
A flow diagram details the planning, implementation, and evaluation of the CS coaching pilot (Figure 2).CS learning support included a range of structured activities adaptable to identified student needs (Figure 2).A coaching agreement upheld CS coaching as a non-evaluative confidential experience.
Clerkship students (cohort 1) attended three three-hour required group coaching sessions and could opt for individual coaching during clerkships.Individual sessions were optional to avoid overburdening students and to gauge student engagement in co-creating their learning.Assessment of the clerkship cohort's CS included two formative OSCEs, each with three patient encounters incorporating immediate faculty feedback.These formative OSCEs were part of the routine HMS schedule at three and nine months after clerkship entry, corresponding to the close of the coaching pilot and six months post-pilot.A summative OSCE with nine patient encounters (and no formative feedback) required for graduation corresponded to two years post-pilot.
Each post-clerkship student attended one three-hour required group coaching session with optional individual coaching.Reassessment entailed a summative OSCE retake with new clinical case scenarios.
Student engagement and feedback was monitored throughout the pilot as key indicators of creating a normalized learning support environment.This was done through frequent check-in

Faculty satisfaction & reflections
• Created a safe space for student learning.
• Created an opportunity for reflection on clinical skills gaps.
• Responsive to rapid time frame, program and individual learning needs • Dynamic faculty engagement to continually align priorities.impactful for the post-clerkship cohort two group sessions.Survey respondents (cohort one and two) unanimously agreed that individual coaching was "impactful to my clinical learning and practice".
Faculty and HMS leadership fully supported the pilot as a destigmatized and effective approach to remediation.Feasibility was demonstrated by student progress, sustained student and faculty engagement across and beyond the six-month pilot, and subsequent approval as the "HMS Clinical Learning Coaching Program" to provide longitudinal clinical learning support.

Discussion and conclusions
Remediation has an essential and growing role in medical schools 21 .This pilot demonstrates that a CS coaching program is a feasible and scalable way to reconceptualize remediation, mitigate perceived stigma, and support progress in CS development for 'at risk' medical students.CS coaching is an underutilized resource 8 and this six-month pilot provides "proof of concept" at one medical school with sustained CS improvement demonstrated by OSCE performance in clerkship students (cohort one) over two post-pilot years.The skills of reflective practice and self-directed learning were iteratively emphasized through appreciative inquiry and strengths-based coaching strategies, and it is likely the students applied these strategies to other aspects of their learning, resulting in sustained improvement in OSCE scores over two years.
This coaching pilot builds on existing work that addresses clinical skills remediation 22 , notably for students identified as 'at risk' based on a multi-station summative OSCE 23 or for graduate medical education learners 24 , by specifically designing coaching-based activities for two levels of undergraduate learners (clerkship students and post-clerkship students).It aims to further the learner-driven aspects of prior studies, in this case allowing targeted student remediation 25 to be guided by faculty through a dynamic and iterative co-creation process 26 .The pilot contributes towards a standardized approach to active remediation, with trained faculty coaches to support struggling learners.Key aspects of a targeted remediation 25 were embedded within the coaching pilot, whereby the individual student performance, stated goals, and engagement in individual and group coaching sessions shaped a clear process that was locally available and integral to the standard Harvard Medical School curriculum.
It is important to note that this pilot study did not enroll participants based on pre-defined inclusion or exclusion criteria other than participants' demonstrated need for clinical skills remediation in medical school.All students identified as needing learning support were included in the pilot and the outcome of the pilot looked at quantitative clinical exam scores.This small feasibility education pilot study was not powered to look at sex and gender differences, though this can be considered in future studies that include larger cohorts.
A key lesson is that CS coaching reframed the learning process from a 'gap' or remedy-focused approach to one of guided self-directed learning and growth as master adaptive learners 19 .Additional lessons include the value of appreciative inquiry to redirect individual student concerns about having "failed" 27 .
Strengths-based coaching strategies build towards improved CS and learning while fostering a growth mindset 28,29 .Group coaching allowed students to adjust to the pilot with their peers, while individual sessions at clerkship sites provided coaching in settings integral to daily clinical learning.The alignment of coaching with the medical school curriculum enabled students to simultaneously participate in classes and clerkships, thereby remaining on track with peers.Identifying clear endpoints and student transition planning is essential.
Next steps include expansion of the program within the HMS curriculum 30 as a formal longitudinal learning support program with a focus on early identification and engagement of students during pre-clerkship training, to avoid simultaneous remediation of core CS during clerkship learning 31 .A CS coaching "implementation template" (Figure 2) with suggested activities and timelines can be locally adapted to guide remediation in UME.Further adaptations of the template are encouraged to collaboratively develop best practices in CS coaching across medical schools.
As a limitation, the small total numbers (n=23) in this feasibility "proof of concept" pilot reflect a specific cohort of 'at risk' learners for whom remediation was most critically needed.Nevertheless, the findings demonstrate the benefits of destigmatized remediation and offer a pathway to reimagine clinical skills learning support for larger cohorts of struggling or 'at-risk' students.A further limitation of the pilot was the exclusive use of OSCE assessment scores to evaluate student progress.This measure is a standardized metric by which all HMS students are evaluated, though future studies may include iterative AAMC EPA ratings, end-of-rotation evaluations, narrative comments, and other assessments for a holistic view of student progress.Finally, we were only able to complete two-year longitudinal follow-up for students in the clerkship cohort (cohort one), as the post-clerkship cohort (cohort two) students were in the final year of training at the time of the pilot.
It The CS coaching pilot demonstrates the feasibility of coaching to address the needs of 'at risk' learners in a destigmatized, supportive, and growth-oriented manner with measurable success.As the coaching interventions of the pilot may also be beneficial to all medical students, we plan to pilot optional student workshop sessions based on the three group sessions for clerkship students this year.CS coaching for remedial learning can offer a structured approach to promote competency and readiness for the UME to GME transition [33][34][35] .

Data availability statement
The

Mary Ellen J. Goldhamer, MD MPH
Dear Dr. Chou, Thank you for your thoughtful and timely review of our manuscript.Your suggestions and literature references have been incorporated and strengthen our "Manuscript, Version 2".We anticipate the updated "Version 2" manuscript will be circulated for your review in the next 1-2 weeks by the editorial staff.We hope you will have time to re-review "Manuscript, Version 2" which accompanies our point-by-point responses to your queries below.We thank you in advance for reconsidering our paper for publication.Sincerely, Jean E. Klig, MD and Mary Ellen J. Goldhamer, MD MPH (on behalf of our author group) Reviewer query #1 I commend the authors on an important description of a remediation coaching program that is thoughtful, methodically designed and implemented, and assessed, with high satisfaction by the learners.This description adds to the literature about effective means to support learners who struggle with clinical skills development.This would start to address the amount of faculty time and effort it takes to shepherd students through processes like this.Author response: #9: The authors agree it is important to track faculty time dedicated to remediation.Under "Results and Analysis" we included: "While group coaching sessions were required, optional hour-long individual coaching sessions were utilized by 57% (8/14) of clerkship students (24 total sessions, range two-seven sessions/student) and 44% (4/9) of post-clerkship students (six total sessions, range one-two sessions/student) during the pilot."Updates to the manuscript were not made as this information was included in "Manuscript Version 1".
Reviewer Query #10 How similar were the formative OSCEs and the summative OSCE?If the formative ones looked very similar to the summative one ("teaching to the test"), then the impressive results of passing could weaken.Author response #10: Students in the remediation coaching program were assessed on the same formative and summative OSCE exams as students who did not participate in the program on the same regularly scheduled intervals.The summative end of medical school comprehensive OSCE examination broadly examines skills all graduating medical students should achieve and is a Harvard Medical School graduation requirement.Coaching sessions targeted clinical skills essential for all graduating students and did not "teach to the test".The Methods section, paragraph 6, was updated to include the following statement: "Assessment of the clerkship cohort's CS included two formative OSCEs, each with three patient encounters incorporating immediate faculty feedback.These formative OSCEs were part of the routine HMS schedule at three and nine months after clerkship entry, corresponding to the close of the coaching pilot and six months post-pilot.A summative OSCE with nine patient encounters (and no formative feedback) required for graduation corresponded to two years post-pilot."Reviewer Query #11 I'm confused by the top panel in Table 1.If all students were selected on the basis of not passing formative or summative OSCEs, shouldn't the pilot entry number be 100%?Or were pre-clerkship evaluations of other types taken into consideration?Author response #11: Students in the clerkship cohort (cohort 1) were identified as 'at risk' based on either low formative OSCE performance compared to the class or concerns on narrative evaluations.34 (2016).https://doi.org/10.1186/s12909-016-0555-yReviewer Query #12 Finally, the results might imply a prevention strategy: to fully destigmatize learners from being labeled as being in remediation, perhaps might you consider adapting the three group interventions for the clerkship students into practice sessions for all medical students?Author response #12: Thank you.The authors agree that ideally the interventions described would likely be beneficial for all medical students and we plan to pilot optional student workshop sessions based on the three group interventions for the clerkship students this coming academic year.The following text was added to the last paragraph of the Discussion and conclusions section, "As the coaching interventions of the pilot may also be beneficial to all medical students, we plan to pilot optional student workshop sessions based on the three group sessions for clerkship students this year."The paper could be improved with clarification of a few points: You state that students were identified as 'at risk' based on OSCE performance of 'below passing'.What exactly does this entail?You seem to be reporting results relative to the class mean.Are you saying everyone below the mean is not passing?Obviously not, but it is unclear.It would be helpful to offer some rationale for your choice of coaching model: why strengths-based framework, as opposed to other options?You state that "Each student attended a three-hour required group coaching session", but figure 2 seems to be imply multiple group sessions.Can you clarify this discrepancy?What was the rationale for the decision to make the individual sessions optional?There were "frequent check-in meetings".How many of these per student?How frequent?Optional or required?Individual or group? Figure 5 suggests 14% of cohort one had preclerkship OSCE performance at or above class mean, and yet earlier you stated that entry to the pilot required a below passing score.How is this discrepancy explained?Or am I misunderstanding the data, in which case more clarity may be needed, perhaps?The sustained improvement beyond the pilot is impressive -readers might be interested to hear why you think this occurred.

Have any limitations of the research been acknowledged? Yes
Are all the source data underlying the results available to ensure full reproducibility?No

Are the conclusions drawn adequately supported by the results? Yes
Competing Interests: No competing interests were disclosed.
I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.
, 34 (2016).https://doi.org/10.1186/s12909-016-0555-yReviewer query #3: It would be helpful to offer some rationale for your choice of coaching model: why strengthsbased framework, as opposed to other options?Author response #3: The following statement was added to the Methods section, paragraph 4: "These [appreciative inquiry methods and a strengths-based coaching] frameworks were chosen over models of behavior change to promote self-reflection, self-directed learning, and co-creation of learning goals.This framework aligned with the pilot's CS (clinical skills) coaching goals which aimed to engage students and promote lifelong learning strategies."Further, in the Discussion and conclusions section, paragraph 2, we include a include a paragraph on how this current pilot builds on previous coaching pilots.The updated manuscript includes the following: "This coaching pilot builds on existing work that addresses clinical skills remediation (Parsons A, Warburton, K et al), notably for students identified as 'at risk' based on a multi-station summative OSCE (Chang,  Chou, Hauer)   2: Early clerkship students (cohort 1) attended three 3-hour group sessions that covered history taking (session 1, 3 hours), hypothesis-driven physical exam (session 2, 3 hours), and clinical reasoning and oral presentation (session 3, 3 hours).
To clarify the "post-clerkship" cohort 2 students; the "Methods" section, paragraph 7 notes: "Manuscript Version 2" was updated to include the following: "Each post-clerkship student attended one three-hour required group coaching session with optional individual coaching."(The word "one" replaced "a" in reference to the group coaching session.)The original Figure 2 stated "Cohort 2 Group coaching: 3-hour session, held twice".The authors would like to clarify that the same session was held twice to ensure attendance.For clarity, the version 2 manuscript includes an updated Figure 2 which states: "Cohort 2: Group Coaching: 3-hour session".Thank you for noting the need for this clarification.Reviewer query #5: What was the rationale for the decision to make the individual sessions optional?Author response #5: The following statement was added to the "Methods" section, paragraph 6: "Individual sessions were optional to avoid overburdening students and to gauge student engagement in co-creating their learning."Reviewer query #6: There were "frequent check-in meetings".How many of these per student?How frequent?Optional or required?Individual or group?Author response #6: The following statement was added to the "Results" section, paragraph

Figure 1 .
Figure 1.Strengths-based coaching cycle for the Clinical Skills (CS) pilot.

Figure 2 .
Figure 2. Implementation template for the coaching pilot.

Figure 5 .
Figure 5. OSCE score performance for the coaching pilot clerkship cohort (cohort one).Pre-clerkship (pilot entry) to close of the six-month pilot (clerkship month three); Six-months post-pilot (clerkship month nine) to the post-clerkship summative graduation OSCE assessment (21-24 months post-pilot).

"Most impactful areas": "Most helpful aspects":
32 important to note that the OSCEs are scored by both trained Standardized Patients (SPs) and HMS OSCE core educator faculty.SPs score the communication skills component of the OSCE only, using a 5-point Likert scale.SPs receive implicit bias training during their new hire orientation session and ongoing refresher training during each training session.HMS core educator faculty score the remainder of the OSCE, and checklists used to calculate an OSCE score are mostly based on a binary response (yes/no) to indicate presence or absence of an observed skill.These binary checklist scores are less likely to reflect implicit or unconscious bias of the evaluator.These biases, however, may occur in narrative evaluation comments, while selecting an AAMC EPA rating32, and/or during feedback sessions.While there is ongoing faculty development aimed at detecting and eliminating unconscious and implicit bias in feedback sessions and on written evaluations, this training was not in place in 2019 at the time of the pilot.The authors appreciate the evidence in the literature that describes these biases32and recognize the importance of ongoing work to detect and extinguish unconscious and implicit bias in medical student assessment and feedback.

the work clearly and accurately presented and does it cite the current literature? Partly Is the study design appropriate and does the work have academic merit? Yes Are sufficient details of methods and analysis provided to allow replication by others? Yes If applicable, is the statistical analysis and its interpretation appropriate? Not applicable Have any limitations of the research been acknowledged? Partly Are all the source data underlying the results available to ensure full reproducibility? Partly Are the conclusions drawn adequately supported by the results? Yes Competing Interests:
Introduction: needs a stronger aim statement: I think you are saying that since currently existing coaching programs in US medical schools focus less on clinical skills development or remediation, you set forth to determine the efficacy of a coaching program for students who struggle with clinical skills.Can you also differentiate between the coaching program you developed and descriptions of others (for example, Parsons et al, South Med J 2022 1 ; Guerrasio et al, Acad Med 2019 2 ; Chang et al, Med Education 2008 3 )?It seems that the "Ethics" subheading in the Methods section is extraneous.The development of the coaching program itself seems robust, and using an appreciative, strengths-based, and master adaptive framework is key; the authors could additionally invoke literature describing the optimal components of such a program (eg, Boileau et al, Adv Med Educ Pract 2017 4 ).Figure2seems straightforward; using Steinert's framework (Med Teach 2013 5 ), one step that seems to be a bit underemphasized is clarification -after identification, clarifying the students' needs: many times, students are subject to forces outside school that play outsized influences on a student's performance.In terms of demographics: what was the proportion of underrepresented or minoritized learners, relative to the classes as a whole?How much of the nonpassing rate for students may have been because of implicit bias in evaluation by SPs?The group interventions that the authors use do not address these sadly common and more systemic influences.How many individual coaching sessions were set up for each student?This would start to address the amount of faculty time and effort it takes to shepherd students through processes like this.How similar were the formative OSCEs and the summative OSCE?If the formative ones looked very similar to the summative one ("teaching to the test"), then the impressive results of passing could weaken.I'm confused by the top panel in Table1.If all students were selected on the basis of not passing formative or summative OSCEs, shouldn't the pilot entry number be 100%?Or were pre-clerkship evolutions of other types taken into consideration?Co-editor of the book Remediation in Medical Education: A Midcourse Correction -but any royalties from my half go directly to the Academy of Communication in Healthcare, not to me.
Finally, the results might imply a prevention strategy: to fully destigmatize learners from being labeled as being in remediation, perhaps might you consider adapting the three group interventions for the clerkship students into practice sessions for all medical students?References 1. Parsons A, Warburton K, Martindale J, Rosenberg I: Characterization of Clinical Skills Remediation: A National Survey of Medical Schools.Southern Medical Journal.2022; 115 (3): 202-207 Publisher Full Text 2. Guerrasio J, Brooks E, Rumack CM, Aagaard EM: The Evolution of Resident Remedial Teaching at One Institution.Acad Med.2019; 94 (12): 1891-1894 PubMed Abstract | Publisher Full Text 3. Chang A, Chou CL, Hauer KE: Clinical skills remedial training for medical students.Med Educ.2008; 42 (11): 1118-9 PubMed Abstract | Publisher Full Text 4. Boileau E, St-Onge C, Audétat MC: Is there a way for clinical teachers to assist struggling learners?A synthetic review of the literature.Adv Med Educ Pract.2017; 8: 89-97 PubMed Abstract | Publisher Full Text 5. Steinert Y: The "problem" learner: whose problem is it?AMEE Guide No. 76.Med Teach.2013; 35 (4): e1035-45 PubMed Abstract | Publisher Full Text Is Reviewer Expertise: feedback and remediation in health professions education I confirm that I

have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.
Author Response 01 Jul 2023

Author response #1: Thank you.
below) but was moved from the "Methods" section to the end of the "Introduction" section.We have included some additional text from the Introduction here for context: "A recent survey of American Medical Association (AMA) Accelerating Change in Medical Education

Can you also differentiate between the coaching program you developed and descriptions of others (for example, Parsons et al, South Med J 2022 1 ; Guerrasio et al, Acad Med 2019 2 ; Chang et al, Med Education 2008 3 )? Author response #3:
Thank you for this query and for suggesting relevant literature citations aimed at strengthening our manuscript, which have been incorporated as noted below.The following paragraph and literature references have been added to "Manuscript Version 2", "Discussion and conclusions" section, "The coaching pilot builds on existing work that addresses clinical skills remediation (Parsons A, Warburton, K et al here), notably for students identified as 'at risk' based on a multistation summative OSCE(Chang, Chou, Hauer)or for graduate medical education learners(Guerrasio, Brooks, Rumack), by specifically designing coaching-based activities for two levels of undergraduate learners (clerkship students and post-clerkship students).It aims to further the learner-driven aspects of prior studies, in this case allowing targeted student remediation (Boileau, St-Onge, Audetat) to be guided by faculty through a dynamic and iterative co-creation process (Steinert Y).The pilot contributes towards a standardized approach to active remediation, with trained faculty coaches to support struggling learners.Key aspects of a targeted remediation (insert Boileau et al) were embedded within the coaching pilot, whereby the individual student performance, stated goals, and engagement in individual and group coaching sessions shaped a clear process that was locally available and integral to the standardHarvard Medical School curriculum."Author note: These additional full references will be incorporated into the manuscript by the MedEdPublish editorial staff per "Manuscript updated version guidelines".References: Parsons A, Warburton K, Martindale J, et al.: Characterization of Clinical Skills Remediation: A National Survey of Medical Schools.Southern Medical Journal.2022; 115 (3): 202-207 | Publisher Full Text 1. Chang, A., Chou, C.L. and Hauer, K.E.(2008), Clinical skills remedial training for medical students.Medical Education, 42: 1118-1119.https://doi.org/10.1111/j.

It seems that the "Ethics" subheading in the Methods section is extraneous. Author response
The development of the coaching program itself seems robust, and using an appreciative, strengths-based, and master adaptive framework is key; the authors could additionally invoke literature describing the optimal components of such a program (eg, Boileau et al, Adv Med Educ Pract 2017 4 ).While it is possible for implicit or unconscious bias to impact SP scores, these important steps have been taken to guard against it.Importantly, students are rarely, if ever, identified for formal remediation due to poor communication skills.Harvard Medical School OSCE core educator faculty score all other aspects of each case on OSCE checklists that are mostly based on a binary response (yes/no) to indicate presence or absence of an observable skill.Thus, the checklists with binary ratings from observed student OSCEs are less likely to reflect implicit or unconscious bias as the skill was either observed or not observed.Unconscious or implicit bias may occur in narrative evaluation comments and/or during feedback sessions.While there is currently ongoing faculty development aimed at detecting and eliminating unconscious/implicit bias in feedback sessions and on written evaluations, this training was not in place in 2019 at the time of the pilot.The authors appreciate the evidence in the literature that describes these biases and recognizes the importance of ongoing work to detect and extinguish unconscious and implicit bias in medical student assessment and feedback.The following paragraph was added to the "Discussions and conclusions" section of paragraph 6: "It is important to note that the OSCEs are scored by both Standardized Patients (SPs) and trained HMS OSCE core educator faculty.SPs score the communication skills component of the OSCE only, using a 5-point Likert scale.SPs receive implicit bias training during their new hire orientation session and ongoing refresher training during each training session.HMS core educator faculty score the remainder of the OSCE, and the majority of the checklists used to calculate an OSCE score are based on a binary response (yes/no) to indicate presence or absence of an observed skill.These binary checklist scores are less likely to reflect implicit or unconscious bias of the evaluator.These biases, however, may occur in narrative evaluation comments, while selecting an AAMC EPA rating [insert Hauer, Park, Bullock reference], and/or during feedback sessions.While there is currently ongoing faculty development aimed at detecting and eliminating unconscious and implicit bias in feedback sessions and on written evaluations, this training was not in place in 2019 at the time of the pilot.The authors appreciate the evidence in the literature that describes these biases[Hauer, Park, Bullock, etal] and recognizes the importance of ongoing work to detect and extinguish unconscious and implicit bias in medical student assessment and feedback."Hauer Karen E. MD, PhD; Park Yoon Soo PhD; Bullock Justin L. MD, MPH; Tekian Ara PhD MHPE Title: "My Assessments Are Biased!" Measurement and Sociocultural Approaches to Achieve Fairness in Assessment in Medical Education, Academic Medicine: Journal of the Association of American Medical Colleges DOI: 10.1097/ACM.0000000000005245Reviewer Query #9 How many individual coaching sessions were set up for each student?
#4: The "Ethics" sub-heading in the Methods section is required by the MedEdPublish Journal as it alerts readers to the section that describes the Institutional Review Board (IRB) process.Reviewer Query #5: many times, students are subject to forces outside school that play outsized influences on a student's performance.Author response #6: Thank you for these important queries.During individual 1:1 meetings with the coaching program director, students were encouraged to share experiences that may impact their learning and where possible, solutions were sought and implemented.As these experiences and solutions are confidential and unique to each student, these discussions and interventions are not described in the manuscript.Reviewer Query #7 In terms of demographics: what was the proportion of underrepresented or minoritized learners, relative to the classes as a whole?Author response # 7: The following statement has been added under "Results and analysis", paragraph 1: "Students who were underrepresented in medicine made up less than 20% of both cohorts in the pilot".Reviewer Query #8: How much of the non-passing rate for students may have been because of implicit bias in evaluation by SPs?The group interventions that the authors use do not address these sadly common and more systemic influences.Author response #8: Thank you for this important raising this important question.The OSCEs are scored by both standardized patients and trained core educator faculty.Standardized patients (SPs) rate communication skills only and ratings are on a 5point Likert scale.All SPs have extensive training on implicit bias as part of their 2-hour new hire orientation training with ongoing refresher training during each training session.

been revised for clarification from
Two students in this cohort (14%) with OSCE scores at the class mean on pilot entry were identified based on narrative evaluations.The revised "Manuscript Version 2" Methods section, paragraph 2, statement has

Table 1 ]
The post-clerkship cohort (cohort two) was identified based scores below the passing threshold score of 68% on the HMS summative graduation OSCE.The 68% total passing score for the HMS summative graduation OSCE was calculated using a modified Hofstee method [Burr, S.A., Whittle, J., Fairclough, L.C. et al.] that incorporates historical data of students' performance and a faculty mean score based on expectations for minimum and maximum passing scores and percentages."Anearlier reference to Table 1 has been added.Reference: Burr, S.A., Whittle, J., Fairclough, L.C. et al.Modifying Hofstee standard setting for assessments that vary in difficulty, and to determine boundaries for different levels of achievement.BMC Med Educ 16, 1. Parsons A, Warburton K, Martindale J, et al.: Characterization of Clinical Skills Remediation: A National Survey of Medical Schools.Southern Medical Journal.2022; 115 (3): 202-207 | Publisher Full Text 2. Guerrasio J, Brooks E, Rumack CM, et al.: The Evolution of Resident Remedial Teaching at One Institution.Acad Med.2019; 94 (12): 1891-1894 PubMed Abstract | Publisher Full Text 3. Chang A, Chou CL, Hauer KE: Clinical skills remedial training for medical students.Med Educ.2008; 42 (11): 1118-9 PubMed Abstract | Publisher Full Text 4. Boileau E, St-Onge C, Audétat MC: Is there a way for clinical teachers to assist struggling learners?A synthetic review of the literature.Adv Med Educ Pract.2017; 8 89-97 PubMed Abstract | Publisher Full Text 5. Steinert Y: The "problem" learner: whose problem is it?AMEE Guide No. 76.Med Teach.2013; 35 (4): e1035-45 PubMed Abstract | Publisher Full Text

You state that "Each student attended a three-hour required group coaching session", but figure 2 seems to be imply multiple group sessions. Can you clarify this discrepancy? Author response #4:
or for graduate medical education learners(Guerrasio, Brooks, Rumack et al), by specifically designing coaching-based activities for two levels of undergraduate learners (clerkship students and post-clerkship students).It aims to further the learner-driven aspects of prior studies, in this case allowing targeted student remediation (Boileau, St-Onge, Audetat) to be guided by faculty through a dynamic and iterative co-creation process (Steinert Y).The pilot contributes towards a standardized approach to active remediation, with trained faculty coaches to support struggling learners.Key aspects of a targeted remediation (Boileau, St-Onge, Audetat) were embedded within the coaching pilot, whereby the individual student performance, stated goals, and engagement in individual and group coaching sessions shaped a clear process that was locally available and integral to the standard Harvard Medical School curriculum."ParsonsA, Warburton K, Martindale J, et al.: Characterization of Clinical Skills Remediation: A National Survey of Medical Schools.Southern Medical Journal.To clarify the differences between the two cohorts we note in the Methods section, paragraph 6, "[Clerkship] Students attended three three-hour required group coaching sessions and could opt for individual coaching during clerkships." Figure

query #7: Figure 5 suggests 14% of cohort one had pre-clerkship OSCE performance at or above class mean, and yet earlier you stated that entry to the pilot required a below passing score. How is this discrepancy explained? Or am I misunderstanding the data, in which case more clarity may be needed, perhaps Author response #7:
1: "Optional individual clerkship student (cohort 1) check-ins occurred in person, over email, and by phone and were offered monthly, with a range of 0-4 check-in meetings per clerkship student."ReviewerThankyouforthisquery.Clerkship students, cohort 1 were identified by below passing pre-clerkship OSCE scores and/or performance evaluations indicating concern(s) in the area of clinical skills.The revised "Manuscript Version 2" Methods section, paragraph 6, statement has been revised for clarification from: "The clerkship student cohort (cohort one) was identified based on below passing pre-clerkship OSCE scores and evaluations" to now state: "Students were identified as'at risk' based on low OSCE performance and/or concerns on narrative evaluations.A 'below passing' OSCE performance was the lowest fifteenth percent of scores for the clerkship cohort (cohort 1).Two students in the clerkship cohort 1 with OSCE scores at the mean on pilot entry were identified based on narrative evaluations.[Table1]Thepost-clerkshipcohort (cohort two) was identified based scores below the passing threshold score of 68% on the HMS summative graduation OSCE.The 68% total passing score for the HMS summative graduation OSCE was calculated using a modified Hofstee method [Burr, S.A., Whittle, J., Fairclough, L.C. et al.] that incorporates historical data of students' performance and a faculty mean score based on expectations for minimum and maximum passing scores and percentages."Anearlierreference to Table1has been added.Reference: Burr, S.A., Whittle, J., Fairclough, L.C. et al.Modifying Hofstee standard setting for assessments that vary in difficulty, and to determine boundaries for different levels of achievement.BMC Med Educ 16, 34 (2016).https://doi.org/10.1186/s12909-016-0555-yReviewer

query #8: The sustained improvement beyond the pilot is impressive -readers might be interested to hear why you think this occurred. Author response #8:
Thank you.The following sentence was added to the "Discussion" section, paragraph 1: "The skills of reflective practice and selfdirected learning were iteratively emphasized through appreciative inquiry and strengths-based coaching strategies, and it is likely the students applied these strategies to other aspects of their learning, resulting in sustained improvement in OSCE scores over two years." Competing Interests: No competing interests were disclosed.