Patient Safety and Quality Improvement: A Curriculum for Physical Medicine and Rehabilitation

Introduction: Quality improvement (QI) is an imperative aspect of healthcare and resident education, and as such, an emphasis has been placed on the development of QI curricula. We set out to describe and analyze the eﬀectiveness of the PSQI Curriculum within the PM&R Residency Program at the University of Kansas Medical Center during the academic years of 2017-2020. Methods: The curriculum includes nine mandatory institute for healthcare improvement modules, the university’s GME core conference series, the departments patient safety conference series, a PSQI lecture series, and annual resident-led QI projects. Resident PSQI knowledge was assessed before and after completion of the annual curriculum through an internally generated questionnaire with optional resident feedback. Results: A total of 16 PM&R residents participated in this curriculum for a minimum of one or more years during the academic years of 2017-2020. We found a signiﬁcant diﬀerence in 9 of 12 domains during the 2017-2018 academic year, 10 of 12 domains during the 2018-2019 academic year, and 10 of 12 domains during the 2019-2020 academic year. Aggregating for all academic years revealed a statistically signiﬁcant diﬀerence across all 12 domains. Discussion: In graduate an residency program, residents must exhibit proﬁciency in PBLI and SBP, which includes the domain of quality improvement. The continuous QI curriculum within the PM&R Residency Program KUMC has proven to be eﬀective in improving residents’ familiarity and comfort with quality assessment and improvement and may serve as a model for other


Introduction
Quality improvement (QI) is an imperative aspect of healthcare and resident education, and as such, an emphasis has been placed on the development of QI curricula in residency training programs. The Clinical Learning Environment Review (CLER) program was founded in 2012 by the Accreditation Council for Graduate Medical Education (ACGME) as a means to examine and subsequently ensure the quality of training in residency programs. The CLER National Report in 2016 found that many academic institutions were routinely failing to engage trainees in patient safety and quality improvement (PSQI) opportunities (Nasca, 2016). As a result, the ACGME mandated that QI instruction and participation be included in residency and fellowship education (ACGME, 2020). The CLER went a step further to include PSQI in its assessment domains for resident and fellow learning environments (Weiss, Wagner and Nasca, 2012). Given that the clinical patterns practiced during residency and fellowship often predict how physicians will care for their patients post-training (Asch et al., 2009) it is inherently essential that training programs implement specialty specific QI curricula that will transcend their trainees' education (Weiss, Bagian and Nasca, 2013).
Currently, no standard exists for quality improvement curricula regardless of level of training or specialty. Studies have shown a wide variation in the content and degree of exposure to QI principles and participation in QI projects among residency programs (Parker et al., 2015;Craig et al., 2014;Mann, Craig and Moses, 2014;Wong et al., 2010;Philibert et al., 2014). Furthermore, the CLER program reported that trainees and program leadership were often distant participants (Schumacher and Frohna, 2016). Some have called for the implementation of QI curricula that involve more active participation of trainees in QI efforts and more institutional support for such endeavors (Liao, Co and Kachalia, 2015) which is essential to optimize resident and fellow education during training.
In order to graduate from an ACGME-accredited residency program, residents must exhibit proficiency in practicebased learning and improvement (PBLI) and systems-based practice (SBP) (ACGME, 2020). Quality improvement and patient safety have come to be included among these domains. We developed a PSQI model designed to meet ACGME requirements and CLER expectations. Our model's foundation is a continuous QI curriculum that recurs annually and is designed specifically for Physical Medicine and Rehabilitation (PM&R) residents. It focuses on core PBLI and SBP competencies while engaging residents in quality improvement initiatives that foster better patient care. While there are QI curricula published (Keefer et al., 2016;Stewart et al., 2016;Wener, 2017;Vinas et al., 2018;Kiger and Bertagnoli, 2018), this resource represents the first QI curriculum specific to PM&R residents.

Methods
The patient safety and quality improvement curriculum within the PM&R Residency Program at the University of Kansas Medical Center (KUMC) was implemented in 2015. The initial (2015) version of the curriculum has undergone marked changes over the ensuing years, reflecting evaluations from learners and faculty instructors in addition to evolving literature in the area of medical education. A total of 16 PM&R residents have participated in this QI curriculum for a minimum of one or more years. Medical students rotating through were invited but not required to participate in QI efforts.
Curriculum structure: Institute for Healthcare Improvement (IHI) The IHI is the international leader in the realm of PSQI and has developed an online Open School that is available free of charge to any medical student, resident, fellow, or faculty member who is involved in teaching health professionals. Their modules incorporate content with video clips, interactive questions, and quizzes in an effort to educate healthcare professionals on the foundational concepts of PSQI. There are nine required modules for all residents at KUMC, five within the quality improvement domain and four within the patient safety domain (Table  1). There are numerous supplementary modules available to our residents in PSQI in addition to opportunities in the areas of medical education, leadership, and patient care. While our residents are encouraged to complete all of the IHI modules by graduation; currently, we are only mandating completion of the nine specified modules in table 1 for all PGY-2 residents by the end of their second year of residency. The IHI modules are introduced to our residents during the first PSQI lecture of the year, which is described further below. Quarterly reminders are subsequently sent to all second-year residents to help ensure completion in a timely fashion.

University of Kansas Graduate Medical Education (GME) Core Competency Conference Series
The GME's primary responsibility at any given institution is to provide an organized educational program that fosters resident and fellow education while safeguarding patient care (Armstrong et al., 2012). KUMC's conference series provides accessible and valuable instruction aligned with the ACGME competencies and supports the University of Kansas Health System's quality and safety initiatives. All residents are required to attend at least 75% of KUMC's GME core conference series, held on the second Tuesday of every month, throughout their residency. Residents on rotations off the main campus are afforded the opportunity to attend remotely or watch the recorded lectures at a later date.

Departmental Patient Safety Conference
Every year, each resident is required to present a patient case, with faculty mentorship, at our departmental patient safety conference held once per month. There are 30 minutes allotted for each presentation with the first 10 minutes designated for the case presentation and respective literature review and the remaining 20 minutes designated for discussion. During the discussion phase of the presentation, residents utilize the Healthcare Matrix developed at Vanderbilt University Medical Center to demonstrate the opportunities for improvement in their patient case. This Healthcare Matrix is a valuable teaching instrument in resident education as it allows for patient care assessment using ACGME and Institute of Medicine (IOM) core competencies (Quinn et al., 2009). Of note, the literature review will reflect the presenting resident's interpretation of the matrix prior to group discussion and should synthesize the current literature trying to address this specific problem.

PSQI Lecture Series and Resident-led Projects
At the start of each academic year, residents form three teams, each comprised of at least one PGY2, PGY3, and PGY4 resident, respectively. Each team nominates a faculty mentor to help them identify and complete a PSQI project throughout the academic year. We elected to utilize this structure for two reasons. First, as residents advance through their training, they experience a tremendous amount of growth in clinical knowledge and skills which should be reflected with gradually increased responsibility in both medical education and patient care. This structure affords senior residents an opportunity to exercise leadership and share their knowledge with junior residents. Secondly, this structure allows for more cohesive implementation of the Plan-Do-Study-Act (PDSA) cycle from one year to the next should non-graduating residents opt to continue with their previously identified project.
The resident PSQI lecture series is divided into four quarterly lectures with the addition of two supplemental lectures comprising a total of six distinct lectures. Each lecture is 30 minutes in duration except for our Mock Root Cause Analysis lecture which is allocated 60 minutes of lecture time. Lectures are given by faculty members throughout the course of the academic year. The focus is on high yield PSQI topics and development of resident led QI projects. The topics are outlined in Table 2.  1 and 2, respectively) are provided at the beginning of the academic year following the introduction and construction of a QI project lecture. Each lecture correlates to specific questions within the worksheets, and residents are expected to have completed the correlating questions prior to lecture within their respective project teams as detailed above. The first 10-15 minutes of each 30-minute lecture is spent on the presentation of a new topic. The remaining 15-20 minutes is spent reviewing QI worksheet updates with the faculty member to ensure appropriate implementation of newly learned topics and timeliness of QI project completion. Following the completion of the PSQI lecture series and projects, each team then presents their project to the department at the end of the academic year during our final departmental patient safety conference. Residents were asked about their previous exposure to QI, their confidence in implementing QI methodologies in future practice, and their confidence in their ability to use QI principles to improve patient care. Additionally, they were asked for qualitative feedback in regard to questions and concerns pertaining to patient safety and quality improvement. The survey is considered exempt by the KUMC Institutional Review Board, and the results are continuously utilized to modify and improve our QI curriculum.

Faculty training and commitment
The primary faculty mentor for the patient safety and quality improvement curriculum has had specific training in QI methodology in their residency program and has completed institutional QI training. Department leadership provides sufficient administrative time for the development, implementation and continuous improvement of this curriculum. On average the faculty mentor spent 2 additional hours of work each month developing content in addition to coordinating and following up resident efforts over the first year of curriculum initiation. This time occurred during previously appointed administrative time. Over the next four years of the curriculum, the faculty mentor spent on average 2 hours of work each quarter updating lectures and coordinating resident efforts.

Results
Resident feedback was gathered via Likert scale relating to how comfortable each resident was with various aspects of PSQI. The following connotations were used on pre and post assessments: 1 -not at all comfortable, 2 -slightly comfortable, 3 -moderately comfortable, and 4 -extremely comfortable. Question statements which residents were asked to rate can be found in Supplementary File 3.
Our data was collected over three academic years, spanning 2017-2020, with a total of 16 different residents (PGY-2 through PGY-4) having completed the pre and post assessment. We had a 100% completion rate during this threeyear collection period (n=29). Importantly, because this curriculum occurs annually, PGY-2 residents during the 2017-2018 academic year went on to complete this questionnaire a second and third time during 2018-2019 and 2019-2020, respectively. The same holds true for PGY-3 residents who completed the questionnaire a total of two times during their residency. The responses were aggregated for each academic year. Given that our survey responses were completely anonymous, we were unable to stratify results based on the number of years a resident had completed the curriculum.
The results of the pre-curriculum responses were compared to the results of the post-curriculum responses for each academic year. As the respondents were the same for each pre-and-post test, a paired t-test was used to compare resident comfort with QI pre-and-post curriculum. We expected the curriculum to have a positive impact on resident familiarity with QI, thus a one-tailed test was utilized (Table 3). We found a significant difference in 9 out of the 12 domains in our QI curriculum during the 2017-2018 academic year, 10 out of 12 domains during the 2018-2019 academic year, and 10 out of 12 domains during the 2019-2020 academic year. Specifically, making changes in a system, identifying whether a change leads to an improvement, and identifying best practices were not statistically significant for the 2017-2018 academic year. Identifying best practices and identifying how data is linked to specific processes were not significant for the 2018-2019 academic year. Writing a clear problem statement and applying the best professional knowledge were not significant for the 2019-2020 academic year. However, aggregating for all academic years, as shown in Figure 1, revealed a statistically significant difference across all 12 domains.

Discussion
We set out to describe and analyze the effectiveness of our PSQI Curriculum within the PM&R Residency Program at the University of Kansas Medical Center over the academic years of 2017-2020. Analysis revealed significant improvement between pre and post assessment in 9 out of 12, 10 out of 12, and 10 out of 12 domains for the academic years of 2017-2018, 2018-2019, and 2019-2020, respectively. Making changes in a system, identifying whether a change leads to an improvement, identifying best practices, identifying how data is linked to specific processes, writing a clear problem statement and applying the best professional knowledge were all areas that residents failed to demonstrate significant improvement at some point during our data collection period. However, there was year-to-year variance pertaining to which domains failed to demonstrate significant improvement. In fact, only identifying best practices spanned multiple academic years, 2017-2018 and 2018-2019. Moreover, aggregated analysis for all academic years demonstrated significant improvement across all domains.
Given that our assessment tool was a questionnaire, it is possible that questionnaire bias impacted our results. Bias may have arisen from the way individual questions were designed and thus interpreted by our residents. This may explain why there was year-to-year variance in regard to which domains residents failed to show significant improvement. On the contrary, it is possible that there are deficiencies in our residents' knowledge, and thus in our curriculum, which would necessitate further modifications to address these discrepancies. However, given the overall improvement seen across all domains for the aggregated data, it would appear this is less likely.
There were two major limitations in this study: 1) the use of a non-validated questionnaire 2) the inability to track individual residents as they progressed through the curriculum from year-to-year. Future considerations would include adapting a validated evaluation tool to be relevant for PM&R, such as the Quality Improvement Knowledge Application Tool (QIKAT), which has previously been described (Kiger and Bertagnoli, 2018). This tool would allow for more accurate assessment of resident QI knowledge and skills. Adjustments could also be made to the wording and design of the pre-and post-PSQI self-assessment form to minimize bias. Allowing for qualitative feedback at the end of the survey in regard to the strengths and weaknesses of the QI curriculum could prompt additional necessary revisions in the curriculum. Lastly, tracking residents as they progress through the curriculum throughout their residency training would provide valuable information on the curriculum's effectiveness from yearto-year. Intake could remain anonymous, but a tracking system nonetheless would need to be implemented.