High Value Care Education: A Needs Assessment at a General Surgery Residency Program

Background: Medical education has long recognized the importance of competence in high value care (HVC) but only recently have these concepts been codified. Comprehensive HVC curricula in surgery have not been reported and little is known about graduates’ perceptions of HVC education. Our objective was to investigate the perceptions of surgery residency graduates regarding education in HVC and its importance in clinical practice. Methods: Our General Surgery graduates (2012-2016, n=23) were surveyed regarding perceptions of HVC education they received during training and the importance of HVC in their current practice. Results: Thirteen of the 23 graduates responded (57%). While only half of respondents reported receiving instruction in HVC during residency, all reported participating in quality improvement (QI) projects. A minority (n=5, 38%) felt that they had adequate knowledge and skills to lead a QI effort in practice. The majority, however, agreed that knowledge and practical skills in HVC are important to their clinical practice (92%, 77% respectively). Conclusion: General surgery graduates value knowledge in HVC concepts in their current practice, however, most do not feel they possess the knowledge and skills to lead QI efforts. Given that QI work is valued in practice, our results suggest a need to improve HVC education in residency.


Introduction
The United States (US) healthcare system is inefficient, providing care that is of variable quality at high costs Martinez-Vargas V, Malhotra N, Presson A, Vargo D, Smith B MedEdPublish https://doi.org/10.15694/mep.2019.000177.1 Page | 2 (Bentley et al., 2008). US healthcare spending continues to grow and has reached 18% of the gross domestic product, far exceeding healthcare spending in other industrialized nations (Scheurer et al., 2016). Expensive healthcare, however, has not resulted in improved health outcomes, prompting payers, such as the Centers for Medicare and Medicaid Services (CMS) and the public to demand change in the current system (Papanicolas, Woskie and Jha, 2018). In addition to cost and quality, a renewed focus on the perspective of the patient has been incorporated into our understanding of value in health care, resulting in the value equation, Value = (Quality + Service) / Cost (Scheurer et al., 2016). Transforming the current healthcare system into one that focuses on measuring and improving "value" will not be easy and will require commitment from all stakeholders, including payers, policymakers, patients, and providers.
The need for physicians to be competent in the principles of high-value healthcare delivery was first highlighted by the Institute of Medicine publication "To Err is Human" in 2000 (Kohn, Corrigan and Donaldson, 2000). Since that time, accrediting bodies in medical education have recognized the importance of physician competency in high value care (HVC) concepts, including quality, safety, service, and cost-containment. The Accreditation Council for Graduate Medical Education (ACGME) Core Competencies, introduced in 2002, include systems-based practice (SBP) and practice-based learning and improvement (PBLI), which encompass systems of healthcare delivery, personal practice improvement and quality improvement(Accreditation Council for Graduate Medical Education, 2002). Although these competencies for graduate medical education (GME) had been in place for a decade, in 2012 the American Hospital Association (AHA) identified significant gaps between physicians' demonstration of these competencies and their relative importance to healthcare organizations (Combes and Arespacochaga, 2012). The most significant gap was noted in the competency of SBP, which may reflect a lack of effective education in HVC during training.
Our General Surgery Residency program is no exception. The 2014, ACGME Clinical Learning Environment Review (CLER) demonstrated major shortcomings in resident knowledge of basic VDH concepts, with most residents unable to articulate basic concepts of quality improvement (QI). Furthermore, a recent article from our institution supports the CLER findings, noting that residents across GME at the our institution lack a common understanding of the meaning of QI and that residents generally perceive QI as "a lot of extra work (Butler et al., 2017)." The aim of this study was to elucidate the perceptions of recent General Surgery residency graduates regarding HVC. Three main areas were investigated including 1) perceptions of HVC education received during residency, 2) the importance of knowledge and skills in HVC concepts in practice, and 3) what an ideal HVC curriculum during general surgery residency should include.

Survey development
In order to assess the needs and issues for the development and implementation of a complete HVC curriculum for in our department, we surveyed recently graduated General Surgery Residents. Survey items were designed with HVC experts including value engineers, clinical operations leaders, the General Surgery Residency Program Director and the Department of Surgery Chief Value Officer (CVO). Questions were piloted with current residents for clarity. The survey instrument was designed to elicit the perceptions of General Surgery graduates regarding the HVC education they received during residency and their need for knowledge and skills in HVC concepts in current practice.

Survey instrument
The survey was divided into six sections (See Appendix 1). The first section included graduates' demographics. The second and third sections included questions related to attitudes and experiences with HVC education during residency and fellowship, respectively. In these sections, respondents were also asked to specify if the value components (quality improvement, safety, cost and service) were addressed during their training time and to characterize their training experience into positive, neutral or negative using a three-point Likert scale. The fourth section asked program graduates to rank the importance of the core HVC concepts from most important to least important. The fifth section included questions that asked respondents to assess if they had adequate knowledge and skills to lead a quality improvement effort and if they considered knowledge about value driven healthcare concepts to be important in their current day-to-day clinical practice using a three-point Likert scale. The last section asked about what topics should be included in an ideal HVC curriculum, with options including health systems operations, the opportunity to lead a team QI project, cost containment in healthcare and exceptional patient experience, among others. Graduates were also given the opportunity to elaborate on their HVC experience, their knowledge and skills with these concepts, and any additional comments using free text responses. At our institution, high value care is often called value driven healthcare (VDH) and this verbiage is reflected in the survey questions.

Survey administration
Surveys were e-mailed to graduates of our General Surgery Residency program who completed residency from 2012-2016 (n=23). Surveys were e-mailed once with reminder e-mail and included a link to the online survey. The survey was hosted and data were collected using the REDCap electronic data capture tools hosted at our institution (Harris et al., 2009).

Data analysis
Descriptive statistics and qualitative analysis were applied to summarize results. Demographic information was summarized using count and percentage. Rank order data, including questions on QI, patient safety, service/patient experience, and cost were summarized using mode, minimum and maximum values. Likert scale data, including questions on knowledge, skills and attitudes were summarized using median, interquartile range (IQR), count and percentage. Finally, comparisons of four questions regarding knowledge and skills between academic and private practice were summarized using median and IQR. This analysis between academic and private practice was conducted using Wilcoxon rank sum tests. Quantitative statistical analysis was conducted using SAS 9.4 (SAS Institute, Cary, NC). Qualitative analysis was undertaken using an inductive, grounded theory approach.

Results/Analysis
Thirteen of the 23 general surgery residency graduates responded to the survey (57% response rate). Most respondents were between the ages of 30-39, participated in additional training after residency, did not pursue formal education in HVC outside of residency/fellowship and work in private practice settings ( Table 1). Half of respondents felt that they received formal instruction in HVC during residency. All were involved in a QI project, and eight respondents (60%) reported that they led a QI project ( Figure 1). Seven respondents (63%) described their QI project as a positive experience. All respondents agreed that they had adequate knowledge to lead a QI project, while fewer respondents (85%) felt that they had adequate skills to do so ( Figure 2). Many respondents agreed that knowledge and practical skills in HVC are important in their day-to-day clinical practice (92% and 77%, respectively). When asked what an ideal HVC curriculum would include, cost containment was ranked first, respectively). When asked to rank core HVC concepts in order of importance, all respondents ranked patient safety as the most important concept (Table 2). No statistically significant differences were found in perceptions of knowledge and skills in private versus academic practice. Qualitative analysis of free text responses revealed lack of understanding of QI methods, with many respondents believing that protocol design and implementation constituted a complete QI project. In addition, comments reflected frustration with being asked to "do more work" during residency and a belief that QI is something that only concerns the business leadership of hospitals.

Discussion
QI is the component of HVC that has been a primary focus of GME in surgery, as evidenced by the formation of the American College of Surgeons (ACS), Quality in Training Initiative (QITI), a "multidisciplinary collaborative of academic affiliates of the ACS National Surgical Quality Improvement Program (NSQIP)," which aims to develop a common surgical QI curriculum (Kelz et al., 2013). Despite the importance placed on QI by the ACS, a survey of surgery program directors revealed that they consistently ranked the SBP competency as the least important of the six core competencies (Kelz et al., 2013). A systematic review of the literature to identify surgery residency QI curricula, published in 2014, identified just six surgery-specific curricula (19% of the 31 articles included for review) (Medbery et al., 2014). None of these articles outlined a complete curriculum. Medbery and colleagues concluded that there is a gap between the current and ideal state of QI education in surgery residency programs nationally (Medbery et al., 2014). While elements of a successful QI program exist in programs and the literature, there is, at present, no comprehensive curriculum available.
Our results further define this gap from the perspective of recent graduates at a high volume, tertiary care, academic institution. One key finding of this work is a discrepancy between formal instruction in QI principles and participation in QI projects. All respondents felt that they participated in a QI project, while only half reported that they received formal instruction in QI principles. This suggests that the intention to foster QI exists, but robust didactics and teaching infrastructure are not yet in place. The issue with this disparity is highlighted by one free text comment describing a residency QI project as "a literature review to perform a grand rounds and cost analysis…paving the way for creation of a new trauma algorithm." Again, the thoughts and intentions are there, but the actual process of QI is lacking. QI principles, whether using Plan Do Study Act, Lean or other popular methodologies, espouse small cycles of change with consistent follow up investigation of impact. Implementation of new protocols appear to be viewed by novices as QI, which suggests that residents may be missing the core concepts of baseline investigation and then analysis of whether or not the intervention resulted in an improvement. This is an important issue for developing and implementing effective educational programs in QI. We must foster the mentality that QI is an ongoing process, rather than a single event.
In order to develop the requisite knowledge to understand QI principles and skills to effectively participate in QI projects, trainees require formal instruction or mentorship. One respondent noted that during fellowship they participated in a QI project, but that "no new skills learned, no professional benefit," again, highlighting the importance of formal instruction and critical reflection in a skill set that can be meaningfully applied in professional practice. An emphasis on developing this skillset may be an important facet of gaining resident buy-in to QI projects, rather than their current view of yet "another hoop to jump through." Educators invested in cultivating a QI culture but lacking the institutional support for a widely implemented formal curriculum should spend time with their trainees after "completion" of a QI project to help trainees reflect on the ongoing process of QI. Just as in simulation-based training, debriefing is an essential component of QI training as it encourages participants to understand their QI experience in the broader context and translate it to their daily and future practice (Gururaja et al., 2008).
Formal didactics and mentorship are traditional models of education, but the nascent field of QI should be open to other models and engagement of residents in QI. The idea behind participation in a QI project is to immerse the residents in the process of QI, to deepen their understanding of underpinning theories, and to develop their practical skills. Another paradigm, championed by Sarwar and colleagues, is to create a resident leadership position for QI in the department (Sarwar et al., 2013). This serves to not only prepare an individual resident for a career in QI but also may improve acceptance of QI mechanisms in the department. The use of peer education, rather than hierarchal dissemination of information, may prove invaluable to gaining resident buy-in. Furthermore, their program employs the use of a resident only QI conference, as they have found that residents may be reluctant to seek out guidance when faced with issues of adverse events, but these are the very issues that benefit from investigation through a QI lens. Though this type of program was initiated in radiology, any surgical educator can see the value of peer support and education in the face of poor outcomes and adverse events. Yet another immersive experience, as promoted by Smith and colleagues, involves residents in a monthly QI conference (Smith et al., 2012). One by one, each resident investigates a patient care issue, completes a limited RCA and then presents at conference and engages in discussion of potential interventions. The success of these interventions is then tracked and then fed-back to the residents. In this case, the residents are not necessarily responsible for design, implementation and analysis, but instead are the ones who simply develop the idea. This model, unlike that of Butler and colleagues, was not found to be particularly burdensome by participants (Smith et al., 2012). It would, however, require a robust change and improvement system to be in place at the institution. Facing a dearth of formal curricula in surgical QI education, surgical educators must look to other disciplines and be willing to employ non-traditional models of teaching to meaningfully engage residents in QI.
Interestingly, despite 50% QI project participation and 60% leadership of a QI project, 85% of respondents felt that they received "adequate instruction" in HVC concepts and 100% felt that they were prepared to lead a QI project in practice. This, again, highlights the dichotomy between education and practice. Given prior research that suggests that residents have difficulty prioritizing QI in relationship to other clinical responsibilities, one could hypothesize that respondents may have expressed that instruction was adequate because, though it was minimal, they do not care to receive more and felt other issues to be more valuable in their schema. This is, sadly, supported by the response to questions related to what an ideal HVC curriculum should include, in which 40% of respondents felt that formal instruction in QI concepts and methods is not needed. It is also possible that while all respondents feel prepared to lead a QI project, there may be a discrepancy between their perceived and true knowledge and skills in QI. Or perhaps residents receive adequate training on QI during medical school; though based on our personal experience with incoming interns this is not the case. Further study is needed to determine whether or not their perception of preparedness reflects their true knowledge and skills.
The ACGME and ACS require that residents lead a QI project to achieve level four of the milestones (the level targeted for graduation)(Accreditation Council for Graduate Medical Education, 2015). Just 60% of respondents reported that they did in fact lead a QI project and only 45% felt that the opportunity to lead a team QI project was important. This suggests that recent general surgery residency graduates do not buy-in to the ACGME Milestones that inform the level of competence in SBP and PBLI that is expected of a general surgeon entering practice. Curricula in QI and other HVC concepts will not be effective without resident engagement. Making a strong case for the importance of leading QI efforts will be an important consideration for any QI curricular innovation.
The majority of respondents (85%) believed that hospital leaders in their current institution felt that knowledge and skills in HVC are important, however, one respondent commented that HVC is "something that the business leadership of hospitals concerns themselves with." This is consistent with existing literature that has identified the effective integration of GME HVC curricula into existing healthcare systems and their established clinical operations as a challenge (Tess et al., 2015). Graduate medical education and clinical operations leadership and their missions have typically existed in silos, hindering cross-talk between these groups. Implementation of effective HVC curricula requires access to the infrastructure and resources for improving value in healthcare, which are likely to reside within clinical operations. Hospital operations and GME alignment will need to be addressed and fostered to support HVC curriculum development and implementation.
Several changes to the HVC curriculum at our General Surgery Residency Program were made to address the results of this survey. First and foremost, a formal curriculum, including learning goals and objectives and scheduled, mandatory instructional sessions, was developed and implemented. The curriculum includes mandatory leadership of a QI project of each residents' choosing during the PGY-2 year of training. These projects are supported by health systems engineering experts as well as faculty mentors. Residents are required to present their projects at an annual grand rounds forum, which is an important opportunity for faculty to show their support and gratitude to the residents for their contributions in providing high value healthcare to our patients. Finally, the importance of a faculty champion to coordinate and oversee HVC curricula cannot be overstated. This individual can provide consistent messaging to learners regarding QI principles, project coaching, and the overall importance of engaging in such work. Assessment of learning and evaluation of this new curriculum must be conducted to understand whether or not these changes have resulted in improved resident learning and engagement.

Limitations
This study has important limitations. First, the sample size is small and does not allow for analysis of statistical significance of responses. Second, this is a single institution study and the results may not be generalizable across general surgery residency training programs. Though we do believe our program is reflective of a large, academic, tertiary care referral hospital system. While the response rate was respectable and similar to, if not greater than, most survey-based studies, there remains the possibility of response bias.

Conclusion
Recent general surgery residency graduates value knowledge in HVC concepts in their current practice, however, many did not have a positive experience with QI projects while in training and even fewer expressed interest in leading QI efforts. Given that QI work is valued by modern health care systems, our results suggest a need to improve, and perhaps formalize our existing HVC curriculum. Further research is underway to assess the scope of this problem nationally.

Take Home Messages
High value care (HVC) principles are of growing importance in medical practice 1.
Formal education in HVC principles is lacking in graduate medical education 2.
Did you participate in additional training (accredited or non-accredited fellowship) after residency? If yes, please name the specialty: What is your current practice: Private, Academic, Mixed, None (still in training) What is your current position/title: Fellow still in training, Clinical Instructor, Assistant Prof, Associate Prof, Other (please name)

Value-driven Healthcare Education in Residency:
Did you receive formal instruction in value-driven healthcare during residency at the University of Utah? If yes, please mark each component of value-driven healthcare that you believe was addressed: quality improvement, safety, service, cost Did you lead a quality improvement project during residency? Did you participate in a quality improvement project during residency? If yes, how would you characterize this experience? Very positive, somewhat positive, neither positive nor negative, somewhat negative, very negative; Explain:

Value-driven Healthcare Education in Fellowship:
Did you receive formal instruction in value-driven healthcare during fellowship training? If yes, please mark each component of value-driven healthcare that you believe was addressed: QI, safety, service, cost Value-driven Healthcare Education, Other: Did you pursue formal education in value-driven healthcare outside of residency and/or fellowship curricula (certificate programs, CME, etc.)? If yes, please name/describe:

Current Need for Value-driven Healthcare Knowledge and Skills:
Rank the following value-driven healthcare concepts in order of importance for your day-to-day clinical practice (1 = most important, 4 = least important):