Evaluation of a Curriculum in an Internal Medicine Residency and Patient-Centered Medical Home Rotation

Introduction: The Patient-Centered Medical Home (PCMH) model aims to transform primary care by delivering high-quality, safe, comprehensive, patient-centered, and accessible care. Few evaluations of PCMH curricula for internal medicine residency programs have been published. The objective of this study was to develop and implement a PCMH curriculum, a new PCMH rotation, and to redesign the resident ambulatory care clinic and private primary care practices of the residency program. Effects were evaluated by examining: 1) residents’ attitudes towards primary care and PCMH, 2) residents’ skills in communicating and managing care and 3) patients’ assessments of practice accessibility. Methods: From 2010-2015, we implemented a multi-modal program combining practice redesign, a lecture series, a one-month PCMH rotation, clinical activities, a quality improvement project, and self-reflection at an academic community hospital, a Federally Qualified Health Center (FQHC) and faculty practice offices to internal medic ne residents. Evaluations were conducted through resident self-assessment, patient assessment of practice accessibility, and objective structured clinical examinations. Results: Twelve residents [41% (12/29)] completed all components of the curriculum. After the rotation, residents thought their PCMH skills improved in 13 components of PCMH (all domains: p<0.02). Residents’ attitudes towards primary care and PCMH did not improve from 2010 to 2012 [response rate: 38% (11/29) and 44% (13/29), respectively]. Patient ratings improved, mostly in practice accessibility (p<0.001) and test notification (p=0.01). Conclusion: By engaging in the PCMH curriculum and practice redesign, residents showed improvement in PCMH skills across all domains, and patients demonstrated increased satisfaction with the practice. This curriculum in PCMH can be utilized as a useful model for other internal medicine programs.


Introduction
The model known as Patient-Centered Medical Home (PCMH) aims to transform primary care by changing how it is organized and delivered [1]. PCMH encompasses these five functions and attributes: Comprehensive care, patient-centered, coordinated care, accessible services, and quality and safety [1]. Patient outcomes have been shown to improve as a result of the PCMH model [2]. PCMH teams can optimize the model's potential if they learn to coordinate, communicate, and collaborate [3]. Since the introduction of the PCMH, residency programs have been challenged to develop curricula and provide a clinical experience that will prepare residents to practice in such settings [4]. Physicians must learn to work in interdisciplinary teams and share decision-making [5] to improve care processes, to enhance quality and safety, and to exercise managerial skills, such as managing relationships and finances. Few residents are familiar with the PCMH model [6] and few residency programs offer their residents a fully realized PCMH [7].
Some residency training programs have reported work in redesigning their curricula and clinical components to train residents to engage with chronically ill patients and to measure their proficiency in doing so [8,9]. Previous publications have also described curricula that address individual components of the PCMH, such as pain management [10], team-based care [11] and group visits [12]. However, prior published studies did not include resident selfassessments of their mastery of specific PCMH components, nor do they provide patient assessments [6]. Lastly, most of the published studies are based on family practice residency programs, rather than internal medicine [12].
practices affiliated with the internal medicine residency program. We evaluated the impact of the PCMH curriculum and practice redesign on 1) residents' attitudes towards primary care and PCMH, 2) residents' skills in communication and patient care management and 3) patients' assessments of practical accessibility and self-care support.

Setting and participants
Griffin Hospital is a 140-bed, mid-size academic community hospital in Connecticut, and it is a teaching affiliate of the Yale School of Medicine. The residency programs consist of an internal medicine program (12 residents); a preliminary year program (8 residents); and a combined internal medicine/preventive medicine program (12 residents). Residents receive training in outpatient medicine primarily at two sites: A Federally Qualified Health Center (FQHC) and the primary care practices of the Griffin Faculty Physicians (GFP). The residents complete approximately 12,000 patient encounters per year at the FQHC. The majority of these patients are Caucasian (63%), followed by Hispanics (22%) and African Americans (15%). Most of these FQHC patients have Medicaid (61%) and 17% are self-pay. The five primary care providers of the GFP offices conduct approximately 8,700 visits per year. Ninety-two percent of GFP patients are Caucasian (92%), 3% are African-American, 3.4% are Hispanic, and 2.4% are Asians. About 20% of these patients are self-paid or have Medicaid.

Practice redesign
Like most previous efforts to implement a new care model [8], ours attempted to combine a curriculum introduction with practice redesign. We collaborated with TransforMED, a proof-of-concept demonstration project through the American Academy of Family Physicians [13], which, through a series of two site visits and regular conference calls, provided a framework for the practice redesign as well as a national comparison group for patient evaluations of the practice. We conducted monthly conference calls with our consultants and semi-annual meetings with residents to redesign the practice. We made multiple changes to both the resident clinic and the faculty practices, including schedule changes, team-based care through the creation of resident provider teams, patient goal documentation, and active management of critical conditions through chart audits. Both the FQHC and GFP sites implemented a series of changes to adopt the PCMH model over the course of three years.

Lecture series
We designed and created a new lecture series in PCMH led by TransforMED and the Connecticut State Medical Society (CSMS). Topics included 'Introduction of PCMH' , 'Basic aspects of the PCMH at Griffin Hospital-what is missing' , 'Team-based Care' , 'Care coordination and clinical integration' , 'Evidence based medicineevaluation and comparative effectiveness within the practice' , and 'Enhanced access to medical care and practice/physician payment' .

Rotation
In their final year of training, residents participated in the four-week PCMH rotation. The rotation included four main components: selfstudy of curricular material, clinical activities, practice management/ quality improvement (QI), and self-reflection ( Figure 1).  [14] and quality improvement [15].
2) Ambulatory primary care: Clinical activities took place at both the FQHC and GFP sites: Residents attended their weekly continuity clinic at the FQHC as usual. Also, residents completed an ambulatory care block at the Griffin Faculty Practices, during which they spent two to three days per week at the GFP. At both sites, residents were expected to observe and reflect on any differences in the workflow and staff and patient behaviors between the FQHC and the GFP. They also observed how experienced clinicians integrate Electronic Health Record (EHR) documentation, coding, and billing in their workflow, and saw policies in action that govern same-day access and optimal use of staff and resources. The supervising physicians observed residents and provided feedback on patient engagement, workflow habits, and documentation.
3) Practice management/quality improvement: Residents learned about practice management tool through their readings (as discussed above), attending practice meetings, shadowing various office staff members, and by performing a quality improvement (QI) project. The resident chose a topic for this project at the beginning of the rotation and could modify it during the rotation. Examples of resident projects include: Analyzing and improving continuity of care at the resident clinic; Assessing the quality of care of diabetic patients; and rewriting educational material distributed at the office to make it more accessible to appropriate for patients with low health literacy. Residents presented progress reports on their QI projects at practice team meetings, and

4) Reflection:
Residents were expected to reflect on their experiences at least twice a week in an email to the rotation supervisor. In these messages, residents reported on their readings or webinars completed that week and reflected on what they had learned about PCMH and how their work at the clinic compared with their understanding of the PCMH. We provided the residents a rotation handbook containing suggested topics for reflection and tip to organize reflections. The rotation supervisor replied to the reflective emails with comments. Resident output for this rotation consisted of the reflective emails, a report on the results of their QI project at the end of the rotation, and an email to their co-residents at the end of the rotation in which they summarized take-home points they felt were important to their peers. Residents had two supervisors during their rotation: their clinical supervisor, who was one of the primary care physicians in the faculty practice office; and the rotation supervisor, who was the medical director of the practice group.

Resident and rotation evaluations/statistical analyses
For the purpose of overall project evaluation, we combined resident self-assessments, patient assessments, and faculty assessments. We used SAS 9.2 (SAS Institute Inc., Cary, NC) to perform statistical analyses. Specific statistical analyses and tests are described in each subsequent section.

Evaluation of residents
The rotation supervisor and the clinical supervisor evaluated the residents at the midpoint and the end of each rotation. Supervisors based the evaluation on direct observation of the residents' interactions with patients and staff, and on the quality of the residents' documentation, the quality of the reflective emails, participation in practice management meetings, and an evaluation of the QI project.

Rotation evaluation of residents
The residents rated their PCMH competencies before and after they participated in the PCMH rotation in a retrospective before-and-after assessment. We used retrospective self-assessment because we reasoned that residents would have had little exposure to many facets of the PCMH before their rotation and would, therefore, have different criteria for assessing their skills and knowledge at the beginning and the end of the rotation. We used similar retrospective self-assessments in faculty development workshops, which correlate well with independent ratings [16], PCMH competencies included 13 domains, such as patient-centered care, ensuring safe care transitions, and fundamentals of practice management. Residents rated these domains on a scale of 1-9 (lowest to highest competency). We analyzed beforeand after-rotation scores using paired t-test.

Resident attitudes towards primary care and PCMH
We assessed residents' attitudes toward primary care and PCMH annually. We used web-based and anonymous surveys to ensure confidentiality. Statements that residents rated, such as "I have the tools and resources to be a physician in a high-performance primary care practice, " were rated on a 5-point Likert scale. We used Wilcoxon rank sum test to compare variations of responses.

Residents' communication and care management skills
In addition to requiring self-assessments, we also assessed residents' communication and care management skills twice a year using Objective Structured Clinical Examinations (OSCEs). To evaluate residents' skills in using the EHR for patient care management in transitions of care, we developed a medication reconciliation station. The scenario asked residents to change or discontinue medications while discharging a patient. A clinical Information Technology specialist rated the results on a scale from 0-100. A score greater than 70 was considered passing. We compared changes in the scores over the years of the program using Fisher's Exact test.

Assessment of the practice redesign: Patient experience assessment test
The patients assessed practice accessibility and primary care support using the Patient Experience Assessment Test (PEAT), a survey developed by TransforMED that has been used by over 30,000 patients nationally. This survey lists statements that patients rate on a five-point Likert scale. The survey listed five statements, such as 'I was able to see the clinician I requested, ' 'My care team contacts me to remind me I need to come in for a checkup, ' and 'I know my rights and responsibilities as a patient of this practice. ' We collected patient assessments in 2011 and again in 2012. We grouped 'strongly agreed' and 'agreed' against those that choose 'neutral,' 'disagreed' , and 'strongly disagreed.' We analyzed differences between the two samples by the Fisher exact test.

Results
Twelve residents completed all components of the curriculum.

Resident rotation evaluation
All twelve residents completed the self-evaluation of PCMH skills and felt they improved significantly after the PCMH rotation in all 13 components ( Figure 2). The largest changes in before and after scores were seen in the areas of open access scheduling, coding, billing, and practicing quality improvement (Table 1).

Resident attitudes towards primary care and PCMH
Our response rate was lower in both years [(38% (11/29)  There was no statistically significant change over the course of the curriculum in residents' attitudes towards primary care or PCMH (Table 1).

OSCE results
We first trialed a medication reconciliation station in 2010, but it was too complicated for the residents. In 2011, ten residents participated in the medication reconciliation station, and 2/10 reached a score over 70 out of 100. In 2012, 19/21 residents scored over 70 (p<0.001).

Patients' ratings of the practice
Forty-seven patients completed the PEAT survey in 2011 and 77 patients completed it in 2012. An overview of the patients' ratings of the practice is given in Figure 3.
Overall, patient ratings improved significantly, most notably in patient engagement, practice accessibility, and test notification ( Table  2).

Discussion
Our experience shows that it is feasible to implement a PCMH curriculum and rotation, and that including this rotation can improve the residents' experience. Such a rotation can be incorporated even if a practice is in the transformation phase of fully converting to PCMH. As in other studies, we found that building a PCMH curriculum in residency practice is labor-intensive, but rewarding [6]. PCMH provides an opportunity to improve clinic practice and residency training [17].
Contrary to our expectations, the residents' attitudes towards their work in the clinic and towards primary care as a career choice did not improve significantly. Several possible explanations come to mind. Our response rates were relatively low which was likely due to our decision to conduct an anonymous web-based survey to obtain objective data. Previously published curricula which did demonstrate a change in resident attitudes towards clinic work [16] were based on surveys of residents immediately after the rotation and were in reference to a pain clinic that had previously been perceived negatively. In contrast, our residents' attitudes towards primary care were already quite positive in most domains, so the potential for improvement was small. Compared to a national sample in which residents described their continuity experiences as stressful [7], our residents' attitudes seem to be mostly positive.
We faced numerous challenges in our efforts to implement the PMCH curriculum. We made several changes as we progressed in our quest for PCMH in response to resident and staff feedback. For example, based on our experiences, we wrote an orientation manual describing the main elements of the program. Organizing and implementing the rotation was time-intensive because it required a lot of one-on-one supervision of residents, both to foster reflections and to oversee the QI project. Once residents had completed their rotations, we were not consistently successful in keeping them involved in redesigning the resident clinic. In the evaluation, we struggled to improve response rates to our web-based surveys and to obtain patient evaluations of individual residents. We had planned to survey patients about their providers, but this proved to be too difficult logistically. Another challenge was how to develop OSCE-stations that would measure behaviors important in care management and population management. The medication reconciliation station we developed depended heavily on assistance from the hospital's Informational Technology (IT) department.
Our study is also subject to several limitations. We implemented it at a single institution and with a small sample, which may undermine the internal and external validity of the study, thus not making it generalizable to other educational programs or institutions. It was also not feasible to have a control group, given the size of our program. Essential Entrustable Professional Activities have recently been defined for PCMH [18]. Many of these have not been evaluated yet in our rotation, such as usage of EHR or use of a disease registry. Nor did we have a robust faculty development component, as in other published curricula [19], and we struggled with a lack of IT support for patient registries or patient evaluations of individual residents. Lack of IT support for higher-level EHR functions has been described previously as a barrier to implementing PCMH in resource-challenged FQHCs [20] and is not unique to our study.
Notwithstanding the above-mentioned challenges, our study has several notable strengths. In order to keep residents involved in redesigning the clinic, we instituted semi-annual meetings with all residents to review clinic procedures. Also, many resident educational activities were synergistic and overlapped with our transformation efforts. For example, residents could educate other team members about PCMH components or research best practices on the TransforMED's web exchange, DeltaNet, to help with team practice meetings. We also used an experiential design in two practices, one at a resident continuity clinic, and one in faculty practice offices which allowed our residents to compare workflows and challenges in different settings. It also helped them distinguish which challenges and stresses are common to primary care, and which are specific to primary care in a resident continuity clinic or a private practice. Overall, our rotation was well received by our residents, and it led to improvements in their self-assessed competencies. We also used a robust evaluation structure, including external assessments and patient assessments of the practice that were compared to a national sample.

Conclusion
This curriculum may provide a useful model for other internal medicine programs to develop, implement, and evaluate PCMH curricula.