INTRODUCTION

While both administrators of pay-for-performance programs and practicing physicians strive to improve healthcare quality, they sometimes disagree on the best approach.1 On one hand, it has been demonstrated that physician behavior can be modified through financial incentives,2, 3 and some real-world experience does demonstrate improved health outcomes for patients.4, 5 On the other hand, other investigators suggest that pay-for-performance (P4P) programs engender burdens that do not justify equivocal results.6 A recent literature review summarized two faults of contemporary P4P programs by stating that “institutions are not responding strategically to P4P incentives” and “safety-net providers are performing worse.”7

The Medicare Access and CHIP Reauthorization Act of 2015 (MACRA) established a physician payment system that incentivizes improvements in US healthcare.8 Under MACRA, physicians participate in the Quality Payment Program either through the default track—the Merit-Based Incentive Payment System (MIPS)—or via Advanced Alternative Payment Models, such as risk-bearing accountable care organizations. In its 2017 form, the MIPS program required about 700,000 physicians9 or their groups to report data related to (1) healthcare quality, (2) cost, (3) improvement activities, and (4) use of certified electronic health record technology to the Centers for Medicare and Medicaid Services (CMS).10, 11 For each calendar year of participation, CMS calculates performance scores for physicians based on submitted data and adjusts Medicare Part B fee-for-service reimbursement rates 2 years in the future to reward good performance or penalize poor performance. For example, based on the first year of MIPS performance data (2017), CMS has adjusted 2019 fee-for-service physician reimbursement rates by up to ± 4%. By 2022, payment adjustment will increase to a maximum of 9%.12

As the newest national P4P program, MIPS entered the P4P debate as a controversial program even before its implementation, primarily because of its extensive and complex quality reporting requirements. 13, 14 While scores from the first year of the program were recently released,15 little is known about how the program is affecting physicians and their practices. Thus, in the interest of guiding federal efforts to improve healthcare quality under MIPS while minimizing burdens for physician practices, we undertook this study to describe PCP perspectives and self-reported practice changes related to quality measurement under MIPS and collect and disseminate PCP suggestions for improving the program. We used a qualitative research design because the MIPS policy is still early in implementation,16 and we believed that an in-depth exploration of physician perspectives would best facilitate exploration of a wide range of physician responses. We interviewed primary care physicians (PCPs) for the following three reasons. First, many MIPS measures pertain to preventive care and chronic diseases managed by PCPs.17 Second, PCPs may have previous experience with quality measurement under previous programs18 such as CMS’s Physician Quality Reporting System19 or the Healthcare Effectiveness Data and Information Set (HEDIS).20 Third, a large percentage of primary care physicians work in small community practices21 and thus are likely to participate in decision-making about MIPS.

METHODS

Design, Setting, and Participants

Between November 2017 and June 2018, we conducted a qualitative study using semi-structured interviews with PCPs trained in family medicine or internal medicine who were expected to report under MIPS in 2017. MIPS eligibility was verified using the online tool at https://qpp.cms.gov/participation-lookup. We used maximum variation purposeful sampling22 to guide recruitment. We began by seeking participants of different practice sizes (solo, 2–14 physicians, 15 or more physicians); practice environments (urban, suburban, or rural); and practice regions (West, Northwest, Midwest, South, Northeast, or Mid-Atlantic). We also attempted to obtain participants with a variety of years in practice; primary compensation model (salary, fee-for-service, or capitation); and percentage of patient panel covered by fee-for-service Medicare. After each interview, we reviewed the characteristics of the study sample for distribution on the above criteria and targeted recruitment of the next participant accordingly. Recruitment was done nationwide by telephone and email.

Description of Interview Sessions

One author (CB) conducted all interview sessions, and sessions lasted between 30 and 60 min. Interviews were conducted in person for participants living in Los Angeles who were available to meet. All other interviews were done by telephone. At the beginning of each interview session, a brief survey was administered to confirm eligibility for the study and characterize physicians based on the above purposeful sampling criteria. Our interviewer used a semi-structured interview guide to ask participants questions relevant to our study objectives, and all interviews were audio-recorded and transcribed by a professional transcription service. (See Appendix A for a copy of the survey and the semi-structured interview guide.) Both the brief survey and semi-structured interview guide were piloted and iteratively edited with the help of five community primary care physicians prior to the beginning of the study.

Analysis

To analyze data from the semi-structured interviews, we used thematic analysis guided by a framework analysis.23 First, we identified all content related to each principal study question. Using the qualitative analysis software Atlas.ti Version 8 (ATLAS.ti Scientific Software Development GmbH, Berlin, Germany), two investigators (CB and ME) independently reviewed and deductively coded the transcripts by the major domains of the study aims and the interview guide. Then, they independently inductively open-coded the data. After 18 interviews had been coded, the coders noted that they were no longer creating new codes. To ensure thematic saturation had been reached, two more interviews were conducted and coded, again with no new codes created. The coders met to discuss their findings and developed a final list of sub-themes and aggregated representative quotes from participants that represented the sub-themes. Discrepancies were adjudicated by discussion and consensus between the two coders.

Ethics

The institutional review boards at Cedars-Sinai Medical Center and the University of California, Los Angeles, both approved the protocol. The institutional review boards deemed this a low risk study, thereby allowing us to electronically send each participant an information sheet electronically prior to phone contact and then obtain oral consent at the time of the beginning of the interview session. Participants were provided a $50 gift card as compensation.

RESULTS

Twenty PCPs participated in the study between November 2017 and June 2018. Eighteen of the twenty participants were board certified at the time of interview (7 in Family Medicine and 11 in Internal Medicine). Nine worked in small practices (groups of < 15 providers), and five worked in rural areas. Most PCPs worked in practices in which the majority of revenue was derived from fee-for-service payment arrangements. In-person interviews were different than phone interviews in that participants all practiced in urban or suburban settings in the greater Los Angeles area; however, data and the tone of the interviews were similar for both interview modalities. Table 1 describes participants and their practice environments in further detail.

Table 1 Characteristics of participating physicians and their practices, N = 20
Full size table

When asked to describe their perspectives related to participation in MIPS, most participants identified at least one advantage and one disadvantage. The most common advantage was that the program had encouraged PCPs to develop systems for quality monitoring and improvement that led to recognizing a care gap and working to improve in that area. In terms of program disadvantages, PCPs expressed concern that the MIPS’s administrative burdens could lead to downstream harms for patients and physicians and that MIPS could lead already strained practices serving the most vulnerable patients to be unfairly penalized. PCPs commonly reported that they were responding to program requirements by improving systems for collection and reporting of quality data by using strategies related to technology and staffing. Some PCPs reported feeling overwhelmed by administrative burdens and said they were considering joining larger practices or retiring early. Suggestions for program improvement included simplifying program requirements to make it less burdensome and more consistent from year to year, modifying methodology to protect practices serving vulnerable patients, and improving communication between program administrators and participants. For the remainder of the results section, we will present these findings in further detail accompanied by quotes from participants. Table 2 presents an overview of the major findings from interviews. More quotes related to advantages and disadvantages of MIPS are available in Appendix B, and quotes related to self-reported practice changes are available in Appendix C.

Table 2 Overview of study questions and findings from interviews
Full size table

Advantages of MIPS and Related Self-Reported Practice Changes

Some participants, most of whom participated in quality management in their organizations, expressed positive feelings about the impact of MIPS. One PCP was enthusiastic about the validity of the quality measures under MIPS, as he perceived that they were evidence-based, with strong process-outcome links: “I reviewed the measures last night and I really agree with them. I think they’ll definitely increase quality. For primary care physicians, the measures include basic things in terms of blood pressure control and smoking and weight loss and statins for cardiovascular disease that I think there’s no argument in my mind. In those domains, there’s a really strong evidence base to reduce morbidity and mortality.” Participants also reported having new technology and staffing resources dedicated to quality measurement, improvement, and reporting, which they perceived as steps toward optimizing healthcare quality. As an example of MIPS encouraging PCPs to acquire new technology, one respondent mentioned that a bonus from MIPS help justify the cost of buying retinal screening equipment for diabetic patients if her clinic could improve screening rates: “We just bought our retinal camera for $5,000. We take pictures of the retina, and now they’re going to get read by a retinal specialist. It was always really difficult to get a diabetic eye exam done at a different clinic and then get the result back from the eye doctors. So, now we’re just going to get that piece done ourselves and maybe next year we’ll pick that as a measure, if we seem to be getting good results.”

In terms of other new and improved uses of technology, nearly all PCPs using electronic health records mentioned that the system had recently been modified, or “tweaked,” to assist with quality measurement related to MIPS. These modifications tended to involve small changes to improve documentation such as using or editing templates, creating order sets or pathways, and ensuring more reliable input of data into the electronic health record: “They’ve kind of tweaked our electronic health record so that certain orders will get captured and documented and then, be able to send into data analysis for MIPS. For example, there’s a current work order to do a better job capturing all the counseling we do for the folks with a BMI greater than 25 and the documentation of follow-up and a plan.” Several PCPs who did not have electronic health record systems reported plans to acquire them: “We shockingly are still on paper which makes complying with MIPS very, very challenging. So, we’re now participating in the process of trying to convert within the next year to an electronic health record.”

PCPs also commonly reported having made or planning to make staffing changes to assist with tasks related to optimizing performance on metrics related to chronic care management: “We have increased our staff. We need more people working, just to manage all this data or at least attempt to manage the data. So, I went from having one medical assistant to having two medical assistants.” Another PCP described a need for an entire population health team as a necessity for improving primary care: “I’ll tell you, you need a team of people to help you do that work. You need other people, all focused on the similar goals, but really they’re there to help you with all of these aspects of population health management and really leaving you to do higher-level care.”

Disadvantages of MIPS and Related Self-Reported Practice Changes

Some PCPs believed that burdens related to data collection and reporting under MIPS were leading to a misguided diversion of resources because staff were now focused on data entry instead of direct patient care: “We are highly trained, highly paid data entry specialists, which is not a good use of our time.” Several PCPs mentioned specifically that they believed patients would be less satisfied with their care because of the diversion of resources: “I do believe that the process of [reporting under MIPS] is going to take up more time, time we don’t have already, and I do think it’s going to take away from patient care. And if I define quality partly as patient satisfaction, I just can’t believe if I spend less time with patients and more time on proving my quality, I think it’s going to be a net loss for the patient, personally.” PCPs provided specific examples of how resources were being misappropriated during the implementation of quality improvement plans. For example, “Our group recently got a grant to do a quality improvement initiative. The doctors in the room wanted to use it either to improve telephone outreach to our patients or to hire [health promotors] to visit patients at home. Then, the chief operating officer came into the room, told us we weren’t doing so great with colon cancer screening, and strong-armed us into ultimately using hundreds of thousands of dollars to purchase stool cards. There are so many better things that that money could have been used for. So, that’s an example of how too much focus on these metrics cost us to lose sight of what’s most important.”

Many PCPs characterized the burdens of MIPS as having a negative impact on physicians and physician practices. One PCP, this one in a small practice, characterized MIPS as only one of several pay-for-performance systems that she participates in, explaining that participating in multiple programs is burdensome: “Yes, so, well, we participate in MIPS. We also participate in a pay-for-performance system with a big commercial insurer in our region …. We also do [Patient-Centered Medical Home (PCMH)] which requires membership in a physician organization that makes sure we stay on track with our PCMH designation. Then there’s another thing through our Medicaid carrier. The types of metrics they focus on have some overlap, but we don’t have a targeted program to address them because it’s all a little bit of a moving target.”

Multiple participants expressed the belief that the administrative burdens of MIPS were negatively impacting professional satisfaction: “I think quality measurement is at least a gray cloud over the profession generally and over primary care.” Sometimes, worsening professional satisfaction was connected to worsening relationships with patients. “I think [MIPS] has been very detrimental to morale. I really do. I would put the doctor-patient relationship as one if the top things that has suffered. I would also put physician autonomy up there. I also feel like sort of the joy in practice, all of those kinds of things, which are harder to measure, have been neglected.”

Some PCPs in small practices described feeling so overwhelmed by the idea of complying with MIPS that they were interested in making structural changes in their practices. One PCP mentioned that he was considering entering into a new affiliation with a larger organization to reduce his small group’s burden of administrative work related to MIPS. “We may affiliate with a new group. We’re in discussions with them and so, in our minds, we’re thinking at least form kind of a loose administrative affiliation with them, then of course, they’re going to be helping us with quality measurement because they have all of that down.” Another said, “One of the reasons we’re tempted to affiliate [with a larger practice] is exactly why we’re on the phone today. Because we’re a really small group, mom and pop, old-fashioned practice. We don’t really want to get diverted too much from spending time on things that are not directly patient care-related. And we recognize that we also want to be quality.”

Participants’ Suggestions for Program Administrators

We asked PCPs to describe recommendations for how program administrators could improve MIPS. Many respondents’ recommendations aligned with their feedback about the disadvantages of the program: common suggestions included simplifying the program to reduce administrative burdens, adding protections for practices serving vulnerable patients, and improving communication between program administrators and participating physicians.

Many respondents recommended reducing burdens of the program, and a common suggestion was to leverage the electronic health record or Medicare claims data to make data collection easier. “I thought there were things that Medicare could find out without us submitting data, just from finding out that I’m the physician and then my patient is getting the flu shot, let’s say, either here or there. So therefore, I get credit for it. Why do I have to do any additional data entry?” Another PCP recommended simplifying the measurement process: “It is very complicated, and I don’t think it needs to be. This is what happens when we abdicate—When the profession has lost control and we end up asking payors and the government to do something for us.”

Participants also voiced fears that practices serving vulnerable patients might be penalized under MIPS, and they recommended changes to methodology that would protect such practices. For example, many believed that the denominators of some measures did not include enough room for exclusion of patients who refused certain services or patients with social needs. Regarding patients who refused services, one PCP explained, “Like my patient with untreated psychosis who we’ve tried a lot of times to plug into mental healthcare but who is still really resistant to it. It sort of isn’t fair to her or to my team to have her be counted in the denominator of people where we’re trying to do vaccinations and colorectal cancer screening.” Regarding patients with complex social needs, one PCP stated, “There’s much higher social complexity and addiction and other issues in the southern part of our city. And so, it’s going to be a lot easier for me to hit 80% colorectal cancer screening than my colleagues down south where there’s a lot more barriers to doing that preventive healthcare.” PCPs said that making additional accommodations for special populations would be necessary, or else physicians might neglect patient preferences or decide to avoid treating patients with complex social needs.

The level of knowledge about the MIPS program varied among those participating in the study, but a common theme was that communication between program administrators and participants should be improved. One provider declined to offer suggestions related to the MIPS program, even after she listened to an explanation of how MIPS functioned. “I’m afraid I’m just not knowledgeable enough about the program to really give any suggestions.” Another respondent reported a higher level of knowledge but remained confused about how CMS obtained data: “I don’t know how the data is extracted. I guess, honestly, I think it’s still a mystery.” Finally, one PCP provided a clear explanation of a challenge that lies ahead for CMS in disseminating accurate and concise information about MIPS: “The more that CMS can sort of make the incentives and the changes sort of understandable and accessible—for primary care providers in particular but really for all clinicians—the better.” See Appendix D for more quotes from participants describing how the MIPS program could be improved.

DISCUSSION

The explicit goal of MIPS is to tether healthcare quality and payment together so that patients experience better health outcomes.24 CMS has publicized numerical goals for the percent of healthcare payments tied to quality over time,25 and our findings demonstrate that physician practices are progressing in collecting healthcare quality data and working to improve performance on MIPS measures. Moreover, many stakeholders have forecast structural changes in physician practices because of MIPS,26 and we do find evidence that physicians in small practices are considering joining larger practices that would relieve them of burdens of quality reporting. Physicians’ fears that small and rural practices will perform worse under MIPS appear to be true,15, 27 and thus the debate surrounding practice consolidation will likely continue for years to come.

There are few published studies of MIPS’s impact to date because 2017 performance data were released only recently, and payment adjustments from the 2017 performance year first took effect in January of 2019. However, existing publications suggest that physicians are divided about the program’s potential for positive impact: Liao et al. performed an online survey of 1431 physicians about the MIPS program’s potential impacts in 2017. The group found that 55% of physicians believed that MIPS would somewhat or significantly improve healthcare quality, 31% believed it would have little or no impact, and 14% believed that it would actually reduce healthcare quality.28 Another recent study rated the quality of the measures available in MIPS and found that many were based on poor quality evidence.29 More than 10 years ago, Young et al. demonstrated that, while physicians felt positively about the concept of quality improvement under pay-for-performance programs, they were “ambivalent about specific features” of methodology.30 It appears that clinicians’ perspectives related to quality measurement and pay-for-performance have changed little over time despite increased exposure to various programs.

MIPS-specific concerns expressed by PCPs in our study elaborate on those recently described by multiple stakeholders. Regarding program design, for example, representatives from MedPAC recently advocated for redesigning MIPS so that it would de-emphasize process-oriented measures and instead focus on patient-oriented ones such as patient experience. Furthermore, they criticized MIPS measurement and reporting methodology as being excessively burdensome and unreliable.13, 14, 31 Regarding patient and provider experience in the program, the survey by Liao et al. described provider concerns related to unintended consequences of MIPS, and relayed the finding that physicians felt they were losing control over key features of their practices.28 These are all concerns that our participants voiced as well, which adds credence to MedPAC’s criticisms. Our participants additionally demonstrated concern that MIPS is leading practices to divert already scarce resources away from direct patient care, and they feared that penalties on practices serving vulnerable patients would reduce critical access, which was also a finding elicited in a study that focused on small rural practices.32

Our participants had many recommendations for how to improve MIPS, including simplifying the program to reduce administrative burdens, minimizing unintended consequences practices serving vulnerable patients, and improving communication between program administrators and participants. These suggestions parallel those of MedPAC31 and Liao et al.28 Notably, Liao et al. also emphasized the need to align program goals with physician perspectives to maximize engagement. Future, larger-scale investigations into physician perspectives, experiences, and practice changes will be necessary so that stakeholders can monitor the MIPS program for unintended consequences and determine how to adjust the program so that it has the best possible impact on physicians, the health system, and patient outcomes. Additionally, program administrators should work to identify practices that are slow to adopt to MIPS so that technical assistance can be well-directed, thereby avoiding adverse consequences for vulnerable patients.

Limitations

A number of special considerations should be taken into account to understand our findings in light of the study design. First, small-sample qualitative studies are useful for developing theories and identifying questions that deserve future investigation. As such, this study may be limited in its transferability, though we attempted to mitigate this by including PCPs from various practice environments across the USA. Second, our study sample was limited to PCPs, while MIPS includes physician and non-physician providers from many specialties. Third, the characteristics of our sample may not reflect those of the population of physicians nationwide; for example, we interviewed a large proportion of salaried physicians, who may not perceive as much of an impact from the program as physicians who are paid purely by fee-for-service. Fourth, some of our respondents spoke in generalities about quality measurement and pay-for-performance programs when asked specifically about the MIPS program. Despite our efforts to explain the program to respondents who were less familiar with it and frame respondents’ perspectives within the context of their knowledge of the program, their answers to our questions may have been limited because of lack of familiarity with MIPS and/or confounding with other programs. Fifth, we made attempts to ensure rigor and limit biases by employing reflexivity and bracketing33 during study design, recruitment, data collection, and analysis, but there remains risk that investigator biases may have limited the reliability and/or validity of our findings.