INTRODUCTION

The mission of accountable care organizations (ACOs) and patient-centered medical homes (PCMHs) is to coordinate the delivery of high-quality care to improve the health of patients while lowering overall costs.1 – 3 A key measure of the success of accountable care is timely access to high-quality primary care.4 – 7 In this setting “no-shows,” or missed appointments, present a clear problem for organizations taking on increased accountability for population health. No-shows result in fragmented continuity of care, reduce access for other patients in the practice, and decrease provider productivity.8 – 16

Studies in primary care and specialty clinics have demonstrated that staff phone calls are an effective intervention for reducing no-shows.17 – 30 However, this convincing work is difficult to apply, because staff phone calls are time-consuming and costly. Moreover, while this intervention is thought to be cost effective in clinics with high baseline no-show rates, the efficacy is less clear when baseline no-show rates are low.18 , 19 , 30 , 31 Nevertheless, even in practices with low baseline no-show rates, the risk of no-show is heterogeneous—that is, some patients are more likely of no-shows than others.9 , 10 , 28 , 29 , 32 – 34 Prior work has effectively demonstrated that no-shows are predictable; validated models can accurately predict the likelihood that a patient will fail to keep a scheduled appointment.9 , 35 – 38

Subspecialty clinics have made use of the predictability of no-shows by integrating predictive modeling to target interventions to those at high risk of no-show. In a study of five oncology practices, a patient navigator successfully reduced no-shows by 42 % in a subset of patients at high risk for non-adherence, and in a dermatology practice, predictive modeling has been used for algorithmic double-booking of appointments.38 , 39 We designed and executed a randomized controlled trial in an academic primary care clinic to answer the following question: Can targeted telephone reminder calls to patients at the highest risk of no-show reduce no-show rates?

METHODS

Study Setting

We conducted this study in the Internal Medicine Associates (IMA), an academic hospital-based primary care clinic at Massachusetts General Hospital (MGH). The IMA has 80,000 appointments scheduled annually, with an average 7 % no-show rate. The IMA is staffed by 66 internal medicine residents, 50 attending physicians, 8 nurse practitioners, 20 nurses, 22 medical assistants, and 36 patient service coordinators (PSCs). PSCs are responsible for answering phones, booking appointments, obtaining records, and coordinating referrals and testing. The practice is organized into 13 pods, each of which has 2–3 PSCs. As a part of usual practice, all patients receive an automated phone call 3 days prior to their appointment using TeleVox® Appointment Reminders (TeleVox Software, Mobile, AL).

Study Design and Implementation

All patients older than 18 years who had an appointment at the Internal Medicine Associates between May and November 2013 were assessed for enrollment eligibility. We calculated the likelihood that a patient would no-show using a predictive model developed using administrative data for outpatient MGH appointments, which was described previously.38 Predictive variables included age, wait days (days between the scheduling and appointment date), insurance type, and appointment adherence history over the previous 5 years. The model was derived and validated specifically for primary care practices (see online methods supplement). The Partners IRB reviewed and approved the study protocol, including waiver of consent prior to the enrollment of any patients.

We randomized patients with a predicted risk equal to or greater than 15 % to each study arm in a 1:1 ratio based upon the last digit of their medical record number. We defined the high-risk threshold based on prior work and internal sensitivity analyses (see online methods supplement).40 Patients in the intervention group appeared in our enterprise population health management platform, TopCare. TopCare is a web-based population health management platform developed as a collaboration between the Laboratory of Computer Science at MGH and SRG Technology (Fort Lauderdale, FL).41 The tool is used to augment cancer screening and improve chronic disease management, independent of office visits, by identifying patients “not at goal” based on processed data from the EMR. We developed a module in TopCare where patients assigned to the intervention would appear in a queue 7 days prior to their appointment. Information needed to contact the patient to remind them of their appointment included, but was not limited to, patient name, phone number(s), appointment date and time, physician, and emergency contact(s). The outcomes of the call—confirmed, left message, canceled, rescheduled, no message, invalid number—could be recorded, as well as any comments. Patients were removed from the calling queue when a call outcome was recorded, the appointment date passed, or the appointment was cancelled. PSCs were responsible for making reminder phone calls and recording the outcomes using TopCare.

Intervention

All 36 PSCs received a brief one-on-one training prior to the start of the trial. The training was designed (1) to teach PSCs to access and use the calling tool and (2) to show PSCs how to use language designed to increase follow-through on good intentions.42 – 45 For example, to engage patients in concrete planning, even after patients confirmed their appointment, the PSCs were asked to follow up by asking patients, “What are you doing before your appointment?” (online appendix). PSCs had no specific clinical or social work training. There were no changes in other appointment booking practices, including double-booking during this period.

Outcomes

The primary outcome was no-show rate as defined by the number of no-shows divided by the sum of no-shows and arrived appointments. Secondary outcomes, defined a priori, included arrival rate (arrived appointments divided by all appointments), cancellation rate (cancelled appointments divided by all appointments), reschedule rate (rescheduled appointments divided by all appointments), and time to cancellation (number of days between the cancellation date and original appointment date). In these measures, “all appointments” is the sum of arrivals, cancellations, reschedules, and no-shows.

Assessing Financial Impact

In the US health care market, physicians bill for care rendered during an office visit using evaluation and management (E/M) codes, which are assigned relative value units (RVUs) that form the basis for payment. The total RVUs is the sum of three components: malpractice RVU, practice expense RVU, and physician work RVU. Work RVU is a consistent measurement of productivity that has been used in previous studies; it is independent of differences in payer mix, geographic adjustment, or specialty.46 – 51 We excluded malpractice and practice expense RVUs because they vary by location, and thus estimates generated using these figures would not be generalizable. We collected data on all professional services claims reimbursed to the practice for patients in the study during the study period. We used the 2013 Physician Fee Schedule to ascertain the work RVUs for each professional fee claim (identified by revenue center code 0983).52

For the main analysis, we reported the mean work RVUs per patient. Since there are multiple ways of assessing the financial impact, we performed two sensitivity analyses to validate our findings. First, we estimated total revenue per patient by arm using the percent-of-charge method based on all bills (technical and professional) originating from the practice during the study period.53 Using this method, charges on outgoing claims are multiplied by a constant that represents a practice’s best estimate of the revenue that will be collected from the submitted charge. Second, we modeled revenue per patient by arm assuming each of the permutations of the following conditions: all visits were evaluation and management (E&M) subsequent level 3 (99213), all visits were E&M subsequent level 4 (99214), all visits were conducted in a hospital-based clinic, and all visits were conducted in an office-based clinic. This second method generated estimates assuming all visits were reimbursed according to the CMS Physician Fee Schedule.

Statistical Analysis

We analyzed the effect of the intervention on the primary outcome, the no-show rate, by comparing the rates between the two arms to calculate an absolute risk difference. Because some patients had multiple appointments during the study period, we calculated confidence intervals and p values accounting for repeated measurements. Specifically, we used the generalized estimating equations approach to take into account correlations between observations from the same patient. We calculated the effect of the intervention on arrival, cancellation, and reschedule rates using the same methodology. We analyzed the effect of the intervention on the time to cancellation using a linear model accounting for repeated measures. Reimbursements were assessed at the patient level; consequently, we used the two-sample Student t test to determine whether differences in reimbursement between the study arms were significant.

RESULTS

Study Participants

Over a 24-week period, we screened 20,955 patients, and 2,247 (10.7 %) patients with a 15 % or greater predicted risk of no-show for an appointment in 7 days were enrolled in the study. We randomized patients to the intervention (n = 1129) or control (n = 1118). Of those assigned to the intervention group 144 (12.8 %) did not receive phone calls prior to any appointments and 162 (14.3 %) received at least one phone call before one visit but did not receive phone calls prior to all visits (Fig. 1).

Fig. 1
figure 1

Study design and allocation.

Full size image

Baseline characteristics such as clinical comorbidities, health care utilization, and measures of population health, as well as risk factors for appointment no-show, were equally distributed between the treatment and control groups (Table 1). The mean age of participants was 51.4 years, and 51.9 % were women. Within the study population, 61.9 % identified as white, and 87.0 % spoke English as their preferred language. The median number of wait days, the number of days between when the appointment was made and the day of the appointment, was 35 days. Resident physicians were the primary care physicians for 39.5 % of patients, and 53.7 % of patients had commercial insurance.

Table 1 Baseline Characteristics of the Participants
Full size table

Chronic conditions such as diabetes and depression were common, affecting 24.1 % and 22.5 % of patients, respectively. In 2012, for every 100 patients in the study, there were 51.8 emergency department visits and 19.6 inpatient hospitalizations. Among those eligible for cancer screening, 29.8 % of patients had incomplete colorectal cancer screening, 23.3 % had incomplete cervical cancer screening, and 21.4 % had incomplete breast cancer screening. Of the patients with diabetes, 25.0 % had glycated hemoglobin (Hb-A1c) greater than 9 %.

Appointment Outcomes

The no-show rate was 22.8 % in the intervention arm compared to 29.2 % in the control arm (absolute risk difference [ARD] −6.4 %, p < 0.001, 95 % confidence interval [CI] −3.0 % to −9.8 %) (Table 2). Arrival rates (54.3 % and 51.9 %; ARD 2.4 %, p = 0.15, 95 % CI −0.9 % to 5.8 %), cancellation rates (13.1 % and 11.5 %; ARD 1.6 %, p = 0.15, 95 % CI −3.7 % to 0.6 %), and reschedule rates (14.2 % and 12.2 %; ARD 2.0 %, p = 0.09, 95 % CI −0.4 % to 4.7 %) were higher in the intervention group than in the control group, but the differences were not statistically significant.

Table 2 Appointment Outcomes by Study Arm
Full size table

Time to Cancellation

Rescheduled and canceled appointments occurred further in advance of the appointment in the intervention group compared to the control group (1.94s vs. 1.59 days, mean difference 0.35 days; 95 % CI 0.07–0.64; p = 0.01) (Fig. 2).

Fig. 2
figure 2

Time to cancellation by study arm.

Full size image

Reimbursement

All outpatient bills were aggregated by study arm to assess the impact on work RVU billing. Among 1129 patients in the intervention arm, there were 1037 arrivals that resulted in 2220 work RVUs. For the 1118 patients in the control arm, there were 953 arrivals, resulting in 2055 work RVUs. When comparing the two groups, there were 1.97 work RVUs per patient generated in the intervention arm compared with 1.84 work RVUs per patient generated in the control group. This difference was not significantly different (mean difference 0.13 95 % CI −0.02 to 0.28, p = 0.10) (Table 3).

Table 3 Work RVUs billed and estimate of return on phone calls
Full size table

DISCUSSION

This trial demonstrated that a telephone call from a patient care coordinator 7 days prior to a scheduled clinic appointment, targeted to patients predicted to be at high risk of no-show, led to a 22 % relative reduction in the no-show rate compared to control patients.

Prior studies have demonstrated that phone calls are an effective intervention for reducing no-show rates. However, generalizability has been limited, since prior interventions have focused on clinics with high no-show rates at baseline. Moreover, these studies have not assessed the financial impact on practices with low no-show rates at baseline. By focusing on patients at high risk of no-show and by using detailed billing data, we have attempted to address both of these limitations. Identifying patients at high risk was accomplished using basic demographic data (age, insurance, wait days, and prior history of appointment adherence) available through administrative data. This parsimonious predictive model was developed specifically to facilitate replication. By using this predictive modeling approach, we targeted the intervention to patients at the highest risk of no-show. Utilizing this method, we found a meaningful reduction in no-show rates in a practice with low no-show rates at baseline. In addition, arrivals, cancellations, and reschedules all occurred at a greater rate in the intervention arm. These findings were not statistically significant, but the study was not powered to detect a difference in these outcomes; this may be an area for further research. Finally, the observed increase in work RVU billing suggests that the intervention is cost effective.

The results from this study also have clear operational implications for provider organizations taking on increasing accountability for population health outcomes, in two domains. First, this intervention represents an approach for improving access to primary care, which is a principal performance measure of ACO quality.54 In addition, as the population ages and insurance coverage expands through full implementation of the Affordable Care Act, the Health Resources and Service Administration projects a shortage of 20,400 primary care physicians in 2020.55 Faced with this shortage in physician supply, the intervention described here is one of many that can be used to make use of existing capacity.

Second, this intervention attempted to engage patients who perform poorly on measures of population health quality in primary care. Prior work from our group revealed that a patient’s likelihood of no-show for a clinic appointment was an independent risk factor for incomplete cancer screening, as well as for poor performance on measures of chronic disease management.56 In this study, baseline characteristics reflect that patients identified as high risk of no-show had high rates of incomplete cancer screening (colorectal and breast) and poor performance on measures of chronic disease management (LDL ≥ 130 mg/dl among patients with diabetes and/or vascular disease, and HbA1c ≥ 9 % among patients with diabetes). Together, these metrics represent five of the 33 quality measures that ACOs are responsible for through the Centers for Medicare & Medicaid Services Pioneer ACO program.54 Since cancer screening and chronic disease management are core to the practice of primary care, it stands to reason that fewer no-shows to primary care clinics may improve these measures of quality. Additional study is needed to assess the impact of reduced no-show rates on these and other measures of health care quality.

The results of this study should be interpreted within the context of the study design. First, the study was conducted at a single hospital-based primary care practice, and the findings may not be generalizable to other practice settings. Second, while the use of predictive analytics was a strength of the study, to implement this intervention, interested sites would need to access appointment data, calibrate the published model, and create a calling queue of patients at high risk of no-show. Small and medium-sized practices tend to be at a relative disadvantage when implementing interventions with fixed costs, as was demonstrated in the PCMH transformation process.57 , 58 Nevertheless, although smaller practices may not be able to implement the full prediction model, they may be able to target pre-appointment outreach to patients based on strong predictors that have been previously described. For instance, the strongest predictor of a future no-show is a prior no-show.38

Third, because of operational limitations, only 72.9 % of patients in the intervention group received the complete intervention (i.e. phone calls before all visits). As a result, the reported results reflect programmatic efficacy and may underestimate the effectiveness of the intervention.

Finally, we presented the financial impact of this study in terms of work RVUs. Limited assessment of financial impact has been cited by systematic reviews as a major limitation of studies examining interventions designed to improve adherence.59 A major hurdle in such analyses is that there are multiple ways that medical practices are structured financially. Specifically, payer mix and accounting practices vary from practice to practice. While the use of work RVUs addressed these issues to provide a generalizable measure of productivity, it does not represent a true estimate of revenue. For instance, to enhance generalizability, we did not include practice expense and malpractice RVUs in this analysis. Their inclusion would have increased the estimate of revenue generated from the intervention. Given multiple methods of measuring the financial impact, we performed two sensitivity analyses. We calculated an increase in per-patient billing of 8.8 % when examining total billing, and 7.8 % when modeling revenue, using standard Medicare charges. These estimates are comparable to the 7.0 % increase in billing reported here. For the reasons outlined above, the figure presented represents an accurate relative (and conservative absolute) measure of the financial impact of the intervention. We did not track, calculate, or report the fixed costs of developing the intervention; this limits the ability to determine the cost-effectiveness of the intervention.

In conclusion, this study demonstrated that a phone call 7 days prior to an appointment led to a significant reduction in no-shows among patients predicted to be at high risk of no-show for a scheduled clinic appointment. The use of a targeted intervention to reduce no-shows among high-risk patients may be of interest to practices taking on increased accountability for population health.