Predictors of Follow-Up Appointment No-Shows Before and During COVID Among Adults with Type 2 Diabetes

Background: The coronavirus disease 2019 (COVID-19) pandemic has rapidly transformed health care delivery into telehealth visits. Attending regular medical appointments are critical to prevent or delay diabetes-related complications. Although telehealth visits have addressed some barriers to in-person visits, appointment no-shows are still noted in the telehealth setting. It is not completely clear how the predictors of appointment no-shows differ between in-person and telehealth visits in diabetes care. Objective: This retrospective study examined if predictors of appointment no-shows differ (1) between pre-COVID (January 1, 2019–March 22, 2020) and COVID (March 23, 2020–December 31, 2020) periods and (2) by health care delivery modes (in-person or telehealth visits) during COVID among adults with type 2 diabetes mellitus (T2DM). Methods: We used electronic health records between January 1, 2019 and December 31, 2020 across four diabetes clinics in a tertiary academic hospital in Baltimore, Maryland. Appointments marked as completed or no-show by established adults with T2DM were included in the analyses. Results: Among 7,276 appointments made by 2,235 patients, overall appointment no-show was 14.99%. Being older and White were protective against appointment no-shows in both unadjusted and adjusted models during both time periods. The interaction terms of COVID periods (i.e., pre-COVID vs. COVID) were significant for when glycated hemoglobin drawn before this visit and for missing body mass index. Telehealth visits during COVID decreased more half of the odds of appointment no-shows. Conclusions: In the context of diabetes care, the implementation of telehealth reduced appointment no-shows. Future studies are needed to address social determinants of health, including access to internet access, to further reduce health disparities among adults with T2DM.


Introduction
T he coronavirus disease 2019 (COVID-19) pandemic has rapidly transformed health care delivery. Since the Centers for Medicare and Medicaid Services expanded telehealth coverage for all Medicare patients, and for the duration of the COVID-19 Public Health Emergency, 1 telehealth visits using phone or video conferencing have been used widely in outpatient settings in the United States, 2 including diabetes clinics. Telehealth is defined as the use of electronic information and telecommunication technologies to provide health care remotely. 3 The implementation of telehealth visits ensures the continuity of diabetes care when social distancing is required to mitigate the spread of COVID-19. 4 Type 2 diabetes mellitus (T2DM), accounting for 90-95% of all diabetes mellitus cases, 5 is a complex and chronic illness that requires active patient engagement in care. 6,7 To prevent or delay serious diabetes-related complications, American Diabetes Association (ADA) recommends that people with T2DM attend regular medical appointments to evaluate glycated hemoglobin (HbA1c) every 3 months 8 and to assess vascular complications annually, 9 on top of actively engaging in lifestyle management. 7,10 During a high-quality patientcentered diabetes appointment, health care teams support and enhance patient engagement in diabetes care for optimal glycemic control, while people with T2DM receive tailored education and treatment plan. 8 Yet, studies show that 12-36% of adults with T2DM missed their regular in-person medical appointments pre-COVID. 11,12 Missed regular appointments among adults with T2DM directly results in poor glycemic control 13 and has been associated with increased risk for rehospitalization. 14 Also, appointment no-shows pose substantial financial burdens for providers and health care systems, [15][16][17] disturbance of operations, as well as longer waitlists for other patients to seek care. 15,17 A recent systematic review of 18 studies identified various individual, health care provider and system, and interpersonal factors associated with appointment no-shows among adults with T2DM in the context of in-person visits. 18 Telehealth visits have addressed some barriers to ''inperson'' appointments (e.g., transportation, time, and work commitment) 19 and decreased appointment no-shows. 20 Nevertheless, appointment no-shows are still noted in the telehealth setting. 21,22 It is not completely clear why appointment noshows occur in the telehealth setting and how the predictors of appointment no-shows differ from in-person appointments. Using electronic health record (EHR) data, this study examined (1) if predictors of appointment no-shows differ by pre-COVID versus COVID among adults with T2DM and (2) if predictors of appointment no-shows differ for health care delivery modes (in-person or telehealth visits) among adults with T2DM during COVID.

STUDY DESIGN OVERVIEW AND SAMPLE
This retrospective study examined predictors of appointment no-shows using scheduled visits from adults (18+ years) with T2DM with at least one prior visit seen at four outpatient diabetes clinics between January 1, 2019 and December 31, 2020 in a tertiary academic hospital in Baltimore, Maryland. The study period was chosen due to the availability of EHR at the time of designing this study in early 2021. Only patients with a zip code within Maryland were included as reimbursement criteria for telemedicine visits differ across states in different time points.
There were no telehealth visits before COVID. All appointments at the clinics were shifted to telehealth visits starting March 23, 2020. Around September 2020, the clinics started scheduling for both in-person and telehealth visits based on patients' preferences and their COVID-19 risk.
Our initial EHR inquiry resulted in 18,259 appointments made by 3,742 unique patients. We then excluded 10,983 of those appointments based on the inclusion criteria. Those visits marked as new patient visits were excluded as the reasons for appointment no-shows might differ between new patients versus established patients. 21 The final cohort included 7,276 appointments (4,728 appointments during pre-COVID [all in-person visits] and 2,548 appointments during COVID [79.36% telehealth visits]) made by 2,235 unique patients ( Fig. 1). Appointment no-show rate was 17.41% (823/ 4,728) during pre-COVID and 10.52% (268/2,548) during COVID, with an overall appointment no-show of 14.99%.

DATA SOURCES
All data were captured by EHR. The Johns Hopkins Medicine Institutional Review Board approved this study design and procedures (IRB00231790).

STUDY VARIABLES AND OUTCOME REVIEW
Outcome. Appointment status was coded as completed or no show.
Predictor variables. Literature review guided the variables chosen. 18 Supplementary Table S1 shows the details of variables and their operational definitions. Covariates included basic characteristics (age, sex, race, and ethnicity). Age was categorized as <46, 46-60, 61-75, or >75 years old to create equal groups.
Additionally, we included the following factors: T2DMrelated and comorbidity, health utilization, health behaviors, and others. Specifically, T2DM-related and comorbidity factors included: HbA1c associated with this visit, when associated HbA1c was drawn before this visit, body mass index (BMI) (missing, normal [£24.9], Overweight, Class 1, Class 2, or Class 3 obesity 23 ), diabetes complications (yes, no), types of T2DM medications (no medications on file, metformin, other than metformin but no insulin, any insulin), depression, or other comorbidities (count of hypertension, hyperlipidemia, coronary heart disease, heart failure, chronic kidney disease, stroke, cancer, Chronic Obstructive Pulmonary Disease emphysema, dementia, HIV/AIDS, liver disease, other advanced illness 24,25 ).
Health utilization factors included: emergency department (ED) or hospitalization 12 months before this visit, and patient portal status. Health behaviors included: smoking status, alcohol status, and recreational drug status. Other variables related to social determinants of health (SDOHs) were also included: marital status, selected preferred language on file, emergency contact on file, distance between home's zip code and clinic's zip code, and insurance type.
Finally, we also included health care provider factors (types of providers), health care system factors (scheduling lag, health care delivery modes), and COVID time periods.

STATISTICAL ANALYSIS
To examine if predictors of appointment no-shows differ by time periods (i.e., pre-COVID vs. COVID), the association of each predictor with appointment no-show and the interaction of each predictor and time period were assessed using random effects logistic regression with visits nested within patients. All variables significant at the p < 0.05 level were then entered into a multivariable random effects logistic regression to examine the relative importance of the factors in predicting appoint no-shows.
To examine if the predictors of no-show differ for health care delivery modes (in-person vs. telehealth) during the COVID, the interaction of the health care delivery mode with each predictor were tested separately with random effects logistic regression. To facilitate interpretation of the interactions and examine difference in effect sizes, stratified analyses by health care delivery mode are presented. All statistical analyses were conducted in Stata 17.0. 26

POWER ANALYSIS
In this sample, there were 4,728 encounters from 1,913 people during pre-COVID and 2,548 encounters from 1,424 people during COVID. Intraclass Correlation Coefficient (ICC) estimated from the data was 0.36 during pre-COVID and 0.18 during COVID. Our power analysis estimated the odds ratios (ORs) detectable accounting for the ICC with power = 0.80 and a = 0.05, varying the distribution of the predictors, we can detect significant ORs ranging from 1.29 to 1.50 for pre-COVID and 1.48-1.83 for COVID. Table 1 shows the distribution of patient-level (fixed input throughout different encounters) and encounterlevel (might be different across encounters for a patient) variables.

CHARACTERISTICS OF STUDY SUBJECTS
Among 2,235 patients, 1,102 patients were seen in both periods, 811 patients only during pre-COVID, and 322 only during COVID. In this dataset, each patient scheduled 3.26 appointments on average (ranged 1-18, standard deviation [SD] 2.21), where during pre-COVID each patient scheduled 2.47 appointments on average (range 1-14, SD 1.56) and during COVID the average scheduled appointments was 1.79 (ranged 1-9, SD 0.98). Overall, 49.53% of the included patients were female, and 48.72% Black with 40.94% White. Majority of the included patients (83.89%) had an activated MyChart account.
More patients during COVID had an activated MyChart account compared with pre-COVID (88.24% vs. 81.59%). During COVID, a higher percentage of appointments were with A1c drawn >6 months, missing BMI, missing smoking status, scheduling within a week. APPOINTMENT NO-SHOWS T2DM PRE-COVID VS COVID

PREDICTORS OF APPOINTMENT NO-SHOW
Unadjusted model. Table 2 shows the unadjusted ORs of appointment no-shows by time periods and the p-value for the interaction between a variable and time periods. Majority of the unadjusted results were similar in direction across both periods if comparing qualitatively. Protective factors against appointment no-shows included older age (vs. <46 years), White or other race (vs. Black), controlled HbA1c (compared with uncontrolled HbA1c), use more than metformin but no insulin (vs. any insulin), patient portal account activation (vs. no), and being married (vs. single, widowed, or unknown). Risk factors of appointment no-shows across both periods were missing BMI (compared with BMI £24.9), ED visit 12 months prior (vs. no) and hospitalization 12 months prior (vs. no), never use alcohol (vs. active alcohol use), having a Medicare or Medicaid (vs. commercial or other), home distance £30 miles (vs. >30 miles), and longer scheduling lag (vs. £7 days).
Several factors were shown as risk factors only during COVID and not during pre-COVID: missing an HbA1c on file or an HbA1c was drawn 4-9 months before this appointment (vs. drawn 1-3 months ago), missing smoking status (vs. active smoker), and having a depression ICD10 (vs. no). Telehealth was a protective factor against appointment no-shows during COVID.
The interaction terms with COVID periods (i.e., pre-COVID vs. COVID) were significant for the following variables: months when HbA1c was drawn before this visit, missing BMI, never smoking, and scheduled with a nurse practitioner (NP). Most of the variables were in same directions but with different magnitude across both periods. For instance, compared with HbA1c drawn within 1-3 months before the scheduled appointments, the HbA1c on file was drawn 4-6 months (pre-COVID vs. COVID ORs: 1.19 vs. 2.51, p-value for interaction: 0.015), and was much stronger risk of appointment no-shows during COVID than pre-COVID. Only the HbA1c on file drawn 7-9 months before this visit had different direction across COVID periods-it was protective against appointment noshows during pre-COVID but a risk factor during COVID (pre-COVID vs. COVID ORs: 0.84 vs. 1.87, p-value for interaction: 0.001).
Adjusted model. Results of adjusted odds ratios (aORs) of appointment no-shows by COVID are shown in Table 3. Similar to unadjusted model, adjusted results were similar in direction across periods with moderate magnitude in the adjusted model.
Comparing adjusted models across periods, 61-75 years old, being White race, having an activated patient portal ac-count, were protective factors against appointment no-shows, while missing BMI on file, ED visits 12 months before the scheduled visit, scheduling with an NP, and scheduling lag ‡91 days were predictive of appointment no-shows.

INTERACTION OF PREDICTOR VARIABLES WITH HEALTH CARE DELIVERY MODES
The stratified unadjusted ORs of appointment no-shows by health care delivery modes (in-person vs. telehealth visits) are presented in Table 4. When testing for interaction between the predictor variables and the health care delivery modes during COVID, none of the interactions was significant. We observed some trends in the magnitude of the OR. Being Hispanic was protective against appointment no-shows for telehealth visits (OR: 0.37) but not for in-person visits (OR: 1.84). We also found longer scheduling lags, where scheduling between 1 and 3 months, was as protective against appointment noshows in telehealth visits (OR: 0.88), but a risk factor in inperson visits (OR: 1.48) and the same in scheduling more than 3 months ago (telehealth vs. in-person ORs: 0.65 vs. 2.18). Moreover, we found that having a missing HbA1c on file had a higher odds of appointment no-show in in-person visits (OR: 1.89) compared with telehealth visits (OR: 1.11). A similar trend was also observed in the variable when HbA1c was drawn 10-12 months before this appointment (telehealth vs. in-person ORs: 1.08 vs. 2.20).

Discussion
This study examined a comprehensive set of multilevel predictors of appointment no-shows among adults with T2DM comparing pre-COVID and COVID period based on the most current systematic review. 16 While previous studies focused on in-person visits, our study findings add to the current literature to examine the impact of COVID and telehealth implementation to predictors of appointment no-shows among people with T2DM.
We found that missing HbA1c and BMI or an HbA1c drawn longer than the recommended 3-month interval 8 was a risk factor of appointment no-shows during COVID. As measuring BMI is a standard procedure for in-person visits, less than 0.5% of encounters in our dataset had a missing BMI during pre-COVID, which could result in an unstable estimate of the OR due to insufficient sample size. Whether or not limited inperson activities during COVID led to worse glycemic control or compromised other diabetes quality indicators remained unanswered as current literature had conflicting results. [27][28][29] However, limited in-person activities do not equal to disengagement from health care. For example, diabetes medication prescribing stayed the same level during COVID as pre-COVID      in a national cohort. 28 It would be important to identify and implement effective strategies to ensure a patient with T2DM are engaged in health care regardless of either delivery method (in-person or telehealth). For example, a recent systematic review revealed that using patient portal was essential for patients to stay engaged in their care, including communicating with their providers. 30 In fact, having an active patient portal account decreased more than 50% of the odds of appointment no-shows during both periods. More patients were introduced to patient portal during COVID than pre-COVID (88% vs. 82%) in our study due to how telehealth visits set up in the diabetes clinics. Still, more than 10% of patients did not have an active account. Among people without an activated patient portal account, more than 70% were >60 years old and non-White race (data not shown). White populations with higher education levels are repeatedly more likely to adopt using a patient portal. 31,32 These findings highlight the need for designing a patient portal for patients with diverse backgrounds in mind (e.g., older patients, lower education levels) to use a patient portal to further engage in their diabetes care.
The implementation of telehealth reduced appointment noshows compared with in-person visits during COVID (aOR: 0.40, 95% confidence interval 0.29-0.57, p < 0.001) in this study. Telehealth seems to provide an opportunity to democratize diabetes care to racial or social minority populations 33 by addressing some of the barriers. 18,19 However, telehealth visits require access to reliable internet and technological devices. It is worth noting that only 78.0% of the household in Baltimore subscribed to Broadband internet compared with 88.5% in Maryland. 34 Additionally, people with limited English proficiency, older adults, and people from racial minorities are less likely to use video visits compared with phone visits in primary care because of limited access to broadband internet and/or digital devices. 35,36 Indeed, the National Academies of Sciences, Engineering, and Medicine warns that the digital divide might exacerbate health care disparities. 37,38 These results underscore the importance of addressing SDOHs in bridging the gap in using technology/telehealth visits in diabetes care, especially for racial and social minorities.
When comparing predictors of appointment no-shows during COVID by health care delivery modes (i.e., in-person vs. telehealth visits), we did not find any statistically significant results. Instead, we found longer scheduling lag for telehealth visits tended to have a OR <1 compared with in-person visits. This trend could be explained by the efforts of outreaching at the health care provider and system level. Providers and staff were required to verify patients' locations to provide care based on individual state regulations 39 , and those calls might have served as a reminder of the scheduled appointment. Additionally, providers would reach out to patients to verify technological problems if patients were not online from our clinical experience. Future research is warranted to better understand if predictors of appointment no-shows differ between in-person and telehealth visits.
In our study, older age and White race were protective against appointment no-shows in both unadjusted and adjusted models during both periods. The finding is consistent with prior research in the United States where being racial minorities with diabetes was associated with higher appointment no-shows. 40,41 More than half of the participants in the analysis dataset were racial minorities. 42 Race in these studies might have been a proxy indicator of SDOHs (e.g., education attainment or income level). 43,44 For instance, comparing Baltimore city with 62.3% Black, where the clinics are located, to Maryland with 31.4% Black, 2020 Census revealed that in Baltimore more residents ‡25 years had a high school degree or less (41.9% vs. 33.6%) and the median household income was lower ($52,164 vs. $87,063 in 2020$). 34 To promote appointment keeping and reduce health disparities, future research should explore the intersectionality of race with SDOHs and patient engagement in diabetes care.
Additionally, the finding suggests the need for better documentation of SDOHs in EHR. ADA recommends routine screening for SDOHs as they contribute to medical and psychosocial outcomes and further affect one's ability to manage their diabetes. 8 Yet, these factors are either unmeasured or poorly documented in the EHR or were sparsely mentioned in free texts, in which a more sophisticated data extraction method is required. 45,46 NPs have been an integral workforce in health care and provided high quality of care. 47 According to a secondary analysis of the Consumer Assessment of Health care Providers and Systems survey, patients rated NPs significantly higher in satisfaction than other health care providers, including physicians. 48 However, contrary to those facts, our study found that scheduling with an NP had 36-120% of increased odds of appointment no-shows in comparison with a physician during both periods. In our diabetes practice, new patients with diabetes are required to see a physician first. Since the reasons behind appointment no-shows might be different between new and established patients due to rapport, we excluded all new patient appointments in the analyses. Additionally, more than 60% of the included appointments were scheduled with a NP. To this end, our finding may be an artifact of sampling bias, not an evaluation on different types of providers. A qualitative inquiry to both providers and patients is warranted to understand this phenomenon and the care transition between team members.
Our study is not without limitations. First, unmeasured predictors of appointment no-shows, such as financial hardship, change of insurance coverage during COVID, access to internet or technical devices, or COVID-related health impact were not captured in EHR. This might lead to uncontrolled biases to our comprehensive model. Additionally, this study was not powered for the interaction analyses. There were more telehealth visits scheduled during the first 9 months of COVID in 2020 compared with current clinical practices. This might potentially over or underestimate the OR in the results. However, the COVID period included in the study were relatively homogeneous compared with calendar year 2021 as COVID vaccine was approved under emergency use authorization on December 11, 2020 49 and the massive vaccine roll out to general populations did not happen until 2021. 50

Conclusions
Keeping regular diabetes appointment is essential in diabetes care for better health outcomes. 8 Since the COVID pandemic, diabetes care and other chronic care delivery mode has rapidly expanded to include telehealth visits. We found that using patient portal and telehealth visits reduced the odds of appointment no-shows in the context of diabetes care. Future studies are needed to address SDOHs, including access to internet, to provide personalized quality care, and to further reduce health disparities among adults with T2DM.