Virtual Education for Patient Self-Testing for Warfarin Therapy Is Effective During the COVID-19 Pandemic

Background Prior to the COVID-19 pandemic, warfarin users were required to complete in-person training in order to participate in approved international normalized ratio (INR) patient self-testing (PST) programs. To minimize in-person contact during the pandemic, a federal waiver of the in-person training requirement allowed new patients to begin PST after completing virtual training. However, it was uncertain whether such patients achieved comparable levels of INR control to patients receiving in-person training. Methods INR results for patients receiving virtual training upon PST commencement between April 1, 2020, and December 31, 2020, were compared to those of patients initiating PST with in-person training between April 1, 2019, and December 31, 2019. The primary outcome was the difference in warfarin time in therapeutic range (TTR) between the groups, with secondary outcomes including differences in the percentages of INR values within individually prescribed INR range and of critical INR values. Results The records of 33,683 patients were included in the analysis (13,568 in the “In-Person” sample; 20,115 in the “Virtual” sample). Patients in the Virtual sample achieved a TTR of 66.78%, compared to the In-Person sample (64.19%; absolute difference 2.59; 95% confidence interval [CI] = 2.50–2.68, p < 0.001). The TTR values were also statistically significantly higher in all subgroups evaluated across categories of patient age, gender, geography, and indication. Similarly favorable results were achieved for INR values in range and critical values. Conclusion Virtual education for PST for warfarin therapy is effective and should continue to be an option for patients and providers throughout the pandemic, and possibly beyond.

T he COVID-19 pandemic has led to rapid expansion of virtual care modalities across health care systems worldwide, presenting challenges and opportunities in maintaining high quality of care across diverse patient populations. 1 , 2 The ability to maintain high-quality services for outpatients prescribed warfarin during the pandemic is of particular interest due to the fact that the drug requires tight control of the international normalized ratio (INR) to minimize serious bleeding and thrombotic events.
Maintaining the INR within therapeutic range is key to maximizing warfarin's effectiveness at preventing thrombotic events while minimizing bleeding risk, and INR control is best maintained through well-designed and controlled care systems. 3 , 4 Patient self-testing (PST) at home is one widely used warfarin management model that has been shown to maintain high levels of INR control, is associated with low rates of adverse outcomes associated with bleeding or thrombosis 5 , 6 and is a recommended guideline by professional organizations. 7 Beyond INR control, PST has other advantages, such as improved patient quality of life as compared to traditional clinical management 8 , 9 and expanded access to high-quality anticoagulation management for pa-tients residing in rural regions. 7 , 10 In the context of the COVID-19 pandemic, PST also minimizes in-person encounters with clinical personnel and travel associated with office visits. 11 However, until recently, not all aspects of PST have been executed virtually. Prior to the COVID-19 pandemic in the United States, Medicare coding and coverage policies required that new patients successfully complete in-person training by a health care provider prior to initiating PST. 12 Recognizing the risk of continuing this requirement in the face of the pandemic, a federal waiver was announced in March 2020 that allowed for a temporary transition to completely virtual training for new PST patients. 13 The change from in-person to virtual training provided an excellent opportunity to explore the impact of virtual training on INR time in therapeutic range ( T TR). This analysis compares the quality of INR control of new PST patients trained virtually during the COVID-19 pandemic to that of patients receiving traditional in-person training prior to the COVID pandemic.

METHODS
This pre-post retrospective observational study compares the INR control of patients who initiated PST and received in-person PST training immediately before the COVID-19 pandemic (April 1, 2019, to December 31, 2019; "In-Person" sample) with those who initiated PST and received virtual training during the pandemic (April 1, 2020, to December 31, 2020; "Virtual" sample). The study met criteria for exemption from Institutional Review Board (IRB) review as determined by Advarra IRB (Columbia, Maryland).
All patients included in the analysis were enrolled in the PST program provided by Alere Home Monitoring, Inc. (dba Acelis Connected Health, "ACH"). A subsidiary of Abbott Laboratories, ACH is a Centers for Medicare & Medicaid Services (CMS)-approved Independent Diagnostic Testing Facility that receives referrals from clinicians for patients to initiate PST. ACH then performs the required patient education and provides the patients with point-ofcare testing devices and necessary supplies. Through the program, patient INR values are reported back to the referring clinicians, who are responsible for all warfarin dosing and patient management decisions.
All patients in the analysis resided in the United States and were 19 years of age and older at the time of referral by their clinicians. The ACH data set includes records of patients with services reimbursed through all major insurance types. For Medicare eligibility, beneficiaries must have required chronic oral anticoagulation with warfarin for an approved indication (for example, mechanical heart valve, chronic atrial fibrillation, venous thromboembolism), taken warfarin with regular outpatient (non-PST) monitoring for at least three months prior to use of the home INR device, and been instructed by their physicians to conduct home testing with the device no more frequently than once a week. 12 These eligibility requirements were similar for patients with other insurance coverage types and were not affected by the COVID-19 pandemic. In the process of evaluating referrals, ACH documents a number of standard patient characteristics, including date of birth, gender, and primary diagnosis for warfarin (as ICD-10). For patients proceeding through training to perform PST, ACH also documents the date, time, and value of every individual INR test reported through the program.

In-Person Sample
For the current retrospective analysis, the In-Person sample (that is, pre-COVID-19) was limited to patients referred for and receiving PST services between April 1, 2019, and December 31, 2019, who recorded their first PST INR value in that interval and who completed two or more INR tests in the measurement period. Patients with gaps between INR tests of greater than 60 days in the measurement period were excluded from analysis.
In-person patients referred to the Acelis Connected Health (ACH) Face-2-Face ® training program in this interval had an INR monitor (Roche CoaguChek ® XS or Roche CoaguChek Vantus) shipped (via FedEx, 2-day air) to their homes. ACH-approved trainers receiving the assigned case engaged patients by phone within 48 hours to schedule an in-home training appointment to be completed within 10 days from the date of referral.
Trainers were instructed to spend at least 60 minutes with the patient addressing key points according to a comprehensive training and documentation manual. Sessions included live demonstrations facilitated by the trainer with a noncalibrated "demo" meter, as well as multiple live INR tests performed by the patient on their assigned meter. Testing was considered successful if the patient achieved the following: • Demonstrated no physical or cognitive barriers precluding successful testing and reporting of results • Demonstrated the ability to perform two or more INR tests and report them via the patient's preferred method (HealthCheck app for tablets and smart phones, ACHHealthCheck.com for computers, Interactive Phone Recognition for telephones) • Completed all items in the Key Points and Knowledge Assessment ( Table 1 )

Virtual Sample
The Virtual sample was limited to patients referred for and receiving PST services between April 1, 2020, and December 31, 2020, who recorded their first PST INR value in that interval and who completed two or more INR tests in the measurement period. Patients with gaps between INR tests of greater than 60 days in the measurement period were likewise excluded from analysis. Despite the pandemic, the underlying eligibility requirements for PST under the Medicare program did not change (that is, number of prior months using warfarin). However, given the restrictions put in place by national and state health authorities, and in alignment with applicable emergency orders and waivers for health services, the training process for patients referred to the program in this interval was converted from in-person to virtual encounters. 13 All of the necessary supplies were shipped to patients in their homes, and all required training and assessment activities ( Table 1 ) were performed remotely by ACH personnel according to a modified training manual using the Webex ® videoconferencing platform (Cisco Systems, Inc., San Jose, California). Virtual training was interactive and performed in real time using bidirectional video and voice features of the platform. As with the in-person training, trainers were to spend at least 60 minutes per session. The modified training manual included additional resources for trainers regarding the following: • Establishing and maintaining an adequate Webex connection • Ensuring patient receipt of all necessary resources • Leading an engaging and effective virtual training session • Assessing patient physical and cognitive aptitude for PST • Executes all acknowledgments and assessments relating to receipt of necessary resources, physician-prescribed monitoring plan, sharing of medical information and billing for services • Both the trainer and the patient must attest to completion of all of the above steps for training to be considered successful.

• Evaluating successful completion of all Key Points and
Knowledge Assessment items • Troubleshooting and problem solving during virtual training encounters

Outcome Testing
The primary outcome was the difference in warfarin TTR (Rosendaal method 14 ) between the Virtual and In-Person samples, with secondary outcomes including differences in the percentages of INR values within individually prescribed INR range (PINRR) and of critical INR values (that is, INR < 1.5 or > 5.0). The z -test for proportions for two independent dichotomous samples was performed for each measure, with a p value of < 0.05 being considered statistically significant. 15 Two-sided 95% confidence intervals (CIs) were also constructed for each outcome measure. Baseline patient characteristics for the measurement intervals were compared using the t -test for continuous variables and the chi-square test for categorical variables. Statistical calculations were performed with R, version 3.6.0 (R Foundation for Statistical Computing, Vienna), and TTR calculations were performed using Microsoft SQL 13.0 (Microsoft Corp., Redmond, Washington). Univariate analysis was also performed for subgroups based on age bands, gender, and primary indication for anticoagulation (by ICD-10). In addition, subanalysis was performed based on patient geography. Patient zip code was used to assign 1 of 10 US Department of Agriculture (USDA) rural-urban commuting area (RUCA) codes to each patient. 16 The codes were then aggregated into three categories: metropolitan (RUCA codes 1-3), micropolitan (codes 4-6), and small town/rural (codes 7-10). Approximately 0.4% of patient addresses failed to match to an RUCA region, and associated records were excluded from subanalyses relating to geography. Overall, the patients were of mean age of 70.8 years with nearly equal proportions of males and females included ( Table 2 ). The predominant indications for warfarin therapy were atrial fibrillation (54.80%) and cardiac implants; that is, valves (15.99%). Small but statistically significant differences were seen in patient age, days of therapy, number of completed tests, and referral diagnosis. Similar proportions of patients were excluded from analysis due to greater than 60 days gaps between INR readings in the In-Person (7.92%) and Virtual (8.19%) samples.

Primary Measure
In the In-Person sample there were 1,721,423 evaluable care days, among which 1,104,896 were imputed to be within prescribed therapeutic range by the Rosendaal method ( T TR 64.19%) ( Table 3 ). In the Virtual sample there were 2,806,340 evaluable care days, among which 1,873,946 were imputed to be in range ( T TR 66.78%), which was statistically significantly greater than that of the In-Person sample (absolute difference 2.59; CI = 2.50-2.68, p < 0.001). The TTR values were also statistically significantly higher in all subgroups evaluated across categories of patient age, gender, geography, and indication.  Table 5 ), which was significantly fewer than observed in the In-Person sample (absolute difference -0.95; CI = -1.08--0.81, p < 0.001). The proportion of critical values was also numerically lower in all Virtual sample subgroups evaluated across categories of patient age, gender, geography, and indication, with the differences achieving statistical significance in all subgroups except for "Micropolitan," "Pulmonary Embolism," "Cerebral Infarction," "Other Pulmonary Heart Diseases," and "Other ICD-10." The proportions of low vs. high critical values were similar in the In-Person and Virtual cohorts, with the majority in both groups (72.05% in the In-Person cohort and 71.29% in the Virtual cohort) being low.

DISCUSSION
The COVID-19 pandemic has dramatically affected the provision of health care services in the United States and globally, as evidenced by emergency authorizations and evolving guidance issued by US federal agencies. 17-19 Although temporary federal authorizations allowed for rapid expansion of telemedicine services, 18 it is important to critically evaluate the quality and safety of health care services provided via these new modalities. Careful appraisals are essential to driving continuous quality improvements for care Virtual Education for Patient Self-Testing processes as the pandemic persists, and will also inform the maintenance of safe, effective, and efficient care services after the pandemic. The evaluation of systems supporting warfarin patient management during the pandemic is particularly important, as maintenance of the INR within a narrow therapeutic range is critical to avoid life-threatening thromboembolic and hemorrhagic events. The current analysis describes the quality of INR control among a sample of more than 20,000 patients from across the United States initiating PST during nine peak months of the COVID-19 pandemic. Compared to patients initiating PST in the same calendar months from the year prior, the level of INR control was not only equivalent, but overall was statistically significantly superior across the domains of TTR, 14 PINRR, and critical values. Further, a comparable level of INR control was maintained across every subgroup evaluated, with numerical and statistical superiority being achieved among the Virtual sample in the majority of subanalyses.
Our study noted an overall increase in patients referred for PST during the COVID-19 pandemic compared with the preceding years, consistent with the broader pandemicassociated expansion of virtual health care modalities. This increase in enrollment was consistent across available demographics (age and gender), indication for anticoagulation, and type of location (metropolitan, micropolitan, or small town/rural). This analysis demonstrates that the transition from face-to-face to virtual patient training did not negatively affect patients' ability to master PST and achieve treatment goals (that is, maintain INR within prescribed therapeutic range) regardless of age, gender, indication, or geographic setting.
The value of converting components of warfarin management to virtual options is substantiated by evaluations of processes implemented in the COVID era in other countries. A tertiary care teaching hospital in India achieved a statistically significantly higher level of INR control among patients using their new virtual management program as compared to those remaining in traditional in-office management ( T TR 75.4% vs. 71.2%, p < 0.001; PINRR 66.7% vs. 62.4%, p < 0.001). 20 Another tertiary care center in India implementing a remote management program ( N = 1,214) in response to the pandemic demonstrated no significant differences in rates of patients experiencing supratherapeutic or subtherapeutic INRs during the measurement interval. 21 In addition, they identified no signifi- cant differences in rates of hospitalization, bleeding events, or thromboembolic episodes. Similarly, a regional outpatient hematology center in Brazil converted warfarin patients to telephonic management and retained comparable levels of INR control ( T TR 63% vs. 62%, p = 0.78). 22 To our knowledge, this study is the largest evaluation of COVID-era TTR quality available and the only study of its kind in the United States to date. The current analysis has a number of strengths, including the size and diversity of the evaluated patient samples, the standardization of training materials, the uniformity in INR testing technology and results reporting, and, most importantly, the standardized measure of INR time in range as a validated quality indicator.
However, the analysis also has limitations. As ACH is entirely dependent on clinician referrals for PST services, it is possible that the In-Person and Virtual patient populations differ in ways that cannot be measured with available data. Likewise, the levels of warfarin management expertise of referring clinicians and their decision-making processes relating to PST referrals and warfarin management before and during the pandemic were impossible to assess.
Although the analysis showed an overall increase in number of patients referred for PST across the categories of pa-tient characteristics studied, the data set does not include information on socioeconomic status, education level, or other characteristics that might have affected patients' access to or engagement with virtual training modalities. With the In-Person and Virtual cohorts, patients were referred only after an initial three-month period of stability on warfarin. This analysis therefore does not include patients who were not stable for that initial period-potentially a particularly vulnerable population. However, we would not expect significant differences in these two groups with regard to this characteristic.
The data set also lacks information on comorbid illnesses and medication use beyond warfarin that may affect INR control (for example, renal function, antibiotic use) and does not include information on adverse clinical outcomes, such as actual bleeding and thrombotic events. As such, inferences regarding actual patient outcomes between the In-Person and Virtual intervals cannot be drawn directly from the available data. However, other studies have demonstrated an association between high TTR and improved clinical outcomes among patients with atrial fibrillation, 23 prosthetic heart valves, 24 and venous thromboembolism, 25 making the high level of TTR control achieved during the COVID-19 pandemic clinically relevant. Finally, the data set does not contain variables that can account for clinical and social factors related to the COVID-19 pandemic that may have influenced the quality of INR control. It is possible that pandemic-related factors such as telecommuting, unemployment, reduced access to restaurants, limitations on travel, and other pandemicrelated factors may have affected warfarin adherence and INR control, but the impact of such factors cannot be evaluated with available data to the ACH PST program. Although the conversion from in-person to virtual training was the only control variable evaluated in the available data set, the improvements seen in TTR, PINRR, and critical values may not be clearly attributable to that transition. Additional research into other pandemic-related factors that affect patient medication adherence and INR control is warranted.

CONCLUSION
Patients receiving virtual training for warfarin PST during the COVID-19 pandemic achieved equivalent or superior levels of INR control than patients initiating PST with inperson training immediately preceding the pandemic. PST with virtual training should continue to be an option available to well-suited patients requiring warfarin therapy. Virtual training for warfarin PST may help improve access to care for patients with geographic or scheduling limitations and may serve as a model for other educational interventions to support patient care.
Disclosures. Personnel from Acelis Connected Health Services (the clinical laboratory service provider) provided data analytic support and information characterizing the patient self-testing program. Acelis staff reviewed the draft manuscript for accuracy regarding descriptions of their program. Neither author received financial reimbursement from Acelis for work relating to the manuscript.