Evaluation of an automated feedback intervention to improve antibiotic prescribing among primary care physicians (OPEN Stewardship): a multinational controlled interrupted time-series study

ABSTRACT Tools to advance antimicrobial stewardship in the primary health care setting, where most antimicrobials are prescribed, are urgently needed. The aim of this study was to evaluate OPEN Stewarship (Online Platform for Expanding aNtibiotic Stewardship), an automated feedback intervention, among a cohort of primary care physicians. We performed a controlled, interrupted time-series study of 32 intervention and 725 control participants, consisting of primary care physicians from Ontario, Canada and Southern Israel, from October 2020 to December 2021. Intervention participants received three personalized feedback reports targeting several aspects of antibiotic prescribing. Study outcomes (overall prescribing rate, prescribing rate for viral respiratory conditions, prescribing rate for acute sinusitis, and mean duration of therapy) were evaluated using multilevel regression models. We observed a decrease in the mean duration of antibiotic therapy (IRR = 0.94; 95% CI: 0.90, 0.99) in intervention participants during the intervention period. We did not observe a significant decline in overall antibiotic prescribing (OR = 1.01; 95% CI: 0.94, 1.07), prescribing for viral respiratory conditions (OR = 0.87; 95% CI: 0.73, 1.03), or prescribing for acute sinusitis (OR = 0.85; 95% CI: 0.67, 1.07). In this antimicrobial stewardship intervention among primary care physicians, we observed shorter durations of therapy per antibiotic prescription during the intervention period. The COVID-19 pandemic may have hampered recruitment; a dramatic reduction in antibiotic prescribing rates in the months before our intervention may have made physicians less amenable to further reductions in prescribing, limiting the generalizability of the estimates obtained. IMPORTANCE Antibiotic overprescribing contributes to antibiotic resistance, a major threat to our ability to treat infections. We developed the OPEN Stewardship (Online Platform for Expanding aNtibiotic Stewardship) platform to provide automated feedback on antibiotic prescribing in primary care, where most antibiotics for human use are prescribed but where the resources to improve antibiotic prescribing are limited. We evaluated the platform among a cohort of primary care physicians from Ontario, Canada and Southern Israel from October 2020 to December 2021. The results showed that physicians who received personalized feedback reports prescribed shorter courses of antibiotics compared to controls, although they did not write fewer antibiotic prescriptions. While the COVID-19 pandemic presented logistical and analytical challenges, our study suggests that our intervention meaningfully improved an important aspect of antibiotic prescribing. The OPEN Stewardship platform stands as an automated, scalable intervention for improving antibiotic prescribing in primary care, where needs are diverse and technical capacity is limited.


Study enrollment and follow-up
In Canada, primary care physicians in the Greater Toronto Area were offered enrollment via email or presentations to groups of providers and were considered eligible if their prescribing data were available in the UTOPIAN database.In Israel, half of primary care physicians within Clalit Health Services (192 physicians) were offered enrollment via a letter from the chief physician, with follow-up by an interviewer.
In total, we enrolled 43 physicians in total to the intervention, 22 from Canada and 21 from Israel.In Canada, all 22 intervention participants received the intervention, but a data access issue arising during the analysis phase rendered follow-up prescribing data unavailable for 11/22 participants and an unknown number of controls.In total, complete follow-up data were available for 11 intervention participants and 361 controls in Canada and 21 intervention participants and 364 controls in Israel.Of the 11 Canadian intervention participants with available data, 4 belonged to wave 1 (beginning October 2020) and 7 belonged to wave 2 (beginning April 2021); all 21 intervention participants from Israel belonged to wave 2.

Data acquisition
Monthly aggregated antibiotic prescribing data by provider were acquired from the University of Toronto Practice-Based Research Network (UTOPIAN) for Canada and Clalit Health Services (CHS) for Israel.Israeli data covered visits by adults (aged 18+) insured by CHS.For each provider-month, the following data were available on antibiotic prescribing: • The overall number of prescriptions • The number of prescriptions for the study-defined indication categories • The overall number of visits • The number of visits for the study-defined indication categories • The total duration of therapy for all prescriptions • The overall number of prescriptions with duration greater than 7 days (Canada only) The ICD-9 codes corresponding to the study-defined indication categories, viral respiratory conditions (and associated indications) and acute sinusitis, are detailed in Supplementary Table 2. Prescriptions were linked to visits based on local linkage algorithms, and many prescriptions could not be linked to a specific indication (but were included in the overall prescription numbers).Due to differences in the underlying datasets, duration of therapy was defined as days of therapy in the Canadian dataset and Defined Daily Doses (DDDs) in the Israeli dataset.Missing data for duration of therapy was handled locally: filtered out in Canada and imputed based on default values in Israel.Finally, we obtained basic demographic characteristics (age and sex) for participating and non-participating providers.
Survey data were also collected from intervention participants.One month after receiving each of the three reports, intervention participants received a survey to capture their thoughts on the intervention.The results of these surveys will be reported in a forthcoming manuscript.

Data cleaning
The combined datasets (which included data from Canada and Israel for both intervention and control participants) were cleaned to remove implausible/likely erroneous values.For the overall prescribing dataset, we removed 129 provider-months where the average duration of therapy per prescription exceeded six weeks, as these represent either extremely atypical community prescribing or data errors.For all datasets, we removed 18 provider-months where the number of prescriptions exceeded the number of visits (all observations were in the overall prescribing dataset).Most of these observations implied ≥2.5 prescriptions per visit, which likely indicated errors in how visits were calculated in those provider-months.

Statistical analysis
For the overall and indication-specific prescribing outcomes, we fit multilevel logistic regression models including terms for the study site (Canada versus Israel), study month (e.g., November 2020, July 2021, etc.), and the interaction between study site and study month.This model specification accounts for the temporal trends in prescribing that may differ across study sites.Since the vast majority of primary care visits will result in one or zero prescriptions, we performed logistic regression using the number of prescriptions for a particular outcome as the number of successes and the number of visits (overall or indication-specific, depending on the outcome) as the number of trials.Random intercepts for each provider, nested within the study group (Canada intervention wave 1, Canada intervention wave 2, Canada control, Israel intervention wave 2, Israel control), nested within the study site, were included.Finally, we added an AR(1) term to account for temporal autocorrelation among the residuals for each provider.
For the mean duration of therapy outcome, we fit a zero-truncated Poisson model using the same model structure but with total duration of therapy as the outcome and the log of total prescriptions as an offset.A zero-truncated negative binomial model was considered but was rejected by comparing model AIC values.
We fit a pair of models for each of the four outcomes.For the total intervention period model, we created a binary variable taking a value of 1 for months within the nine-month intervention period (for intervention participants) and a value of 0 otherwise (Figure 1).This coefficient (total) estimates the deviation from the expected outcome that is attributable to the intervention.
Participants in wave 1 of the intervention were censored following the end of their intervention period (to avoid contaminating the baseline expectation); the end of the intervention period in wave 2 coincided with the end of the study.To examine each of the three three-month intervention sub-periods, we fit an alternative set of models with three intervention effects, one for each sub-period (1, 2, 3).We created three binary variables corresponding to the three three-month intervention periods: these take a value of 1 for months within the respective intervention sub-period (for intervention participants) and a value of 0 otherwise (Figure 1).Model residuals were assessed using the R (1) package DHARMa version 0.4.6 (2).

Assessment of pre-intervention trends
We assessed the similarity of prescribing trends in intervention and control participants prior to the intervention period by fitting an additional set of models.Within the two study sites, models were fit for each of the four study outcomes using data beginning January 2019 and ending the month prior to the first intervention wave in the site (September 2020 in Canada and March 2021 in Israel).The multilevel regression models included terms for enrollment status (intervention versus control), study month, and the interaction between enrollment status and study month.Random intercepts for each provider, nested within enrollment status, were included, as was an AR(1) term to account for temporal autocorrelation among the residuals for each provider.
To compare the pre-intervention trends for each region-outcome pair, we plotted the predicted values of intervention and control participants and visually assessed their similarity.

Sensitivity analyses
For Canada only (due to data availability), we considered an alternative metric for the duration of therapy outcome: the percentage of antibiotic prescriptions with a duration greater than 7 days.We fit a logistic regression model using the number of prescriptions with duration greater than 7 days as the number of successes and the total number of prescriptions as the number of trials.

Changes to intervention
Since data on days of therapy was only available for Canada (Israel provided DDDs as its measure of duration of therapy), the first intervention report was different between the two study sites.As detailed in Supplementary Table 1, the first intervention report was intended to include peer benchmarking figures for both the overall antibiotic prescribing rate and the percentage of prescriptions with duration greater than 7 days.However, the latter figure could not be generated for Israeli participants and was thus included in the first intervention report only for Canadian participants.

Changes to study timeline
In the original protocol, the study timeline figure displayed a 12-month control period prior to the intervention as an illustration; however, to accurately capture each provider's baseline prescribing rate, we used prescribing data beginning from January 2019 in the analysis (Figure 1).For each study participant, their intervention reports were generated using their prescribing data from January 2019-December 2019, as described in the original protocol.
The evaluation periods for each outcome were initially described as being the six months following the receipt of the report targeting that outcome.However, this had two issues: 1) the three reports are all components of a single intervention, rather than three independent interventions and 2) for reports two and three, the "control" period preceding the evaluation period was contaminated by previous reports (e.g., for the second report on prescribing for viral respiratory conditions, participants had already received the first report targeting their overall prescribing; however, the three months between the receipt of the first and second reports was considered part of the control period for evaluating the viral respiratory condition prescribing outcome).Instead, in this analysis, we define three intervention sub-periods of three months each following the receipt of each intervention report and define the total nine-month period as the intervention period (Figure 1).With the view that each report forms an incremental part of the whole stewardship intervention designed to improve appropriate antibiotic prescribing, we fit models to estimate the intervention effects for all outcomes 1) during the total nine-month intervention period and 2) during the three three-month intervention sub-periods, as detailed in the Statistical analysis section of the main manuscript.

Changes to statistical analysis
Prior to the COVID-19 pandemic, we anticipated that the temporal trends in antibiotic prescribing would follow a predictable seasonal cycle with a gentle longer-term trend.However, it later became clear that the COVID-19 pandemic and evolving response to it was driving the dramatic changes in prescribing rates that we observed during the study period.Thus, the statistical models were simplified and re-focused on estimating the temporal trends at each site via the inclusion of a term for study month (e.g., November 2020, July 2021, etc.) with an interaction for the study site, as described in the Statistical analysis section of the main manuscript.
The protocol originally proposed negative binomial models for prescribing rate outcomes (overall and indication-specific), using the log of the number of visits as the offset in the model.However, since the vast majority of visits in primary care result in either zero or one antibiotic prescriptions, the logistic regression model better captures the underlying distribution of the outcome.While it is theoretically possible for the number of antibiotic prescriptions to exceed the number of visits in a month (an observation incompatible with the logistic regression model), most observations of this nature in our dataset clearly result from erroneous data (as described in the Data cleaning section above).
Since the guidelines in the first report specifically targeted prescribing with duration greater than 7 days, the protocol originally proposed the outcome of the percentage of prescriptions with duration greater than 7 days as the primary model (logistic regression) and the outcome of the mean duration of therapy as an alternative model (zero-truncated Poisson regression).However, since the data to perform the former analysis was only available for Canada, we used the mean duration of therapy model as the primary analysis for this prescribing metric.The zero-truncated Poisson model also preserves more information by avoiding dichotomization of the outcome, improving power.