104. Improving Efficiency of Antimicrobial Stewardship Reviews Using Artificial Intelligence Modelling

Abstract Background Antimicrobial stewardship programs (ASP) in hospitals improve antibiotic prescribing, slow antimicrobial resistance, reduce hospitalisation duration, mortality and readmission rates, and save costs. However, the strategy of prospective audit and feedback is laborious. In Singapore General Hospital (SGH), 10 reviews are required to identify 2 inappropriate cases. Limited manpower constraints ASP audits to only about 30% of antibiotics prescribed. This proof-of-concept study explored the feasibility of developing a predictive model to prioritise inappropriate antibiotic prescriptions for ASP review. Methods ASP-audited adult pneumonia patients from January 2016 to December 2018 in SGH were included. Patient data e.g., demographics, allergies, past medical history, and relevant laboratory investigations at each antibiotic use episode were extracted from electronic medical records and re-assembled through linking for analysis. Ground truth for model training was based on ASP-defined appropriateness for each encounter. The dataset was split into 80% and 20% for training and testing respectively. Three modelling techniques, XGBoost, decision tree and logistic regression, were assessed for their relative performance in terms of precision, sensitivity and specificity. Results There were 12471 unique patient encounters. Training was done on 10459 encounters and 39 data elements were included. When tested on 2012 encounters, the logistic regression model performed the best (86.7% sensitivity, 71.4% specificity). The model correctly classified 1377 out of 1388 (99.2%) encounters as “appropriate” (do not require ASP intervention). 624 antibiotic use encounters were classified as “inappropriate”, of which only 72 were truly inappropriate (positive predictive value for ASP intervention, PPV 11.5%). The low PPV was likely due to inadequate representation of “inappropriate” cases in the training dataset (4.1%). Applying this model would prioritise the number of immediate ASP reviews needed to identify cases for intervention by two-thirds, from 2012 to 624 (Figure 1). Figure 1. Illustration of AI benefits in ASP Conclusion ASPs can leverage on machine learning capabilities to improve audit efficiency. This can increase ASP’s productivity and staff’s job satisfaction as they are freed up to perform other work. Disclosures All Authors: No reported disclosures

. Distribution of the Antibiotic Spectrum Index / Day of Therapy by Days of Therapy / 1000 Days Present for 124 Acute-Care VHA Hospitals during 2018. Black line: Median values of DOT/1,000 DP and ASI/DOT, respectively. Figure 2. (a) Distribution of the rankings in DOT/1,000 DP and ASI/1,000 DP. Blue line: the position of same ranking between ASI/1,000 DP and DOT/1,000 DP. (b) Distribution of the differences in each hospital's ranking for DOT/1,000 DP and ASI/1,000 DP Conclusion. Our findings suggest that hospitals using fewer days of antibiotic therapy did not necessarily use narrower-spectrum antibiotics. ASI/1,000 DP, as a combined measure of antibiotic consumption quantity and average spectrum, provided a different view of hospital performance than DOT/1,000 DP alone. Future work is needed to define how this new metric relates to the quality of antibiotic use.
Disclosures. Background. Antimicrobial stewardship programs (ASPs) have traditionally focused interventions on inpatient care to improve antibiotic prescribing. Support of effective interventions for ASPs targeting antibiotic prescriptions at hospital discharge is emerging. Our objective was to expand stewardship services into the outpatient setting through implementation of a process by the antimicrobial stewardship team (AST) to verify antimicrobials prescribed at discharge.
Methods. This quality improvement initiative incorporated a discharge order verification queue managed by AST pharmacists to review electronically prescribed antimicrobials Monday through Friday, from 8:00 am to 4:00 pm. The queue was piloted Sep 2020 and expanded hospital-wide Feb 2021. Patients < 18 years old and those with observation or emergency department status were excluded. The AST pharmacist reviewed discharge prescriptions for appropriateness, intervened directly with prescribers, and either rejected or verified prescriptions prior to transmission to outpatient pharmacies. Complicated cases were reviewed with the AST physician to evaluate intervention appropriateness. Interventions were categorized as either dose adjustment, duration, escalation or de-escalation, discontinuation, or safety monitoring.
Conclusion. An AST pharmacist review of antimicrobial prescriptions at discharge improved appropriate prescribing. The discharge queue serves as an effective stewardship strategy for inpatient ASPs to expand into the outpatient setting.
Disclosures. Background. Antimicrobial stewardship programs (ASP) in hospitals improve antibiotic prescribing, slow antimicrobial resistance, reduce hospitalisation duration, mortality and readmission rates, and save costs. However, the strategy of prospective audit and feedback is laborious. In Singapore General Hospital (SGH), 10 reviews are required to identify 2 inappropriate cases. Limited manpower constraints ASP audits to only about 30% of antibiotics prescribed. This proof-of-concept study explored the feasibility of developing a predictive model to prioritise inappropriate antibiotic prescriptions for ASP review.
Methods. ASP-audited adult pneumonia patients from January 2016 to December 2018 in SGH were included. Patient data e.g., demographics, allergies, past medical history, and relevant laboratory investigations at each antibiotic use episode were extracted from electronic medical records and re-assembled through linking for analysis. Ground truth for model training was based on ASP-defined appropriateness for each encounter. The dataset was split into 80% and 20% for training and testing respectively. Three modelling techniques, XGBoost, decision tree and logistic regression, were assessed for their relative performance in terms of precision, sensitivity and specificity.
Results. There were 12471 unique patient encounters. Training was done on 10459 encounters and 39 data elements were included. When tested on 2012 encounters, the logistic regression model performed the best (86.7% sensitivity, 71.4% specificity). The model correctly classified 1377 out of 1388 (99.2%) encounters as "appropriate" (do not require ASP intervention). 624 antibiotic use encounters were classified as "inappropriate", of which only 72 were truly inappropriate (positive predictive value for ASP intervention, PPV 11.5%). The low PPV was likely due to inadequate representation of "inappropriate" cases in the training dataset (4.1%). Applying this model would prioritise the number of immediate ASP reviews needed to identify cases for intervention by two-thirds, from 2012 to 624 (Figure 1).

Figure 1. Illustration of AI benefits in ASP
Conclusion. ASPs can leverage on machine learning capabilities to improve audit efficiency. This can increase ASP's productivity and staff 's job satisfaction as they are freed up to perform other work.
Disclosures. All Authors: No reported disclosures