Comparative analysis of a rapid diagnostic test and scoring tools for ESBL detection in Enterobacterales bloodstream infections for optimizing antimicrobial therapy

ABSTRACT Extended-spectrum beta-lactamase (ESBL)-producing Enterobacterales bloodstream infections (BSI) are associated with increased cost, morbidity, and mortality compared with BSI by non-ESBL-producing bacteria. Phenotypic susceptibility data from blood cultures can take up to 72 hours to result, leading to delays in appropriate antimicrobial administration and unnecessary carbapenem usage. We sought to evaluate the performance of a rapid diagnostic test (RDT) and two published diagnostic scoring tools for predicting ceftriaxone resistance in BSI by Enterobacterales. This was a retrospective observational cohort study evaluating adult patients with BSI by select Enterobacterales species (Escherichia coli, Klebsiella oxytoca, Klebsiella pneumoniae group, Proteus mirabilis, and Salmonella spp.). We compared the performance of a multiplex-PCR panel by BioFire (BCID2) with two diagnostic scoring tools for predicting ceftriaxone resistance. Among 356 patients, ceftriaxone resistance was observed in 41 of 356 (11.5%) isolates. BCID2 accurately predicted ceftriaxone susceptibility in 99.2% of isolates (92.7% sensitivity and 100% specificity), compared with <90% for the two scoring tools (P < 0.05 for both). In ceftriaxone-resistant isolates (n = 41), BCID2 led to fewer instances of undertreatment with ceftriaxone compared with scoring tools (7.3% vs 68.3% and 92.7%). In ceftriaxone-susceptible isolates (n = 315), BCID2 led to no instances of overtreatment with carbapenems compared with 3.8% and 5.7% with the scoring tools. The BCID2 panel outperformed both ESBL scoring tools in our population. Local evaluation of RDT and scoring tools should be performed in conjunction with institutional implementation. IMPORTANCE Our study addresses a significant issue in the medical and scientific community—the delayed administration of appropriate antimicrobial treatments due to the time-consuming process of phenotypic susceptibility data collection in gram-negative bloodstream infections. Our research indicates that a multiplex PCR rapid diagnostic test (RDT) significantly outperformed two clinical scoring tools in predicting ceftriaxone susceptibility. Multiplex PCR also led to reduced instances of undertreatment with ceftriaxone and minimized overtreatment with carbapenems. Furthermore, multiplex PCR demonstrated high sensitivity and specificity in predicting ceftriaxone susceptibility. The results of our study underscore the potential RDTs to reduce the time to appropriate antimicrobial therapy, leading to improved patient outcomes and reduced healthcare costs.

Reviewer #2 (Comments for the Author): In this study, the authors evaluated the performance of a RDT (BCID2) in predicting ceftriaxone resistance in BSI caused by select ESBL-E.The authors also compared the performance of BCID2 in predicting ceftriaxone resistance compared to two diagnostic scoring tools.The manuscript is well-written and easy to follow.Below are some possible points of clarification/elaboration specific to the current manuscript: 1.The scoring tools by Augustine and Lee were published in 2017 and it is unclear the clinical relevance of either scoring tool, especially since the current paper's methods and results align closely with the paper by Cwengros et all (2020; DOI: 10.1093/ofid/ofaa278) that identified similar benefits with another commonly used RDT (Verigene) compared to the same scoring tools.Consider adding some background information in the introduction or the discussion sections that address the current real-world relevance of these scoring tools.For example, have they been referenced in other recent ESBL-E papers that would lead clinicians to use the scoring tools over the seemingly more common practice of RDT results and CTX-M identification?2. Please provide a breakdown of information specific to the PPV and NPV differences between groups.Was there a commonality between the RDT and scoring tools that consistently led to either correct identification or misidentification among all three (e.g., organisms [ESBL-E], source of infection, etc)?From those results, is there a better scoring tool that could be created using data from the current study and that from Cwengros et al. that would facilitate appropriate therapy in the first 24 hours of therapy? 3. Please clarify how ceftriaxone susceptibility/resistance was defined prior to the release of MIC data.Was it strictly based on CTX-M presence or absence? 4. Authors may consider incorporating a two-tier NPV and PPV approach.Tier 1 would be CTX-M results and Tier 2 would be CRO-R vs. CRO-S.5. Please update the text in lines 74-75 to fix the presumed typo.

Preparing Revision Guidelines
To submit your modified manuscript, log onto the eJP submission site at https://spectrum.msubmit.net/cgi-bin/main.plex.Go to Author Tasks and click the appropriate manuscript title to begin the revision process.The information that you entered when you first submitted the paper will be displayed.Please update the information as necessary.Here are a few examples of required updates that authors must address: • Point-by-point responses to the issues raised by the reviewers in a file named "Response to Reviewers," NOT IN YOUR COVER LETTER.
• Upload a compare copy of the manuscript (without figures) as a "Marked-Up Manuscript" file.
• Each figure must be uploaded as a separate file, and any multipanel figures must be assembled into one file.• Manuscript: A .DOC version of the revised manuscript • Figures: Editable, high-resolution, individual figure files are required at revision, TIFF or EPS files are preferred For complete guidelines on revision requirements, please see the journal Submission and Review Process requirements at https://journals.asm.org/journal/Spectrum/submission-review-process.Submissions of a paper that does not conform to Microbiology Spectrum guidelines will delay acceptance of your manuscript." Please return the manuscript within 60 days; if you cannot complete the modification within this time period, please contact me.If you do not wish to modify the manuscript and prefer to submit it to another journal, please notify me of your decision immediately so that the manuscript may be formally withdrawn from consideration by Microbiology Spectrum.
If your manuscript is accepted for publication, you will be contacted separately about payment when the proofs are issued; please follow the instructions in that e-mail.Arrangements for payment must be made before your article is published.For a complete list of Publication Fees, including supplemental material costs, please visit our website.
Corresponding authors may join or renew ASM membership to obtain discounts on publication fees.Need to upgrade your membership level?Please contact Customer Service at Service@asmusa.org.
Thank you for submitting your paper to Microbiology Spectrum.
In this study, the authors evaluated the performance of a RDT (BCID2) in predicting ceftriaxone resistance in BSI caused by select ESBL-E.The authors also compared the performance of BCID2 in predicting ceftriaxone resistance compared to two diagnostic scoring tools.The manuscript is well written and easy to follow.Below are some possible points of clarification/elaboration specific to the current manuscript: 1.The scoring tools by Augustine and Lee were published in 2017 and it is unclear the clinical relevance of either scoring tool, especially since the current paper's methods and results align closely with the paper by Cwengros et all (2020; DOI: 10.1093/ofid/ofaa278) that identified similar benefits with another commonly used RDT (Verigene) compared to the same scoring tools.Consider adding some background information in the introduction or the discussion sections that addresses the current real-world relevance of these scoring tools.For example, have they been referenced in other recent ESBL-E papers that would lead clinicians to use the scoring tools over the seemingly more common practice of RDT results and CTX-M identification?2. Please provide a breakdown of information specific to the PPV and NPV differences between groups.Was there a commonality between the RDT and scoring tools that consistently led to either correct identification or misidentification among all three (e.g., organisms [ESBL-E], source of infection, etc)?From those results, is there a better scoring tool that could be created using data from the current study and that from Cwengros et al. that would facilitate appropriate therapy in the first 24 hours of therapy? 3. Please clarify how ceftriaxone susceptibility/resistance was defined prior to release of MIC data.Was it strictly based on CTX-M presence or absence? 4. Authors may consider incorporating a two-tier NPV and PPV approach.Tier 1 would be CTX-M results and Tier 2 would be CRO-R vs. CRO-S.5. Please update text in lines 74-75 to fix the presumed typo.
Below, we have listed each comment and corresponding response, including selections of manuscript text where the relevant changes have been made.Reviewer comments are denoted by boldface text and direct responses to the reviewers are denoted by italics.All edits to the manuscript have been highlighted in yellow.

Reviewer Comments:
Reviewer #1 (Comments for the Author): Thank you for conducting this study comparing the BCID2 panel with risk scoring tools.I feel like there are a few different concepts being brought into this study that make the overall message diluted a bit.Some of the concepts I'm picking up include stewardship impact on RDT use/effectiveness, ensuring local validation of publicly available risk scoring tools, and accuracy of RDTs to scoring tools.While these concepts are all linked, I would highly encourage the authors to focus on 1 or maybe 2 of those concepts at most to strengthen the take-home message.That said, the take-home message of this paper seems to be that BCID2 is accurate and should be used (which is known information), hence, spinning the paper a touch more to RDT accuracy vs just using a scoring tool (with or without validating it) might make this paper a bit more novel/useful to existing literature.

Thank you for your thoughtful feedback on our manuscript. We appreciate your comments and the effort you've invested in reviewing our work.
You correctly pointed out that there are multiple concepts embedded within our study.We acknowledge that intertwining these concepts might make the primary message less clear to our readers.We intended to provide a comprehensive analysis but understand the importance of maintaining clarity and focus.
We have adjusted the positioning in the abstract, and conclusion to downplay the stewardship impact.We believe both the performance of RDT compared to culture and risk scoring tools are critical messages to maintain and thus have left those as is.

Reviewer #2 (Comments for the Author):
In this study, the authors evaluated the performance of a RDT (BCID2) in predicting ceftriaxone resistance in BSI caused by select ESBL-E.The authors also compared the performance of BCID2 in predicting ceftriaxone resistance compared to two diagnostic scoring tools.The manuscript is well-written and easy to follow.Below are some possible points of clarification/elaboration specific to the current manuscript: We appreciate your review and considerate feedback.We have added more background information in the discussion on the clinical relevance of the scoring tools with improved context more clearly delineated with the reference of a recent systematic review and meta-analysis on this topic (https://pubmed.ncbi.nlm.nih.gov/37760748/) which reflects the commonality of the development and real-world use of the risk score tools.

2.
Please provide a breakdown of information specific to the PPV and NPV differences between groups.Was there a commonality between the RDT and scoring tools that consistently led to either correct identification or misidentification among all three (e.g., organisms [ESBL-E], source of infection, etc)?From those results, is there a better scoring tool that could be created using data from the current study and that from Cwengros et al. that would facilitate appropriate therapy in the first 24 hours of therapy?Derivation of a novel risk scoring or prediction model was beyond the scope of this paper.As noted above, we have added expanded background on the use of these tools in the discussion with a referenced systematic review.Additionally, to this point we have added discussion around the challenges in performance of risk prediction tools, particularly with external validation given most risk factors are only low to moderately sensitive or specific beyond previous resistant organism infection or colonization.

Please clarify how ceftriaxone susceptibility/resistance was defined prior to the release of MIC data. Was it strictly based on CTX-M presence or absence?
Yes, the ceftriaxone susceptibility prior to release of MIC data was based on CTX-M presence or absence.We have added a sentence to the methods on this clarification.
4. Authors may consider incorporating a two-tier NPV and PPV approach.Tier 1 would be CTX-M results and Tier 2 would be CRO-R vs. CRO-S.
While this is an interesting approach to analyzing our data, we feel that this is beyond the scope of our research, as this places the focus on the diagnostic scoring tools rather than comparing the accuracy of a rapid diagnostic test with that of the scoring tools.Your manuscript has been accepted, and I am forwarding it to the ASM production staff for publication.Your paper will first be checked to make sure all elements meet the technical requirements.ASM staff will contact you if anything needs to be revised before copyediting and production can begin.Otherwise, you will be notified when your proofs are ready to be viewed.
Data Availability: ASM policy requires that data be available to the public upon online posting of the article, so please verify all links to sequence records, if present, and make sure that each number retrieves the full record of the data.If a new accession number is not linked or a link is broken, provide production staff with the correct URL for the record.If the accession numbers for new data are not publicly accessible before the expected online posting of the article, publication may be delayed; please contact ASM production staff immediately with the expected release date.
Publication Fees: For information on publication fees and which article types have charges, please visit our website.We have partnered with Copyright Clearance Center (CCC) to collect author charges.If fees apply to your paper, you will receive a message from no-reply@copyright.com with further instructions.For questions related to paying charges through RightsLink, please contact CCC at ASM_Support@copyright.com or toll free at +1-877-622-5543.CCC makes every attempt to respond to all emails within 24 hours.
ASM Membership: Corresponding authors may join or renew ASM membership to obtain discounts on publication fees.Need to upgrade your membership level?Please contact Customer Service at Service@asmusa.org.
PubMed Central: ASM deposits all Spectrum articles in PubMed Central and international PubMed Central-like repositories immediately after publication.Thus, your article is automatically in compliance with the NIH access mandate.If your work was supported by a funding agency that has public access requirements like those of the NIH (e.g., the Wellcome Trust), you may post your article in a similar public access site, but we ask that you specify that the release date be no earlier than the date of publication on the Spectrum website.

Embargo Policy:
A press release may be issued as soon as the manuscript is posted on the Spectrum Latest Articles webpage.The corresponding author will receive an email with the subject line "ASM Journals Author Services Notification" when the article is available online.
The ASM Journals program strives for constant improvement in our submission and publication process.Please tell us how we can improve your experience by taking this quick Author Survey.
Thank you for submitting your paper to Spectrum.

1.
The scoring tools by Augustine and Lee were published in 2017 and it is unclear the clinical relevance of either scoring tool, especially since the current paper's methods and results align closely with the paper by Cwengros et all (2020; DOI: 10.1093/ofid/ofaa278) that identified similar benefits with another commonly used RDT (Verigene) compared to the same scoring tools.Consider adding some background information in the introduction or the discussion sections that address the current real-world relevance of these scoring tools.For example, have they been referenced in other recent ESBL-E papers that would lead clinicians to use the scoring tools over the seemingly more common practice of RDT results and CTX-M identification?

5.
Please update the text in lines 74-75 to fix the presumed typo.
-23R1 (Comparative Analysis of a Rapid Diagnostic Test and Scoring Tools for ESBL Detection in Enterobacterales Bloodstream Infections for Optimizing Antimicrobial Therapy) Dear Dr. Sam Rose Andrews: