Multicenter Evaluation of the BIOFIRE Blood Culture Identification 2 Panel for Detection of Bacteria, Yeasts, and Antimicrobial Resistance Genes in Positive Blood Culture Samples

ABSTRACT Diagnostic tools that can rapidly identify and characterize microbes growing in blood cultures are important components of clinical microbiology practice because they help to provide timely information that can be used to optimize patient management. This publication describes the bioMérieux BIOFIRE Blood Culture Identification 2 (BCID2) Panel clinical study that was submitted to the U.S. Food & Drug Administration. Results obtained with the BIOFIRE BCID2 Panel were compared to standard-of-care (SoC) results, sequencing results, PCR results, and reference laboratory antimicrobial susceptibility testing results to evaluate the accuracy of its performance. Results for 1,093 retrospectively and prospectively collected positive blood culture samples were initially enrolled, and 1,074 samples met the study criteria and were included in the final analyses. The BIOFIRE BCID2 Panel demonstrated an overall sensitivity of 98.9% (1,712/1,731) and an overall specificity of 99.6% (33,592/33,711) for Gram-positive bacteria, Gram-negative bacteria and yeast targets which the panel is designed to detect. One hundred eighteen off-panel organisms, which the BIOFIRE BCID2 Panel is not designed to detect, were identified by SoC in 10.6% (114/1,074) of samples. The BIOFIRE BCID2 Panel also demonstrated an overall positive percent agreement (PPA) of 97.9% (325/332) and an overall negative percent agreement (NPA) of 99.9% (2,465/2,767) for antimicrobial resistance determinants which the panel is designed to detect. The presence or absence of resistance markers in Enterobacterales correlated closely with phenotypic susceptibility and resistance. We conclude that the BIOFIRE BCID2 Panel produced accurate results in this clinical trial.

B loodstream infections are associated with high morbidity and mortality, and the rapidity of detection, characterization, and chemotherapeutic management of these microbes impacts the clinical outcome of patients with bacteremia and fungemia. Therefore, the discipline of clinical microbiology continues to focus on developing tools that can more rapidly detect and characterize microbes causing bloodstream infections. The BIOFIRE Blood Culture Identification 2 (BCID2) Panel is an in vitro diagnostic device that uses nucleic acid amplification testing to detect and identify common pathogens and contaminants isolated from blood cultures.
Standard-of-care (SoC) detection and reference standard detection of bacteremia and fungemia are currently based on broth blood cultures of the microbes from blood specimens. Blood specimens are collected directly into broth bottles and then incubated upon receipt in the laboratory using a continuously monitored blood culture instrument, which identifies microbial presence in the bottles by detecting metabolic by-products produced by the microbes during their growth and replication. When microbial presence is detected by the instrument, a portion of the positive blood culture (PBC) is examined microscopically and subcultured to nutrient agar media. Additionally, rapid tests such as the BIOFIRE BCID2 Panel can be performed at the time microbial growth is detected in the PBC. These rapid tests have the greatest clinical impact and financial value when employed in close partnership with caregivers outside the laboratory who are ready to quickly act on these rapid test results (1,2).
The current study describes the prospective clinical evaluation of the BIOFIRE BCID2 Panel that was submitted to the U.S. Food and Drug Administration (FDA) for clearance as an in vitro diagnostic test. The BIOFIRE BCID2 Panel is a second-generation assay, which is an iterative improvement of the BIOFIRE Blood Culture Identification (BCID) Panel (3,4). The BIOFIRE BCID2 Panel is differentiated from the original BIOFIRE BCID Panel by increased specificity of some taxonomic identification, additional identifiable taxa, and additional genetic resistance markers (Table 1).

MATERIALS AND METHODS
Study overview. The study was conducted at nine sites across the United States and Europe over a period of approximately 8 months (October 2018 to May 2019). At each study site, a waiver of the requirement for informed consent was obtained from the Institutional Review Board (IRB) for the use of residual blood culture samples created during SoC.
Aerobically and anaerobically cultured samples identified as positive for microbial growth by an automated continuous monitoring blood culture system (CMBCS) were frozen or tested using the BIOFIRE BCID2 Panel within 24 h of positivity. PBC samples collected from October 2018 through February 2019 were immediately frozen for future testing, and fresh PBC samples were collected between January and May 2019. Frozen samples were tested immediately after thawing, and fresh samples were tested within 24 h of positive blood bottle indication. All samples were tested between January 2019 and May 2019. Sample enrollment was not discriminated based on subject age, sex, patient location, or diagnosis; however, PBC bottles containing charcoal in the culture medium were excluded, which is in agreement with the FDA-cleared instructions for use, and only one sample per subject was used in the study.
SoC subculture and isolate identification were performed per each laboratory's routine practice, and this identification was used as the comparator method for organism identification. The comparator method for antimicrobial resistance (AMR) genes was not obtained using each laboratory's SoC testing. Instead, AMR genes were identified by comparing to FDA-cleared predicate devices or PCR assays performed in a research setting.
BIOFIRE BCID2 Panel testing. An investigational-use-only (IUO) version of the BIOFIRE BCID2 Panel that is identical to the commercial (i.e., FDA-cleared [5], CE-marked) in vitro diagnostic (IVD) version was used in this study. All specimen manipulation was performed with good laboratory biosafety practices and in accordance with the manufacturer's instructions (https://www.online-ifu.com/ITI0048). The BIOFIRE BCID2 Panel incorporates internal controls, automated nucleic acid extraction, amplification, detection, and qualitative result interpretation with a sample-to-answer time of approximately 1 h. The BIOFIRE BCID2 Panel simultaneously tests for 43 targets (Table 1), 18 of which are modified or new to the BIOFIRE BCID2 Panel compared to the BIOFIRE BCID Panel. The detection or lack of detection of each bacterial AMR gene is reported only if a potential microbial carrier of the gene is also detected ( Table 2). Both FilmArray 2.0 and FilmArray Torch instruments were used with the BIOFIRE BCID2 Panel in the study. Quality control testing was performed each day of testing at each site. Environmental swab wipe tests were performed regularly on the BIOFIRE BCID2 Panel to ensure the absence of amplicon contamination near the testing location.
Comparator testing. (i) Organism identification. SoC methods to identify bacteria and fungi isolates from PBC were followed at each study site. A summary of each site's SoC culture and organism identification methods is provided in Table 3. Discrepancies in identification between the BIOFIRE BCID2 Panel and SoC were investigated further. Additionally, all Candida species were further characterized by bidirectional sequencing to confirm the SoC identification.
(ii) AMR gene detection. (a) BIOFIRE BCID Panel. All samples were interrogated for resistance markers using the BIOFIRE BCID2 Panel's predicate device, which is the FDA-cleared BIOFIRE BCID Panel (BIOFIRE, Salt Lake City, UT). The BIOFIRE BCID Panel was the comparator test used for mecA/C, vanA/B, and KPC.
(b) PCR assays. AMR targets not included on the BIOFIRE BCID Panel (CTX-M, IMP, NDM, OXA-48-like, VIM, and mcr-1) were interrogated using single, nested PCR assays followed by bidirectional sequencing of amplicons. When feasible, the comparator PCR assays were designed to target different regions of the AMR genes than are targeted by the BIOFIRE BCID2 Panel. Comparator PCR assays were designed to generate sufficient sequence information for conclusive AMR gene identification. The comparator sequences for CTX-M, IMP, NDM, VIM and mcr-1 were all downstream of the BIOFIRE BCID2 Panel target; most sequences contained some overlap between the panel target and the comparator target, but the mcr-1 comparator region did not overlap. OXA-48-like used two comparator sequences: one was upstream with no overlap with the BIOFIRE BCID2 Panel target, and one was downstream with significant overlap. (iii) ESBL phenotypic detection and antimicrobial susceptibility test interpretations. Phenotypic detection of ESBL and susceptibility to cefepime, piperacillin-tazobactam, and meropenem were determined for all Enterobacterales isolates using broth microdilution (BMD) antimicrobial susceptibility testing. Customdesigned frozen BMD Sensititre trays (Thermo Fisher, Waltham, MA) were used for MIC testing of the isolates.   Phenotypic detection of extended-spectrum b-lactamases (ESBL) was identified if clavulanic acid decreased the MIC of ceftazidime or cefotaxime by more than two doubling dilutions. The Clinical and Laboratory Standards Institute (CLSI) document Performance Standards for Antimicrobial Susceptibility Testing (6) was used to interpret MIC results. Enterobacterales isolates with an MIC of #2 mg/mL of cefepime, #8/4 mg/mL of piperacillin-tazobactam, and #1 mg/mL of meropenem were interpreted as susceptible. Enterobacterales isolates with MICs of 4 to 8 mg/mL of cefepime and 16/4 mg/mL of piperacillin-tazobactam were interpreted as susceptible-dose dependent (SDD).
Results and discrepancy analysis. A BIOFIRE BCID2 Panel result was considered a true positive (TP) or true negative (TN) when it agreed with the result from the comparator method. A result was considered a false positive (FP) or false negative (FN) when it disagreed with the result from the comparator method. Sensitivity or positive percent agreement (PPA) was calculated as 100 Â TP/ (TP 1 FN), while specificity or negative percent agreement (NPA) was calculated as 100 Â TN/(TN 1 FP). When sufficient PBC specimen volume and/or the isolate was available, discordant results were investigated using additional, independent molecular testing. Note that the performance data for sensitivity or PPA and specificity or NPA presented here consist of unresolved data, as presented in the IFU for the FDA-cleared test; discrepancy investigation is provided but was not used to recalculate performance data.
Statistical analysis. The exact binomial two-sided 95% confidence intervals (95% CI) were calculated for performance measures according to the Wilson score method (7).

RESULTS
A single external control was successfully performed each day of testing at each of the nine testing sites. Each site rotated through one negative and four positive controls in succession. If a successful external control result was not obtained, all BIOFIRE BCID2 Panel runs from the site on that day were excluded from analysis. A total of 146 swab wipe tests were performed on the BIOFIRE BCID2 Panel to monitor the laboratory environment for possible contamination, and all results were reported as "not detected" for all reportable targets.
In total, 1,093 clinical samples were enrolled in the intent-to-test group. Of the 1,093 samples, 17 (1.5%) were excluded for not meeting the defined inclusion criteria, such as being tested more than 24 h after being flagged as positive or being a second sample from the same subject. One additional sample (0.1%) was excluded due to protocol deviation, and one additional sample (0.1%) was excluded because successful external control results were not obtained. Of the intent-to-test samples, 1,074 (98.3%) were included in the final analysis; 1,005 (93.6%) were fresh samples tested between January and May 2019, and 69 (6.4%) were banked (frozen) samples that were later thawed and immediately tested. No difference in assay performance was recognized between fresh and frozen samples. Of the 1,074 samples included in the final analysis, 24.3% (261/1,074) were from individuals less than 18 years of age. Of the 1,074 samples included in the final analysis, 976 (90.9%) had microbes detected by the BIOFIRE BCID2 Panel. All except one of the organism targets on the BIOFIRE BCID2 Panel had a sensitivity of 92.3% or greater, and all but one organism target had a specificity of 96.6% or higher (Table 4). Overall, the sensitivity of the BIOFIRE BCID2 Panel for the detection of on-panel microbes was 98.9% (1,712/1,731). One thousand one hundred twenty organism targets were detected in the 976 positive samples, which included 125 (12.8%) samples that had multiple organism targets detected by the BIOFIRE BCID2 Panel (see Table S1 in the supplemental material).
In the 1,074 samples included in the final analysis, 118 off-panel microbes were detected by the comparator (i.e., SoC) method that were not included on the BIOFIRE BCID2 Panel (Table 5). By comparison, 128 organisms detected by the comparator (i.e., SoC) were off panel for the predicate BIOFIRE BCID Panel. Eighty-eight off-panel microbes were present in samples that had no targets detected by the BIOFIRE BCID2 Panel, and 30 off-panel microbes were detected in a total of 30 samples that had additional taxa detected by the BIOFIRE BCID2 Panel.
In the 1,074 samples included in the final analysis, 327 AMR genes were detected and able to be associated with a detected taxon. AMR gene results demonstrated a PPA of 91.2% or higher and an NPA of 97.9% or higher (Table 6). Overall, the sensitivity of detecting on-panel AMR genes was 97.9% (325/332). No detections were observed for IMP, OXA-48-like, or mcr-1.
Discrepancy analysis, which included but was not limited to additional PCR and sequencing direct from the sample as well as additional identification methods performed on the isolates, was performed for all FN and FP results (Table S2). Of all FN, the BIOFIRE BCID2 Panel result was confirmed for 38%, and the comparator result was confirmed in 42%. Five FN result investigations were inconclusive, meaning that no evidence of the presence of the analyte was found in the sample via additional PCR testing or other investigations. For all FP results, the BIOFIRE BCID2 Panel results were confirmed for 55% and the comparator result was confirmed for 45%. As indicated in the footnote to Table S2, the 45% of FP results in which the comparator results were confirmed were all attributed to the presence of nucleic acid from nonviable Escherichia coli in specific lots of blood culture bottles.
Gram-negative bacilli were recovered from 307 PBC specimens using SoC; at least one Enterobacterales isolate was recovered in 87.9% (270/307) of these specimens. Of the 270 specimens from which at least one Enterobacterales isolate was recovered, the BIOFIRE BCID2 Panel detected Enterobacterales in 269. BIOFIRE BCID2 detected Enterobacterales in 54 additional samples in which no isolate was recovered by culture; 53 of these were attributed to the presence of nonviable E. coli in specific lots of blood culture bottles.

DISCUSSION
The BIOFIRE BCID2 Panel is differentiated from the original BIOFIRE BCID Panel in that it has increased specificity of some taxonomic identifications, additional identifiable taxa, and additional genetic resistance markers. Notably, the BIOFIRE BCID2 Panel has the following modifications from the BIOFIRE BCID Panel: expanded detection of the Acinetobacter calcoaceticus-Acinetobacter baumannii complex, Enterobacterales, Staphylococcus spp., and Streptococcus spp.; individual species/genus assays instead of higher-level interpretations for Enterococcus faecalis, Enterococcus faecium, Klebsiella aerogenes, Salmonella spp., Staphylococcus epidermidis, and Staphylococcus lugdunensis; and the addition of Bacteroides fragilis, Candida auris, Cryptococcus neoformans/Cryptococcus gattii, and Stenotrophomonas maltophilia. Further, the AMR gene menu on the BIOFIRE BCID2 Panel has been expanded to include targets for CTX-M, IMP, NDM, OXA-48-like, VIM, mecA/C, and MREJ (MRSA), and mcr-1, which can help to guide antimicrobial selection before phenotypic antimicrobial susceptibility testing (AST) results and interpretations are available.
Although C. auris and Cryptococcus neoformans/Cryptococcus gattii fungemia is rarely encountered and was not present in any samples from this clinical study, the performance of the panel for these pathogens was evaluated using archived and seeded PBC samples,  and these analytes are included in the FDA-cleared assay. Rapid identification of these yeasts should provide clinical value due to their resistance to antifungal agents. B. fragilis group and S. maltophilia were identified by the BIOFIRE BCID2 Panel in 100% (6/6) and 87.5% (7/8) of the samples containing these bacteria in the current study (Table 4). Rapid identification of S. maltophilia is important because of its resistance to commonly used empirical Gram-negative antibiotics (8).
Potentially the most clinically valuable addition to the BIOFIRE BCID2 Panel is the CTX-M target, which was identified in 17% of samples containing Enterobacterales. The presence of CTX-M correlated closely with resistance to cefepime (Table 8), and piperacillintazobactam is often avoided when an ESBL like CTX-M is present because of the findings from the MERINO trial (9). Enterobacterales were reliably susceptible to meropenem when carbapenemase genes were not detected (Table 10). Similar to the findings of a study by Spafford and colleagues, genetic resistance markers like CTX-M and KPC are able to identify many Enterobacterales isolates that lack susceptibility to relevant beta-lactams, but a substantial minority of Enterobacterales isolates that are not susceptible to third-or fourth-generation cephalosporins do not have CTX-M (10). When the presence or absence of a single gene marker (e.g., CTX-M) is used to guide empirical antimicrobial therapy, it is important to consider not only the sensitivity of an assay but also its negative predictive value, which is dependent upon local prevalence of overall resistance and also resistance mechanisms (11). In the future, characterization and analysis of full genomic data rather than single gene markers could be used to more accurately predict phenotypic susceptibility and resistance (12). With the addition of the expanded Enterobacterales AMR targets, local laboratories in collaboration with local antimicrobial stewardship programs will be able to design optimized empirical antibiotic regimens that can be informed by the BIOFIRE BCID2 Panel AMR results.
A limitation of the BIOFIRE BCID2 Panel revealed by the study is the ability of the assay to detect nucleic acid from nonviable Enterobacterales (Table 4 and Table S2). In practice, this limitation can be potentially mitigated by requiring Gram stain findings to be congruent with an "Enterobacterales" identification when no specific taxa within Enterobacterales are identified by the BIOFIRE BCID2 Panel. This mitigation strategy would be in line with the IFU, which states that BIOFIRE BCID2 Panel results are intended to be interpreted in conjunction with Gram stain results. Total 32 10 218 a The MICs were determined using broth microdilution and interpreted using the CLSI standards (6) as susceptible (S), susceptible-dose dependent (SDD), or resistant (R). Six specimens with KPC, VIM, and/or NDM were excluded from this analysis (n = 260). a Four additional specimens had at least one Enterobacterales isolate recovered but did not have phenotypic susceptibility results available. b KPC was detected in four specimens; NDM and VIM were detected in one specimen. c VIM was detected in one specimen.
The greatest limitation of the study is that not all targets were thoroughly interrogated, as some targets are rarely encountered in clinical practice. This is a common challenge in clinical studies that evaluate broad, multitarget panels. This limitation is mitigated by evaluating the performance of the test with contrived samples. The performance of the BIOFIRE BCID2 Panel with contrived samples was performed, and the results can be found in the BIOFIRE BCID2 Panel IFU (https://www.online-ifu.com/ITI0048).
We conclude that the BIOFIRE BCID2 Panel will be a valuable addition to the growing repertoire of IVD tools to rapidly identify and characterize microbes recovered by blood culture.

SUPPLEMENTAL MATERIAL
Supplemental material is available online only. SUPPLEMENTAL FILE 1, DOCX file, 0.03 MB.