Derivation and validation of a simple score to predict the presence of bacteria requiring carbapenem treatment in ICU-acquired bloodstream infection and pneumonia: CarbaSCORE

Background The recommendations of learned societies mention risk factors for the presence of multidrug resistant bacteria in hospital-acquired infections, but they do not propose a scoring system to guide empiric antibiotic therapy. Our study was aimed at developing a simple score for predicting “the presence of bacteria requiring carbapenem treatment” in ICU-acquired bloodstream infection and pneumonia. Methods Between December 2011 and January 2015, we conducted a retrospective study using a prospectively collected French database of nosocomial infections in the polyvalent intensive care unit of a French university hospital. All patients with ICU-acquired bloodstream infection or pneumonia were included in the study. Bivariate and multivariate analyses were performed to develop the CarbaSCORE, and this score was internally validated. Results In total, 338 patients were analyzed, including 27 patients requiring carbapenem treatment. The CarbaSCORE was composed of four criteria: “presence of bloodstream infection” (as opposed to pneumonia) scored 2 points, “chronic hemodialysis” scored 4 points, “travel abroad in the last 6 months” scored 5 points, and “MDR-colonization or prior use of a β-lactam of class ≥ 3” scored 6 points. Internal validation by bootstrapping showed an area under the receiver operating characteristic curve of 0.81 [0.73–0.89]. Sensitivity was 96% at the 6-point threshold and specificity was 91% at the 9-point threshold. Conclusions The CarbaSCORE is a simple and efficient score for predicting the presence of bacteria requiring carbapenem treatment. Further studies are needed to test this score before it can be used in practice. Electronic supplementary material The online version of this article (10.1186/s13756-019-0529-z) contains supplementary material, which is available to authorized users.


Background
Effective antibiotic therapy is urgent in severe infections, and studies have shown that a delay in effective antibiotic therapy is associated with an increase in the mortality of patients with these infections [1,2]. For example, a 2008 meta-analysis of 12,296 intensive care patients with ventilator-associated pneumonia (VAP) or blood stream infection (BSI) found a link between delay in effective antibiotic therapy and mortality for both VAP (odd ratio 3.03 [1.12-8.19], p = 0.03) and BSI (odd ratio 2.28 [1.43-3.65], p = 0.006) [2]. Other authors have found statistically significant links between delay in effective antibiotic therapy and other outcome variables, such as length of stay in intensive care unit (ICU) and medical management costs [3,4]. In view of the above, empiric antibiotic therapy must effectively target the bacteria that cause infection.
The recommendations of learned societies for the antibiotic treatment of hospital-acquired infections mention risk factors for the presence of infection-causing multidrug resistant (MDR) bacteria [12][13][14][15][16][17][18][19][20][21][22][23]. However, there are discrepancies between these different recommendationsfor instance, regarding the presence of a state of septic shock or that of colonization with MDR bacteria. Moreover, these various risk factors have not been studied together, which raises the possibility of redundancies and multicollinearities [24]. Lastly, no score has been recommended to guide prescribing physicians in the choice of empiric antibiotic therapy for ICU-acquired infections.
The aim of this study was to develop a simple score for predicting "the presence of bacteria requiring carbapenem treatment" in ICU-acquired bloodstream infection and pneumonia.

Study population
Between December 2011 and January 2015, we conducted a retrospective study using a prospectively collected French database of nosocomial infections in the 23-bed medical/surgical adult ICU of a French university hospital (Centre Hospitalo-Universitaire Félix Guyon, Saint-Denis). The study was approved by the Ethics Committee of the Centre Hospitalo-Universitaire de La Réunion (reference R15016). We included all patients with ICU-acquired bloodstream infection or pneumonia that occurred more than 2 days after admission and that were reported to the national network for the surveillance of ICU-acquired infections (REA-raisin network).
Excluded from the analysis were patients under 18 years of age, patients with non-bacterial or undocumented infection, and patients whose incomplete records did not allow for analysis. Only the first episode of ICU-acquired infection was analyzed.

Data collection and processing
We analyzed demographic data, status on admission, microbiological examination results, severity scores at the time of infection, modified Clinical Pulmonary Infection Score (CPIS), therapies performed, and patient outcome (death, duration of ventilation, and length of stay in ICU) [25,26] All bloodstream infection and pneumonia occurring more than 2 days after ICU admission are reported to the REA-raisin network.
The selected date of infection was the date of microbiological sampling (blood culture or lung sample). The primary endpoint was "the presence of bacteria requiring carbapenem treatment," as defined by antimicrobial susceptibility testing analysis of the infection-causing bacteria. Antimicrobial susceptibility testing analysis was performed by one of the investigators (LT).
Colonization with MDR/XDR at the time of ICU-acquired infection was defined by knowledge of one positive hygiene or clinical sample for MDR/XDR at least 2 days before the date of infection.
Following recommendations of the French Society of Anesthesia and Intensive Care Medicine (SFAR) and the French Institute for Health Surveillance (InVs), ICU-acquired pneumonia was defined by: radiological signs with two successive images suggesting a focus of pneumonia (a single image being sufficient in the absence of a history of heart disease or of an underlying lung disease), and at least one of the following signs: temperature > 38.3°C with no other cause, leukocyte concentration < 4 G/L or ≥ 12 G/L, and at least two of the following signs: purulent tracheobronchial secretions, cough or dyspnea, and oxygen desaturation or increased oxygen requirement or ventilation demand [20,29].
The diagnosis had to be confirmed quantitatively by deep respiratory sampling: bronchoalveolar lavage with a positivity threshold ≥10 4 colony-forming units (CFU)/ml or protected distal sampling with a positivity threshold ≥10 3 CFU/ml or, failing this, tracheobronchial aspiration with a positivity threshold ≥10 6 CFU/ml. ICU-acquired bloodstream infection was defined by the presence of at least one positive blood culture-with the exception of coagulase-negative Staphylococci and other saprophytic microorganisms, for which two positive blood cultures were required.
Following the classification proposed by Weiss et al., β-lactams were ranked into six classes according to spectrum of activity (Additional file 1: Table S1) [30].
"The presence of bacteria requiring carbapenem treatment" was defined by the identification of one or more bacterial strains requiring treatment with a β-lactam class 6, with no possibility of using a lower class of β-lactams.
Carbapenem overuse was defined by empiric treatment with a β-lactam class 5 or 6, with the possibility of using a lower class of β-lactams based on antimicrobial susceptibility testing results.
Carbapenem treatment failure was defined by one (or more) bacteria causing infection was resistant to carbapenem (class 6), according to the antimicrobial susceptibility testing.

Statistical analysis
Results were expressed as medians and interquartile ranges for quantitative variables and as frequencies and proportions for qualitative variables. Quantitative variables were compared using the Mann-Whitney test. Qualitative variables were compared using the Pearson Chi-square test or Fisher's exact test, as appropriate. All analyses were performed with a bilateral alpha risk of 5%.
Variables collected at the time of empiric prescription of antibiotic therapy that were statistically associated in bivariate analysis with "the presence of bacteria requiring carbapenem treatment" (with a p < 0.05) were entered into a backward stepwise logistic regression model. A score was then developed from the model obtained by regression analysis, using as weight the logistic regression coefficient rounded to the nearest whole number. The calibration of the model was evaluated using the Hosmer-Lemeshow test, and the Nagelkerke and Cox/ Snell R 2 were calculated. Score performance was measured using the area under the receiver operating characteristic (ROC) curve and its 95% Confidence Interval. Finally, score performance was validated by bootstrapping (2000 replicates). The software SPSS version 15 (SPSS Inc., Chicago, Ill, USA) and R (pROC package) were used for data analysis [31][32][33]. This article follow the STROBE reporting guidelines for observational studies.

Results
Over the 37-month study period, 2584 patients were admitted to the ICU. Among these, 410 (15.8%) presented at least one ICU-acquired bloodstream infection or pneumonia more than 2 days after ICU admission, and were consequently included in the study. Seventy-two of these patients were excluded from the analysis, and the remaining 338 patients formed the study cohort ( Fig. 1).

Characteristics of patients at ICU admission
The characteristics of the 338 analyzed patients are presented in Table 1. Antimicrobial susceptibility testing of the bacterial strain responsible for infection in the intensive care unit indicates that carbapenem treatment was required in 27 cases (8%). The variables significantly associated with the variable "presence of bacteria requiring carbapenem treatment" were: Chronic renal failure (p = 0.03) and chronic hemodialysis (p = 0.01) Travel abroad in the last 6 months (p < 0.001) Colonization with MDR/XDR known on ICU admission (p = 0.02) or 2 days before the date of nosocomial infection (p < 0.001) Prior use of a β-lactam of class ≥3 (p = 0.02) A delay of more than 7 days between ICU admission and date of nosocomial infection (p = 0.05) Type of ICU-acquired infection (bloodstream infection as opposed to pneumonia, p = 0.02)  Table 2, along with their susceptibility profile to β-lactams. All infections combined, 61.8% of the bacteria found were Enterobacteriaceae, 39.9% were non-fermenting Gram-Negative Bacilli (GNB), 6.5% were other GNB, and 27.5% were Gram-positive bacteria. Eighty-four percent of the ICU-acquired pneumonia cases in our cohort were identified by deep respiratory sampling (protected distal sampling or bronchoalveolar lavage), and 19.6% were identified by tracheobronchial aspiration. In addition, the median modified CPIS score was 7 [6][7][8] for the 198 pneumonia cases.

ICU-acquired infections
As regards microbial resistance, 7.1% of the strains were ESBL-E and 0.9% were carbapenem-resistant Enterobacteriaceae. Among non-fermenting GNB strains, 2.36% were resistant to ceftazidime and 8.6% were resistant to imipenem. A total of 48 bacteria were resistant to carbapenems, representing 14.20% of all ICU-acquired infections. Of these 48 strains, 41 were susceptible to a β-lactam of a lower class (including 12 Pseudomonas with resistance to imipenem).

Colonization
In our cohort, 55 patients (16.3%) were colonized with MDR/XDR at least 2 days before the date of ICU-acquired infection. This status was known on ICU admission for 11 patients (3.2%). In the 55 patients colonized with MDR/ XDR, the analysis found 81 bacterial strains, 68 of which were sensitive to carbapenems and 13 were resistant to  carbapenems (Additional file 2: Table S2). There was a statistically significant association between colonization with MDR/XDR and "the presence of bacteria requiring carbapenem treatment" (p < 0.001).

Empiric antibiotic therapy
Of the 342 antibiotic therapies administered to the 338 cohort patients, 40.6% were monotherapies and 44.7% were dual therapies ( Table 3). The molecule most commonly administered as monotherapy was piperacillin-tazobactam (19.3%), followed by carbapenems (11.1%). Of all antibiotic therapies (whether monotherapies or multidrug therapies), 125 (36.5%) were empiric and included a carbapenem. Analysis of the adequacy of empiric antibiotic therapy with a carbapenem is detailed in Table 4. Antimicrobial susceptibility testing analysis of the infection-causing strains found that 68.8% of the 125 empiric antibiotic therapies with a carbapenem were ineffective. In addition, 20% of the patients who received empiric antibiotic therapy with a carbapenem had an infection with carbapenem-resistant bacteria.

Subgroup analysis
Subgroup analysis according to type of ICU-acquired infection was conducted and is presented in Additional file 3: Table S3. In this subgroup analysis, the significant association found in the total cohort between "the presence of bacteria requiring carbapenem treatment" and "prior use of a β-lactam class ≥ 3" was no longer observed. Moreover, the statistical link for chronic renal failure was no longer found in the pneumonia subgroup. In the subgroup of bloodstream infection, "chronic hemodialysis", "colonization with MDR/XDR bacteria known on admission" and "delay of more than 7 days between ICU admission and date of nosocomial infection" were no longer statistically associated with "the presence of bacteria requiring carbapenem treatment".

Multivariate logistic regression analysis and CarbaSCORE derivation
After multicollinearity analysis, variables collected at the time of empiric prescription of antibiotic therapy that were Including any immunosuppressive diseases, hematologic diseases, treatment with immunosuppressive drugs within the previous 30 days, or corticosteroids at daily doses of at least 10 mg/day of a prednisone equivalent for more than 2 weeks, b Selected threshold for chronic renal failure: Glomerular Filtration Rate (GFR) less than 60 ml/min/1.73 m 2 associated in bivariate analysis with "the presence of bacteria requiring carbapenem treatment" (with p < 0.05) were entered into a multivariate logistic regression analysis. These variables were, namely, chronic hemodialysis, bloodstream infection, prior use of a β-lactam class ≥3, travel abroad in the last 6 months, MDR colonization and delay of more than 7 days between ICU admission and date of nosocomial infection.
The variable "chronic renal failure" was not selected due to collinearity with chronic hemodialysis. Since "MDR colonization" was strongly related to "use of a β-lactam class ≥ 3" (p < 0.001), and because we wanted a score that can be calculated in ICUs that do not analyze MDR colonization, we decided to enter a composite item in the model: "colonization with MDR bacteria at least 2 days before the date of ICU-acquired infection or use of a β-lactam class ≥ 3 (in the 3 months prior to infection)" in the same item of the score. The independent variables selected in the final model are presented in Table 5. Thus, according to the chosen methodology, "presence of bloodstream infection" (as opposed to pneumonia) scored 2 points in the Carba-SCORE, "chronic hemodialysis" scored 4 points, "travel abroad in the last 6 months" scored 5 points and, finally, "colonization with MDR bacteria at least 2 days before the date of ICU-acquired infection or use of a β-lactam class ≥ 3 (in the 3 months prior to infection)" scored 6 points. The obtained CarbaSCORE thus varied from 0 to 17 points.
Internal validation of this score by bootstrapping (2000 replicates) showed an area under the ROC curve of 0.81 [0.73-0.89]. The ROC curve obtained for the CarbaSCORE is shown in Fig. 2. Table 6 presents the diagnostic performance of the CarbaSCORE in terms of sensitivity, specificity, positive and negative predictive values (with 95% Confidence Intervals determined by bootstrapping (2000 replicates)) for the main thresholds. We can see that sensitivity was 96 [89-100] % at the 6-point threshold and specificity was 91 [88-94] % at the 9-point threshold. The French recommendations of 2014 had a sensitivity and a specificity of 100 and 19%, respectively [19]. Amoxicillin-clavulanic acid and aminoglycoside 1 (0.

Discussion
The ongoing problem of emerging MDR and XDR involves two risks: (i) the risk that empiric antibiotic therapy will fail in cases of nosocomial infection with resistant bacterial strains; and (ii) the risk that over-prescription of carbapenems will favor the development of XDR. The aim of this study was to develop a simple score to help physicians prescribe carbapenems more sparingly in the treatment of major ICU-acquired infections, an issue that has not been well studied in the literature. The Car-baSCORE is a simple tool in that data for three of its four criteria are easily obtained from the patient. Data for the last criterion, "presence of bloodstream infection" (as opposed to pneumonia), are generally obtained from the clinical presentation or from laboratory results at the onset of ICU-acquired nosocomial infection. The diagnostic performance of the CarbaSCORE, which was validated in our cohort by bootstrapping, is interesting. On the one hand, a satisfactory sensitivity of 96 [89-100] % was achieved at the 6-point threshold, making it an appropriate choice for the most severe forms of infection. On the other hand, a specificity of 91 [88-94] % was achieved at the 9-point threshold, suggesting that it is a more appropriate choice for less severe forms. It should be noted that up to 2014, the French recommendations had a sensitivity and a specificity of 100 and 19%, respectively [19]. While updated recommendations propose taking into account case severity (acute respiratory distress syndrome, septic shock), the performance of the new criteria has not been assessed in the literature. Two other scores have been proposed in the literature to address this issue, but they all have several major limitations. First, the score proposed by Vasudevan et al. in 2014 to predict a resistant GNB infection [34]. Definition of resistant GNB was problematic (Acinetobacter baumannii, Pseudomonas aeruginosa, Klebsiella pneumoniae, or Escherichia coli with resistance to one or more agents among at least three classes of antibiotics, without further details being provided). Moreover, the 76 patients with ICU-acquired resistant GNB infection were compared with 1398 patients with systemic inflammatory response syndrome criteria; and not with patients with non-resistant GNB infection. The validation cohort had an area under the ROC curve of 0.77[0.68-0.89]. Second, the score developed by Sonti et al. in 2017, was also aimed at predicting resistant bacterial infection in ICU patients, but the bacterial ecology reported in the study was composed of nearly 45% MRSA and GRE strains, making the score inapplicable in ICUs where the local bacterial ecology does not have this resistance profile. Moreover, the study did not include any internal or external validation [35]. Finally, these two scores included poorly defined items and the calculation of the score proposed by Vasudevan et al. was quite complex.
The cohort presented in our study was large given the topic studied. The most recent studies on this topic focused on smaller or similar patient cohorts [36][37][38]. For example, a recent study of risk factors for resistant GNB bloodstream infection in ICU used a cohort of 177 patients [36]. Our study population was a standard ICU population composed of patients aged 61 years on average and recruited from medical and surgical units. However, only documented infections were examined. Not only was the observed bacterial ecology similar to that found in other French studies on the topic, but it was very similar to the figures presented in the last national survey of prevalence in France [39]. The ecology of bacterial infections, such as those with a high prevalence of CPE, may be considerably different elsewhere in Europe or in the world, and the CarbaSCORE would probably not apply to them [7]. The rate of ICU-acquired infection (15.8%) found in our study was higher than the national average (10.4%), which may be explained by the severity profile of patients in our cohort: indeed, severity scores on admission were 54 for SAPS II and 12 for SOFA. The mortality rate (31.1%) and the high proportion of patients ventilated on admission (89.1%) also reflects the severity profile of our patients. Other factors associated with increased risk of infection should be emphasized: use of extracorporeal circulation techniques (cardiac surgery center, renal replacement therapy in 44.7% of patients), and cancer/immunocompromised status on admission (17.5% of patients). There was also a high proportion of patients with diabetes, which is a well-known risk factor for infection [40,41].
Another interesting finding of our study is that more than one third of the treatments administered empirically included a carbapenem. However, antimicrobial susceptibility testing of the bacterial strains responsible for the infection revealed that only 11.2% of cases actually required carbapenem treatment. In a French observational study published in 2015, carbapenems were used three times out of four as empiric treatment for hospital-acquired infections; because this treatment seemed appropriate, it was continued in 21% of cases in 2009 and in 38% of cases in 2011 [42]. These observations confirm the interest of our study aimed precisely at improving empiric antibiotic therapy, given that the latter appears to perform poorly at the moment. Furthermore, empiric antibiotic therapy is not uncommon in cases where infection is not confirmed. It is therefore likely that overuse of carbapenems is even more important than suggested by the analysis of antibiotic therapies used to treat documented infections. As regards risk factors for resistant bacterial infection, our results suggest that some of the criteria mentioned in national and international recommendations are of little interest. Although we did not observe the presence of acute respiratory distress syndrome or of septic shock, the SOFA score on the date of infection was not statistically associated with appropriate carbapenem prescription in bivariate analysis. We wish we had evaluated these two specific criteria, but the latest recommendations were published after the start of our study.
Similarly, the scientific basis for using prior colonization with MDR/XDR bacteria as a criterion to guide empiric antibiotic therapy seems questionable. For example, a 2013 meta-analysis focusing on 791 cases of VAP drawn from 13 studies found that infection-causing bacteria were effectively predicted by routine surveillance of the bacteria identified in deep respiratory samples. However, there appears to be an important bias, as colonization and infection were diagnosed by the same examination [43]. It should also be noted that practice recommendations do not include routine surveillance of respiratory ecology in the treatment of VAP [19,38]. Moreover, a recent study investigating the predictive power of surveillance of rectal colonization with ESBL-E found a statistical link between rectal colonization with ESBL-E and the presence of ESBL-E in respiratory samples, with a sensitivity of 75.3[72.6-78.1] % and a specificity of 71[68.1-73.9] %. However, these samples were taken routinely, and not in the context of infection, thereby limiting the clinical relevance of the study [44].
In addition, most studies focusing on colonization with MDR bacteria as a criterion to guide empiric antibiotic therapy do not analyze the other risk factors for infection with MDR bacteria, thus raising the possibility of multicollinearities. Indeed, the analysis of our results yielded a strong collinearity between prior colonization with MDR/XDR bacteria and the variables "prior use of a β-lactam class ≥ 3". Screening for colonization with MDR/XDR bacteria is not universal, and some authors even propose to abandon the practice. For example, a recent before-and-after study found that discontinuation of routine surveillance of colonization with ESBL-E in ICU (with continuation of recommended hygiene measures, including routine isolation of patients) was not associated with an increase in the incidence of ESBL-E infection (p = 0.64), although it was accompanied by a significant decrease in patient exposure to carbapenems during the stay in ICU (p = 0.01) [45].
Another strength of our study is that cases of bloodstream infection and pneumonia were analyzed globally so as to propose a single score for both types of infection. Moreover, we did not limit our analysis to infections caused by Enterobacteriaceae. Many studies focus exclusively on the analysis of risk factors for ESBL-E infection [46]; but in practice, when prescribing empiric antibiotic therapy, ICU physicians cannot know whether infections are caused by Enterobacteriaceae or, for example, by non-fermenting GNB.
Our study has many limitations. First, our methodology involves biases and limits extrapolation of results. Although we have shown that the demographic and microbiological characteristics of our patients were standard in France, this study remains a monocentric study with results that are difficult to extrapolate, especially where ecology is quite different. Also, the statistical power of the cohort was moderate, with 27 cases of ICU-acquired infection requiring carbapenem treatment. Second, in our methodology, treatment with a β-lactam class < 5 was considered as therapeutic failure in patients infected with ESBL-E, which can be discussed. Indeed, the literature shows that there are effective alternatives for the treatment of mild ESBL-E infections with no deep infection foci and if the inhibitory concentration (MIC) is low [45][46][47][48]. Lastly, we could not analyze some of the criteria proposed by the latest American and French recommendations for the management of hospital-acquired pneumonia, because our study was initiated before these recommendations were published (in 2016 and 2017, respectively). It should be noted, however, that the criteria "acute respiratory distress syndrome during pneumonia" and "septic shock during pneumonia" are not based on a solid review of the literature. In the same way, we did not evaluate the scores of Vasudevan and Sonti of on our cohort because of the many biases of these studies [34,35]. Additional studies are therefore needed to assess the actual performance of the CarbaSCORE for all ICU-acquired infections, but also for infections acquired outside the ICU setting.

Conclusions
Our study proposes the CarbaSCORE, which is aimed at helping ICU physicians prescribe carbapenems sparingly in the empiric antibiotic therapy of the first episode of ICU-acquired bloodstream infection or pneumonia. This score is composed of four simple criteria, and its accuracy is good. Moreover, the score varies from 0 to 17 points, which may allow for choosing a different threshold depending on case severity and on level of risk-taking. Additional studies, including validation studies, are required for the CarbaSCORE to be used in practice.

Additional files
Additional file 1: