A simple mortality prediction model for sepsis patients in intensive care

Background: Sepsis is common in the intensive care unit (ICU). Two of the ICU’s most widely used mortality prediction models are the Simplified Acute Physiology Score 3 (SAPS-3) and the Sequential Organ Failure Assessment (SOFA) score. We aimed to assess the mortality prediction performance of SAPS-3 and SOFA upon ICU admission for sepsis and find a simpler mortality prediction model for these patients to be used in clinical practice and when conducting studies. Methods: A retrospective study of adult patients fulfilling the Sepsis-3 criteria admitted to four general ICUs was performed. A simple prognostic model was created using backward stepwise multivariate logistic regression. The area under the curve (AUC) of SAPS-3, SOFA and the simple model was assessed. Results: One thousand nine hundred eighty four admissions were included. A simple six-parameter model consisting of age, immunosuppression, Glasgow Coma Scale, body temperature, C-reactive protein and bilirubin had an AUC of 0.72 (95% confidence interval (CI) 0.69–0.75) for 30-day mortality, which was non-inferior to SAPS-3 (AUC 0.75, 95% CI 0.72–0.77) (p = 0.071). SOFA had an AUC of 0.67 (95% CI 0.64–0.70) and was inferior to SAPS-3 (p < 0.001) and our simple model (p = 0.0019). Conclusion: SAPS-3 has a lower prognostic value in sepsis than in the general ICU population. SOFA performs less well than SAPS-3. Our simple six-parameter model predicts mortality just as well as SAPS-3 upon ICU admission for sepsis, allowing the design of simple studies and performance monitoring.


Background
Sepsis is one of the most frequent diagnoses in intensive care units (ICUs), accounting for around 30% of all ICU admissions in Europe. 1,2It is also one of the leading causes of death in the world. 3The criteria for sepsis are suspected or documented infection and an acute increase of two Sequential Organ Failure Assessment (SOFA) points or more. 4he high incidence and mortality of sepsis make reliable and easy ICU prognostication of septic patients crucial.The Acute Physiologic and Chronic Health Evaluation II (APACHE II) and the Simplified Acute Physiology Score 3 (SAPS-3) are two of the most commonly used models for the assessment of illness severity and prediction of outcomes in the ICU. 5 While SAPS-3 has been shown to have high sensitivity and specificity, it is developed for a mixed ICU cohort. 6However, the ICU population is heterogeneous, and neither SAPS-3 nor APACHE scores are designed to estimate mortality rates for any specific ICU subgroup.Both SAPS-3 and APACHE II have demonstrated good mortality prediction in postoperative surgical patients. 7][10][11][12] However, studies have indicated that SAPS-3 is challenging to use in clinical practice due to its many variables. 13The SOFA score has previously demonstrated good prognostic accuracy in predicting hospital mortality in patients with suspected infection admitted to the ICU.Still, it is mainly intended for measuring and monitoring organ failure. 14,15

Objectives
This study aimed to assess the prognostic performance regarding mortality of SAPS-3 and SOFA score for patients with sepsis (according to Sepsis-3) admitted to the ICU and to find a simpler model with fewer parameters but maintained predictive value.Such a mortality prediction model could facilitate future studies and performance monitoring.

Study design and setting
This study was a retrospective multicentre observational study including adult patients fulfilling the Sepsis-3 criteria admitted to one of four mixed surgical and medical ICUs between 2015 and 2018.This study did not include patients admitted to specialised ICUs (cardiothoracic or neurosurgical).The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines were followed. 16

Participants
Patients fulfilling the Sepsis-3 criteria with a total SOFA score of 2 or more on ICU admission were included.All admitted patients had an assumed baseline SOFA score of 0. To establish the presence of suspected infection, included admissions had to fulfil the following criteria: At least one blood culture collected within 24 h of ICU admission and antibiotics administrated within 24 h before to 72 h after blood culture collection.These criteria were in accordance with the Sepsis-3 guidelines. 4Patients directly transferred from another ICU, planned ICU admissions following elective surgery and patients with cardiac arrest within 6 h prior, to 1 h after ICU admission were excluded.

Data sources
Data collection was performed by multiple trained data collectors using standardised and clearly defined guidelines.Data such as comorbidities, physiological parameters and laboratory findings were collected from electronic medical records.In addition, SOFA parameters, survival data, age and sex were collected from the Patient Administrative System for Intensive Care Units (PASIVA).

Variables
The worst recorded value 6 h prior, to 1 h after ICU admission was documented regarding physiological and laboratory parameters.SOFA and SAPS-3 scores were calculated based on the worst recorded values 1 h prior, to 1 h after ICU admission.Immunosuppression was defined as immunosuppressive treatment such as chemotherapy or neutropaenia with absolute neutrophil count <1.5 × 10 9 /L before ICU admission.Corticosteroid treatment was only recorded for a treatment duration of more than 3 days before ICU admission.Septic shock was defined as vasopressor use and lactate >2 mmol/L at ICU admission.The patients were divided into two subgroups based on the outcome: Alive more than 30 days after ICU admission (survivors) and dead within 30 days from ICU admission (non-survivors).We used the Swedish 2016 SAPS-3 calibration, which estimates the 30-day mortality, to calculate the estimated mortality ratio (EMR). 17Therefore, we used 30-day mortality as the mortality measure when dividing the patients.

Statistics
The two-sample Wilcoxon rank-sum test (Mann-Whitney U-test) was used to test for a difference in the continuous variable's median.The two-proportions Z-test (Chisquared test) was used to test for a difference in proportions.p-Values of less than 0.05 were considered significant.Backward stepwise multivariate logistic regression was performed with 30-day mortality as the outcome and the parameters with the highest significance from the univariate logistic regression as factors.This method begins with a complete model including several variables and then removes one variable after another to create a reduced model with the most significant independent variables.The area under the receiver operating characteristic (ROC) curve (AUC) was used to assess the predictive power of our models.Differences in AUC were tested with a method by DeLong et al. 18

Ethics
This study was approved by a Regional Ethical Review Board and conformed to the Declaration of Helsinki.

Participants
A total of 7556 ICU admissions were identified, where 2223 admissions fulfilled the inclusion criteria, 222 admissions were excluded, and 17 patients were lost to follow-up.Therefore, the total number of patients included in the study was 1984.

Descriptive statistics
The 30-day mortality was 25.3%.Survivors were younger than non-survivors (p < 0.001).Septic shock was more common in non-survivors than survivors (p < 0.001).Survivors were likelier to have a higher body mass index (BMI) than non-survivors (p = 0.028).No difference between the sexes regarding 30-day mortality could be found.The median SAPS-3 score at ICU admission was 63 for survivors and 74 for non-survivors.The median SOFA score at ICU admission was seven for survivors and nine for non-survivors.See Table 1.
Our analyses found almost all documented comorbidities to be more frequent in non-survivors than survivors, except for asthma or chronic obstructive pulmonary disease (COPD), which was more common among survivors than non-survivors.Only 14.9% of patients in the non-survivor group had a diagnosis of asthma or COPD, compared to 20.5% of patients in the survivor group (p = 0.0075).The prevalence of documented comorbidities did not differ between survivors and non-survivors regarding diabetes mellitus (type I and II), ischaemic heart disease, atrial fibrillation and renal disease.See Table 1.
Non-survivors had lower systolic blood pressure, higher respiratory rate, lower Glasgow Coma Scale (GCS) score, lower peripheral oxygen saturation (SpO 2 ) and lower body temperature than survivors.A higher percentage of non-survivors than survivors needed vasopressor support (44.2% and 25.3%, respectively).There was no difference in heart rate between the two groups (p = 0.32).Non-survivors had higher C-reactive protein (CRP) levels, lower platelet counts, higher lactate levels, higher creatinine levels and higher bilirubin levels.No difference was observed between the two groups regarding white blood cell count (WBC).See Table 1.

Multivariate logistic regression
Out of the 36 parameters in Table 1, 24 were significant in univariate logistic regression analyses (see Supplemental Material).Subsequently, these 24 were used in a backward stepwise multivariate logistic regression analysis.Several simpler models were created using backward stepwise multivariate logistic regression, each including four to six variables, and the simple model with the highest AUC was chosen.A six-parameter model was found with odds ratios (OR) of more than one for age, immunosuppression, CRP and bilirubin and OR of less than one for GCS score and body temperature.See Figure 1.SAPS-3, on ICU admission, had an AUC of 0.75 (95% CI 0.72-0.77).Our six-parameter model had an AUC of 0.72 (95% CI 0.69-0.75)with no difference in predictive strength compared to SAPS-3 (p = 0.071).SOFA score upon ICU admission had an AUC of 0.67 (95% CI 0.64-0.70).Both SAPS-3 and our model had a better predictive value than the SOFA score (p < 0.001 and p = 0.0019, respectively).See Figure 2.
Internal validation of the six-parameter model was undertaken through five-fold cross-validation with 50 repeats, which yielded an AUC of 0.71 (95% CI 0.66-0.76).

Discussion
SAPS-3 is a well-known and widely used model in ICUs worldwide.[10][11] This is further supported by a recent study in a large general ICU population, which showed an AUC of 0.85 for SAPS-3. 19However, SAPS-3 is not explicitly designed for sepsis patients, and the discriminative power of SAPS-3 was notably lower in our analyses in a sepsis population, with an AUC of 0.75.Thus, our findings suggest that the performance of SAPS-3 in a sepsis population is lower than in a general ICU population, which is important since sepsis is common in the ICU. 1,2SAPS-3 also requires extensive clinical information, which limits its use.Another potential mortality prediction model for sepsis patients in the ICU is the SOFA score.The SOFA score has previously been shown to predict hospital mortality in ICU patients adequately. 20Like our model, it also consists of six parameters and is thus simpler than SAPS-3.Both the SOFA score and our model include the GCS score and bilirubin levels.However, the SOFA score did not perform well in this study compared to our model and SAPS-3.The SOFA score is not mainly intended for mortality prediction but rather to estimate and monitor the degree of organ dysfunction.One could argue that the SOFA score complements mortality prediction tools rather than replaces them. 15e found a simple six-parameter model that was as good as SAPS-3 in the Sepsis-3 population in our large cohort.This model suggests a higher risk of death within 30 days of ICU admission with advanced age, immunosuppression, low GCS score, low body temperature, increased CRP level and increased bilirubin level.It combines measures of systemic inflammation (CRP), immune dysregulation (normo-or hypothermia during sepsis) and  organ dysfunction (GCS score and bilirubin).Finding age and immunosuppression to be important prognostic factors was no surprise.This model is more straightforward and takes less time to document than SAPS-3, which contains 20 parameters.The parameters needed to predict 30-day mortality with this model are usually recorded at admission and daily on patients with sepsis in the ICU.Only immunosuppression is needed from the medical history.It includes only two laboratory values and does not require blood gas analysis.
Additionally, fewer parameters also minimise the risk of error.Since sepsis accounts for around a third of all ICU admissions, a simple predictive model for mortality adapted for patients with sepsis is relevant.Such a model would benefit low-and middle-income countries (LMIC) ICUs.Common challenges for ICU prognostic models in LMICs are a lack of laboratory testing resources and a lack of staff training and availability, which results in missing or inaccurate parameter values. 21A model containing only six parameters would facilitate quick and correct documentation.Both CRP and bilirubin can be analysed relatively rapidly and cheaply.Age, immunosuppression, GCS score and body temperature can all be obtained bedside quickly.
3][24] As with SAPS-3 and SOFA, our model also includes bilirubin.Previous studies have found that elevated bilirubin levels at ICU admission are an early independent predictor of mortality in patients with sepsis. 25,26Interestingly, our analyses suggest that CRP and lactate have important prognostic values in sepsis.Both CRP and lactate are variables that are routinely analysed but not included in any standard mortality prediction model in the ICU.We found elevated CRP levels to be prognostic and thus included it in our simplified model.In a separate publication, our research group previously explored the finding that CRP is a strong independent prognostic marker for 30-day mortality. 27levated lactate levels were strongly correlated with 30-day mortality in this study's univariate logistic regression results, and similar results can be seen in previous studies. 28,29Though, lactate levels did not add enough independent predictive power to be included in our simple model.Lactate levels have been reported to be a good predictor of mortality in sepsis and have good discriminative power, with an AUC of 0.66. 30Although lactate was not included in the simple prognostic model of this study, further research on the predictive value of lactate levels is of interest.
In contrast, our analyses showed no significant difference in WBC between survivors and non-survivors.WBC is included in SAPS-3, and although it is clinically used to follow the course of an infection, the prognostic value of WBC in sepsis is poorly studied.One prior study has indicated that WBC is not of predictive value in sepsis patients. 31Likewise, while tachycardia is included in SAPS-3, heart rate was not shown to correlate with 30-day mortality in our analyses.Although the correlation between heart rate and mortality in sepsis is poorly studied, some studies have suggested that tachycardia and relative tachycardia (the ratio between heart rate and body temperature) are associated with increased mortality in sepsis. 32,33ince we did not have access to data on baseline SOFA scores and the Sepsis-3 criteria include an acute SOFA score increase of two points or more, we assumed all admitted patients to have a baseline SOFA score of 0. This means that patients with chronic organ failure and a baseline SOFA score of 1 or more may have been falsely categorised as having sepsis due to the study design.The relatively high median SOFA score in our results implies that most patients would likely have had an acute SOFA score increase of at least two points anyway.
Our results noted a higher prevalence of asthma/COPD in survivors than non-survivors.Exacerbations of COPD may be clinically challenging to distinguish from sepsis.Furthermore, COPD patients also tend to obtain antibiotic treatment more liberally than others, and antibiotic treatment 24 h prior, to 72 h after ICU admission was one of the inclusion criteria.Thus, it is possible that some patients with COPD were falsely included as septic patients in this study.Since COPD patients admitted to the ICU because of respiratory failure have relatively low mortality, this may explain an overrepresentation of asthma/COPD in survivors. 34Another minor limitation in this study was the lack of documentation of diabetes type.According to earlier publications, patients with type 2 diabetes have increased mortality in sepsis. 35,36Our study showed no such increase, perhaps because the two types of diabetes were not distinguished in the documentation process.
This study was conducted at multiple centres and included a relatively large patient sample, strengthening our results.Furthermore, the two databases used for data collection were not explicitly designed for this study but had multiple purposes, making them less susceptible to confirmation bias.Several data collectors documented data using one standardised and clearly defined guideline, which reduced the risk of systematic errors.However, it is worth noting that the AUC of our simple six-parameter model, like the AUC for SAPS-3 in this study, indicates only a moderately performing model.Our model needs external validation in other Sepsis-3 cohorts in the ICU before its utility can be confirmed.
Furthermore, this six-parameter model can only be used on a group level, not to estimate individual patients' mortality risk.
Further research on the calibration of this model is needed to make the mortality prediction of individual patients possible, which is necessary to make this model an equal alternative to SAPS-3.Simple and accurate riskadjustment models are important for easy but accurate quality control of sepsis care and the design of simple yet robust studies on sepsis in intensive care.Our suggested six-parameter model is undoubtedly a good candidate for such a model.

Conclusion
Although sepsis is very common in the ICU, SAPS-3 has a lower prognostic value in ICU admissions fulfilling the Sepsis-3 criteria than in the general ICU population.SOFA score upon ICU admission has poorer performance than SAPS-3 but was not designed with mortality prediction in mind.A simpler six-parameter method using age, immunosuppression, GCS score, body temperature, CRP and bilirubin on admission predicts 30-day mortality upon ICU admission for sepsis patients just as well as SAPS-3.These findings could allow the design of studies with access to less data without compromising quality and facilitate performance monitoring.

Figure 1 .
Figure 1.Multivariate logistic regression of a simple sixparameter model consisting of age, immunosuppression, Glasgow Coma Scale, body temperature, C-reactive protein and bilirubin in predicting 30-day mortality in sepsis patients admitted to the ICU.ICU: intensive care unit; GCS: Glasgow Coma Scale; CRP: C-reactive protein.

Figure 2 .
Figure 2. ROC curves and AUC for SAPS-3, SOFA score, and a simple six-parameter model consisting of age, immunosuppression, Glasgow Coma Scale, body temperature, C-reactive protein and bilirubin in predicting 30-day mortality in sepsis patients admitted to the ICU.ROC: receiver operating characteristic; AUC: area under curve; SAPS: Simplified Acute Physiology Score; SOFA: Sequential Organ Failure Assessment; ICU: intensive care unit.

Table 1 .
Characteristics of the studied patient population.Values presented as medians and interquartile range or percentages, with the exception of GCS, which is presented as means and standard deviations (SD).*Calculated using the two-samples Wilcoxon rank-sum test (Mann-Whitney U-test) and the two-proportions Z-test (Chi-squared test) as appropriate.