Four Assessment Tools for Predicting Mortality and Adverse Events in Surgical Patients With Sepsis and Septic Shock: A Comparative Study

ABSTRACT Background The mortality rate for sepsis and septic shock in surgical patients is approximately 36%, which is higher than that of other medical patients. Predisposition, infection/injury, response, and organ dysfunction (PIRO) is currently the most widely used tool for assessing patients with surgical sepsis. However, it is not a standardized assessment tool for surgical patients in general. Purpose The purposes of this study were to (a) create a modified PIRO (mPIRO) that adds a count of platelets and does not include a body temperature reading; (b) test the sensitivity and specificity of the mPIRO for predicting mortality and adverse events among patients with surgical sepsis; and (c) compare the predictive accuracy of the mPIRO, sequential organ failure assessment (SOFA), quick SOFA, and PIRO tools. Methods A retrospective observational cohort study was conducted. Two thousand fifty-five patient medical records were reviewed, with 103 identified as meeting the inclusion criteria. Results Compared with the other tools, mPIRO ≥ 4 achieved better sensitivity (90.5%) in predicting mortality and high sensitivity (72%) and specificity (80%) in predicting adverse events. mPIRO was the most accurate predictor of mortality (area under the receiver operating characteristic curve [AUC] = 0.83) among the tools considered. SOFA and mPIRO were the first and second most accurate predictor of adverse events, respectively, with respective AUC values of 0.86 and 0.82. Conclusions/Implications for Practice mPIRO, which employs an easy-to-use scoring system, is a valid assessment tool with good sensitivity and AUC for predicting both mortality and adverse events in patients with surgical sepsis. We recommend using mPIRO ≥ 3 as an indicator of potential adverse events.


Introduction
Sepsis with septic shock is the major cause of death in hospitals and a major reason for admission to the intensive care unit (ICU; Evans et al., 2021).According to the World Health Organization, the global incidence of sepsis is 189 per 10,000 patients, with approximately 20% of patients with sepsis dying and more than half of patients with sepsis admitted directly to the ICU (World Health Organization, 2020).In the United States, medical expenses related to sepsis management total approximately 16-25 billion USD annually (Rodríguez et al., 2021).As patients with sepsis often encounter issues related to aging and chronic diseases, their medical expenses and risk of mortality increase dramatically with length of stay (Fathi et al., 2019).Furthermore, mortality and ICU stay ≥ 3 days are both recognized as ICU adverse events (Evans et al., 2021) and may be used as clinical criteria for identifying patients with sepsis.The early identification and evaluation of risks of early interventions in patients with sepsis have become important.
Sepsis is characterized by highly variable and nonspecific signs and symptoms, making its diagnosis and evaluation of severity very complicated (Villegas & Moore, 2018).The mortality rate for sepsis and septic shock in surgical patients is approximately 36% (Vincent et al., 2019), which is higher than the 12%-18% rate observed in other medical patients (Fernando et al., 2018).Different from other patients, monocyte dysfunction, which puts patients at a high risk of death and secondary infections, has been found to be present in patients with sepsis during the first 3-5 days after surgery (Baëhl et al., 2016).Intraoperative blood loss volume also affects T-lymphocyte proliferation (Albertsmeier et al., 2015).Moreover, degree of T-lymphocyte proliferation severity is positively associated with the complexity of the surgical procedure performed, as patients undergoing surgery cannot effectively defend themselves against bacteria and endotoxins and may develop organ failure or death (Albertsmeier et al., 2015(Albertsmeier et al., , 2017)).Furthermore, broadspectrum antibiotics are commonly prescribed to treat or prevent infections in surgical patients, which further complicates the identification, assessment, and monitoring of septic-related changes (Martínez et al., 2020).Sepsis assessment tools specific to surgical patients have been rarely reported in the literature (Posadas-Calleja et al., 2018).However, early identification and monitoring of septic condition severity in surgical patients are imperative to initiate timely treatment and reduce mortality (Kim & Park, 2019).

Assessment Tools
Over the past two decades, several assessment tools for sepsis have been developed that quantify sepsis severity and predict mortality risk and prognosis as an aid in allocating medical resources (Evans et al., 2021).However, these tools are lengthy and difficult to apply, making them difficult to use regularly and repeatedly in clinical practice.The most commonly used tools with reasonable numbers of assessment items include the sequential organ failure assessment (SOFA); quick SOFA (qSOFA); and predisposition, infection/injury, response, and organ dysfunction (PIRO).
SOFA, developed by the European Society of Intensive Care Medicine, is a widely used tool consisting of six systemic variables: ratio of partial pressure of oxygen to fraction of inspiration O 2 (PaO 2 /FiO 2 ), mean arterial pressure, administration of vasopressors, bilirubin, platelets, creatinine, and Glasgow Coma Scale (GCS; Vincent et al., 1996).Higher scores indicate a higher risk of mortality.However, this tool does not consider host factors such as age and comorbidities, which are also known to significantly affect mortality in patients with sepsis (Evans et al., 2021).
qSOFA was developed in 2016 using only three criteria: GCS, blood pressure, and respiration rate.Although qSOFA does not require laboratory data and may be used quickly and repeatedly (Finkelsztein et al., 2017), its sensitivity in predicting mortality has been found to be lower than that of the other assessment tools (Finkelsztein et al., 2017).In addition, the feasibility of using this tool with patients undergoing surgery with multiple organ failure has yet to be confirmed.PIRO, developed by Levy et al. (2003), consists of four dimensions.Because of its theoretical nature, criteria weightings may differ for different patient groups.Thus, it is not an assessment tool standardized for clinical practice.PIRO has been applied to patients undergoing surgery with intra-abdominal sepsis.Posadas-Calleja et al. (2018) found that eight factors, including age, comorbid condition, white blood count, body temperature (BT), systolic blood pressure, GCS, creatinine, and PaO 2 /FiO 2 , were significantly related to mortality.PIRO has been shown to be a good predictor of mortality and severity in surgical patients, with results suggesting this tool may be applicable to other surgical intensive patients.
Several recent studies have shown that, in the host defense mechanism, BT is not a good predictor of mortality in infectious patients (Schuttevaer et al., 2019), as BT in patients with infections is often within the normal range.Thus, normal BT may lead to the mis-assessment of disease severity, delaying the initiation of antibiotics and increasing mortality risk (Inghammar & Sunden-Cullberg, 2020).According to O' Grady et al. (2008), patients with infections, normal BT, and positive blood culture tend to experience higher levels of disease severity.The cutoff BT is 36°C in current assessment tools.Although hypothermia is often diagnosed in the late phase of sepsis (Ostadi et al., 2019), it is not a valid indicator for the early detection of sepsis.Furthermore, it is difficult to determine the actual influence of antipyretics and antibiotics on BT (Schuttevaer et al., 2022).Hence, using BT as a predictor of mortality in patients with infections should be done cautiously.
SOFA and Acute Physiology and Chronic Health Evaluation II both use thrombocytopenia as an important parameter for predicting mortality in critical patients.Thrombocytopenia is often defined as a platelet count of < 150 Â 10 3 /μl (Claushuis et al., 2016).The incidence of thrombocytopenia in patients with sepsis is approximately 70% and is even higher in patients requiring ICU treatment (Giustozzi et al., 2021).Platelets are inflammatory and coagulation elements that function in hemostasis and immune mechanisms (Giustozzi et al., 2021).The large quantities of inflammatory and bioactive molecules released by platelets help control and modulate the immune cells (Ribeiro et al., 2019).During sepsis, platelets inhibit inflammation and promote tissue repair in a receptor-and organ-dependent manner (Ribeiro et al., 2019).Moreover, patients with thrombocytopenia have a higher risk of surgical bleeding and organ dysfunction (Assinger et al., 2019), which leads to higher risks of hospital mortality and increased length of stay (Lyons et al., 2018).Detecting thrombocytopenia in a timely manner and initiating treatment early can effectively improve the rate of survival in patients with sepsis (Giustozzi et al., 2021).
In light of the above, this study was designed to create a modified Posadas-Calleja PIRO (mPIRO) that (a) includes a count of platelets and (b) eliminates consideration of BT.Furthermore, because adverse events have rarely been assessed in previous studies and are now considered a key outcome in patients with sepsis (Evans et al., 2021), the predictive accuracy of mPIRO in terms of mortality and adverse events in a sample of patients with surgical sepsis was assessed and compared against the results obtained by SOFA, qSOFA, and PIRO.

Study Design and Setting
This retrospective, observational study was conducted using a review of the electronic medical records of a surgical ICU (SICU) in southern Taiwan from January 2016 to December 2017.The inclusion criteria were as follows: age > 20 years, severe sepsis or septic shock diagnosis, surgery, and SICU admission.Patients who had been diagnosed with secondary infection or readmitted to the SICU were excluded.Patients who had stayed in the SICU for < 8 hours were also excluded because detecting early changes in either their physiological response or organ dysfunction would be difficult.Patients with altered consciousness before admission were excluded because it would be difficult to identify the impact of sepsis.Of the 2,055 medical records reviewed, 103 met the inclusion criteria.The study design flow is shown in Figure 1.

Measurements
A three-part checklist for data collection was created, including demographic variables, assessment tools, and outcome indicators.Demographic data included age, gender, diagnosis at ICU admission, and source of infection.The parameters of SOFA, qSOFA, PIRO, and mPIRO and their respective scoring methods are depicted in Table 1.Outcome indicators included ICU length of stay, mortality, and adverse events.Adverse events were defined as ICU death, a stay greater than 3 days, or both and were identified by researchers based on length-of-stay and mortality data.

Data Collection Procedures
The protocol for this study was approved by the institutional review board of the affiliated institution (KMUHIRB-E[I]-20190020). Electronic medical records were retrieved from the electronic database and hospital information system in the medical center.All of the parameters with extreme values within the first 8 hours of ICU admission were collected and recorded in an Excel file.

Statistical Analysis
Baseline characteristic data were analyzed using descriptive statistics.We used the area under the receiver operating characteristic curve (AUC) to determine the discrimination among the scores.Discrimination is the ability of a score to distinguish between survivors and nonsurvivors.Pairwise comparisons between the AUC of each score in the one-tailed t test were performed based on the method described by Hanley and McNeil (1983).The optimal cutoff point was decided for each assessment tool based on Youden's index.The values of sensitivity, specificity, positive likelihood ratio (LR+), and negative likelihood ratio (LR−) were also calculated based on the optimal cutoff point in predicting mortality and adverse events in patients with surgical sepsis.IBM SPSS Statistics 22.0 (IBM Inc., Armonk, NY, USA) was used for all statistical analyses, and significance was set at p < .05.

Results
The characteristics and outcomes are presented in Table 2. Most were men (64.1%), and the mean age was 66.56 (SD = 12.87) years.Forty-eight (46.6%) patients were diagnosed with sepsis, and 55 (53.4%) had septic shock.The most frequent infection site was the gastrointestinal tract (47.6%), followed by the skin or soft tissue (22.3%).No significant differences were found between survivors and nonsurvivors in terms of age ( p = .333),gender ( p = .431),diagnosis ( p = .063),or infection source ( p = .439).The means and standard deviations of the scores for SOFA, qSOFA, PIRO, and mPIRO were 6.7 (4.57), 1.7 (1.03), 3.21 (1.61), and 3.26 (1.74), respectively.The scores of the four assessment tools for predicting mortality all differed significantly for survivors and nonsurvivors ( p < .001).The overall mortality rate in the ICU was 20.4%, and the rate of adverse event occurrence (mortality or length of stay in the ICU ≥ 3 days) was 85.4%.
In this study, the optimal cutoff points for predicting mortality in mPIRO, SOFA, qSOFA, and PIRO were 4, 6, 3, and 4, respectively.The optimal cutoff points for predicting adverse events in mPIRO, SOFA, qSOFA, and PIRO were 3, 4, 2, and 4, respectively.On the basis of the optimal cutoff point, the LR+ and LR− were calculated for ruling in and ruling out mortality and adverse events (Table 3).Of these four assessment tools, mPIRO was the most reliable in terms of predictive results.

The Journal of Nursing Research
Yi-Chin CHU et al.

Discussion
In this study, the inclusion of the parameter platelet count into PIRO for the early severity assessment of patients with surgical sepsis was validated.The results showed no significant differences among these assessment tools, indicating mPIRO to be the equal of the other three assessment tools considered.mPIRO's 90.5% sensitivity for mortality indicates this tool is an accurate detector of high-risk patients, supporting its appropriateness for clinical care application in the early detection and continuous monitoring of changes.
Although the specificity of 72% is not high, according to Oduncu et al. (2021), sensitivity is significantly more important than specificity in predicting sepsis mortality.With regard to adverse events, the high sensitivity and specificity values indicate that mPIRO is valid for detecting patients with sepsis at a high/low risk of adverse events, which can enhance the effectiveness of clinical care resource allocation.Among the four tools, mPIRO was shown to be a valid assessment tool, with an AUC > 0.8 for predicting both mortality and adverse events in patients with surgical sepsis.According to Nahm (2022), AUC > 0.8 indicates good discrimination.Furthermore, the consideration of platelet count in mPIRO increased the predictive accuracy for patients with surgical sepsis.In previous studies, the AUC values of SOFA, qSOFA, and PIRO for mortality in patients with sepsis in the ICU were 0.57-0.76(Kądziołka et al., 2019;Songsangjinda & Khwannimit, 2020), 0.6-0.71(Raith et al., 2017;Songsangjinda & Khwannimit, 2020), and 0.64-0.86(Macdonald et al., 2014;Songsangjinda & Khwannimit, 2020), respectively.Of the three, qSOFA had the lowest discrimination ability, which may result from its lack of renal and liver function assessment (Finkelsztein et al., 2017).For patients with sepsis, organ failure is the most critical parameter to be evaluated (Songsangjinda & Khwannimit, 2020).Note.SOFA = sequential organ failure assessment score; qSOFA = quick SOFA; PIRO = predisposition, infection/injury, response, and organ dysfunction; mPIRO = modified PIRO; ≥ 2 infection sources = patients who suspect or show two or above two infection sources; ICU = intensive care unit; LOS = length of stay.
In terms of LR+, qSOFA was better than mPIRO (3.64 vs. 3.23) based on the optimal cutoff point for mortality prediction.However, qSOFA had the poorest LR− and sensitivity, which may be attributable to qSOFA having the fewest evaluation items (Finkelsztein et al., 2017).The LR− of SOFA was better than mPIRO (0.09 vs. 0.13).However, the complex nature of the scoring system in SOFA makes it significantly more difficult to use in repeated patient assessments (Finkelsztein et al., 2017).With regard to adverse event prediction, mPIRO performed well in both LR+ and LR−.Thus, mPIRO is a reliable tool for ruling in and ruling out mortality and adverse events.
The discrimination of mPIRO for mortality was better than that of PIRO.The findings of this study support those of previous studies showing thrombocytopenia to be an independent predictor of mortality and a poor prognostic indicator of clinical outcomes in patients with surgical sepsis (Assinger et al., 2019;He et al., 2022).In patients with sepsis, platelet consumption is commonly seen during bacterial infections (Giustozzi et al., 2021).Thrombocytopenia severity is strongly associated with sepsis severity (He et al., 2022).
The discrimination of mPIRO was better than that of SOFA, indicating the potential importance of patient predisposition factors.In Posadas-Calleja et al.'s (2018) PIRO study, age was identified as an important factor predicting mortality, with a cutoff age of 65 years (Posadas-Calleja et al., 2018).Previous epidemiological studies have reported that approximately 60% of patients with sepsis are > 65 years old (Kotfis et al., 2019;Lineberry & Stein, 2014).Older patients typically have at least one comorbid condition, frequent invasive procedures, insufficient immunity, and prolonged hospitalizations (Fathi et al., 2019).The risk of sepsis-related death increases with age (Kotfis et al., 2019).
The AUC values of SOFA and qSOFA for adverse events in patients with sepsis admitted in the ICU have been reported as approximately 0.74 and 0.61, respectively (Raith et al., 2017).However, these values were higher in this study.In this study, the incidence of adverse events in patients with surgical sepsis was 85.4%, which is higher than reported in a previous study (55.7%;Raith et al., 2017).This difference may relate to the average length of ICU stay.Adverse events were defined in this study as mortality or ICU stay of ≥ 3 days (Singer et al.,

The Journal of Nursing Research
Yi-Chin CHU et al.

2016)
. The average ICU stay length in this study was 12.38 days, which is similar to the average for patients with sepsis in Taiwan (10.96 days according to the Taiwan Health Insurance Academic Database; Chen et al., 2019) and higher than that reported in other sepsis studies (2.7-3.1 days; Raith et al., 2017;Songsangjinda & Khwannimit, 2020).Differences in sepsis duration between countries may be influenced by type of ICU, causes of sepsis, clinical setting differences, and sepsis treatment experience (Chen et al., 2019).Higher average hospital stay lengths are associated with higher adverse event incidences.

Limitations
This study was affected by several limitations.First, all study data were obtained from the laboratory, nursing assessments, and medical records, and the possibility of data entry errors cannot be ruled out.Second, this study focused on patients with sepsis in SICUs only and cannot be generalized to other sepsis groups.Third, as the rates of mortality and adverse events were relatively high in this study, the findings may not be applicable to patients with less severe sepsis.

Conclusions and Implications for Practice
mPIRO was shown to have better mortality discrimination than PIRO, SOFA, or qSOFA.Organ failure, age, comorbidity, white blood count, and platelets were identified as important factors affecting sepsis severity and prognoses.The scoring system used in mPIRO is simpler than that used in SOFA.Thus, mPIRO is a quick and valid tool for predicting mortality and adverse events in patients with sepsis.Early detection of sepsis severity promotes early intervention, the prevention of disease deterioration, and improved clinical prognoses.In terms of clinical implications, nurses should be familiar with the different sepsis-related assessment tools and indicators.mPIRO may be applied at frequent intervals to better capture dynamic changes as well as the risks of mortality and adverse events in patients with surgical sepsis.We suggest mPIRO ≥ 3 be used as a warning sign for adverse events and a trigger for initiating appropriate early interventions to prevent clinical deterioration and improve health prognosis.In addition, mPIRO can help improve nurse workload management, patient transition decision making, and quality of care in the ICU.The predictive utility of mPIRO for mortality and adverse events should be validated on patients with other surgical procedures such as cardiovascular and orthopedic to facilitate its generalization.

Table 1
Comparison of Assessment Tool Scoring Systems

Table 3
Sensitivity and Specificity at Different Thresholds for Predicting Mortality and Adverse Event in Four Assessment Tools for Patients With Surgical Sepsis