A simple nomogram for predicting the mortality of PICU patients with sepsis-associated encephalopathy: a multicenter retrospective study

Background As one of the serious complications of sepsis in children, sepsis-associated encephalopathy (SAE) is associated with significantly poor prognosis and increased mortality. However, predictors of outcomes for pediatric SAE patients have yet to be identified. The aim of this study was to develop nomograms to predict the 14-day and 90-day mortality of children with SAE, providing early warning to take effective measures to improve prognosis and reduce mortality. Methods In this multicenter, retrospective study, we screened 291 patients with SAE admitted to the PICU between January 2017 and September 2022 in Shandong Province. A least absolute shrinkage and selector operation (LASSO) method was used to identify the optimal prognostic factors predicting the outcomes in pediatric patients with SAE. Then, multivariable logistic regression analysis was performed based on these variables, and two nomograms were built for visualization. We used the area under the curve (AUC), calibration curves and decision curves to test the accuracy and discrimination of the nomograms in predicting outcomes. Results There were 129 patients with SAE in the training cohort, and there were 103 and 59 patients in the two independent validation cohorts, respectively. Vasopressor use, procalcitonin (PCT), lactate and pediatric critical illness score (PCIS) were independent predictive factors for 14-day mortality, and vasopressor use, PCT, lactate, PCIS and albumin were independent predictive factors for 90-day mortality. Based on the variables, we generated two nomograms for the early identification of 14-day mortality (AUC 0.853, 95% CI 0.787–0.919, sensitivity 72.4%, specificity 84.5%) and 90-day mortality (AUC 0.857, 95% CI 0.792–0.923, sensitivity 72.3%, specificity 90.6%), respectively. The calibration plots for nomograms showed excellent agreement of mortality probabilities between the observed and predicted values in both training and validation cohorts. Decision curve analyses (DCA) indicated that nomograms conferred high clinical net benefit. Conclusion The nomograms in this study revealed optimal prognostic factors for the mortality of pediatric patients with SAE, and individualized quantitative risk evaluation by the models would be practical for treatment management.


Introduction
Severe sepsis, normally treated in the pediatric intensive care unit (PICU), is a leading threat to the life and health of children worldwide (1).Sepsis-associated encephalopathy (SAE) is defined as a diffuse brain disorder without direct central nervous system (CNS) infection and is one of the most critical complications in children with sepsis (2,3).Previous studies have indicated that in all sepsis patients, the incidence of SAE varies from 8% to more than 70% (4).Importantly, SAE can cause a dramatically poorer prognosis, and the mortality rate of sepsis patients significantly increases with increasing SAE severity to a maximum of 70% (5).Furthermore, surviving patients with SAE are likely to develop permanent neurocognitive impairment, which can significantly reduce the quality of life and bring great burden to their families and even society (5)(6)(7).Based on these findings, early identification and medical intervention for SAE is very important to improve the prognosis and reduce the mortality of patients with sepsis, especially children.Li et al. proposed an accurate discriminative model and a simplified scoring model, which are applicable tools for assessing the severity in pediatric sepsis patients (8).However, the risk factors for predicting the mortality of SAE patients, especially pediatric patients, remain incompletely understood to date.Huang et al. (9) proposed that the hemoglobin-to-red cell distribution width ratio (HRR) was associated with the prognosis and mortality of SAE, but the limitation was that there was no long-term out-of-hospital follow-up.Peng et al. (10) developed machine learning (ML) models to predict the risk of 30-day mortality of SAE, but external validation was lacking.In addition, few high-quality studies have focused on mortality prediction of SAE in children.
The nomogram model is a powerful and practical decisionmaking tool that has been proven to provide highly accurate risk assessment in individual patients (11).In the current study, the main objective was to develop nomograms based on routine variables measured easily within a few hours of PICU admission to help clinicians identify children with SAE at high risk of death.
2 Materials and methods

Study design
This retrospective, multicenter observational study was performed in three hospitals designated for PICU patients with SAE in Shandong Province.Among the three centers, Qilu Hospital of Shandong University with 28 PICU beds was included as the training cohort, while Jinan Children's Hospital of Shandong University with 30 PICU beds and Jining First People's Hospital with 27 PICU beds were included as external validation cohorts.The study was registered at the Chinese Clinical Trial Registry (ChiCTR2100048600) and approved by the institutional ethics review board of Qilu Hospital of Shandong University (KYLL-202202-027-1).Informed consent was not required because of the observational and retrospective nature of the study.

Participants
SAE is defined as sepsis combined with alterations in mental status, cognitive or behavioral abnormalities, and a Glasgow Coma Score (GCS) ≤14 during the patient's stay in the PICU (2,3).Cognitive and neuropsychiatric symptoms of SAE include lack of focus, confusion, agitation, heightened sensitivity, reduced psychomotor function, drowsiness, stupor, and coma.These symptoms were thoroughly recorded in the medical documentation by healthcare professionals.SAE patients were enrolled during the period from January 2017 to September 2022 in the training cohort and from January 2017 to May 2022 in the validation cohorts.We enrolled patients with the following inclusion criteria: (1) age between 28 days and 18 years; (2) a diagnosis of sepsis according to the Sepsis 3.0 definition (12); and (3) altered mental status with cognitive or behavioral abnormalities accompanied by GCS ≤ 14 during the PICU stay.The exclusion criteria were as follows: (1) primary cerebral disease, including brain trauma, cerebrovascular disease, epilepsy, intracranial infection and autoimmune encephalitis; (2) metabolic encephalopathy caused by diabetic ketoacidosis, hypoglycemia, hepatic encephalopathy and inherited metabolic diseases; and (3) missing laboratory data.Most of the SAE patients were diagnosed within 72 h after PICU admission, while others were diagnosed longer after PICU admission.The earliest time for SAE diagnosis was 8 h after PICU admission.

Data collection and study outcomes
Demographic and clinical data were retrieved from the electronic medical record systems of the three hospitals.Two researchers who were totally unaware of the study design were responsible for the data collection.Vital signs, the pediatric critical illness score (PCIS) and laboratory data were recorded during the first 24 h after PICU admission.If there were multiple measurement results within the first 24 h, we would choose the worst one.The PCIS system mainly includes heart rate, systolic blood pressure, respiration, arterial partial oxygen pressure, pH, and creatinine, etc.We collected data on the suspected infection site and microorganisms, the use of vasopressors (adrenaline, norepinephrine, dopamine, dobutamine and other vasopressors) and advanced life support (including mechanical ventilation and renal replacement therapy) during the PICU stay.The primary outcomes include death within 14 days and 90 days after PICU admission.

Statistical analysis
The normal distributions of variables were checked by Shapiro-Wilk tests.Continuous variables with normal distributions were tested by unpaired Student's t test and are presented as the mean ± standard deviation (SD), whereas nonnormally distributed continuous variables were tested by the Mann-Whitney U-test and are presented as the median (interquartile range, IQR).Categorical variables were tested by chi-square analysis or Fisher's exact test accordingly and described as frequencies (proportions).To identify the relative importance of the variables and build prediction models, a least absolute shrinkage and selection operator (LASSO) algorithm was tested in the training cohort.Then, we performed a multivariable logistic analysis based on the LASSO regression model, presenting odds ratios (ORs) and 95% confidence intervals (CIs).The selected variables were used to build the nomogram and online calculator to predict the death risk of SAE patients in the PICU.We used the area under the curve (AUC) to quantitatively evaluate the discriminative performance of the nomogram.Calibration curves were plotted by bootstraps of 1,000 resamples to assess the association between the observed incidence and the predicted probability.The clinical usefulness of the nomogram was determined by decision curve analysis (DCA) in training and validation sets, which quantified the net benefits at various threshold probabilities.Statistical analyses were conducted using R version 3.4.3(The R Foundation for Statistical Computing, Vienna, Austria). 1  Statistical significance was defined as a two-sided p-value of <0.05.

Demographic and clinical characteristics
The selection of the study cohort is illustrated in Figure 1.A total of 291 patients were enrolled in our study, including 129 patients at Qilu Hospital of Shandong University (training cohort), 103 patients at Jinan Children's Hospital of Shandong University (validation cohort 1), and 59 patients at Jining First People's Hospital (validation cohort 2).The baseline and clinical characteristics of the training and validation cohorts are shown in Table 1.The training cohort consisted of 79 (61.24%) males and 50 (38.76%)females with a median age of 51 months (IQR 13-137 months).The 14-day and 90-day mortality rates in the training cohort were 44.96 and 50.39%, respectively.Validation cohort 1 comprised 52 (50.49%) males and 51 (49.51%) 1 https://www.r-project.org1).
The baseline and clinical characteristics of SAE patients in the training cohort are shown in Table 2.There were 65 nonsurvivors with a median age of 74 months (IQR 15-138 months).No significant differences were found in age or sex between survivors and nonsurvivors (p > 0.05).The respiratory rate of nonsurvivors was significantly higher than that of survivors (32 with IQR 25-40 breaths/ min vs. 27 with IQR 22-33.25 breaths/min, p = 0.015).For both nonsurvivors and survivors, respiratory infection occupied the highest proportion among suspected infection foci.Interestingly, blood infection seemed to be more common in the nonsurvivors, but the difference did not reach statistical significance (18.46% vs. 14.06%,p > 0.05).In addition, gram-negative bacteria were the most common microorganisms in the nonsurvivors, while gram-positive bacteria were the most common microorganisms in the survivors.Nonsurvivors tended to have more vasopressor use within 24 h after PICU admission (47.69% vs. 21.88%,p = 0.004) and more mechanical ventilation therapy during the PICU stay (60% vs. 12.5%, p < 0.001).Moreover, nonsurvivors were more likely to experience septic shock, and they had longer hospital and PICU stay times (p < 0.05).Furthermore, the levels of alanine transaminase (ALT), total bilirubin (TBIL), creatine kinase-MB (CK-MB), lactic dehydrogenase (LDH), international normalized ratio (INR), prothrombin time (PT), D-dimer, brain natriuretic peptide (BNP), procalcitonin (PCT) and lactate were significantly higher in nonsurvivors (p < 0.05).The levels of platelets (PLT), hemoglobin (HB), albumin, fibrinogen (Fib) and the pediatric critical illness score (PCIS) were significantly lower in nonsurvivors (p < 0.05).

Construction and validation of the nomogram
We then built nomogram models based on the logistic regression to predict 14-day mortality (Figure 4A) and 90-day mortality (Figure 4B).The performance of the models was identified by receiver operating characteristic (ROC) curves.In the training cohort, the AUC calculated by the bootstrap self-sampling method was 0.853 (95% CI 0.787-0.919)with a sensitivity of 72.4% and specificity of 84.5% for 14-day mortality (Figure 5A) and 0.857 (95% CI 0.792-0.923)with a sensitivity of 72.3% and specificity of 90.6% for 90-day mortality (Figure 5D), indicating high predictive power.Additionally, the calibration curve (Figures 6A,D) and decision curve (Figures 7A,D) revealed that the nomogram exhibited a good predictive ability for 14-day mortality and 90-day mortality, respectively.
The performance of the models was also assessed in validation cohorts.In validation cohort 1, the AUC was 0.835 (95% CI 0.758-0.912)with a sensitivity of 75.6% and specificity of 81.0% for 14-day mortality (Figure 5B) and 0.841 (95% CI 0.761-0.921)with a sensitivity of 80.6% and specificity of 75.6% for 90-day mortality (Figure 5E).In validation cohort 2, the AUC was 0.833 (95% CI 0.726-0.940)for patients with a sensitivity of 63.6% and specificity of 86.5% for 14-day mortality (Figure 5C) and 0.823 (95% CI 0.711-0.935)with  5F).The calibration curves also presented high agreement between prediction and observation in the risk of mortality in the two validation cohorts (Figures 6B,C,E,F).

Clinical application of the nomogram
DCA based on the net benefit and threshold probabilities was performed to test the clinical applicability of the nomogram models.Figure 7 shows that the risk prediction nomograms had a superior net benefit with a wide range of threshold probabilities in both the training cohort and validation cohorts.

Development of the online calculator
Based on the nomogram models, two Internet browser-based online calculators were developed (Figure 8).After the user inputs the requested information of specific items, the probability of 14-day mortality2 or 90-day mortality 3 in patients with SAE can be estimated.

Discussion
Our study revealed several key factors affecting the short-and long-term mortality of SAE, such as lactate, PCT, albumin, PCIS, and the use of vasopressors.These predictors were analyzed and integrated into the most suitable prediction model visualized as a prediction nomogram.Based on the external validation, our model showed relatively high AUC, sensitivity and specificity, indicating good reliability and discrimination.Our study is the first to investigate independent risk factors associated with mortality in SAE patients in the PICU across multiple centers.
Lactate is a critical prognostic indicator of sepsis in children (13,14).A prospective cohort study among children with sepsis found that lactate levels within 4 h after admission were strongly associated with the occurrence of persistent organ dysfunction (15).We found that there was increased mortality within 90 days after PICU admission in SAE patients with higher lactate levels.Mechanistically, elevated serum lactate levels indicate microcirculatory dysfunction in patients, which results in tissue ischemia and hypoxia (16).Insufficient oxygen supply and inadequate nutrient delivery to the affected tissues may have severe consequences, particularly if vital organs such as the CNS are involved.Furthermore, prolonged insufficiency of oxygen supply leads to irreversible neuronal cell death and even permanent neurological disability, ultimately contributing to the occurrence of SAE (17).Overall, the measurement of lactate serves as a critical tool in assessing disease severity and predicting outcomes in pediatric sepsis and SAE patients.
As a calcitonin propeptidyl, serum PCT is elevated during the early stage of inflammation and infection and is thus always used as an early diagnostic test for infection (18).In fact, sepsis triggers a significant surge of proinflammatory mediators and leads to the disruption of anti-inflammatory defense in the body.The presence of these inflammatory factors, such as cytokines and chemokines, may disrupt the delicate balance of the body's immunity mechanism, thereby exacerbating inflammation and causing subsequent tissue damage.Meanwhile, inflammatory mediators also have a direct impact on cerebral function by modulating cerebral blood flow and oxygen delivery, which constitutes the underlying mechanism of SAE (19,20).Moreover, increasing evidence indicates that the surge of inflammatory factors during sepsis can compromise the integrity of the blood-brain barrier (BBB), leading to brain edema and brain injury, which is also the potential pathogenesis of SAE (19).In our study, PCT showed good predictive value for the mortality of SAE patients and was included in the nomogram model.Therefore, for patients with high levels of PCT, physicians should provide timely Feature selection using the least absolute shrinkage and selection operator (LASSO) regression in the training cohort.LASSO coefficient profiles of the 35 features for predicting 14-day mortality (A) and 90-day mortality (C).Identification of the optimal penalization coefficient (λ) in the LASSO model was performed via 10-fold cross-validation for predicting 14-day mortality (B) and 90-day mortality (D).
Forest plots showing the results of multivariate analyses in the 14-day mortality (A) and 90-day mortality (B) groups.PCT, procalcitonin; PCIS, pediatric critical illness score; OR, odds ratio; CI, confidence interval.The receiver operating characteristic (ROC) curves of the nomograms in the training cohort, validation cohort 1, and validation cohort 2 for predicting 14-day mortality (A-C) and 90-day mortality (D-F).
anti-infection and anti-inflammation therapy to reduce the mortality of SAE.
Our study demonstrated that the use of vasopressors was significantly higher in the nonsurvivors and was an independent risk factor for SAE death.For sepsis, a cascade of macrocirculatory and microcirculatory alterations may result in impaired vasoconstriction regulation, ultimately leading to severe hypotension (21).When sepsis-induced hypotension is refractory, the prognosis of septic shock is exceedingly poor.Impaired perfusion leads to dysfunction or failure of multiple vital organs, including the kidney, lung, liver, brain and heart (22), which may be associated with the development of SAE.The primary cause of death is typically attributed to resistant hypotension Construction of nomograms predicting 14-day mortality (A) and 90-day mortality (B) in the training cohort.The points of all features are added to obtain the total point, and a vertical line is drawn on the total point to obtain the corresponding 'predicted value', which indicates the risk of outcomes.PCT, procalcitonin; PCIS, pediatric critical illness score.
Frontiers in Neurology 09 frontiersin.orgthat fails to respond to volume resuscitation or to a high dose of vasopressors (23,24).Moreover, there has been a shift in the preferred drug choice for fluid-refractory SAE patients from dopamine to either epinephrine or norepinephrine (25).Indeed, two randomized controlled trials comparing epinephrine and dopamine for SAE patients demonstrated that epinephrine yielded a better outcome (25).
In conclusion, by identifying this independent risk factor, it may be possible to make more informed decisions about medication management.At present, the commonly used pediatric critical scores include the PCIS, Pediatric Logistic Organ Dysfunction Score 2 (PELOD-2), Pediatric Multiple Organ Dysfunction Score (P-MODS) and Pediatric Early Warning Score (PEWS), among which the PCIS is extensively employed in China to assess the severity and prognosis of critically ill children (26).The comprehensive evaluation indicators included in the PCIS system are all objective variables obtained from laboratory examinations and physical examinations.The first PCIS after PICU admission can accurately reflect the severity of children's condition and has certain value in judging the prognosis of diseases.Fang et al. (27) proved that the PCIS was negatively correlated with the number of combined systemic injuries for patients with severe Mycoplasma pneumoniae pneumonia, suggesting that a lower PCIS implied a higher possibility of multiple organ dysfunction.Our study confirmed that the PCISs were significantly decreased in nonsurviving patients compared with the surviving group, further supporting the accuracy of the PCIS as an independent predictor for 14-day and 90-day mortality in SAE patients.
Our research found that albumin occupies a substantial weight in the nomogram, indicating that it is also an important variable, particularly for predicting 90-day mortality in children with SAE.Previous studies have suggested that there is a correlation between serum albumin and the prognosis of patients with sepsis (28), but few studies have found a correlation between albumin and SAE.Higher albumin levels have been reported to be associated with a lower risk of delirium in patients with sepsis (29,30), which supports our conclusion.Recently, many studies have demonstrated that in addition to being an indicator of malnutrition, there also exists a robust association between albumin and severe infection (31, 32).Low levels of albumin are prevalent among patients with severe sepsis and are associated with more severe inflammation (33-35).Reduced serum albumin levels predispose patients to brain edema, thereby leading to impaired consciousness and SAE (29,30).Overall, albumin serves not only as an important prognostic marker but also as a potential therapeutic target for improving the outcome of critically severe sepsis and SAE patients.Appropriate patient evaluation followed by early initiation of enteral nutrition may be imperative for mitigating the risk of infection and organ dysfunction, thereby potentially reducing the incidence of SAE (30).
Recently, an increasing amount of evidence has suggested that PLT plays a crucial role not only in hemostasis and thrombosis but also in other various physiological processes, such as microbe-induced host defense, inflammatory processes, wound healing and tissue remodeling (36)(37)(38).Our study demonstrated a significant difference in PLT values between the deceased and surviving groups.The The calibration curves of the nomograms in the training cohort, validation cohort 1, and validation cohort 2 for predicting 14-day mortality (A-C) and 90-day mortality (D-F).Two Internet browser-based online calculators for nomograms predicting 14-day mortality (website: https://johny2020.shinyapps.io/Dyn_14_mortality/) and 90-day mortality (website: https://johny2020.shinyapps.io/Dyn_90_mortality/).
decreased PLT count observed in nonsurvivors may be related to the depletion of clotting factors and platelets during sepsis (39).Once platelets and coagulation factors are depleted, the formation of plasmin is increased, and then disseminated intravascular coagulation (DIC), a life-threatening syndrome, occurs.Therefore, early identification of thrombocytopenia and optimal management of these patients are essential keys to improving prognosis.There are some limitations to this study that must be acknowledged.First, this is a retrospective analysis of the recorded data, and prospective studies are needed to identify the Frontiers in Neurology 11 frontiersin.orgvalue of the nomogram in predicting mortality of SAE.Retrospective analyses rely on the existing data and may not provide a comprehensive understanding of all factors influencing the outcomes.Additionally, as this study was limited to PICU patients, the generalizability to other populations may be constrained.
Although the developed models have already been validated in two hospitals, external validation across diverse regions is still necessary.Thus, further research involving diverse populations from various health care settings would enhance the applicability and validity of the findings.

Conclusion
A nomogram was established for predicting the 14-day and 90-day mortality of SAE patients admitted to the PICU, which served as a valuable tool in the medical field.After multicenter external validation, the nomogram showed good reliability and discrimination.It will be particularly beneficial in preventing SAE deterioration and ultimately improving the prognosis of children with SAE.

FIGURE 1 Flowchart
FIGURE 1Flowchart of study participants in the training and validation groups.SAE, sepsis-associated encephalopathy.

FIGURE 7
FIGURE 7Decision curve analysis (DCA) of the nomograms.DCA compares the net benefits of three scenarios in predicting the risk of outcome: a perfect prediction model (gray line), screen none (horizontal solid black line), and screen based on the nomograms (ride line).The DCA curves were depicted in the training cohort, validation cohort 1, and validation cohort 2 for predicting 14-day mortality (A-C) and 90-day mortality (D-F), respectively.

TABLE 1
Baseline and clinical characteristics of the training and validation cohorts.

TABLE 2
Baseline and clinical characteristics of the survivors and nonsurvivors in the training cohort.