Prediction model of emergency mortality risk in patients with acute upper gastrointestinal bleeding: a retrospective study

Background Upper gastrointestinal bleeding is a common presentation in emergency departments and carries significant morbidity worldwide. It is paramount that treating physicians have access to tools that can effectively evaluate the patient risk, allowing quick and effective treatments to ultimately improve their prognosis. This study aims to establish a mortality risk assessment model for patients with acute upper gastrointestinal bleeding at an emergency department. Methods A total of 991 patients presenting with acute upper gastrointestinal bleeding between July 2016 and June 2019 were enrolled in this retrospective single-center cohort study. Patient demographics, parameters assessed at admission, laboratory test, and clinical interventions were extracted. We used the least absolute shrinkage and selection operator regression to identify predictors for establishing a nomogram for death in the emergency department or within 24 h after leaving the emergency department and a corresponding nomogram. The area under the curve of the model was calculated. A bootstrap resampling method was used to internal validation, and decision curve analysis was applied for evaluate the clinical utility of the model. We also compared our predictive model with other prognostic models, such as AIMS65, Glasgow-Blatchford bleeding score, modified Glasgow-Blatchford bleeding score, and Pre-Endoscopic Rockall Score. Results Among 991 patients, 41 (4.14%) died in the emergency department or within 24 h after leaving the emergency department. Five non-zero coefficient variables (transfusion of plasma, D-dimer, albumin, potassium, age) were filtered by the least absolute shrinkage and selection operator regression analysis and used to establish a predictive model. The area under the curve for the model was 0.847 (95% confidence interval [0.794–0.900]), which is higher than that of previous models for mortality of patients with acute upper gastrointestinal bleeding. The decision curve analysis indicated the clinical usefulness of the model. Conclusions The nomogram based on transfusion of plasma, D-dimer, albumin, potassium, and age effectively assessed the prognosis of patients with acute upper gastrointestinal bleeding presenting at the emergency department.


INTRODUCTION
Acute upper gastrointestinal bleeding (AUGIB) is a common presentation at emergency departments (EDs), with hematemesis, melena, and syncope representing as the major symptoms. Although the survival rate of patients with AUGIB has improved with advances in drug therapy and endoscopic diagnosis and treatment, it remains a potentially life-threatening gastrointestinal emergency (Balaban et al., 2014). It is paramount that treating physicians have access to tools that can effectively evaluate the patient risk, allowing quick and effective treatments to ultimately improve their prognosis (Balaban et al., 2014;Barkun et al., 2019).
Several previous models have been developed to estimate the mortality risk of patients with UGIB, such as the Glasgow-Blatchford score (GBS), Rockall score (RS), and the AIMS65 score (Rockall et al., 1996;Blatchford, Murray & Blatchford, 2000;Saltzman et al., 2011;Gaduputi et al., 2014;Abougergi et al., 2016). However, existing predictive models are not commonly used in clinical care for various reasons. First, some of these models require endoscopic information that is not readily available at the time of presentation. However, given the availability of endoscopy, the use of either GBS, modified GBS (mGBS), Pre-Endoscopic RS (PERS), or AIMS65 score is currently recommended, as these models are based on data that are easy to acquire. For example, GBS includes the measurement of systolic blood pressure (SBP), hemoglobin (HB), blood urea nitrogen (BUN), albumin, and international normalized ratio (INR), pulse, as well as evaluation of the presence of symptoms such as melena, syncope and liver disease, and heart failure, while PERS considers age, state of shock, and concomitant disease. Second, recent studies have found inconsistent results when using these models (Yaka et al., 2015;Budimir et al., 2016;Martinez-Cara et al., 2016;Park et al., 2016;Stanley et al., 2017;Zhou et al., 2017;Tang et al., 2018;Kim, Choi & Shin, 2019;Robertson et al., 2020). Third, it is difficult to obtain a personalized mortality assessment from these systems. More recently, Zhou et al. developed a prognostic nomogram for cirrhosis patients with UGIB admitted to intensive care units that they named upper gastrointestinal bleeding-chronic liver failure-sequential organ failure assessment (UGIB-CLIF-SOFA) (Zhou et al., 2017). However, as cirrhosis accounts only for a subset of patients with UGIB, the use of this model is limited. Moreover, the type of clinical intervention administered is an important factor affecting patients' prognosis. To the best of our knowledge, a prognostic model that includes patient characteristics and clinical interventions as risk factors affecting prognosis remains to be developed. In addition, no current prognostic model can predict individual level mortality probability for patients with AUGIB presenting at EDs.
Various factors influence the prognosis of patients with AUGIB, including primary disease, volume and site of bleeding, age and intervention administered at the time of admission (Lahiff et al., 2012;Ko et al., 2017;Barkun et al., 2019). Data on bleeding-related factors, such as details of the site, cause, or type of bleeding, are difficult to obtain for patients treated in emergency care. The extent of bleeding should be ascertained on the basis of the patient's chief complaint, vital signs, and laboratory test results. The aim of this study was to develop a model to predict the mortality of individual patients presenting at EDs, based on routinely collected, objective parameters, and clinical interventions administered. The model is accompanied by a nomogram.

Research design
This was a retrospective cohort study of consecutive patients with AUGIB treated at an ED between July 2016 and June 2019. The Ethics Committee of Jinhua Municipal Central Hospital approved this study . The data were anonymous, and the requirement for informed consent was therefore waived. The study adhered to the principles of the Declaration of Helsinki.

Study population
The medical records were used to recognize participants. Patients older than 18 years of age who had hematemesis (or bloody nasogastric aspirate), melena, or both, as confirmed by the hospital staff, were considered for inclusion (Villanueva et al., 2013). Patients were excluded from the present study if they sought treatment for melena but are diagnosed with lower gastrointestinal bleeding. A total of 991 patients were included in this study.

Data collection
Structured data extraction forms were used by trained researchers for data collection. Medical record review and data extraction were performed by nursing staff with >10 years of experience at the ED. Following data extraction, quality control was performed on randomly selected data subsets. The following data were extracted: age (years), gender, chief complaints (syncope, hematemesis, melena), first effective vital signs and pulse oxygen saturation (%; in room air) obtained at the time of emergency admission, laboratory tests of blood, comorbidities, type of blood transfusion (mL), endoscopic reports, and survival status at discharge from ED (death in ED or within 24 h after leaving the ED). AIMS65, GBS, mGBS, and PERS were calculated for every patient based on the first available data. At the study site, the first vital signs are recorded in the emergency triage system, which constitutes part of the electronic medical records. Data on blood test results and clinical interventions can also be accessed through electronic medical records.

Statistical analysis
Data were analysised as previously described in Wang, Tu & Dong (2019), especially the process of model building and model validation. We used descriptive statistics to summarize the patients' baseline characteristics. For categorical variables, counts (percentages) were reported, while for continuous variables, mean ± standard deviation (SD) or median (interquartile) were reported. Comparisons of parameters between survivors and non-survivors were performed using Student's t-test and the Mann-Whitney U test for continuous variables with or without a normal distribution and using the χ 2 -test for categorical variables. The least absolute shrinkage and selection operator (LASSO) regression, which is suitable for the regression of high-dimensional data (Huang et al., 2016), was used for predictor selection and regularization. Logistic regression analysis was used to establish a model predicting death in ED or within 24 h after leaving the ED, and a nomogram was constructed on the basis of this model. The area under the curve (AUC) was calculated to evaluate the discrimination performance of the model and a 1,000 bootstrap method was applied for internal validation. For further validation, enrolled patients were randomly divided into two groups, 50% for training and 50% for validating. Statistical differences in the AUCs were compared using the method of DeLong. Decision curve analysis (DCA) was performed to evaluate the model's clinical utility by quantifying the net benefits at different threshold probabilities (Vickers et al., 2008). Furthermore, we calculated the AUC and DCA to evaluate the predictive performance of the nomogram in undergoing and not undergoing endoscopy subgroup. The predictive performances of the nomogram, AIMS65, GBS, mGBS, and PERS were compared by AUC and net reclassification improvement (NRI). Statistical analysis was conducted using EmpowerStats (www.empowerstats.com) and R 3.5.1 (www.r-project.org). P-values < 0.05 were considered statistically significant.

RESULTS
A total of 1,008 patients were initially recruited into the study and 991 remained after the exclusion criteria were applied (Fig. 1). Table 1 presents the patient characteristics and types of clinical interventions administered. Among the 991 patients, 41 (4.14%) died in ED or within 24 h after leaving the ED. Univariate analysis showed that death in ED occurred more frequently in patients with low red blood cell (RBC) count, hematocrit (HCT), HB, platelet (PLT) count, 993 patients met the inclusion criteria 1008 patients sought treatment at the emergency department due to melena, hematemesis, coffee ground vomiting, or anaemia 991 patients were included in this study 15 patients were excluded 9 diagnosed with lower gastrointestinal bleeding 6 aged <18 years 2 patients was lost to follow-up after discharge albumin (ALB), albumin/globulin ratio (A/G), total protein, in patients with high C-reactive protein (CRP), prothrombin time (PT), D-dimer, INR, glutamic oxaloacetic transaminase (AST), serum creatinine, in patients with chronic liver diseases, in patients with lower pulse oxygen saturation and in patients receiving plasma transfusion. Among the 42 collected variables shown in Table 1, included age, gender, chief complaints, first effective vital signs and pulse oxygen saturation, laboratory tests of blood, comorbidities, type of blood transfusion, five were filtered on the basis of non-zero coefficients calculated by the LASSO regression analysis using the minimum criteria (Fig. 2). These variables were transfusion of plasma, D-dimer, albumin, potassium, and age. We constructed a mortality risk prediction model based on the aforementioned five predictors ( Table 2). The AUC for the predictive model was 0.847 (95% confidence interval (CI) [0.794-0.900]), whereas the AUC for the internal validation model was 0.858 (95% CI [0.806-0.898]) (Fig. 3). The AUC for training group and validating group was 0.839 and 0.838, respectively (Fig. S1). The AUC of this model was also greater than that of the four other models (Table 3). We then calculated continuous NRI for the nomogram and four other models for death in ED or within 24 h after leaving the ED. The results also show that the nomogram was superior to other models (Table S1). The corresponding nomogram is presented in Fig. 4. The AUC for undergoing and not undergoing endoscopy subgroup was 0.874 and 0.822, respectively (Fig. S2).
The calibration curve of the predictive model showed a good fit between the predicted risk of death and observed outcomes in patients with AUGIB (Fig. 5). The decision curve (Fig. 6) demonstrated that this model, by helping assess the risk of death in ED and informing interventions, improved patient outcomes compared with either treat-all or treat-none strategies. Specifically, when the model assessed the risk of death in ED as within 3-76%, the benefits of this model to patient outcomes were most robust.

DISCUSSION
This study proposed a novel and easy-to-use model for predicting death in ED or death within 24 h after leaving the ED of patients with AUGIB. This model combined five predictors selected by LASSO regression: transfusion of plasma, D-dimer, albumin, potassium, age. Based on patients' characteristics and clinical interventions, all five predictors were readily accessible at EDs. Our nomogram may be used to infer the risk of mortality and therefore aid in the screening of patients with AUGIB who are at high risk. Thus, this could be a useful aid for physicians in determining a treatment regime. The model showed a good discriminatory ability of 0.847 (95% CI [0.794-0.900]). Moreover, it showed potential clinical usefulness. Although the calibration curve showed that among patients with actual mortality of great than 8%, the model overestimated mortality risk by 4% or greater, which may cause the waste of medical resources, it also would likely promote the active and efficient management of emergency patients.
The ability to quickly identify and appropriately treat patients at high risk is paramount in emergency care. Infusion of large plasma volumes was associated with poor survival outcomes (Subramaniam et al., 2016). Patients who received freshly frozen plasma transfusions had significantly greater rates of overall complications, acute respiratory distress syndrome, and pneumonia (Zhang et al., 2017), which resulted in a relatively higher mortality rate among these patients. D-dimer, as a fibrinolytic activity monitor, is generally used to evaluate the overall fibrinolytic condition in UGIB patients (Yue et al., 2020). Patients were considered to have hyperfibrinolysis if they concomitantly had high values of D-dimer and t-PA activity (Violl et al., 1996). Thus D-dimer is a stronger predictor for severity and prognosis both in Nonvariceal UGIB and variceal UGIB (Primignani et al., 2008;Yue et al., 2020). Albumin has multiple biological characteristics, and hypoalbuminemia is a risk factor of mortality in certain diseases. The correlation between hypoalbuminemia and the prognosis of UGIB has also been reported (Lyles, Elliott & Rockey, 2014;Bunchorntavakul, Yodket & Singhasena, 2017). Many studies used the receiver operating characteristic (ROC) curve to find the best value of albumin for predicting mortality. Lower than 3.2 or The definition of comorbidities. Renal failure: previous history of kidney failure, on dialysis or GFR < 15 ml/min/1.73 m 2 ; Respiratory failure: PO 2 < 60 mmHg or SO 2 <90% on room air; Heart failure: current or past clinical symptoms (limitation of activity, fatigue and dyspnea or orthopnea), signs (edema, elevated jugular venous pressure, rales, or S3 gallop), or radiologic evidence of pulmonary congestion. HR, heart rate; SBP, systolic blood pressure; DBP, diastolic blood pressure; RBC, red blood cell; HCT, hematocrit; HB, hemoglobin; PLT, platelet count; CRP, C-reactive protein; PT, prothrombin time; INR, international normalized ratio; ALT, alanine aminotransferase; AST, glutamic oxaloacetic transaminase; BUN, blood urea nitrogen; ALB, albumin; A/G, albumin /globulin ratio; GFR, glomerular filtration rate. 2.6 was used as criteria for hypoalbuminemia and were considered to be independent predictors of death (Gonzalez-Gonzalez et al., 2011Chandnani et al., 2019). Our study showed that the lower the albumin, the greater the risk of death in the ED. With increasing age, the function of the human body gradually decreases, and comorbidities also increase. Therefore, advanced age is a risk factor for the prognosis of various diseases (Rajan et al., 2019). In the present study, the risk of death increases with age.
Interestingly, high serum potassium, which has never been clearly described as an outcome predictor of UGIB, was found to be an independent predictor for death in ED. Mortality risk progressively increased with dyskalemia in patients with and without heart failure, chronic kidney disease, and/or diabetes (Collins et al., 2017). Many studies have shown that serum potassium concentration correlates with the severity of cirrhosis, and mortality was also significantly higher in patients with hyperkalemia (Uslan, Olt & Eminler, 2015;Maiwall et al., 2016). Accordingly, it is likely that serum potassium can be another valuable predictor for death in patients presenting with UGIB.
There are many different scoring systems available to stratify patients with AUGIB at risk of rebleeding or death and to predict the need for clinical interventions such as transfusion and endoscopic therapy. GBS was established to identify patients at low or high risk of needing treatment (Blatchford, Murray & Blatchford, 2000), while RS and AIMS65 were developed to predict in-hospital mortality (Rockall et al., 1996;Saltzman et al., 2011). However, only some scores from these models are suitable for ED, and the results are inconsistent. The most widely studied scores are PERS, GBS, mGBS, and AIMS65 (Rout et al., 2019;Tham & Stanley, 2019). When compared with RS and PERS, GBS has been shown to be the best for predicting intervention or death (Park et al., 2016;Stanley et al., 2017). For predicting in-hospital mortality, other studies have shown that the area under the ROC curve (AUROC) of AIMS65 is superior to or similar to the GBS, RS, and PERS for patients with AUGIB (Yaka et al., 2015;Budimir et al., 2016; Martinez-Cara Figure 2 (continued) A coefficient profile plot was created against the log (lambda) sequence. Dotted vertical lines were drawn at the optimal values by using the 1-SE criteria. In the present study, predictors were chosen according to the 1-SE criteria, a total of five non-zero coefficients were filtered and used to construct the predictive model. SE, standard error.
Full-size  DOI: 10.7717/peerj.11656/fig-2    Tang et al., 2018;Kim, Choi & Shin, 2019;Robertson et al., 2020). However, the predictive effects of these scoring systems for death ranged from 0.56 to 0.91 (Uslan, Olt & Eminler, 2015;Maiwall et al., 2016;Collins et al., 2017;Rajan et al., 2019;Rout et al., 2019;Tham & Stanley, 2019;Shung et al., 2020). The machine learning model developed by Shung et al., which aimed to identify patients meeting a composite endpoint of hospital-based intervention or death within 30 days, performed better than the GBS, admission-RS, and AIMS65 (Shung et al., 2020). However, an app is required in order to access this tool. In addition, most of these studies have focused on predicting hospital mortality or death within 30 days. An effective emergency death risk assessment model may be more valuable to emergency care providers. In our study, we included the parameters considered in the established scoring system and added other parameters that may be relevant to the prognosis, such as CRP, coagulation function tests, and laboratory test results that reflect the functions of important organs such as the liver and kidney (Makhlouf & Morsy, 2012;Tomizawa et al., 2014;Bunchorntavakul, Yodket & Singhasena, 2017). Also, we included transfusion of blood. Among the existing models, our results show that PERS is most effective for predicting in-ED mortality. Nevertheless, our new model outperformed PERS in both the original AUC and corrected AUC. Nomograms are graphical depictions of individual-level prognosis predictions and have been shown to be more accurate than conventional models for predicting prognosis (Balachandran et al., 2015;Cooperman et al., 2016;Huang et al., 2016;Touijer & Scardino, 2009). We established a nomogram based on five predictors including four clinical patient characteristics, which can be readily obtained in clinical settings, and one medical intervention that can be used for the better management of patients. LASSO is a statistical formula mainly used for feature selection and regularization of data model. The LASSO method regularizes the parameters of the model by shrinking the regression coefficient and reduces some of them to zero. The feature selection phase occurs after the shrinkage, where each non-zero value is selected for use in the model (Chang et al., 2021). LASSO, which is considered superior to the conventional approach of choosing predictors, is commonly used in studies with a small sample size and a large number of predictors (Huang et al., 2016).
This study is the first to construct a nomogram that combines both patient characteristics and clinical interventions for predicting the clinical outcome of patients Figure 6 Decision curve analysis of the predictive model. Net benefit was produced against the high-risk threshold. The red solid line represents the predicted estimates. The decision curve indicates that when the threshold probability was within 3-76%, using this model in decision-making benefitted patient outcomes compared with either treat-all or treat-none strategies.
Full-size  DOI: 10.7717/peerj.11656/ fig-6 presenting with AUGIB at an ED. The nomogram shows a good discriminative ability. However, this study has several limitations. First, this was a single-center retrospective study, and therefore its clinical usefulness remains to be assessed through external validation. Second, we only used the first available data obtained at the time of emergency admission for each patient, and therefore did not consider any subsequent changes. Thus, further research is required to validate this model. Third, not all patients underwent endoscopy and the diagnosis of UGIB was mainly based on symptoms, signs, and laboratory tests. Although patients' electronic medical records were carefully reviewed, it is possible that the exact site of the bleeding of some patients may not be able to be identified. Moreover, we did not collect creatinine clearance and electrocardiogram, which maybe associated with blood potassium concentration. Therefore, we could not determine the correlation between these factors and death. However, we built a model with good performance. The addition of other variables would not significantly improve the prediction effect of the model. Finally, the outcome in our study was emergency mortality risk. Whether this model is applicable to predict in-hospital death, needs further research.

CONCLUSIONS
We established and internally validated a nomogram to predict mortality among patients with AUGIB presenting at an ED. The nomogram was based on five readily available clinical predictors and details of clinical interventions. This model may be of value for individual risk assessments and for guiding staff in selecting interventions that are most likely to improve patient outcomes.

ADDITIONAL INFORMATION AND DECLARATIONS Funding
This work was supported by the Youth Research Fund of Jinhua Municipal Central Hospital (No. 2016-02-06) to Han Zheng. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Saibin Wang conceived and designed the experiments, performed the experiments, analyzed the data, prepared figures and/or tables, authored or reviewed drafts of the paper, and approved the final draft.

Human Ethics
The following information was supplied relating to ethical approvals (i.e., approving body and any reference numbers): Ethics Committee of Jinhua Municipal Central Hospital approved this research .

Data Availability
The following information was supplied regarding data availability: The raw measurements are available in the Supplemental File.

Supplemental Information
Supplemental information for this article can be found online at http://dx.doi.org/10.7717/ peerj.11656#supplemental-information.