Quick SOFA vs Rockall preendoscopy scores for risk assessment in patients with nonvariceal upper gastrointestinal bleeding: a retrospective cohort study

Abstract

Background

Several scoring systems are used to evaluate the severity of nonvariceal upper gastrointestinal bleeding (NVUGB) and the risk of rebleeding or death. The most commonly used scoring systems include the Rockall score, Glasgow-Blatchford score, and Forrest classification. However, the use of simpler definitions, such as the quick Sequential Organ Failure Assessment (qSOFA) score, to make a clinical decision is reasonable in areas with limited time and/or material resources and in low- and middle-income countries.

Methods

Patients with NVUGB whose medical records included information required to calculate the qSOFA and Rockall preendoscopy scores at the time of bleeding in the emergency department or another non-intensive care unit department were included in the study. The area under the receiver operating characteristic curve (AUROC) and 95% confidence interval (95% CI) were estimated for the ability of the qSOFA and Rockall preendoscopy scores to predict mortality.

Results

The qSOFA and Rockall preendoscopic scores at the time of bleeding confirmation could be calculated for 218 patients. The mortality rate increased from 3.4% in patients with a qSOFA score = 0 to 88.9% in patients with a qSOFA score = 3 (P < 0.001). The AUROC for prediction of mortality was 0.836 (95% CI 0.748–0.924) for the qSOFA score and 0.923 (95% CI 0.884–0.981) for the Rockall preendocopy score (P = 0.059).

Conclusions

An increase in the qSOFA score is associated with adverse outcomes in patients with NVUGB. The simple qSOFA score can be used to predict mortality in patients with NVUGB as an alternative when Rockall preendoscopy score is incomplete for which the comorbidity is unknown.

Background

Nonvariceal upper gastrointestinal bleeding (NVUGB) is a potentially life-threatening disease that requires urgent intervention. Multiple scoring systems are used to evaluate the disease severity and risk of rebleeding or death. The most commonly used scoring systems are the Rockall score [1], Glasgow-Blatchford score [2], and Forrest classification [3]. The Rockall and Glasgow-Blatchford scores include many clinical and laboratory signs. The Forrest classification allows prediction of the rebleeding risk using endoscopic images isolated from the clinical situation. However, the use of simpler definitions to make clinical decisions is reasonable in areas with limited time and/or material resources and in low- and middle-income countries. Recently, as a result of international collaboration between the North American Society of Critical Care Medicine and the European Society of Intensive Care Medicine, the quick Sequential Organ Failure Assessment (qSOFA) score has been integrated into clinical practice [4, 5]. Three variables independently associated with the risk of death and a length of stay in the intensive care unit (ICU) of more than 3 days were identified by studying data from large patient populations and included a respiratory rate ≥ 22 min⁻¹, systolic arterial pressure ≤ 100 mmHg, and alteration of mental status (Glasgow Coma Scale score < 15). The results of several meta-analyses show that the simple-to-use qSOFA score allows identification of infected patients with a higher risk of death outside of the ICU [6,7,8,9]. Recent studies showed that the qSOFA could be used to predict mortality in noninfected patients in the emergency department (ED) [10] and trauma patients [11]. Quick assessment of the patient’s condition severity with the qSOFA score can contribute to clinical decision-making by the rapid response team and optimize the administration of time and material resources in urgent situations [12].

The aim of our retrospective study was to define whether the qSOFA score calculated in the ED or another non-ICU department was comparable to the Rockall preendoscopy score for the prediction of adverse outcomes in patients with NVUGB.

Methods

Study design

Our study was a retrospective cohort study of patients with NVUGB admitted to the Yekaterinburg City Clinical Hospital No. 40 during the period from 2014 to 2016. Yekaterinburg City Clinical Hospital No. 40 is a major acute university hospital with 1506 beds and approximately 45,000 emergency admissions per year, of which 90–120 admissions are patients with NVUGB. Paper charts are filled out for all admitted patients. All patients admitted to the hospital with NVUGB were included in the initial analysis of our study, but only patients with appropriately filled out medical records, including the information needed to calculate the qSOFA and Rockall preendoscopy scores at the time of bleeding in the ED or another non-ICU department, were included in the analysis. One point was assigned to the patients for the presence of each criterion included in the qSOFA score (respiratory rate ≥ 22 min⁻¹, systolic arterial pressure ≤ 100 mmHg, and Glasgow Coma Scale score < 15); thus, the total qSOFA score ranged from 0 to 3 points. The Rockall preendoscopy score ranged from 0 to 7 points (Table 1) [1].

Table 1 Rockall preendoscopy score

The Charlson comorbidity index was used to evaluate patients’ comorbidities. According to this index, a set number of points is applied for each concomitant disease, and 1 point is applied for every decade of life after 49 years [13]. The NVUGB treatment protocol in the study facility complied with the recommendations of the International Consensus Upper Gastrointestinal Bleeding Conference Group [14]. After an initial assessment of the vital signs, a manual rectal examination was performed and a nasogastric tube was inserted. At the next step, one or two peripheral venous catheters were placed, and infusion therapy using Ringer solution was provided until the clinical condition stabilized. Endoscopic examination of the gastrointestinal tract with a Video Gastrointestinal Scope GIF-XP170N (OLIMPUS®) was performed for every patient included in the study at 30–60 min after confirmation of bleeding in the ED or ICU. The diagnosis was defined according to the 10th revision of the International Statistical Classification of Diseases and Related Health Problems (ICD-10). During the endoscopy procedure, epinephrine injection therapy and argon plasma coagulation were performed if active arterial bleeding/spurting, an oozing hemorrhage (without a visible vessel), or a nonbleeding visible vessel was found. Epinephrine injection therapy and argon plasma coagulation were also performed when a nonbleeding adherent clot with a high risk of rebleeding was observed. In other cases, only a diagnostic endoscopic examination was performed. Other methods of endoscopic bleeding therapy (i.e., sclerosant agents, clip/band ligation, cautery, and thrombin/fibrin cyanoacrylate glue) were not available in our facility at the time of treatment of the patients enrolled in the study (2014–2016). In addition to initial resuscitation and endoscopic therapy, the NVUGB treatment protocol included IV infusion of a proton pump inhibitor with subsequent oral administration, prokinetics administration for improvement of endoscopic visualization, eradication therapy if Helicobacter pylori was found and other disease-specific treatment options. For patients without additional risk factors, a hemoglobin level < 7 g/dL was considered an indication for red blood cell (RBC) transfusion. For patients with additional risk factors (refractory shock, ischemic heart disease, or infection complications), a more liberal RBC transfusion strategy was applied. Fresh frozen plasma was used in the following cases: when a patient required more than 3–4 units of RBCs within an hour, as part of a massive transfusion protocol, and for correction of severe coagulopathy. In case of rebleeding, the endoscopic treatment was repeated. In rare cases when endoscopic hemostasis was inadequate, open surgery was performed (gastroduodenostomy with sewing of the bleeding vessel at the bottom of the ulcer or tumor).

Sample size estimation

The sample size was estimated with the statistical software EZR in R commander version 1.37 [15] according to a previous assessment of mortality rates in patients with different Rockall preendoscopy scores in the original study published by Rockall et al. [1]. The following observations were included in the sample size calculation: the mortality rate was 0.2% in patients with a Rockall preendoscopy score 0 point and 16.7% in patients with Rockall preendoscopy score ≥ 1 point. The ratio between patients groups was 1:5.7. The estimated sample size was 194 patients. The sample size in our study exceeded this number because we included all patients with appropriate and relevant data for the evaluation of the preendoscopic qSOFA and Rockall scores, who were hospitalized in the studied facility during the period from 2014 to 2016.

Statistical analysis

Categorical variables were presented as numbers and percentages and compared with Fisher’s exact test. Continuous variables were presented as the median with interquartile range (IQR) and compared with the nonparametric Mann-Whitney U test. The sensitivity, specificity, positive predictive value (+PV), negative predictive value (−PV), positive likelihood ratio (+LR), and negative likelihood ratio (−LR) with 95% confidence intervals (95% CIs) for the ability of a qSOFA and Rockall preendoscopy scores to predict mortality were calculated. The optimal cutoff points for the scores were estimated with the Youden J statistics. The area under the receiver operating characteristic curve (AUROC) with 95% CI was estimated for prediction of mortality by the qSOFA and Rockall preendoscopy scores. A two-tailed P value < 0.05 was considered statistically significant. The analysis was performed using the statistical software EZR in R commander version 1.37 [15].

Results

In total, 312 patients with NVUGB were admitted to Yekaterinburg City Clinical Hospital No. 40 during the period from 2014 to 2016. A total of 31 patients died, for a mortality rate of 9.9%. We excluded patients, whose medical records did not contain information about comorbidity or vital signs (as level of consciousness, respiratory rate, or blood pressure) at the time of bleeding confirmation. Appropriate and relevant data required for the evaluation of the preendoscopic qSOFA and Rockall scores at the time of bleeding confirmation could be calculated for 218 patients. After endoscopy, the following conditions were found to cause NVUGB: peptic gastric/duodenal ulcer (ICD-10 codes K25.4, K26.4) in 79 (36.2%) cases; gastric/duodenal erosion (K25.0, K26.0) in 64 (31.1%) cases; Mallory-Weiss tear (K22.6) in 38 (17.4%) cases; malignancy (C16.0, C16.1, C16.2, C16.3, C16.8) in 20 (9.2%) cases; erosive gastritis (K29.0) in 14 (6.4%) cases, and other (K22.1, E16.4) in 3 (1.4%) cases. A total of 145 (66.5%) patients were the male gender. The median age was 59 (IQR 43–71) years, and the median Charlson comorbidity index was 3 (IQR 1–4). The baseline characteristics did not differ significantly between the groups of patients with different qSOFA scores except for the Charlson comorbidity index, which was higher in patients with a higher qSOFA score (Table 2). The bleeding severity was associated with the qSOFA score. Patients with higher qSOFA scores had significantly lower hemoglobin levels; most of these patients had a hemoglobin level < 7 g/dL and required vasopressors. (Table 2).

Table 2 Baseline characteristics of the patients enrolled in the study (n = 218)

The qSOFA and Rockall preendoscopic score were associated with the outcome. Patients with a higher qSOFA score required hospitalization in the ICU or open surgery and experienced rebleeding more often. The mortality rate increased from 3.4% in patients with a qSOFA score = 0 to 88.9% in those with a qSOFA score = 3 (P < 0.001) (Tables 3 and 4).

Table 3 Outcomes of the patients enrolled in the study by qSOFA score (n = 218)

Table 4 Outcomes of the patients enrolled in the study by Rockall preendoscopic score (n = 218)

Figures 1 and 2 demonstrate the calibration analysis for the qSOFA and Rockall preendoscopic scores. Mortality is strongly associated with both qSOFA and Rockall preendoscopic scores (Fisher P value < 0.001 for both scores).

Fig. 1

In-hospital mortality by qSOFA score (n = 218). The mortality rate was 3.4% in patients with a qSOFA score = 0, 15.6% in those with a qSOFA score = 1, 42.1% in those with a qSOFA score = 2, and 88.9% in those with a qSOFA score = 3 (P < 0.001)

Fig. 2

In-hospital mortality by Rockall preendoscopic score (n = 218). The mortality rate was 1.3% in patients with a Rockall score ≤ 3, 9.7% in those with a Rockall score = 4, 47.8% in those with a Rockall score = 5, 80.0% in those with a Rockall score = 6, and 100.0% in those with a Rockall score = 7 (P < 0.001)

The accuracy of the qSOFA and Rockall preendoscopic scores for prediction of mortality was compared using ROC analysis. Some characteristics (sensitivity, likelihood ratio for negative results, and negative predictive value) for prediction of mortality by the qSOFA score ≥ 1 and Rockall preendoscopic score ≥ 5 did not differ in the study, but the specificity, likelihood ratio for positive result, and positive predictive value were higher for the Rockall score ≥ 5 in comparison with the qSOFA score ≥ 1 (Table 5). For the prediction of mortality, the area under the receiver operating characteristic curve was 0.836 (95% CI 0.748–0.924) for the qSOFA score and 0.923 (95% CI 0.884–0.981) for the Rockall preendocopy score (P = 0.059) (Fig. 3).

Table 5 Characteristics of the prediction of mortality by the qSOFA and Rockall preendoscopic scores

Fig. 3

Comparison of the ability of the qSOFA and Rockall preendoscopic scores to predict in-hospital death. qSOFA AUROC = 0.836 (95% CI 0.748–0.924), Rockall AUROC = 0.923 (95% CI 0.884–0.981); P = 0.059. qSOFA quick Sequential Organ Failure Assessment, AUROC area under the receiver operating characteristic curve, CI confidence interval

Discussion

Well-studied scoring systems to predict the prognosis of mortality or rebleeding in patients with NVUGB include the Rockall score [1, 16, 17], Glasgow-Blatchford score [2, 17], and Forrest classification [3]. The Rockall preendoscopic score includes the following clinical signs: age, hemorrhagic shock severity, and concomitant diseases. The Glasgow-Blatchford score includes the hemoglobin level, blood urea level, systolic blood pressure, concomitant diseases, and other clinical signs of bleeding. The Forrest classification is based only on endoscopic signs of bleeding that contribute to the prognosis and treatment course. When faced with limited time and/or resource conditions, a simpler scoring system based only on clinical parameters is a useful approach. The qSOFA score includes the respiratory rate, systolic blood pressure, and level of consciousness. Our study established that the qSOFA score was strongly associated with the mortality rate in patients with NVUBG. Evaluation of the patient’s condition with the qSOFA score can help with management of ED resources, such as prompt transfer of patients to the ICU and/or initiation of early endoscopic treatment. Initially, qSOFA was validated in a cohort of patients with infections [4, 5]. Recently, studies have focused on noninfected patients in the ED [10], trauma patients [11], and patients with liver cirrhosis [18]. We found that the qSOFA score was associated with the bleeding severity, rebleeding frequency, hospitalization in the ICU, requirement for open surgery, and mortality rate (see Tables 2 and 3). The mortality rate was 3.4% in patients with a qSOFA score = 0, 15.6% in those with a qSOFA score = 1, 42.1% in those with a qSOFA score = 2, and 88.9% in those with a qSOFA score = 3 (P < 0.001).

The qSOFA and Rockall preendoscopic score AUROCs were compared in our study. For the prediction of mortality, the qSOFA AUROC was 0.836 (95% CI 0.748–0.924) and the Rockall AUROC was 0.923 (95% CI 0.884–0.981); P = 0.059 (see Fig. 3). The prognostic prediction of mortality of the Rockall AUROC was higher than that of the qSOFA AUROC, and the P value was 0.059 which was slightly greater than the significance level and revealed that the difference between the qSOFA and Rockall score AUROCs was not significant. We defined the optimal cutoff value with the Youden index (J) analysis. The optimal cutoff value for the qSOFA score was ≥ 1 point and for the Rockall score was ≥ 5 points. The specificity of the qSOFA score ≥ 1 was lower than of the Rockall preendoscopic score ≥ 5 [73.7 (95% CI 66.8–79.8) vs 92.6% (95% CI 87.9–95.9%)]. The sensitivity of both scoring systems was moderate and comparable: 82.1 (95% CI 63.1–93.9) and 82.1% (95% CI 63.1–93.9%) for the qSOFA score ≥ 1 and Rockall preendoscopic score ≥ 5, respectively. The likelihood ratios for the negative results and the negative predictive values were also comparable between the qSOFA and Rockall preendoscopic scores (see Table 5). In addition to the prognosis of mortality, the qSOFA was useful for prognostic prediction of rebleeding. The rebleeding rate was more than two times higher in patients with a qSOFA score = 1 and more than three times higher in patients with a qSOFA score ≥ 2 compared to those with a qSOFA score = 0 (Table 3). Moreover, patients with a qSOFA score > 0 more often required open surgery because of inadequate endoscopic hemostasis (Table 3). Our study had limitations. First, some nonelectronic medical records for the studied patients did not contain sufficient and relevant information to calculate both the qSOFA and Rockall preendoscopic scores simultaneously at the time of bleeding; thus, these patients were not included in the major analysis. Our study was shifted to more severe cases because 31 (9.9%) of the 312 initially screened patients and 28 (12.8%) of the 218 patients included in the major analysis died. Secondly, we did not perform a comparison of the qSOFA and Rockall preendoscopic scores with scoring models such as the Sequential Organ Failure Assessment Score [19], Acute Physiology and Chronic Health Evaluation Score [20], and others, because the medical records of the patients in the studied facility contained the necessary information only if they were admitted to the ICU during the hospitalization period. Glasgow-Blatchford score [2] is also a well-validated scoring system. However, it is necessary to make a blood urea nitrogen (BUN) test to calculate this score, which in our facility is not an obligatory standard for the treatment of NVUGB and is made only in patients with the high risk of death or comorbidity because of limited material resources. Third, we did not take into account important risk factors, such as administration of nonsteroidal anti-inflammatory drugs, anticoagulants, and other drugs, which could influence the bleeding severity and mortality rate. Fourth, our study is a single-center study performed in a middle-income country with low healthcare costs and thus does not reflect the general population of patients with NVUGB.

Conclusions

An increase in the qSOFA score is associated with adverse outcomes in patients with NVUGB. The simple-to-use qSOFA score can be used for the prediction of mortality in patients with NVUGB as an alternative when Rockall preendoscopy score is incomplete for which the comorbidity is unknown. A multicenter study is required before making recommendations for use of the qSOFA score in defining the prognosis of patients with NVUGB.