Performance of the PEdiatric Logistic Organ Dysfunction-2 score in critically ill children requiring plasma transfusions

Background Organ dysfunction scores, based on physiological parameters, have been created to describe organ failure. In a general pediatric intensive care unit (PICU) population, the PEdiatric Logistic Organ Dysfunction-2 score (PELOD-2) score had both a good discrimination and calibration, allowing to describe the clinical outcome of critically ill children throughout their stay. This score is increasingly used in clinical trials in specific subpopulation. Our objective was to assess the performance of the PELOD-2 score in a subpopulation of critically ill children requiring plasma transfusions. Methods This was an ancillary study of a prospective observational study on plasma transfusions over a 6-week period, in 101 PICUs in 21 countries. All critically ill children who received at least one plasma transfusion during the observation period were included. PELOD-2 scores were measured on days 1, 2, 5, 8, and 12 after plasma transfusion. Performance of the score was assessed by the determination of the discrimination (area under the ROC curve: AUC) and the calibration (Hosmer–Lemeshow test). Results Four hundred and forty-three patients were enrolled in the study (median age and weight: 1 year and 9.1 kg, respectively). Observed mortality rate was 26.9 % (119/443). For PELOD-2 on day 1, the AUC was 0.76 (95 % CI 0.71–0.81) and the Hosmer–Lemeshow test was p = 0.76. The serial evaluation of the changes in the daily PELOD-2 scores from day 1 demonstrated a significant association with death, adjusted for the PELOD-2 score on day 1. Conclusions In a subpopulation of critically ill children requiring plasma transfusion, the PELOD-2 score has a lower but acceptable discrimination than in an entire population. This score should therefore be used cautiously in this specific subpopulation.


Background
Mortality is a frequent outcome in clinical trials in critically ill adults [1][2][3][4][5]. However, as mortality is lower in pediatric intensive care unit (PICU) patients [6][7][8], other outcome measures have been developed. Multiple organ dysfunction syndrome (MODS), frequently observed in PICU, is a good candidate marker of severity of illness because MODS is the main cause of death in adult ICU [9] and in PICU patients [7]. MODS scores can be used to assess the presence and severity of organ dysfunction on admission and throughout the stay [9].
In a general PICU population, the PEdiatric Logistic Organ Dysfunction-2 score (PELOD-2) and the daily PELOD-2 scores had both a good discrimination and calibration, allowing to describe the clinical outcome of critically ill children throughout their stay [8,10].
Little is known regarding plasma use in children. Our recent international observational study shows that nonbleeding patients represent more than half of the critically Open Access *Correspondence: oliver.karam@hcuge.ch 1 Pediatric Intensive Care Unit, Geneva University Hospital, Geneva, Switzerland Full list of author information is available at the end of the article ill children receiving plasma transfusions [11]. This marked heterogeneity in plasma transfusion patterns might be due to the absence of randomized controlled trials (RCTs) that could guide plasma transfusion strategies [12].
The first version of the PELOD score [7] has been used over the last few years as an outcome measure in studies in specific subpopulations, such as sepsis [13], hematopoietic stem cell transplant [14], acute respiratory dysfunction syndrome [15], extracorporeal life support [16], drowning [17], or seizure [18]. However, some authors have voiced their concern regarding using organ dysfunction scores in specific subpopulations [9].
Our hypothesis was that the PELOD-2 score would have the same performance in the subpopulation of patients receiving at least one plasma transfusion, validating its use as a surrogate outcome in a future RCT.
Our objective was to assess the performance of the PELOD-2 score in a subset of critically ill children requiring plasma transfusions [11] during their PICU stay.

Study sites and population
This is an ancillary study of a large point-prevalence study conducted in 101 PICUs in 21 countries. The complete methods have already been published elsewhere [11].
In brief, six 1-week periods were randomly predefined over six consecutive months (April to September 2014) for each study site. All critically ill children aged 3 days to 16 years old admitted to a participating PICU on one of the study days were considered eligible. Any eligible patient for whom at least one plasma transfusion was administered on any study day was included unless one of the exclusion criteria (i.e., plasmapheresis and gestational age less than 37 weeks at the time of PICU admission) was present. If a patient was readmitted within 24 h of PICU discharge, this was considered part of the same admission.

Variable of interest
The primary variable of interest of this analysis is the daily PELOD-2 scores [8,10]. This score evaluates five organ functions using ten items: neurologic (Glasgow coma score and pupillary reaction), cardiovascular (lactatemia, mean arterial pressure), renal (creatinine), respiratory (PaO 2 /FiO 2 ratio, PaCO 2 , invasive ventilation), and hematologic (white blood cell count and platelets). Data were collected on days 1 (i.e., the day of first transfusion), 2, 5, 8, and 12. These time points were previously identified as the first optimal time points to estimate the daily PELOD-2 scores [8,10].
As for previously published severity and MODS scores, the most abnormal value of each variable observed during each of these time points was considered to calculate the PELOD-2 score. No laboratory tests were performed solely to meet the needs of this research; as recommended, a non-collected value was considered normal [8].
We also collected demographic data and PICU mortality, our primary outcome, which was censored 28 days after the end of the enrollment period.

Ethics approval
Ethics committees or boards at all 101 sites approved this study.

Statistical analysis
Descriptive statistics are reported as mean ± standard deviation (SD), median and interquartile range (IQR), or proportions with their 95 % CI.
The association between PELOD-2 score and death was assessed by comparing the PELOD-2 score between survivors and non-survivors with a Mann-Whitney test. We also tested this association, adjusting for the baseline risk (PELOD-2 score at day 1) and the daily change in PELOD-2 score using a logistic regression model [10].
Discrimination refers to the ability of the score to separate non-survivors from survivors across the whole group [19]. We calculated the area under the receiver operating characteristic curve (AUC) of the PELOD-2 scores with its 95 % CI, for each time point. It is usually considered that an AUC of 1, 0.90-0.99, 0.80-0.89, 0.70-0.79, 0.60-0.69, and <0.60 is considered to be perfect, excellent, very good, good, moderate, and poor, respectively [20].
The calibration was assessed by directly comparing the observed and customized predicted mortality across subcategories of risk. We employed the Hosmer-Lemeshow goodness-of-fit test, where a p value >0.05 indicates acceptable calibration [20].
All tests were two sided, with an alpha level of 0.05. All statistical analyses were performed with SPSS version 20 for Mac (SPSS, Chicago, IL, USA).

Population
Over the 30 study days, 13,192 patients were admitted and hence eligible and 443 (3.4 %) critically ill children receiving at least one plasma transfusion were included. The center which included the largest number of patients contributed to 11.1 % (49/443) of the results. Two hundred and fifty-three patients were from Europe, 134 from North America, and 56 from other continents.
The primary indication for plasma transfusion was critical bleeding in 22 % of patients, minor bleeding in 21 %, planned surgery or procedure in 12 %, and high risk of postoperative bleeding in 11 %. No bleeding or planned procedures were reported in 34 % of patients.
Median length of mechanical ventilation was 5 days (IQR 1-16), and median PICU length of stay was 10 days (IQR . The median time to death was 6 days (IQR 1;17).

PELOD-2 score
PELOD-2 score was collected in all patients on day 1 and in all surviving patients still in the PICU at posttransfusion days 2, 5, 8, and 12. There were no missing data.
The PELOD-2 score on day 1 was a significant prognostic factor: The odds ratio for death was 1.30 (95 % CI 1.22-1.39) for each PELOD-2 point. Similarly, the serial evaluation of the changes in the daily PELOD-2 scores from day 1, adjusted for baseline value (PELOD-2 at day 1), demonstrated a significant association with death, for each of the observation days (Table 2).

Calibration
The Hosmer-Lemeshow Chi-square value was 5.02 (p = 0.76) for PELOD-2 score on day 1. The results for the other time points, according to plasma transfusion indication, and according to the reason for admission are given in Tables 3,4,and 5,respectively. Adding the INR to the PELOD-2 score resulted in Hosmer-Lemeshow Chi-square value of 10.6 (p = 0.23).

Discussion
Our results indicate that in a subpopulation of critically ill children requiring plasma transfusions, the PELOD-2 score has an acceptable performance. These results of performance of the PELOD-2 scores are observed with the performance both according to the indications to plasma transfusion (Table 4) and according to the 5 days of the PELOD-2 scores (Tables 2, 3).
The PELOD-2 score seems to have a lower, although acceptable discrimination power compared to a general PICU population, where it had an excellent discrimination power, based on an AUC of 0.93. As the PELOD-2 score has been advocated to be "used as a surrogate outcome measure in randomized clinical trials" [7,8], many recent trials have used these scores to assess patients [13][14][15][16][17][18]. As these studies enrolled only specific PICU subpopulations, their results should be interpreted cautiously, in light of our findings. Similar caveats have been made for adult organ dysfunction scores regarding their use in specific subpopulations [9].
The lower discrimination of the PELOD-2 score in a subset of critically ill children transfused with plasma might be explained by different observations. First, our population was sicker than the initial PELOD-2 study, with a mortality rate of 26.9 versus 6.0 %. The PICU length of stay was also longer (10 vs. 2 days), as the casemix was different: In our population compared to the PELOD-2 population, the reasons for admission were less frequently respiratory (32 vs. 47 %, p < 0.001) and more frequently cardiovascular (64 vs. 19 %, p < 0.001) or hepatic (10 vs. 1 %, p < 0.001) [8]. Moreover, in the study of Pollack et al. (10,078 patients from U.S. PICUs), the mortality rate was 2.7 %, with a median age of 3.7 years and a median hospital length of stay of 4.9 days (PICU length of stay was not provided) [21]. The fact that all patients received plasma transfusions might also explain this lower discrimination, as observational studies have suggested that plasma transfusions were independently associated with increased risk of morbidity and mortality [9,[22][23][24], but neither coagulopathy nor plasma transfusions are items of the PELOD-2 score. Third, except for mechanical ventilation, PELOD-2 score does not take into account the support that can be offered for each organ, such as vasoactive drugs, continuous renal replacement therapies, or extracorporeal life support (ECLS). Given that the mortality rate associated with ECLS is close to 50 % [25,26], one could hypothesize that a lower discrimination may be due at least partly to the fact that PELOD-2 score does not take this variable into account. Similarly, the PELOD-2 score does not incorporate coagulopathy, which might be associated with increased risk of bleeding or disseminated intravascular  Table 2 Serial evaluation of the change in the daily PELOD-2 score from day 1, adjusted for baseline value (PELOD-2 score on day 1) Odds ratio (OR) for death is provided with a 95 % CI. The cumulative OR of death can be calculated as follows: (OR of PELOD-2 score on day 1) PELOD-2 score on day 1 × (OR of change in score from day 1 to specific day) PELOD-2 score on specific day -PELOD-2 score on day 1 . For example, for a child whose score is 10 on day 1 and 6 on day 12 (change in score −4), the OR for death would be 1. coagulopathy, which are also known to be associated with increased mortality [27].
As observed in the validation study, severity of illness is a major component to evaluate the performance of the daily PELOD-2 scores [10]. Therefore, in this study, the serial evaluation of the change in the daily PELOD-2 score has to be adjusted to the baseline value (PELOD-2 score on day 1). Our results indicate that changes in the PELOD-2 score over time are also associated with changes in the probability of death. These are important findings, as they validate the use of sequential measuring of the PELOD-2 score to assess clinical outcome. These results also highlight the relationship between organ failure and death.
Some limitations must be recognized. First, our results are only applicable to critically ill children requiring plasma transfusions and not to other PICU subpopulations. Second, our study days were not days after admission but days after plasma transfusion. It is of note, however, that the median length of PICU stay before the first plasma transfusion was 1 day. Third, although scores have classically been evaluated by their association with mortality, they might still adequately describe the clinical situation related to organ failure, as it has solid physiological basis and is clinically meaningful. Therefore, the ability of the PELOD-2 score to describe organ failure and its change over time might be more important that its ability to predict death. Fourth, additional variables to the PELOD-2 score could have been evaluated. Unfortunately, some interesting variables, such as platelet count, were not available in our database. However, adding the INR to the PELOD-2 score did not improve its performance.

Conclusions
In a subpopulation of critically ill children requiring plasma transfusion, the PELOD-2 score has a lower but acceptable discrimination than in an entire population. Although using this score as an outcome in a RCT seems reasonable, it should be interpreted cautiously.