A comparison of existing risk prediction models in patients undergoing venoarterial extracorporeal membrane oxygenation

Background: Patients undergoing consideration for venoarterial extracorporeal membrane oxygenation (VA-ECMO) require an immediate risk profile assessment in the setting of incomplete information. A number of survival prediction models for critically ill patients and patients undergoing elective cardiac surgery or institution of VA-ECMO support have been designed. We assess the ability of these models to predict outcomes in a cohort of patients undergoing institution of VA-ECMO for cardiogenic shock or cardiac arrest. Methods: Fifty-one patients undergoing institution of VA-ECMO support were retrospectively analyzed. APACHE II, SOFA, SAPS II, Encourage, SAVE, and ACEF scores were calculated. Their ability to predict outcomes were assessed. Results: Indications for ECMO support included postcardiotomy shock (25%), ischemic etiologies (39%), and other etiologies (36%). Pre-ECMO arrest occurred in 73% and 41% of patients underwent cannulation during arrest. Survival to discharge was 39%. Three survival prediction model scores were significantly higher in nonsurvivors to discharge than surivors; the Encourage score (25.4 vs 20; p = .04), the APACHE II score (23.6 vs 19.2; p = .05), and the ACEF score (3.1 vs 1.8; p = .03). In ROC analysis, the ACEF score demonstrated the greatest predictive ability with an AUC of 0.7. Conclusions: A variety of survival prediction model scores designed for critically ill ICU and VA-ECMO patients demonstrated modest discriminatory ability in the current cohort of patients. The ACEF score, while not designed to predict survival in critically ill patients, demonstrated the best discriminatory ability. Furthermore, it is the simplest to calculate, an advantage in the emergent setting. © 2020 The Author(s). Published by Elsevier Inc. This is an open access article under the CC BY license. (http://creativecommons.org/licenses/by/4.0/)


Introduction
Venoarterial extracorporeal membrane oxygenation (VA-ECMO) is increasingly utilized in the setting of cardiogenic shock to provide hemodyamic and respiratory support to the acutely decompensating patient. Common scenarios include cardiogenic shock following acute myocardial infarction, acute exacerbation of chronic congestive heart failure, myocarditis, and postcardiotomy shock among others. 1 This extremely aggressive therapy can be life-saving when appropriately applied, but inappropriate use can lead to unnecessary prolongation of patient and family suffering and significant costs and resource utilization. [2][3][4] Prognosis after institution of VA-ECMO support is heavily dependent on a patient's comorbidities and degree of decompensation prior to institution of mechanical support. Numerous survival prediction models exist for critically ill patients (APACHE II [acute physiology and chronic health evaluation II], SAPS II [simplified acute physiology II], SOFA [sequential organ failure assessment] scores) as well as for patients undergoing elective cardiac surgery (ACEF [age, creatine, ejection fraction] score). These scores have demonstrated variable results in patients undergoing institution of VA-ECMO support. [5][6][7][8][9] While several institutional studies and few registry studies have described predictors of survival after VA ECMO for cardiogenic shock or cardiac arrest, [10][11][12] very few survival prediction models specifically designed for VA-ECMO exist, with varying degrees of complexity. [5][6][7][8][9][13][14] We describe our experience with VA ECMO for cardiogenic shock and cardiac arrest with the goal of comparing the utility of the previously described survival prediction models for critically ill patients and patients undergoing institution of VA-ECMO support.

Patients and methods
This was a retrospective study of all patients undergoing institution of VA-ECMO support at our institution from 2010 to 2017. Data regarding pre-ECMO demographic, clinical, laboratory, and hemodynamic parameters, cannulation strategies, and outcomes including complications, and weaning were collected. The primary binary endpoint was survival to discharge. The study was approved by the New York Methodist Hospital Institutional Review Board (Board Ref # 1382246; date of approval 2/5/2019). The requirement for individual patient consent was waived.
Patients underwent VA-ECMO for cardiogenic shock or cardiac arrest in the setting of postcardiotomy shock, acute myocardial infarction/ischemic cardiomyopathy, nonischemic cardiomyopathy, or rarely pulmonary or other etiologies. Cardiogenic shock was typically defined as 1) a systolic blood pressure of less than 90 mmHg despite the use of maximum doses of vasopressors (norepinephrine 32mcg/min, epinephrine 20mcg/min, and vasopressin 0.04 U/.min), 2) a cardiac index less than 2.0 l/m/m2 despite the use of inotropic agents and other forms of mechanical support including intraaortic balloon pump (IABP) counterpulsation or impella (Abiomed, Danvers, MA), 3) elevated filling pressures including pulmonary capillary wedge pressure greater than 16 mmHg or central venous pressure greater than 18 mmHg, or pulmonary edema on chest xray, and 4) evidence of end organ malperfusion including oliguria, elevated lactate levels and markers of renal or hepatic dysfunction. Pre-ECMO hemodynamic and laboratory variables collected were the last value available within 24 h of VA-ECMO placement. A creatinine level of 4 mg/dL was assigned to any patient on hemodialysis at the time of VA-ECMO cannulation. Contraindications for VA-ECMO included the presence of 1) advanced age (>80 years), 2) neurologic injury, 3) active bleeding or absolute contraindications to anticoagulation, 4) active malignancy, 5) prolonged downtime after cardiac arrest (>30 min prior to ECMO team activation), 6) severe sepsis (hypotension, tachycardia, hyperthermia or hypothermia, and end organ dysfunction secondary to a known or suspected infection), or any other factor thought to represent a profound limitation of life expectancy. Neurological injury is difficult to quantify in shock patients as they are typically intubated and sedated. However, any patient who, at the last assessable time, was documented to not wake up when off sedation, open eyes, move extremities, and follow commands or who demonstrates abnormal brainstem reflexes would be considered at risk for neurologic injury. The final decision to institute VA-ECMO was made by the treating physicians including cardiothoracic surgeons, interventional and heart failure cardiologists, and cardiac intenstivists.
The VA-ECMO circuit consisted of either the Biomedicus (Abiomed, Danvers, MA) or the Centrimag (Levitronix, Waltham, MA) pump and a Quadrox D oxygenator (Maquet, Wayne, NJ). Femoral, axillary, and central aortic arterial cannulation and femoral, internal jugular, and central right atrial venous cannulation strategies were utilized depending the clinical scenario and surgeon preference. Bedside femoral cannulation was most commonly performed in emergent scenarios precluding patient transfer as in the intensive care unit or emergency room, or in the cardiac catheterization suite at the time of percutaneous coronary intervention. Central aortic and right atrial cannulation was most commonly used in the setting of postcardiotomy shock after cardiac surgery in the operating room. Axillary cannulation was utilized semi-electively when a patient failed to wean from femoral VA-ECMO as a preferred long-term cannulation strategy to allow for patient mobilization or in the setting of lower extremity vascular complications secondary to femoral cannulation.
Ipsilateral antegrade limb perfusion sheaths as a general institutional policy were used in all cases of femoral cannulation, either by percutaneous placement with or without ultrasound guidance or via surgical cutdown. If unable to be placed, and the limb thought to be ischemic, the patient was converted from femoral to another cannulation access strategy. Patients were heparinized to an activated clotting time of 250 s for cannulation and maintained at a partial thromboplastin time of 45À60 s during the course of VA-ECMO support except in the presence of bleeding complications or severe coagulopathy secondary to thrombocytopenia or hepatic dysfunction with an elevated international normalized ratio, in which case heparin dosages were reduced or held.
Scores for three previously existing survival prediction models for critically ill patients including the APACHE II score, the SAPS II score, and the SOFA score, as well as for patients undergoing elective cardiac surgey, the ACEF score, were calculated for all patients in the current study. Scores for two additional survival prediction models designed for patients undergoing VA-ECMO, the Save and the Encourage scores were also calculated for all patients.

Statistical analysis
Continuous variables were described as the mean and compared with the students t-test. Categorical variables were described as frequency and percentages and compared with the chi squared test or fisher exact test as appropriate. All variables were assessed for correlation with survival to discharge on univariate analysis. APACHE II, SAPS, SOFA, ACEF, Encourage, and Save scores were calculated for all patients.
Predictive accuracy was assessed on univariate analysis and using the area under the receiver operating characteristic (ROC) curve. The ROC maps the true positive rate (sensitivity) versus the false positive rate (1-specificity) of a given test. As the sensitivity (y-axis) increases, the false positive rate (1-specificity; x-axis) will increase (to catch

ARTICLE IN PRESS
more patients with the disease, one will inadvertently include more patients without the disease). In a totally useless test, the false positive rate will increase in a linear fashion with sensitivity, leading to a diagonal line on the graph. The better the test, the further to the left the curve will lie from that diagonal line (sensitivity will increase a lot with a minor increase in the false positive rate), and the area under the curve (AUC) between the curve and the diagonal line will be greater. Thus, a larger AUC represents a better test or score.
A p value of 0.05 or less was considered significant. All analyses were performed with Stata 13 (StataCorp LLC College Station, TX).

Results
Between 2010 and 2017, 51 patients underwent VA ECMO placement for cardiogenic shock or cardiac arrest at our institution. Mean age was 57 years and 46% were female. Indications for VA-ECMO support included postcardiotomy shock after cardiac surgery in 25%, ischemic cardiomyopathy (including acute myocardial infarction) in 39%, nonischemic cardiomyopathy in 27%, and other etiolgies in 8%. Cardiac arrest prior to VA-ECMO occurred in 73% with an average downtime of 30 min, and 41% of patients underwent cannulation during cardiac arrest (ECPR À extracorporeal cardiopulmonary resuscitation). Peripheral cannulation was utilized in 94% of cases, and central aortic/right atrial cannulation in 6%. Of those undergoing peripheral cannulation, 96% were via femoral cannulation and 4% were via axillary cannulation. Of those undergoing initial femoral cannulation, 7% underwent conversion to central or axillary cannulation for a variety of reasons, including lower extremity ischemia, differential cyanosis, or to allow for mobilization.
Pre-ECMO factors significantly associated with survival to discharge included creatinine level (2.1 vs 1.3 mg/dL; p = .02) and white blood cell count (18.5 vs 11.1 K/uL; p = .05). Demographic, hemodynamic, and laboratory parameters are listed in Table 1. While age wasn't associated with survival to discharge, very advanced age was detrimental as no patient over the age of 75 years survived to discharge (seven patients). Similarly, while disease chronicity (as determined by duration of pre-ECMO mechanical ventilation) wasn't associated with survival to discharge, very prolonged periods of illness were detrimental as no patient with a duration of pre-ECMO mechanical ventilation of five or more days survived to discharge (five patients).
Patients remained on VA-ECMO support for an average of 7.5 days. Two patients underwent emergent cardiac surgery while on VA ECMO support (one coronary artery bypass surgery and one pulmonary embolectomy). Both remained on VA-ECMO support postoperatively and did not survive to discharge. One patient was weaned from VA-ECMO support, and underwent subsequent mitral valve replacement and coronary artery bypass grafting and survived to discharge. Fifty-three percent of patients were successfully weaned from VA-ECMO support, and of these, 74% survived to discharge. Overall survival to discharge was 39% (Table 2).
Observed survival rates for all scores tested weakly paralleled expected survival rates, although the correlation was strongest for the APACHE II and Encourage scores (Figs. 1À2). On univariate analysis, three survival prediction model scores were significantly higher in nonsurvivors to discharge than survivors; the Encourage score (25.4 vs 20; p = .04), the APACHE II score (23.6 vs 19.2; p = .05), and the ACEF score (3.1 vs 1.8; p = .03 [ Table 3]). In ROC curve analysis, the ACEF score demonstrated the greatest predictive ability with an AUC of 0.7 (Table 4; Fig. 3).

Discussion
In the current analysis, a variety of survival prediction models were assessed for their ability to predict outcomes in a cohort of patients undergoing VA-ECMO support for cardiogenic shock. The Encourage score, APACHE II score, and the ACEF score were significantly associated with survival to discharge, while the SAPS, SOFA and SAVE scores demonstrated no difference between survivors and nonsurvivors. Of the three former scores, the ACEF score demonstrated the strongest discriminatory ability by ROC analysis.
VA-ECMO is increasingly utilized in the setting of cardiogenic shock or cardiac arrest to restore hemodynamic stability and end-organ function. VA-ECMO serves as a short term ventricular assist device that can be rapidly placed at the bedside in the emergency room, intensive care unit, cardiac catheterization suite, or operating room. A bridge to decision approach allows for the assessment of neurologic status, renal and hepatic function, and subsequent recovery of cardiac function. Further treatment is tailored appropriately, including bridging to percutaneous coronary intervention or cardiac surgery, recovery with decannulation, long term implantable left ventricular assist device or heart transplant in the setting of adequate end organ function but inadequate cardiac recovery, or withdrawal of care in the unfortunate setting of irreversible multiorgan failure or neurologic injury.   Despite its life-saving potential, VA-ECMO is fraught with complications including vascular complications from cannulation. In addition, neurologic injury, renal failure, liver failure, and sepsis are all well described sequelae of the post-cardiogenic shock or cardiac arrest VA-ECMO patient. [2][3][4] Allocation of this heroic resource to those most likely to benefit is crucial in order to appropriately allocate limited precious financial and human resources as well as to avoid prolonging patient suffering and the associated emotional trauma inflicted on the family in the setting of medical futility. 15 Age, lactate levels, etiology of cardiogenic shock, end-organ function, and prior cardiac arrest are all known to predict survival after VA-ECMO. [5][6][7][8][9][13][14] However, the circumstances surrounding a patient in cardiogenic shock or cardiac arrest with regards to the need for an immediate risk profile assessment in the setting of incomplete or no information creates unique challenges. 16 The APACHE II, SAPS II, and SOFA scores are widely described scores used to predict hospital survival in critically ill patients admitted to the intensive care unit (ICU). All three utilize a combination of hemodynamic and laboratory variables, age, and neurologic status (glasgow coma scale [GCS]) on admission to the ICU. They have been externally validated in a variety of settings. In patients undergoing institution of VA-ECMO support, variable results were demonstrated, with AUCs ranging from 0.5 to 0.7. In the current study, similar findings were noted. Although the APACHE II score did demonstrate significant differences between survivors and nonsurvivors and the SAPS II and SOFA scores did not, all demonstrated only modest discriminatory ability with AUCs in the same range as prior studies. However, these scores were not derived from, nor intended to predict outcomes in VA-ECMO patients.
Several survival prediction models for severe respiratory failure requiring venovenous ECMO have been devised. [17][18][19][20][21] Only a few such models exist for VA-ECMO. Schmidt et al. created the SAVE scoring system utilizing the Extracorporeal Life Support Organization (ELSO) registry to predict survival to discharge in VA-ECMO patients. 5 While incorporating a large number of patients from a registry, limitations include exclusion of ECPR patients and the cumbersome nature of the score, which requires several variables to calculate. ECPR patients represent a unique subgroup of patients undergoing VA-ECMO support with regards to neurologic outcome, the degree of end organ dysfunction, and also the technical difficulty of obtaining access in a pulseless patient rapidly. The ENCOURAGE score was created utilizing a bi-institutional database and predicts survival to ICU discharge in VA-ECMO patients. 7 While the model demonstrated good discriminatory power, it was somewhat cumbersome for use in the emergent setting and is only applicable to patients in cardiogenic shock from acute myocardial infarction.
In the current study, although the Encourage score did demonstrate significant differences between survivors and nonsurvivors and the SAVE score did not, both demonstrated similar discriminatory ability with AUCs in the same range (0.61 and 0.66, respectively). Not surprisingly, in the original study from which the Encourage was derived, strong discriminatory ability was demonstrated with an AUC of 0.84. 7 Pabst. et al. attempted to improve on the score by creating a modified version replacing "prothrombin activity h 50%" with an "INR i 2 00 , and demonstrated an AUC of 0.74, slightly higher than in the current study. 13 Interestingly, in the original study by Schmidt et al. in which the SAVE score was derived, only a modest discriminatory ability was noted in the ELSO registry derivation cohort of 3846 patients with an AUC of 0.68. However, in their validation cohort of 161 patients from a single institution, excellent discriminatory ability with an AUC of 0.9 was noted. 5 A subsequent study by Chen at. al. attempting to modify the SAVE score to predict survival of emergency department patients undergoing institution of VA-ECMO support found an AUC of 0.73 for the original SAVE score, although this was improved to 0.84 by incorporating lactate into a "modified" SAVE score. 6 The original Encourage score study demonstrated an AUC of 0.71 for the SAVE score. 7 Our results with regards to these scores designed specifically for patients undergoing institution of VA-ECMO support are in accordance with these prior results. They demonstrate only a modest discriminatory ability, not much improved over those scores designed for critically ill patients in general (APACHE II, SAPS, SOFA).
The ACEF score was designed to predict mortality after elective cardiac surgery. 22 The score is calculated by taking the ratio of the age in years and the ejection fraction, and adding one point if the creatinine is >2 mg/dL. Interestingly, although the only score in the current analysis not designed to predict mortality in critically ill patients, the ACEF score demonstrated the best discriminatory ability. This is likely due to significant overlap in risk factors commonly correlated with outcomes after elective cardiac surgery (age, ejection fraction, and renal function) VA-ECMO. Furthermore, it is by far the simplest of the above scores to determine, requiring only three easily obtainable variables to calculate. One major limitation occurs in the setting of eCPR, in which case the baseline ejection fraction is likely Fig. 3. Area under the receiver operating characteristic curves (ACEF: age, creatinine, and ejection fraction; APACHE II: acute physiology and chronic health evaluation II; SAPS II: simplified acute physiology II; SOFA: sequential organ failure assessment).

ARTICLE IN PRESS
to be unknown. Another limitation is that it oversimplifies the complex clinical scenario of most patients under consideration for VA-ECMO as the chronicity (or lack thereof) of cardiomyopathy or renal failure are not taken into account, nor are other factors associated with active shock. Acute derangements are more of an indication of the need for VA-ECMO, whereas acute derangements that will not be corrected and suggest a poor overall prognosis are contraindications.
Other risk prediction models for cardiac surgery including the Society of Thoracic Surgeons Predicted Risk of Mortality Score (STS-PROM) or the EuroScore are widely used to predict outcomes after elective or emergent cardiac surgery but are not practical to calculate in the emergent setting of a patient undergoing evaluation for institution of VA-ECMO support.
Future research will need to be conducted utilizing larger datasets to better analyze the risks associated with poor outcomes on VA ECMO support to allow for creation of more accurate risk prediction models. Modifying current registries or creating new ones with the addition of more granular data and incorporating methods to audit such registries to ensure accuracy of data will be crucial to devise such scores in a useful way.
Limitations of this study include those inherent to a retrospective analysis utilizing chart review including incomplete data, potential inaccuracies in data, and potential for selection bias. In particular, determining the GCS of critically ill patients in cardiogenic shock or cardiac arrest, who are frequently intubated and sedated, by retrospective chart review is subject to error. A major limitation was the small sample size, and the inhomogenous nature of the patient cohort. In addition, due to differences in clinical practice across centers, extrapolation of results may be of limited value.

Conclusions
Mortality after VA-ECMO as a salvage therapy for cardiogenic shock remains high. A variety of survival prediction model scores designed for critically ill ICU patients and VA-ECMO patients demonstrated a weak to modest discriminatory ability in the current cohort of patients undergoing institution of VA-ECMO support for a variety of indications. The ACEF score, while not designed to predict survival in emergent scenarios, demonstrated the best, albeit only a modest degree of discriminatory ability. Furthermore, it is by far the simplest to calculate, an advantage in the setting of an acutely decompensating or arresting patient requiring rapid decision making. Further studies are greatly needed with regards to risk prediction models for patients undergoing institution of VA-ECMO support as no optimal currently available model exists, but the current study suggests that simpler may be better.

Declaration of Competing Interest
The authors have no conflicts of interest to declare.