Predicting survival post-cardiac arrest: An observational cohort study

Introduction Over 400,000 out-of-hospital cardiac arrest (OHCA) occur each year in Canada and the United States with less than 10% survival to hospital discharge. Cardiac arrest is a heterogenous condition and patient outcomes are impacted by a multitude of factors. Prognostication is recommended at 72 hours after return of spontaneous circulation (ROSC), however there may be other factors that could predict patient outcome earlier in the post-arrest period. The objective of our study was to develop and internally validate a novel clinical prediction rule to risk stratify patients early in the post-cardiac arrest period. Methods We performed a retrospective cohort study of adult (≥18 years) post-cardiac arrest patients between 2010 and 2015 from the Epistry Cardiac Arrest database in Toronto. Our primary analysis used ordinal logistic regression to examine neurologic outcome at discharge using the modified Rankin Scale (mRS). Our secondary analysis used logistic regression for neurologic outcome and survival to hospital discharge. Models were internally validated using bootstrap validation. Results A total of 3432 patients met our inclusion criteria. Our clinical prediction model was able to predict neurologic outcome on an ordinal scale using our predefined variables with an AUC of 0.89 after internal validation. The predictive performance was maintained when examining neurologic outcome as a binary variable and survival to hospital discharge. Conclusion We were able to develop a model to accurately risk stratify adult cardiac arrest patients early in the post-cardiac arrest period. Future steps are needed to externally validate this model in other healthcare settings.


Introduction
Despite improvements in resuscitation science and treatment, survival rates from out-of-hospital cardiac arrest (OHCA) remain low. 1 Two-thirds of patients who have a return of spontaneous circulation (ROSC) and are admitted to hospital will die, most as a result of neurologic injury. 2 Guidelines recommend waiting at least 72 hours post-ROSC to make prognostication decisions. 35][6] This uncertainty during the early post-cardiac arrest period makes discussions regarding goals of care and therapeutic planning difficult with patient families.Risk stratification of patients can help to reduce uncertainty during this period and inform discussions around patient care.In addition, risk stratification can have many important roles for patient identification in future research studies and in quality improvement initiatives to help benchmark within and across institutions.
Many patient characteristics, intra-arrest and early post-arrest variables are associated with patient outcomes.For example, the Utstein data elements (age, sex, location of arrest, witness status, bystander CPR, initial cardiac rhythm, EMS response time) have been shown to predict 72% of the variability in patient outcomes. 7ther prediction models have been developed which have shown good to excellent predictive performance in the derivation population, https://doi.org/10.1016/j.resplu.[10][11] Our goal was to derive and internally validate a clinical prediction model for use early (24 hours) after return of spontaneous circulation to predict survival and good functional outcome at hospital discharge.

Study design and setting
We performed a retrospective cohort study using data from the Toronto Regional RescuNET Epistry Cardiac Arrest Database (Rescu Epistry).Rescu Epistry is compliant with the Resuscitation Outcomes Consortium (ROC) Epistry-Cardiac Arrest and based on the Strategies for Post Arrest Resuscitation Care (SPARC) methodologies described elsewhere. 12,13Rescu Epistry is a prospective population-based registry of consecutive OHCAs that occur within 7 geographical regions in southern Ontario.Toronto Regional Rescu-NET has a population of 6.6 million people and is served by 7 land paramedic agencies (Toronto, Peel, York, Halton, Durham, Simcoe, and Muskoka), a provincial air ambulance service (Ornge) and 42 destination hospitals.This protocol was approved by the lead hospital Research Ethics Board (REB) at St. Michael's Hospital.

Population
We included consecutive adult (!18 years) OHCA treated by paramedics that occurred between January 1, 2010 and December 31, 2015 and had a documented ROSC in hospital for greater than 20 minutes.We excluded OHCA that were not treated by paramedics due to criteria for obvious irreversible death (e.g.rigor mortis, lividity), patients with cardiac arrests of obvious non-cardiac etiology (e.g.trauma), patients who had a valid "do not resuscitate" (DNR) order in the prehospital setting or within the first 2 hours after ROSC in the hospital, and patients who died within the first 24-hours after ROSC as they were not eligible to be assessed for some of our variables of interest.

Outcomes and included variables
Variables of interest (age, sex, initial cardiac rhythm, location, bystander CPR, witness status, EMS response time interval, etiology of cardiac arrest) were all consistent with Utstein definitions. 14,15MS response time interval was defined as the time from call to 911 dispatch to arrival on scene of the first 911-initiated first responder (paramedics or fire department personnel).Duration of resuscitation was defined as the time from EMS initiation of CPR to the time of first ROSC (prehospital or in-hospital), or termination of resuscitation in those patients without a documented ROSC.In cases where the timing of ROSC was not recorded any patient who obtained ROSC in the prehospital setting the timing of ROSC was imputed as the time of arrival at the ED.For patients without ROSC in either the prehospital or in-hospital setting where the timing of termination of resuscitation was not recorded the resuscitation was assumed to stop 30 minutes after arrival at the ED.Blood gas values (pO2, pCO2, HCO3-were defined as the lowest obtained value within the first 24 hours.If multiple pH values were obtained we used the first obtained value.GCS values were defined as the highest total score and highest score for the motor component recorded over the first 24 hours post-ROSC. The primary outcome was functional survival at hospital discharge.Functional survival was evaluated using the modified Rankin Scale (mRS) 16 and defined as a good functional outcome (0-2), poor functional outcome (3-5) and dead (6).Our secondary outcomes were survival to hospital discharge and functional survival as a binary outcome [good (mRS 0-2) and poor (mRS 3-6)]. 16

Analysis
Our goal was to derive and internally validate a clinical prediction model for early post-cardiac arrest patients.We used descriptive bivariate analyses (ANOVA [or Student's t-test] and Pearson's chisquared test) to examine the association of predetermined variables of interest and patient outcomes.

Model derivation
For our primary analysis we derived our early predictive model using ordinal logistic regression.Our outcome for this model was mRS score at hospital discharge categorized as "good" (mRS 0-2), "poor" (mRS 3-5) and "dead" (mRS 6).To avoid inherent biases generated from using stepwise variable reduction 17 we included predetermined variables of interest based on a review of the literature and expertise: age (continuous), EMS response time (continuous), sex (male/ female), location (public/private), witness status (not witnessed, bystander, EMS), bystander CPR (yes/no), and initial cardiac rhythm (ventricular fibrillation/ventricular tachycardia vs. pulseless electrical activity/asystole), duration of resuscitation, pupillary response to light during the first 24-hours post-cardiac arrest, maximum Glasgow coma scale (GCS) motor score in the first 24-hours post-cardiac arrest.We examined use of corneal reflexes, however there was a large proportion of missing values and so this variable was removed from the analysis.In addition, we derived a second early prediction model that included arterial blood gas values during the first 24hours post-cardiac arrest (pO2, pCO2, pH, and bicarbonate).
For our secondary analysis, we derived early prediction models examining functional status as a binary variable, with good outcome defined as mRS 0-2 and poor outcome mRS 3-6, and for survival status at hospital discharge (Alive/Dead).The results of our regression models are presented with odds ratios (OR) and corresponding 95% confidence intervals.The odds ratio for an ordinal regression model is the odds of moving from one level to the next within each variable.Our models met the assumptions for proportional odds ordinal regression and therefore we assumed that the odds of moving across each level was proportional within variables.Continuous variables were maintained in their original form and we used restricted cubic splines to examine for non-linearity.We performed a likelihood ratio test, as a global test for additivity, to compare models with and without non-linear and interaction terms as well as to compare models with additional arterial blood gas variables.The significance of the inclusion of these additional terms was represented with a P value <0.30.

Internal validation
Internal validation of our final model was performed using Bootstrap 0.632 validation. 18We performed 100 bootstrap repetitions and model performance statistics were averaged across the samples and subtracted from the original performance metrics to determine the "optimism-corrected" performance metrics for the model.

Model performance
We assessed model performance using Brier statistic as an overall measure of performance and specific measures of model discrimination and calibration. 19Discrimination (ability of model to differentiate between those with and without the outcome) was assessed using cstatistic and area under the receiver operating characteristic (AUC/ ROC) curve.Calibration (agreement between predicted and observed outcomes) was evaluated with the use of calibration curves as well as calibration slope and intercept.Our final model is presented as a nomogram.
All statistical computing was done using R: A language and environment for statistical computing (Vienna, Austria).

Results
Overall 5167 OHCA patients had a return of spontaneous circulation during our study time period, of which 3432 met our inclusion criteria.(Fig. 1) Of the included patients, 1656 (48%) survived to hospital discharge and 1452 (42%) had a good functional outcome (mRS 0-2).Seventy-seven patients were missing functional outcome and were removed from further analysis.
The median age of patients treated by paramedics was 65.6 years (IQR 54.5, 77.1) and 70% of patients were male.The descriptive statistics of included patients based on functional status at hospital discharge are shown in Table 1.There was a significant difference found based on functional status at discharge (good functional status vs poor neurological status vs dead) in terms of all of our predetermined variables of interest.
The results of our predictive model are presented in Fig. 2 and Table 2. Our primary model was found to have high predictive ability for functional outcome at hospital discharge.The overall predictive performance, measured with Brier statistic, was 0.127.The discrimination of the model was excellent with a c-statistic (AUC/ROC) of 0.890 (Table 3 and Fig. 3) and the calibration plot showed excellent calibration across the range of predicted probabilities.(Fig. 3) Our model performance statistics were maintained on internal validation suggesting very little overfitting in our model (Table 3).EMS response time and duration of resuscitation were kept as continuous variables and non-linear terms were found to improve the predictive performance of our model (LR test P < 0.001).The addition of blood gas values (pCO2, pO2, pH, HCO3-) did not increase the predictive performance of our model, however increased the model complexity and were subsequently removed (Appendix A).
As a secondary analysis both models had a similar performance as the primary analysis for the outcomes of functional status at hospital discharge as a binary variable and for survival to hospital discharge (Appendices B and C).
The primary analysis was repeated after performing multiple imputation for missing data.There were only very minor differences noted between the imputed model and the original complete case model supporting the robustness of our analysis (Appendix D).

Discussion
In this analysis of prospectively collected, consecutive post-cardiac arrest patients we derived and internally validated an early clinical model to predict functional outcome at hospital discharge with excellent discrimination and calibration.The model performance was maintained when functional outcome was examined as a binary variable, for survival to hospital discharge, and with imputation for missing data.
There are a number of other prediction models that have been derived in the cardiac arrest literature. 20,21Few of the prediction models, however, have undergone rigorous external validation, and those that have suffer in their performance on external data. 20Our model development has some strengths compared to previous research.In order to be applied in clinical practice, clinical prediction rules must undergo rigorous methodological development.In this study we completed the initial derivation and internal validation of our prediction model on a large, research-quality prospective registry with pre-specification of model parameters using previous literature and clinical expertise.Our derivation was completed avoiding the use of traditional stepwise variable selection strategies which can result in overly optimistic models that perform poorly on external validation. 17We also avoided the use of treatment variables which can be subjective and vary over time with changes in practice, making prediction models obsolete.Pupillary response is one variable that may change over time, as the ability to assess pupillary response during the first 24 hours becomes easier as treatment moves from temperature management to fever control.More accurate assessment of pupillary response we feel may improve the performance of our model as this has been shown to have high prognostic perfor-mance post re-warming after TTM. 4,5Our model maintained its performance accuracy on internal validation as well demonstrating the robustness of our model and the potential for high predictive performance on future external validation.
A properly derived and validated clinical prediction rule may serve many purposes in resuscitation science.Risk stratification of patients in future cardiac arrest research may help to identify groups of patients who are more likely to benefit from specific interventions.Cardiac arrest patients are a heterogenous group where characteristics of the event and of the patient are associated with patient outcomes and poor overall survival.Further, ischemic injury during cardiac arrest depends on a number of factors such as underlying    health status, time until treatment, duration of cardiac arrest.Within the cardiac arrest population there are patients with extremely low probability of survival (such as those with asystole initial cardiac rhythm) in which treatment may not be beneficial.There is also a subset of patients (such as those with very short down time) in which the probability of a good outcome is high regardless of the treatment provided.There may be a third subset of patients which may have the largest degree of benefit from treatment.Enrolment of diverse cardiac arrest patients in clinical trials in which the majority of patients die may washout the effect of an intervention.3][24][25] A validated clinical prediction rule could be reliably applied by trialists to identify the patients who are more likely to benefit from the treatment.This would refine the trials at the point of enrolment based on their clinical prediction to have a favourable outcome.This approach may enable trialists to identify treatments that can improve outcomes for specific groups of cardiac arrest patients and perhaps contribute to reducing the number of neutral trials in resuscitation.
Second, risk stratification models can be used to compare riskadjusted mortality within, and across institutions for quality improvement, benchmarking, and reporting.Risk-adjusted outcomes could help to advance the science behind specialized cardiac arrest centres for post-cardiac arrest care.Further, it could be used for reporting purposes to compare across institutions for things such as quality improvement initiatives and funding for specialized centres, similar to how TIMI risk scores is used for risk-adjustment of cath labs for STsegment myocardial infarction (STEMI) patients. 26ird, risk stratification models could provide some direction at the bedside during the early post-ROSC period.While there are currently no prediction models that have consistently shown high enough accuracy to be used at the bedside for post-ROSC patients a highly accurate validated clinical prediction rule would provide guidance during the early post-cardiac arrest period to have conversations with families on patient prognosis.The gender bias towards withdrawing care early or choosing a less invasive approach in women post arrest could be addressed by a validated early clinical prediction rule. 27It would be easier to guide discussions with the family based on a predicted favourable outcome.It could also be used to help guide decisions on treatment that may be useful for subsets of post-cardiac arrest patients, or to inform decisions on the continuation of post-arrest management.In addition, something that has gained attention during the COVID-19 pandemic is the allocation of scarce resources.One difficulty during these times is determining how to use a finite number of resources for a large influx of ICU and critically ill patients.In order to properly triage resources during these extreme times it is critical to accurately determine the probability of a patient's outcome to allow for appropriate resource utilization for the greatest number of patients.

Limitations
There are some important limitations with our study.It is a retrospective cohort study and is limited by the use of previously collected data.As this is an observational design we are unable to determine cause and effect relationships between our variables and outcomes of interest.We had a high number of missing values in some key  variables of interest such as corneal reflexes and were unable to examine these variables in our model.While the model had excellent predictive performance there are still unknown variables that could improve the overall performance.Lastly, our model requires external validation prior to use.

Conclusion
Using a novel combination of intra-arrest and early post-cardiac arrest variables we were able to derive and internally validate a prediction model that had excellent predictive accuracy for predicting functional outcome at hospital discharge.

Funding
The

Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Fig. 2 -
Fig. 2 -Nomogram of final model.Note: CPR Status 0 = no bystander CPR; 1 = bystander CPR; 2 = EMS Witnessed/ CPR.Witness Status 0 = unwitnessed; 1 = witnessed Age and Resuscitation Duration are both nonlinear terms The nomogram is used by drawing a line from a given value in each predictor to the points line.The sum of all the points is then mapped to the Total Points axis.The linear predictor and the probability of survival can than be obtained from the corresponding axis.

Fig. 3 -
Fig. 3 -Model performance for functional outcome at hospital discharge using ordinal regression.

Table 1 -
Comparison of post-cardiac arrest patients by functional outcome at hospital discharge (n = 3355).

Table 2 -
Ordinal regression analysis for determining functional outcome in post-cardiac arrest patients (n = 2414).

Table 3 -
Summary performance statistics for original ordinal regression model and internal validation.
Resuscitation Outcomes Consortium Epistry study is supported by a cooperative agreement (5U01 HL077863) with the National Heart, Lung, and Blood Institute in partnership with the National Institute of Neurological Disorders and Stroke, Canadian Institutes of Health Research-Institute of Circulatory and Respiratory Health, Defense Research and Development Canada, Heart and Stroke Foundation of Canada, and American Heart Association.Rescu Epistry is funded by a centre grant from the Laerdal Foundation, and knowledge translation collaborative grants and operating grants from Canadian Institutes of Health Research and the Heart and Stroke Foundation of Canada.IRD was funded by the Canadian Institute of Health Research (CIHR) Banting and Best Doctoral Research Award.