Artificial intelligence based prediction model of in-hospital mortality among females with acute coronary syndrome: for the Jerusalem Platelets Thrombosis and Intervention in Cardiology (JUPITER-12) Study Group

Introduction Despite ongoing efforts to minimize sex bias in diagnosis and treatment of acute coronary syndrome (ACS), data still shows outcomes differences between sexes including higher risk of all-cause mortality rate among females. Hence, the aim of the current study was to examine sex differences in ACS in-hospital mortality, and to implement artificial intelligence (AI) models for prediction of in-hospital mortality among females with ACS. Methods All ACS patients admitted to a tertiary care center intensive cardiac care unit (ICCU) between July 2019 and July 2023 were prospectively enrolled. The primary outcome was in-hospital mortality. Three prediction algorithms, including gradient boosting classifier (GBC) random forest classifier (RFC), and logistic regression (LR) were used to develop and validate prediction models for in-hospital mortality among females with ACS, using only available features at presentation. Results A total of 2,346 ACS patients with a median age of 64 (IQR: 56–74) were included. Of them, 453 (19.3%) were female. Female patients had higher prevalence of NSTEMI (49.2% vs. 39.8%, p < 0.001), less urgent PCI (<2 h) rates (40.2% vs. 50.6%, p < 0.001), and more complications during admission (17.7% vs. 12.3%, p = 0.01). In-hospital mortality occurred in 58 (2.5%) patients [21/453 (5%) females vs. 37/1,893 (2%) males, HR = 2.28, 95% CI: 1.33–3.91, p = 0.003]. GBC algorithm outscored the RFC and LR models, with area under receiver operating characteristic curve (AUROC) of 0.91 with proposed working point of 83.3% sensitivity and 82.4% specificity, and area under precision recall curve (AUPRC) of 0.92. Analysis of feature importance indicated that older age, STEMI, and inflammatory markers were the most important contributing variables. Conclusions Mortality and complications rates among females with ACS are significantly higher than in males. Machine learning algorithms for prediction of ACS outcomes among females can be used to help mitigate sex bias.


Introduction
Although mortality associated with Acute Coronary Syndrome (ACS) has decreased in recent years thanks to improvements in prevention as well as better pharmacologic and interventional therapies, ACS and Ischemic Heart Disease (IHD) continue to be a major cause of death and disability (1)(2)(3).Recent epidemiological studies point out that the burden of this syndrome is increasing with more than 7 million people diagnosed with ACS annually worldwide (4).
Sexbiases in ACS have received increasing attention in recent decades, with numerous studies reporting significant sex-based differences in diagnosis, management, and outcomes of ACS patients (5-9).Contemporary data demonstrates that in-hospital mortality rates and the risk of recurrent cardiovascular events are higher among females with ACS when compared with males (10-12).Several factors contribute to this disparity, including increased time between symptom onset and diagnosis (13), less aggressive treatment upon diagnosis (14), and poorer in-hospital quality of care (15).
Artificial Intelligence (AI) algorithms have revolutionized healthcare by addressing a wide range of challenges, particularly in predictive tasks (16,17).The use of AI in cardiology has been increasingly prominent, encompassing the prediction of cardiovascular disease outcomes, non-invasive diagnostics, and identification and risk assessment of life-threatening conditions (18,19).Hence, we sought to investigate and report sex disparities in the management and outcomes of ACS patients at a tertiary care medical center using an AI-based algorithms.The aim of the current trial was to provide a proof of concept for the use of AI algorithms that are specifically designed to predict in-hospital mortality among females with ACS, and to highlight their possible role in reducing sex bias among this population of patients.

Study population
All patients diagnosed with ACS who were admitted to a tertiary care intensive cardiac care unit (ICCU) at Shaare Zedek Medical Center between July 2019 and July 2023 were prospectively recruited.The diagnosis of ACS was based on clinical symptoms of myocardial ischemia, with or without new ECG ischemic changes, and with or without acute elevation in high-sensitivity troponin I (hs-cTnI) concentrations, according to the ESC guidelines for ACS (20).

Data collection
Data were anonymously documented in the ICCU by the local coordinator and prospectively submitted into an electronic case report form (eCRF). Data were checked for accuracy and out-ofrange values by the coordinating unit.Demographic data, presenting symptoms, comorbid conditions, physical examination, and laboratory data were systematically recorded.
The Institutional review board approved the study based on strict maintenance of participants' anonymity by de-identifying during database analysis.No individual consent was obtained.Moreover, the authors have no conflicts of interest to declare.No funding was applied to the study.All methods were performed in accordance with the relevant guidelines and regulations.

Study outcomes
The primary outcome was in-hospital mortality that was recorded as an outcome for every ACS patient.

Development and evaluation of machinelearning models
For the development of the model, only variables available at patient presentation were included, so features like culprit vessel and angiographic results were not used for model construction.The entire variables were eligible for selection in the predictive models, and no feature selection method was applied before the models training.Variables that contained missing values were not included in the analysis and are not reported here, all of the reported variables did not had missing values.The cohort of female patients with ACS was partitioned into distinct nonoverlapping sets, with 70% of patients allocated to the trainingvalidation set, and 30% assigned to the test set.The trainingvalidation set was used for training and optimization using 5-fold cross validation.The selection of patients for each set was conducted randomly.To address the imbalance between labels, we down-sampled the training-validation set of patients who did not experience mortality during admission.All models development and parameter selection procedures were carried out exclusively on the training and validation sets, and the ultimate performance of the final model was reported based on the imbalanced test set.Our analysis includes three prediction algorithms, including gradient boosting classifier (GBC), random forest classifier (RFC), and logistic regreesion (LR).The models were optimized through a Bayesian optimization process on a set of model-specific parameters.The optimization process was carried out using a 5fold cross-validation technique, and the best iterations were selected based on the mean area under the curve of the receiver operating characteristics curve (AUROC).We further evaluated the models based on various prediction scores including sensitivity, specificity, positive predictive value (PPV), and area under precision recall curve (AUPRC).In addition to the ROC curve, we plotted the PPV against the sensitivity (precision-recall curve).This curve enabled us to assess the clinical utility and added value of the proposed model.In order to highlight the features that influence the forecasts generated by the GBC, SHAP values (21) were calculated.SHAP values delineate the decomposition of prediction outcomes for each individual sample into the contributions attributable to distinct constituent feature values.This decomposition process is achieved through the estimation of variations between models built upon subsets of the feature space.Through the process of sample-wise averaging, SHAP values provide an assessment of the impact of each feature on the aggregate model predictions.The predictive model was developed, validated, and evaluated using Python programming language version 3.6 (Python Software Foundation).

Statistical analysis
Continuous variables were expressed as mean ± standard deviation if normally distributed or median with interquartile range if skewed.Categorical variables were presented as frequency (%).Continuous data were compared with the Student's t-test and Mann-Whitney test for comparison of normally and non-normally distributed continuous variables, respectively.Categorical data were compared with the use of the chi-square test or Fisher exact test.
All statistical analyses were performed using R software version 3.4.4(R Foundation for Statistical Computing).An association was considered statistically significant for a two-sided P value of less than 0.05.

Interventions and complications during ICCU admission
Procedures that were performed during the ICCU admission course are reported in Table 2. Percutaneous coronary intervention (PCI) was performed in 1,863 (80%) patients, coronary angiography without intervention was performed in 346 (14.7%), coronary artery bypass grafting (CABG) was performed in 79 (3.4%) patients, and conservative therapy alone was assigned to only 58 (2.5%) of ACS patients.Stratification by sex demonstrated that female patients were treated more conservatively with lower urgent PCI (<2 h) rates (40.4% vs. 50.6%,p < 0.001).Rates of usage of more advanced therapies such as mechanical ventilation, intra-aortic balloon pump (IABP), Impella, and extra-corporeal membrane oxygenation (ECMO) were similar between sexes.The overall complication rate Bar plot of ACS cases by subtype and Sex.This bar plot demonstrates the relative portion of female patients in each of the subtypes of ACS.ACS, acute coronary syndrome; F, females; M, males; NSTEMI, non-ST segment elevation myocardial infraction; STEMI, ST segment elevation myocardial infraction; UAP, unstable angina pectoris; p value for the difference between sexes in STEMI < 0.001, in NSTEMI = 0.001, and in UAP = 0.52.

In-hospital mortality and models performance
In-hospital mortality was observed in 58 (2.5%) patients.The mortality rate was found to be higher among females as compared with males (5% vs. 2%, respectively, HR = 2.28, 95% CI: 1.33-3.91,p = 0.003) as presented in Figure 2.Each of the three algorithms wereevaluated on the unseen test set.The GBC outperformed the other two algorithms with AUROC of 0.91 and an optimal operational threshold affording 83.3% sensitivity and 82.4% specificity.Figure 3 illustrates the ROC curves, complete with the corresponding AUROC values.Additionally, Figure 4 presents the precision-recall curves (PRC), illustrating the Positive Predictive Value (PPV) against sensitivity.The GBC outscored RFC and LR in this parameter as well, yielding an AUPRC of 0.92.The ranking of the GBC's most influential features is summarized in Figure 5. Notably, advanced age, presentation with STEMI, evidence of diminished nutritional status (as evidenced by low serum albumin levels), and elevated inflammatory markers, were found to be strong indicators for predicting in-hospital mortality in female ACS patients.Furthermore, elevated levels of high-sensitivity cardiac troponin I (hs-cTnI), reduced serum hemoglobin levels, and heightened lactate levels were also identified as significant contributing factors.

Discussion
Our analysis offers several important findings: First, it uses contemporary data to confirm and expand upon previous observations concerning sex disparities regarding outcomes of ACS patients.Second, this study demonstrates the potential of AI-based prediction models to mitigate these biases by providing an accurate risk estimator for sex-dependent outcomes such as in-hospital mortality.Lastly, this analysis provides an explainability layer with the use of SHAP values, which allows  Bar plot of ACS cases by Sex and mortality Status.This bar plot demonstrates the relative portion of in-hospital mortality in each sex group, highlighting the unproportional death rates within the females subgroup.ACS, acute coronary syndrome; F, females; M, males; Unadjusted HR for sex: HR = 2.28, 95% CI: 1.33-3.91,p = 0.003.for the detection of important contributing variables for in-hospital mortality among females with ACS.
There are numerous sex-based differences in ACS patients.These range from basic biological features (5) (i.e., epicardial coronary artery diameter, myocardial blood flow, and estrogendependent endothelial mediators), as well as clinical features including risk factor profiles (7, 22), and clinical presentation and outcomes (5, 10-12).Hao et al. ( 6) studied sex differences in acute management, medical therapies, and in-hospital mortality in a large cohort from China.They found that females with ACS were less likely to receive evidence-based therapie than males, including reperfusion therapy.In a comprehensive review that focused on sex differences in patients with ACS in the current era (8), the researchers demonstrated higher prevalence of certain complications among females following ACS events, that included cardiogenic shock, bleeding, and post-discharge mortality.Our study findings further support the above investigations by showing that females with ACS received less aggressive treatment, most notably lower rates of urgent PCI (<2 h), and had higher rates of in-hospital complications and mortality.In our study there are several baseline characteristics that differ between males and females.The most notable difference is the older age of females compared to males, which provides a reasonable explanation for the lower rates of invasive treatment, and the higher rates of complications and in-hospital mortality between the two groups.
To the best of our knowledge, this is the first study to develop and train a machine-learning model for the prediction of in-  Receiver operating characteristic curves (ROC).Receiver operating characteristic (ROC) curves of the machine learning models on the selected variable set.This plot illustrates the performance of the models, including sensitivity, false positive rate, and AUROC.AUROC, area under the curve of receiver operating characterisitc curve; GBC, gradient boosting classifier; LR, logistic regression; RFC, random forest classifier.hospital mortality exclusively for females with ACS.Prior studies have constructed models for the prediction of in-hospital mortality among all ACS patients (23,24).The utilization of explainability methods for the exploration of feature importance in the suggested model serves as further validation of our results.Older age and ST-segment elevation are well known risk factors for in-hospital mortality, which have previously been validated in several studies, including the most commonly used risk score for in-hospital mortality in ACS, the Global Registry of Acute Coronary Events (GRACE) (25)(26)(27)(28)(29). Wenzel et al. (29) developed the GRACE 3.0 score on over 400,000 patients by utilizing the GRACE parameters and applying machine learning algorithms for the prediction of inhospital mortality, reporting and AUC of 0.91 and 0.87 for males and females with NSTEMI, respectively.Herein, we have evaluated a much smaller number of patients from a single center, but included not only NSTEMI patients but also STEMI and UAP.Moreover, we have used a variety of features in order to predict the desired outcome and not only the factors from the original GRACE score.Importantly, our study has confirmed older age and ST-segment elevation to be strong predictors of inhospital mortality in females.Diabetes mellitus and arterial hypertension are two comorbidities that have been proposed in the Thrombolysis in Myocardial Infraction (TIMI) risk score for STEMI (27), and were also identified by our feature importance analysis as key predictors of in-hospital mortality.Interestingly, our analysis revealed the significance of other non-overlapping features linked to inflammatory markers including immature platelets fraction (IPF) (30), white blood cells (WBC) count, and D-dimer levels.These features have been associated in prior investigations with ACS pathogenesis and outcomes (31-33).Interestingly, elevated TSH and lower albumin levels were also among the most influential factors.These factors are not usually taken into account when discussing ACS prognosis but have previously reported as having prognosting implications (34,35).Albumin and TSH are disturbed in numerous severe diseases, a situation which reflects both the poor baseline of the patients as well as an adaptation reaction for the disease state.
An important obstacle to implementing machine learning prediction algorithms in healthcare is clinician skepticism, largely because these algorithms are often not transparent.The explanatory analysis adds substantial value due to its ability to bridge this gap, making it easier for healthcare professionals to use these models and integrate them into operational healthcare systems.

Study limitations
Our study has several limitations: (1) it was conducted in a single tertiary-care ICCU, with all its inherent limitations including referral bias.Our proposed model is based on data from our ICCU and currently lacks external validation.(2) Our analysis was based on overall in-hospital mortality rather than cardiovascular mortality.Though mortality statistics in Israel closely resemble those of the European Union, where cardiovascular death is the second most prevalent cause of death following cancer (36).(3) There may be unmeasured laboratory and clinical variables that could have been used to Precision recall curves (PRC).Precision Recall Curves (PRC) of the machine learning models on the selected variable set.This plot illustrates the performance of the models, including precision [positive predicted value (PPV)], recall (sensitivity), and AUPRC.AUPRC, area under the curve of precision recall curve; GBC, gradient boosting classifier; LR, logistic regression; RFC, random forest classifier.
enhance the performance of our prediction model, including the time elapsed between symptom onset to ACS diagnosis, Killip class, NYHA functional class, and BNP.(4) While independent associations have been demonstrated, causality could not be established due to study design, hence the utilization of the proposed model in real-life clinical practice demands further prospective work.

Conclusion
While significant efforts have been made to mitigate sex biases in the management and outcomes of patients with ACS, contemporary data indicate the persistence of such disparities.In our study, we have demonstrated the performance of an AI model in predicting in-hospital mortality among female ACS patients.Additionally, we have conducted a comprehensive feature importance analysis, highlighting the key contributing factors to this unfavorable outcome.Our study provides a proof of concept regarding the possible role of AI algorithms in reducing sex bias among females with ACS.By incorporating this model into the early stages of ACS management for female patients, we envision a potential pathway for addressing the disproportionate mortality rates experienced by this specific demographic with the aim of improving outcomes.Further prospective studies together with external validation are warranted to explore the practical application of this model in real-world healthcare settings, and to evaluate its potential role in combating sex biases in the management and outcomes of females with ACS.

TABLE 2
Interventions during admission.

TABLE 3
Complications during admission.