Surgical Mortality Risk Scores in Transcatheter Aortic Valve Implantation: Is Their Early Predictive Value Still Strong?

Background: Surgical mortality risk scores, even if not properly designed and rarely tested in the transcatheter aortic valve implantation (TAVI) setting, still guide the heart team in managing significant aortic stenosis. Methods: After splitting 1763 consecutive patients retrospectively based on their mortality risk thresholds, the composite endpoint early safety (ES) was adjudicated according to Valve Academic Research Consortium (VARC)-2 and -3 consensus documents. Results: ES incidence was higher if VARC-2 rather than VARC-3 defined. Despite only patients showing VARC-2 ES had significantly lower absolute values of all three main risk scores, these last still failed to foresee both VARC-2 and -3 ES in intermediate-risk patients. The receiver operating characteristic analysis also showed a significant correlation, but with poor diagnostic accuracy, among the three scores and only VARC-2 ES; moreover, the absence of VARC-2 ES and low-osmolar contrast media administration were identified as independent predictors of 1-year mortality and absence of VARC-3 ES, respectively. Finally, even a single complication included in the ES definition could significantly affect 1-year mortality. Conclusion: Currently, the most used mortality risk scores do not have adequate diagnostic accuracy in predicting ES after TAVI. The absence of VARC-2, instead of VARC-3, ES is an independent predictor of 1-year mortality.


Introduction
Aortic stenosis is the most common primary valve disease in Europe and the U.S.A., whose gold standard treatment has been surgical aortic valve replacement (SAVR) for decades. Despite at its outset, transcatheter aortic valve implantation (TAVI) was considered an attractive, less invasive treatment option only for high surgical risk or selected inoperable patients, randomized controlled trials, such as PARTNER 2 [1], PARTNER 3 [2], the OBSERVANT study [3], NOTION [4], and Evolut Low Risk [5], have recently

Study Population
This multicenter observational study assessed all consecutive patients who underwent TAVI at five Italian heart centers (Policlinico University Hospital, "Anthea" Clinic and "Mater Dei" Hospital of Bari, "V. Fazzi" Hospital of Lecce, and "Montevergine" Clinic of Mercogliano) involved in the "Magna Graecia" TAVI registry.
Between March 2011 and September 2021, 1763 consecutive patients (982 females, mean age 80.94 ± 5.73, 1667 transfemoral access) suitable for TAVI were enrolled. They underwent preprocedural assessment with transthoracic echocardiography; coronary angiography; computed tomography scan of the heart, aorta, and peripheral arteries; pulmonary function testing (if necessary); carotid artery ultrasonography; and multidisciplinary evaluation by the Valve Team. Most procedures were performed in a standard cardiac catheterization laboratory with the support of anesthesiology and surgical backup by experienced operators. Iodixanol was the only iodinated iso-osmolar CM administered; the other low-osmolar contrast media (LOCM) used for the procedure were iopromide, iobitridol, iohexol, and iomeprol. The deployed valves were balloon-expandable (Edwards Sapien XT and Sapien 3; Meril Myval), self-expanding (Medtronic CoreValve, Engager, Evolut R, and Evolut PRO; Boston Acurate and Acurate neo; Abbott Portico; JenaValve), and others (Boston Lotus; Direct Flow Medical).
Each participating site maintains a prospective database of all TAVI patients treated at that center using the same dedicated archiving software. All baseline demographics; clinical, laboratory, echocardiographic, intraprocedural, and postprocedural data; and hospital outcomes were collected from each patient's health record; pre-TAVI logistic EuroSCORE, EuroSCORE II, and STS-PROM score were prospectively calculated online using the official websites and calculators based on previously published data, whereas the analysis was performed retrospectively.
Patients' population was retrospectively split according to the most widespread mortality risk thresholds [7]: low if logistic EuroSCORE, EuroSCORE II and STS-PROM were <10%, <4%, and <4%, respectively; intermediate if they were 10-20%, 4-8%, and 4-8%, respectively; and high if they were ≥20%, ≥8%, and ≥8%, respectively. All the adverse events and the ES composite endpoint were also re-adjudicated retrospectively, by an external committee of interventional cardiologists, according to both VARC-2 and VARC-3 criteria [16,17]. The clinical outcomes analyzed and compared included observed and predicted absence of ES; this last was calculated using the cumulative score percentage of each risk model divided by the number of subjects in the study.

Statistical Analysis
Statistical analysis was performed using SigmaStat 3.5, SPSS 25.0, and STATA 13.0 software. Continuous variables were expressed as the mean ± standard deviation and median (interquartile ranges) of absolute numbers; categorical variables were expressed as frequencies and percentages. As appropriate, comparisons were made by t-test, Mann-Whitney's U-test, Fisher's exact test, or χ 2 test. The normal distribution was assessed with Kolmogorov-Smirnov tests. A receiver operating characteristic (ROC) curve analysis was performed to establish the threshold levels of the mortality risk scores that provided the best cutoff for the absence of ES according to VARC-2 and VARC-3 definitions, but also to establish the minimum number of complications needed to cause 1-year mortality after TAVI. Area under the curve (AUC) values were calculated with confidence intervals (CIs) through concordance statistics to measure test accuracy. The DeLong test was used to identify AUC standard errors. The calibrations of these mortality risk models were evaluated by comparing the mean predicted probability and the mean observed frequency of absence of ES with goodness-of-fit R-squared and Cochran-Armitage tests, calibration plots, and estimation of a calibration slope. After this, new optimal cutoff points for the absence of ES were selected using Youden's tests, reporting Youden's indexes; we evaluated sensitivity and specificity according to these new cutoff points. All AUC of mortality risk scores were then compared using the ROC-regression test. Finally, the absence of ES and 1-year mortality predictors was tested in a univariable logistic regression model; all variables with a p-value < 0.05 at univariable regression were tested for multicollinearity in a stepwise multivariable model. Only variables with a variance inflation factor <4 were incorporated in the multivariable logistic regression model. The odds ratios (ORs) with 95% CIs were estimated. All statistical tests were two-sided. For all tests, a p-value < 0.05 was considered statistically significant.

Patient Characteristics
A total of 1515 (85.9%) and 1050 (59.6%) patients presented VARC-2-and VARC-3defined ES, respectively; Supplementary Table S1 and Supplementary Figure S1 describe the distribution of the several component adverse events if the absence of ES was adjudicated with VARC-2 or VARC-3 criteria. All clinical, preprocedural, procedural, and postprocedural data of the study population are shown in Table 1 and Supplementary Table S2.
Body surface area (BSA) and peripheral artery disease (PAD) had a significant impact on the absence of VARC-2 ES (p = 0.018 and p = 0.008, respectively), whereas the absence of VARC-3 ES occurred more frequently in older (p = 0.03) patients already implanted with a permanent PM (p = 0.05); no significant differences in sex were observed among ES and no-ES patients (p = 0.310 for VARC-2 ES, p = 0.872 for VARC-3 ES). Absence of both VARC-2 and VARC-3 ES was detected more in patients with lower baseline values of creatinine clearance (p = 0.019 and p = 0.015, respectively) and a higher rate of critical preoperative state (p = 0.003 and p = 0.002, respectively). ES and no-ES patients did not differ significantly with respect to other baseline clinical characteristics. Moreover, some procedural details, such as the postdilation (p < 0.001, for both VARC-2 and VARC-3 ES) of a bioprosthesis other than balloon-expandable (p = 0.044 for VARC-2 ES, p < 0.001 for VARC-3 ES), the fluoroscopy time (p < 0.001 for VARC-2 ES, p = 0.018 for VARC-3 ES), the radiation dose (p = 0.007 for VARC-2 ES, p = 0.037 for VARC-3 ES), and the contrast medium (CM) amount (p < 0.001, for both VARC-2 and VARC-3 ES), were all significantly associated with the absence of ES. VARC-2 ES occurred less frequently in the case of valve-in-valve implantation (p = 0.014), whereas no VARC-3 ES patients were more likely to receive a small-sized (p < 0.001) self-expanding bioprosthesis (p < 0.001), administering LOCM (p < 0.001).
Concerning complications and outcomes, patients without both VARC-2 and VARC-3 ES exhibited a higher rate of new-onset atrial fibrillation (AF) (p < 0.001 for VARC-2 ES, p = 0.003 for VARC-3 ES) and mostly needed permanent PM implantation (p = 0.016 for VARC-2 ES, p < 0.001 for VARC-3 ES) after TAVI. Besides a shorter hospital stay observed in ES patients (5.27 vs. 8.48 days for VARC-2 ES, 4.67 vs. 7.17 days for VARC-3 ES, p < 0.001 for both), the occurrence of such early composite endpoints was associated with lower 1-year mortality, too (p = 0.002 for VARC-2 ES, p = 0.025 for VARC-3 ES). Table 1 also describes the relation between the three surgical mortality risk scores and the absence of ES. As absolute values, these scores were significantly lower (p < 0.001) in the VARC-2 ES group; in fact, we found that a significant proportion of patients at higher surgical mortality risk did not develop VARC-2 ES compared with those at lower risk. On the contrary, no statistical difference was detected between patients with or without VARC-3 ES in terms of absolute values of the logistic EuroSCORE (p = 0.334), EuroSCORE II (p = 0.097), and STS-PROM (p = 0.177). Notwithstanding, after splitting the population according to its operative risk profile, the aforementioned significant correlation between the 3 scores and the absence of VARC-2 ES was maintained only in low-and high-risk  Table 1 also describes the relation between the three surgical mortality risk scores and the absence of ES. As absolute values, these scores were significantly lower (p < 0.001) in the VARC-2 ES group; in fact, we found that a significant proportion of patients at higher surgical mortality risk did not develop VARC-2 ES compared with those at lower risk. On the contrary, no statistical difference was detected between patients with or without VARC-3 ES in terms of absolute values of the logistic EuroSCORE (p = 0.334), EuroSCORE II (p = 0.097), and STS-PROM (p = 0.177). Notwithstanding, after splitting the population according to its operative risk profile, the aforementioned significant correlation between the 3 scores and the absence of VARC-2 ES was maintained only in low-and high-risk  Moreover, the ratio of observed-to-expected absence of VARC-2 ES (with expected risk calculated on the basis of logistic EuroSCORE, EuroSCORE II, and STS-PROM models) was high for all scores, being 0.890, 0.918, and 0.953, respectively; these scores also correctly estimated the absence of VARC-3 ES with observed-to-expected ratios of 0.992, 0.996, and 0.994, respectively. The ROC analysis also showed a significant correlation between these scores and the absence of ES, only if defined according to the VARC-2 consensus document [15] (Figure 2), but without a significant difference among them (p = Moreover, the ratio of observed-to-expected absence of VARC-2 ES (with expected risk calculated on the basis of logistic EuroSCORE, EuroSCORE II, and STS-PROM models) was high for all scores, being 0.890, 0.918, and 0.953, respectively; these scores also correctly estimated the absence of VARC-3 ES with observed-to-expected ratios of 0.992, 0.996, and 0.994, respectively. The ROC analysis also showed a significant correlation between these scores and the absence of ES, only if defined according to the VARC-2 consensus document [15] (Figure 2), but without a significant difference among them (p = 0.194). Nevertheless, based on the AUC of the new cutoff values established with the higher Youden's indexes, none of the risk scores was significantly performant in detecting the absence of both VARC-2 (logistic EuroSCORE: AUC 0.595, 95% CI 0.571-0.618, sensitivity 50%, specificity 70%, p < 0.001; EuroSCORE II: AUC 0.578, 95% CI 0.554-0.601, sensitivity 63%, specificity 51%, p < 0.001; STS-PROM: AUC 0.575, 95% CI 0.552-0.558, sensitivity 66%, specificity 46%, p < 0.001) and VARC-3 ES (logistic EuroSCORE: AUC 0.514, 95% CI 0.490-0.538, sensitivity 30%, specificity 76%, p = 0.334; EuroSCORE II: AUC 0.523, 95% CI 0.499-0.547, sensitivity 53%, specificity 51%, p = 0.097; STS-PROM: AUC 0.519, 95% CI 0.495-0.543, sensitivity 33%, specificity 75%, p = 0.177).
In the end, ROC analysis was also performed to evaluate the correlation between the number of complications included in the composite endpoint ES and 1-year mortality: depending on the newly established cutoff value (1 ± 0.062), even a single complication could significantly affect 1-year mortality, with sufficient predictive performance (AUC 0.646, 95% CI 0.624-0.669, sensitivity 53%, specificity 86%, p < 0.001) ( Table 2).

Discussion
Despite early and late outcomes also depending on several TAVI-specific factors, including patient frailty, experience of the heart team, prosthesis type, procedural volume, and learning curve, surgical mortality risk scores are still the most used to predict them. The main findings of our study were (1) the accuracy of the commonly used risk scores in predicting TAVI-related ES is not as efficient as expected; (2) after splitting the population according to risk scores' thresholds, VARC-2 and VARC-3 ES cannot be foreseen for the intermediate-risk patients; (3) absence of VARC-2, instead of VARC-3 ES, is an independent predictor of 1-year mortality; (4) each adverse event included in the composite endpoint ES, by itself, can have a significant influence, without an additive effect, on 1-year mortality.

Early Safety
After the new definition of ES in the last updated consensus document [16], ours is the largest TAVI cohort in which this early composite endpoint, defined with both VARC-2 and VARC-3 criteria, has been analyzed. Our incidence of ES, if VARC-2 defined, is in line with data reported in the literature [19], and its absence was mostly related to lifethreatening bleedings and major vascular complications. Conversely, type 2-4 bleedings; major vascular, access-related, or cardiac structural complications; moderate-to-severe aortic regurgitation; and permanent PM implantation were the most frequently responsible for the absence of VARC-3 ES: the lower incidence of VARC-3 ES with respect to VARC-2 ES could be properly interpreted as being the last definition of this composite endpoint much more inclusive of complications.
To date, other studies have used the VARC-3 consensus document to adjudicate events: our data confirm what Costa et al. [20] have already highlighted in a small cohort of patients about the higher rate of the composite endpoint ES after balloon-expandable prostheses implantation. Moreover, another important known contributor to reducing VARC-2 ES occurrence is the administration of large amounts of LOCM: in fact, in line with what was previously stated in the literature [21,22], low instead of iso-osmolality of CM seems to have an unfavorable effect on renal function, raising acute kidney injury incidence. Notwithstanding, only osmolality, rather than CM volume, seems to be an independent VARC-3 ES predictor in our TAVI population.

Surgical Mortality Risk Scores and Absence of Early Safety
Consistent with its main aim, this study provides for the first time an original use of currently adopted surgical mortality risk scores to predict the absence of ES after TAVI, as these scores have only been tested so far to predict 1-month and 1-year mortality in this setting, with limited diagnostic accuracy [13]. Although only the STS-PROM score seemed to foresee 1-month mortality [22], some authors have already demonstrated its lack of discriminatory power and no difference in prediction accuracy compared to other scores [23,24]. In our cohort, all the risk scores did not demonstrate significant differences between the predicted and observed absence of VARC-2 and VARC-3 ES. However, although all patients who failed to achieve VARC-2 ES had significantly higher absolute values of the three risk scores, these last showed limited diagnostic accuracy, as their AUC was never above 0.6; if ES was adjudicated according to the VARC-3 definition, the diagnostic accuracy of the 3 mortality risk scores was even lower, as their AUC was never above 0.5. All these findings are almost certainly because such risk scores have been developed and validated in different settings. In fact, they showed adequate diagnostic accuracy in predicting operative mortality after isolated SAVR [25].
Finally, no study has analyzed the predictive performance of such risk scores based on the preprocedural risk category, i.e., low, intermediate, or high. Instead, our lowand high-risk patients showed a significant association between their surgical risk scores and the absence of VARC-2 ES. In contrast, no significant correlation was found in any preprocedural risk category if the absence of ES was VARC-3-defined.

One-Year Mortality Predictors
An attempt to identify predictors of 1-year mortality after TAVI was made, too. Despite the findings of the small study by Compagnone et al., the STS-PROM score was the only score able to stratify long-term all-cause and cardiovascular mortality, and EuroSCORE II was the only risk score significantly related to the 1-year mortality of our population. Nevertheless, we agree with other studies about the lower performance of these scores in predicting long-term events in the post-TAVI rather than post-SAVR setting [13,26], leading to the urgent need for a specific TAVI-tailored risk score [24].
According to our analysis, other procedural or postprocedural features can have a negative impact on 1-year mortality, first of all, and in line with the literature [27], selfexpanding technologies, probably due to their deployment characteristics. In addition, new-onset AF has been identified as another independent predictor of 1-year mortality, too. In fact, AF is independently associated with an almost twofold increased risk of all-cause mortality [28]; such a novel finding could be likely related to the higher frailty of AF patients, their anticoagulation need, and their higher risk of thromboembolic events.
Finally, even the absence of VARC-2 ES has been shown to be an independent predictor of 1-year mortality. This information can properly be explained by the possible limits of new VARC-3 criteria: the more complications are included in this composite endpoint, the more its long-term predictive performance will decrease. In fact, the newsworthy twist of our analysis is that adverse events do not have an additive effect in influencing 1-year mortality: a single adverse event is enough to affect 1-year mortality.

Study Limitations
Although it was obtained from a prospectively collected database, this is an unspecified post hoc analysis. Therefore, we cannot exclude that potential confounding factors not considered in the model may have influenced the results. The effect of a learning curve and changes in treatment strategy is also heterogeneous, as the study spanned more than a decade, during which several updates to the STS-PROM score have also been conducted. Furthermore, we believe that aspects of management not controlled or specified may have been a source of bias. Finally, an independent committee did not adjudicate all clinical events that were site-referred.

Conclusions
Unfortunately, available surgical risk scores have demonstrated limits in predicting both short-and long-term mortality and the absence of ES after TAVI, especially in intermediate-risk patients, who account for the largest proportion of patients currently undergoing this transcatheter procedure. Defining the composite endpoint ES according to the most recent VARC-3 consensus document significantly reduced its incidence, probably due to the inclusion of additional frequent adverse events, which have also been shown to influence 1-year mortality.
Moreover, EuroSCORE II and the absence of VARC-2, instead of the absence of VARC-3 ES, are independent predictors of 1-year mortality, albeit LOCM administration is the only independent predictor of the absence of VARC-3 ES, confirming the paramount importance of choosing an iso-osmolar contrast medium also in this setting of patients.
Nevertheless, until a better performing TAVI-dedicated risk score is created, individual clinical judgment in a heart team approach should be considered the fundamental weapon to stratify TAVI patients in a precise and detailed manner.