Deep Learning to Estimate Left Ventricular Ejection Fraction From Routine Coronary Angiographic Images

Background Cine images during coronary angiography contain a wealth of information besides the assessment of coronary stenosis. We hypothesized that deep learning (DL) can discern moderate-severe left ventricular dysfunction among patients undergoing coronary angiography. Objectives The purpose of this study was to assess the ability of machine learning models in estimating left ventricular ejection fraction (LVEF) from routine coronary angiographic images. Methods We developed a combined 3D-convolutional neural network (CNN) and transformer to estimate LVEF for diagnostic coronary angiograms of the left coronary artery (LCA). Two angiograms, left anterior oblique (LAO)-caudal and right anterior oblique (RAO)-cranial projections, were fed into the model simultaneously. The model classified LVEF as significantly reduced (LVEF ≤40%) vs normal or mildly reduced (LVEF>40%). Echocardiogram performed within 30 days served as the gold standard for LVEF. Results A collection of 18,809 angiograms from 17,346 patients from Mayo Clinic were included (mean age 67.29; 35% women). Each patient appeared only in the training (70%), validation (10%), or testing set (20%). The model exhibited excellent performance (area under the receiver operator curve [AUC] 0.87; sensitivity 0.77; specificity 0.80) in the training set. The model’s performance exceeded human expert assessment (AUC, sensitivity, and specificity of 0.86, 0.76, and 0.77, respectively) vs (AUC, sensitivity, and specificity of 0.76-0.77, 0.50-0.44, and 0.90-0.93, respectively). In additional sensitivity analyses, combining the LAO and RAO views yielded a higher AUC, sensitivity, and specificity than utilizing either LAO or RAO individually. The original model combining CNN and transformer was superior to DL models using either 3D-CNN or transformers. Conclusions A novel DL algorithm demonstrated rapid and accurate assessment of LVEF from routine coronary angiography. The algorithm can be used to support clinical decision-making and form the foundation for future models that could extract meaningful data from routine angiography studies.

C ine angiography is the cornerstone of invasive cardiology practice.Images obtained from standard angiographic projections are utilized to determine the presence, extent, and characteristics of coronary disease.2][3] This includes possible information about ventricular contractile function, valvular and extracardiac calcifications, lung capacity, and diaphragmatic movement. 4However, noncoronary findings on cine angiography are usually discarded.This is due to the limited available data and the challenges of focusing on noncoronary findings in real time while evaluating coronary disease during routine patient care.
6][7][8] The automatic hierarchical feature extraction capability of CCNs allows them to discover intricate relationships between input and output. 9ditionally, a family of a novel DL framework called transformer has demonstrated exceptional performance in image and video analysis.[12] Recent advances in artificial intelligence provide an opportunity to extract additional information from data routinely acquired in the cath lab.Howard et al showed that DL can facilitate automatic detection of guide catheter dampening while recording aortic pressure waveforms and the prediction of catheterinduced coronary injuries. 13,14There are only a few studies to date that leverage routine coronary angiography images to screen for myocardial dysfunction. 15Therefore, we sought to address this knowledge gap using a large dataset of consecutive and transformers in parallel (Figure 1). Figure 2. The 3D-CNN architecture is a 3D version of the ResNet model, which is widely used for video classification. 16e ResNet solves some of the common issues connected with other architectures including the vanishing gradient problem. 17          present.This will be the focus of further collaborative multicenter studies.Second, the data were all derived from Mayo Clinic catheterization laboratories, which      Angio-derived LVEF

A
There are several versions of ResNet architecture with different numbers of layers including ResNet18, ResNet34, ResNet50, etc.In this study, the architecture with 152 layers, A B B R E V I A T I O N S A N D A C R O N Y M S AUC = area under the receiver operator curve CNN = convolutional neural networks DL = deep learning LAO = left anterior oblique LCA = left coronary artery LVEF = left ventricular ejection fraction MLP = multilayer perceptron RAO = right anterior oblique ResNet152, was utilized.In addition to the 3D CNN, a transformer architecture, TimeSformer, that was proposed in 2021 for video classification tasks was also implemented. 18During the training phase, for each study, 2 cine images were fed into separate 3D CNNs (Figure 1), while there was no weight sharing between them.The output of the 2 networks would be concatenated to create a new feature vector, which was fed into a simple multilayer perceptron (MLP) with a single hidden layer to generate the final output from the 3D CNN part.All 3D CNNs and the MLP model parameters were optimized simultaneously.The same approach was followed using 2 separate transformers to obtain the transformer part output.At the final stage of the classifier, the outputs of 3D CNNs and the transformers were concatenated and fed into a simple MLP to generate the final ensemble classifier's outcome.The final model's output is the predicted label, which is either positive (left ventricular ejection fraction [LVEF] #40%) or negative (LVEF >40%).The ground truth (labels) obtained based on the patients' LVEF as measured on a transthoracic echocardiogram done within 30 days in Mayo Clinic Echo Labs.The hypothesis was that the combination of the 3D-CNN and the transformer models would boost the overall performance.Class weights were assigned to the loss function to handle the imbalanced data issue.All the models were implemented using the Python programming language (version 3.7.11)and in PyTorch (version 1.7.1)framework.MODEL EVALUATION.After the training phase, the trained model was applied on the test set to evaluate the performance.The classification threshold value obtained from the validation samples was utilized during the test phase to predict the labels.Multiple measurements were used to evaluate the model's performance including area under the receiver operator curve (AUC), sensitivity, and specificity. 19To further evaluate the model's performance, the model output was compared to expert human performance on an additional test set.A subset of 290 angiograms (not included in the DL model angiogram) was randomly selected.Two experienced board-certified interventional cardiologists from Mayo Clinic's cath lab provided their prediction on the LVEF of the studies by a visual assessment of 2 views of each angiogram (LAO and RAO).Cardiologists carefully examined the videos and estimated the LVEF, considering the artery displacement and heart motion

FIGURE 1
FIGURE 1 Flow Chart and Model Architecture

FIGURE 2
FIGURE 2 Performance of the CNN, Transformer, and Ensemble Models

Figure 3
Figure 3 illustrates the model's performance with different LVEF thresholds, and Figure 4 shows the confusion metrics for each of the artificial intelligence models utilized.The performance of the artificial intelligence ensemble model was compared to human performance by applying the model on an additional test set.The DL-based algorithm generated superior outcomes when compared to human performance.The human performance was reported with AUCs of 0.76 to 0.77, sensitivity of 0.50 to 0.44, and specificity of 0.90 to 0.93 in comparison with the ensemble model's AUC of 0.86, sensitivity of 0.75, and specificity of 0.77 (Figures 5).Figures 6 and 7 illustrate the

Figures 6 and 7
illustrate the saliency maps from sample frames obtained in the LAO and ROA projections, respectively.DISCUSSION In this study, we examined the feasibility of extracting noncoronary data from routine angiograms using DL.Our hypothesis was that the minute displacement of the epicardial coronary arteries during myocardial contraction could be captured on fluoroscopy and processed with DL algorithms to infer LV contractile function.Our study confirmed the hypothesis and documented that a novel DL method combining 3D-CNN and transformers can discern reduced LVEF from 2 angiographic projects (Central Illustration).These findings have important clinical and investigational implications that deserve further discussion.Clinically, many patients present urgently to the cath lab with no prior echocardiographic examination.Often, those patients require complex high-risk coronary interventions.Knowledge of their LVEF could aid the interventionalist in real-time assessment of the risks and benefits of the intervention.For

FIGURE 3 3 Angio
FIGURE 3 Performance of the Ensemble Deep Learning Model for Various Left Ventricular EF Thresholds

FIGURE 4
FIGURE 4 Confusion Matrices for the CNN, Transformer, and the Ensemble Models

FIGURE 5
FIGURE 5 Performance of the Combined Ensemble Deep Learning Model vs Expert Human Assessment

J
A C C : A D V A N C E S , V O L . 2 , N O .9 , 2 0 2 3 discern these parameters has been limited to echocardiography and electrocardiography. 20-22 Third, the ability of DL to discern local displacement of coronary arteries can be further developed to study other features of the coronary angiogram itself that are understudied.For example, DL could quantify vessel tortuosity and assess its association with device success or clinical outcomes.Similarly, DL could also potentially assess the impact of ectasia or aneurysmal coronary disease on outcomes.STUDY LIMITATIONS.First, for the purpose of this exploratory study, we dichotomized the LVEF values into #40% vs >40%.In clinical practice, it is more meaningful to provide an actual estimated valve of the LVEF.However, conducting such a study would require a much larger sample of angiogram and computational capabilities that are not available at

Using 2
orthogonal views (LAO caudal and RAO cranial) from >17,000 patients who had recent echocardiographic assessment of LV function, a novel deep learning model incorporating convolutional neural network and transformer approaches to identify patients with reduced LV ejection fraction.The models had the best performance (AUC 0.87) for classifying LVEF as >40% or #40%.AUC ¼ area under the curve; CNN ¼ convolutional neural networks; EF ¼ ejection fraction; LAO ¼ left anterior oblique; LVEF ¼ left ventricular ejection fraction; RAO ¼ right anterior oblique.

J
A C C : A D V A N C E S , V O L . 2 , N O .9

J
A C C : A D V A N C E S , V O L . 2 , N O .9 Sample Right Anterior Oblique Cranial Frames With Their Corresponding Saliency Maps FIGURE 6 Sample Left Anterior Oblique Caudal Frames With Their Corresponding Saliency Maps CENTRAL ILLUSTRATION Deep Learning for Left Ventricular Function Assessment on Routine FIGURE 7 Bertasius G, Wang H, Torresani L. Is space-time attention all you need for video understanding?arXiv.2021.https://doi.org/10.48550/arXiv.2102.05095 19.Fawcett T.An introduction to ROC analysis.Pattern Recogn Lett.2006;27:861-874.
TRANSLATIONAL OUTLOOK: Operators could assess left ventricular function in real time in the catheterization laboratory.Future models can also be developed to obtain other meaningful data from the angiogram.18.