A machine learning algorithm for peripheral artery disease prognosis using biomarker data

Summary Peripheral artery disease (PAD) biomarkers have been studied in isolation; however, an algorithm that considers a protein panel to inform PAD prognosis may improve predictive accuracy. Biomarker-based prediction models were developed and evaluated using a model development (n = 270) and prospective validation cohort (n = 277). Plasma concentrations of 37 proteins were measured at baseline and the patients were followed for 2 years. The primary outcome was 2-year major adverse limb event (MALE; composite of vascular intervention or major amputation). Of the 37 proteins tested, 6 were differentially expressed in patients with vs. without PAD (ADAMTS13, ICAM-1, ANGPTL3, Alpha 1-microglobulin, GDF15, and endostatin). Using 10-fold cross-validation, we developed a random forest machine learning model that accurately predicts 2-year MALE in a prospective validation cohort of PAD patients using a 6-protein panel (AUROC 0.84). This algorithm can support PAD risk stratification, informing clinical decisions on further vascular evaluation and management.


INTRODUCTION
Peripheral artery disease (PAD) affects over 200 million people globally and manifests in claudication, rest pain, and tissue loss due to lower extremity arterial atherosclerosis. 1 Despite its significant association with limb amputation and mortality, PAD remains poorly treated. 2 This is partly due to a lack of biomarker-based prognostic tools to identify high-risk patients for further evaluation and treatment. 3everal proteins have been identified to be associated with cardiovascular diseases, including ADAMTS13, 4 ICAM-1, 5 ANGPTL3, 6 alpha 1-microglobulin, 7 GDF15, 8 and endostatin. 91][12][13] Additionally, these 37 biomarkers have shown to be involved in several metabolic processes related to the development of PAD, including inflammation, atherosclerosis, thrombosis, and angiogenesis. 14Their mechanistic relationship to PAD supports their value as disease biomarkers. 14Although previous studies have demonstrated correlations between various proteins and PAD, few have characterized their prognostic value by calculating discriminatory metrics such as area under the receiver operating characteristic curve (AUROC).Furthermore, these proteins have been studied in isolation, and no previous study has investigated the prognostic value of using a combination of these proteins.Given that PAD is a multifactorial and chronic disease, with many metabolic pathways contributing to disease development, 15 we believe that a biomarker-based protein panel, in addition to clinical features, can achieve better accuracy in predicting PAD prognosis than analyzing single proteins in isolation.
Machine learning (ML) is a rapidly evolving field that allows computers to make highly accurate predictions based on large amounts of data. 16Specifically, ML leverages advanced analytics to model complex relationships between inputs (e.g.biomarker levels) and outputs (e.g., PAD outcomes). 16Automated ML algorithms can help clinicians better understand the future clinical course of patients, augmenting the ability to provide patient-centered care with improved outcomes. 17This field has been driven by the explosion of clinical and biomarker-based data combined with increasing computational power. 18The advantage of newer ML techniques over traditional statistical methods is that they can better model complex, multicollinear relationships between covariates and outcomes, 19 which is common in healthcare data. 20In a recent systematic review conducted by our group, we demonstrate that few ML-based tools for PAD prognosis consider novel biomarker data, and many suffer from high risk-of-bias, poor reporting, and inadequate performance. 21For example, Berger et al. ( 2020) developed a Bayesian model to predict 1-year hospitalization in PAD patients, 22 and Chang et al. (2020) built a neural network to predict surgical site infection after PAD intervention. 23However, they did not include biomarker-based data and achieved relatively low AUROC values of 0.63 and 0.61, respectively.Therefore, additional investigation in this area is warranted.Importantly, we plan to use ML methods to link clinical to biochemical data for risk prediction, which is not heavily investigated in PAD.In this study, we used ML to develop an automated prediction tool for PAD prognosis using novel biomarker data in a propensity-score matched cohort, and real-world performance was assessed in a prospective validation cohort.

Model development patient cohort
For ML model development, we recruited 406 patients (254 with PAD and 152 without PAD).Following propensity-score matching, we included 135 patients with PAD matched 1:1 to patients without PAD, for a total of 270 patients.In the matched cohort, the mean age was 68 (SD 10) years, 77 (29%) were female, 193 (72%) had hypertension, 205 (76%) had dyslipidemia, 79 (29%) had CAD, and 196 (73%) were taking statins.There were no differences in baseline demographic or clinical characteristics between patients with and without PAD, demonstrating appropriate matching (Table 1).

Protein levels
From an initial panel of 37 proteins, we identified 6 proteins that were differentially expressed in patients with vs. without PAD.Plasma concentrations of 3 proteins were lower in patients with PAD compared to those without PAD: ADAMTS13 (8.50 [SD 5.47]

PAD-related adverse events
All adverse events occurred in patients with PAD.In the PAD cohort, 42 (31%) patients developed major adverse limb events (MALE), 39 (29%) required vascular intervention, 6 (4%) underwent major amputation, and 21 (16%) had worsening PAD status over a 2-year follow-up period.The low major amputation rate likely reflects the fact that these patients were followed closely by vascular surgeons and underwent early revascularization to prevent limb loss.Therefore, the ML algorithm was not trained to predict this secondary outcome (Table 3).

Model performance
A random forest ML model was developed with the 6 proteins identified to be differentially expressed in patients with vs. without PAD as input features.For prognosis of patients with PAD, the model achieved the following performance metrics: 2-year MALE (AUROC 0.86, Figure 1A), 2-year need for vascular intervention (AUROC 0.85, Figure 1B), and 2-year worsening PAD status (AUROC 0.73, Figure 1C).All 6 proteins contributed to model predictions with the 3 most important features being (1) GDF15, (2) ICAM-1, and (3) ADAMTS13 (Figure 2).3B), and 2-year worsening PAD status (AUROC 0.76, Figure 3C).Of note, the prognostic performance metrics reported thus far are for PAD patients only as all adverse events occurred in PAD patients.To assess generalizability of our model to the general population, we assessed prognostic performance on PAD and non-PAD patients.Our model maintained good performance, with AUROC's of 0.83-0.88 in test set data and 0.73-0.79 on the prospective validation cohort for predicting 2-year MALE, vascular intervention, and worsening PAD status in both PAD and non-PAD patients (Figures S1 and S2).

Supplemental analysis of diagnostic performance
The ML model achieved an AUROC of 0.87 in identifying patients with a diagnosis of PAD.Diagnostic performance remained robust on the prospective validation cohort with an AUROC of 0.85 (Figure S3).

Summary of findings
In this study, we developed a robust ML model using a panel of 6 biomarkers that accurately predicts PAD prognosis in a prospective validation cohort.We showed several key findings.First, from the 37 proteins analyzed, we found 6 to be differentially expressed in patients with vs. without PAD (ADAMTS13, ICAM-1, ANGPTL3, alpha 1-microglobulin, GDF15, and endostatin).Second, we used ML-based predictive modeling to understand the collective impact of these 6 proteins on PAD prognosis.Our random forest ML model achieved excellent predictive ability for 2-year MALE, vascular intervention, and worsening PAD status in both the model development test cohort and prospective validation cohort, achieving AUROCs of 0.73-0.87.Finally, the top 3 predictive features in our algorithm were GDF15, ICAM1, and ADAMTS13.These proteins are most closely related to PAD, thereby identifying potential areas for future research to further characterize the biological relationships between these proteins and PAD development/progression.

Comparison to existing literature
Ross and colleagues (2019) used ML technology to predict major adverse cardiac and cerebrovascular events (MACCE) in PAD patients using data from electronic health records. 24Using retrospectively collected International Classification of Diseases (ICD)-9 codes, Common Procedural Terminology (CPT) codes, and prescriptions, among other clinical data, the authors developed models that predict MACCE that occurs at least 30 days after PAD diagnosis, achieving an AUROC of 0.81. 24There were several limitations to this study.First, the model was not tested on a prospective validation cohort; therefore, its real-world performance remains unclear as developing and testing a model on the same dataset can artificially elevate performance. 25Second, biomarker-based data were not considered as input features, and this information can have an important impact on PAD prognosis given previously demonstrated mechanistic relationships between several biologically active proteins and PAD development. 10,11,14Our study addressed both limitations with consideration of biomarker data as input features in our ML models and performance evaluation on a prospective validation cohort.In our model development test set data, we achieved better performance metrics for PAD prognosis, with AUROC's for predicting 2-year MALE above the 0.81 reported by Ross and colleagues. 24Expectedly, performance declined slightly in the prospective validation cohort, ranging from 0.76-0.85.Therefore, we demonstrate the value of building ML models using biomarker information, which can improve predictive performance.Furthermore, assessing performance on prospective validation datasets can provide a better understanding of real-world model performance.Al-Ramini and colleagues (2022) used gait measurements obtained by a camera system to diagnose PAD and obtained a similar accuracy of 87%. 26Some of these gait measurements could also be used to quantify treatment effectiveness. 27More recently, others have shown that a wearable accelerometer to estimate gait can provide similarly accurate measurements. 28In our study, rather than using gait data, we used biomarker data to predict PAD-related adverse events.It is important to note the cost and expertise needed to run these tests, particularly given that the analyzed biomarkers remain primarily used in the research setting.Therefore, additional research is needed to demonstrate the value of these biomarkers for routine clinical use.Furthermore, future work combining gait and biomarker data using ML techniques may allow for the development of increasingly accurate prediction models for patients with PAD.

Explanation of findings
There are several potential explanations for our findings.First, as of the 37 proteins analyzed, 6 were differentially expressed between patients with and without PAD.These proteins are involved in various mechanistic pathways important for PAD development and progression, including thrombosis (ADAMTS13 4 and ICAM-1 5 ), atherosclerosis (GDF15 8 and ANGPTL3 6 ), inflammation (alpha 1-microglobulin 7 ), and angiogenesis (endostatin 9 ).In our ML model, the most important predictive feature was plasma GDF15 levels.GDF15 (growth differentiation factor 15) is a protein found in circulation that promotes early plaque formation and progression. 29,30Mechanistically, GDF15 interacts with C-C motif chemokine receptor 2 to modulate macrophage chemotaxis, thereby downregulating macrophage apoptosis and inhibiting the proliferation of endothelial cells. 31,32Given that macrophage accumulation and endothelial dysfunction contribute to atherosclerosis, 33 GDF15 plays an important role in PAD development and progression.Interestingly, de Jager et al. (2011) demonstrated that deletion of GDF15 in murine models had a beneficial effect in both early and late atherosclerosis development, indicating the importance of this protein in plaque development and progression. 29Clinically, De Haan and colleagues (2017)  showed that higher GDF15 levels were associated with increased risk of major amputation and death in PAD patients. 8We similarly demonstrated the importance of GDF15 in PAD prognosis in our ML algorithm based on a biomarker panel.Although GDF15 was the most important predictive feature, the other five biomarkers also made important contributions to overall predictions based on variable importance scores.This suggests that using a panel of biomarkers likely improves accuracy in predicting PAD prognosis compared to a single protein alone, given their involvement in various biological pathways of PAD development.Second, we found high rates of adverse limb-related complications in patients with PAD, including over 20% of our cohort developing MALE.This suggests the need for more aggressive medical and possibly surgical management strategies to prevent complications in this high-risk population.Third, our ML model performed well for several reasons.Traditional statistical techniques such as logistic regression assume a linear correlation between the independent variables and the logit of the dependent variable; while advanced ML technology is not restricted by this assumption and can better model complex non-linear relationships between inputs and outputs. 34,35This is of particular importance in healthcare data, where a patient's clinical outcome can be influenced by many factors. 36Given the advantages of ML, including automation, understanding of non-linear relationships, and accurate predictions, this technology will likely outperform traditional statistical techniques in risk prediction. 34,35This is particularly important in biomarker-based models, where different proteins are involved in different biological pathways and may interact in complex ways to contribute to a disease process. 37In our study, random forest likely achieved excellent performance because it is an ensemble learning technique consisting of the aggregation of many decision trees, which (1) reduces variance, (2) handles large datasets efficiently, and (3) reduces overfitting. 38Overall, we demonstrate the advantage of using a synergistic ML-based model that considers a panel of biomarkers, which likely provides better predictive performance than individual biomarker information alone.Given that PAD is a chronic and multifactorial disease involving multiple biological pathways, previous studies have suggested the value of a panel-based approach to improve the prognosis of PAD. 39Our study confirms that applying ML techniques to clinical data in addition to biomarkers involved in atherosclerosis, inflammation, thrombosis, and angiogenesis allows for the development of highly accurate risk prediction tools for PAD.

Implications
Our ML models can be used to guide clinical decision-making in several ways.Our tool can be used to screen patients for asymptomatic PAD.This may be particularly useful in family practice settings, whereby generalists can send a 6-protein plasma panel in addition to routine blood work and use our automated algorithm to understand a patient's PAD risk. 40Patients who screen positive for a PAD diagnosis should be sent for additional vascular evaluation, such as an arterial duplex ultrasound to assess blood flow and confirm a PAD diagnosis. 41Once a PAD diagnosis is confirmed, the ML algorithm can also be used with the same 6 proteins to determine a patient's risk of adverse PAD-related events.Patients at low risk can continue receiving care from a family physician with risk factor optimization including acetylsalicylic acid, statins, and lifestyle modifications. 42Those at high risk of MALE or worsening PAD status should be referred to a vascular surgeon for further evaluation and management. 43Once a PAD referral has been made, vascular surgeons can use the ML algorithm in addition to their clinical judgment to identify those at higher risk of adverse limb events who may be considered for (1) additional vascular imaging to delineate anatomy and disease severity, 44 (2) medical management with low-dose rivaroxaban, 45 and/or (3) interventions for limb salvage in the highest risk patients. 46,47Overall, our automated ML tool can improve care for PAD patients in several ways in both the generalist and specialist settings, including providing efficient PAD screening and stratification of risk, supporting early identification and treatment of patients at high risk for adverse limb events, and reducing the number of unneeded specialist referrals, ultimately improving PAD outcomes and reducing health care costs. 48

Conclusions
In this study, we used a panel of 6 biomarkers to develop an ML model that accurately predicts PAD prognosis in a prospective validation cohort.Our model can be used for PAD screening and risk stratification, thereby improving early identification and targeted management of PAD.Specifically, high-risk patients should be referred for further vascular evaluation and/or receive aggressive medical management.Our ML algorithms also benefit from continuous learning and automation.This tool has the potential for important utility in the care of PAD patients.Our findings also provide insights for future research.Particularly, from a panel of 37 proteins, GDF15 was identified as the most important predictive feature for PAD prognosis.Future basic and translational studies investigating the mechanistic relationship between GDF15 and PAD development/progression may improve our understanding of pathogenesis and potential targeted therapies.Importantly, our study provides impetus for clinical trials evaluating the impact of ML algorithms on PAD outcomes.

Limitations of the study
Our study has several limitations.First, this was a single center study using 2 recruited cohorts, and future validation at other institutions is needed to demonstrate generalizability of our model.Second, we achieved good predictive ability with a relatively small sample size for an ML-based study and future studies with larger sample sizes may improve performance.Third, 2-year outcomes are reported, and longer follow-up is needed to better understand the prognostic value of our algorithm given the long-standing nature of PAD.Fourth, the analyzed biomarkers remain primarily used in the research setting.Therefore, additional translational research and implementation science are needed to demonstrate the value of these biomarkers for routine care to support clinical use.Fifth, the ML algorithm was not trained to predict major amputation due to the low event rate, likely because patients in this study were followed closely by vascular surgeons and underwent early revascularization to prevent limb loss.Future studies are needed to assess the generalizability of the model to real-world clinical practice.PAD and non-PAD patients.Diagnostic performance for discriminating PAD and non-PAD patients was also assessed in the supplemental analysis.The primary metric for assessing model performance was AUROC, a validated metric to assess discriminatory ability that considers both sensitivity and specificity. 100Statistical analysis was performed separately on the model development and prospective validation cohorts.Significance was set at a two-tailed p < 0.05.All analyses were carried out using SPSS software version 23 (SPSS Inc., Chicago, Illinois, USA). 101

Figure 1 .
Figure 1.Receiver operating characteristic curve for random forest machine learning model in predicting (A) 2-year major adverse limb event (MALE), (B) 2-year vascular intervention, and (C) 2-year worsening peripheral artery disease (PAD) status on test set data AUC (area under the receiver operating characteristic curve).*Note: the prognostic models were built on PAD patients only because all adverse events occurred in the PAD cohort.This figure shows prognostic performance on PAD patients only.

Figure 2 .
Figure 2. Variable importance scores (gain) for the 6 proteins used as input features for random forest machine learning model *Note: higher score indicates greater importance in contributing to an overall prediction.

Table 1 .
Baseline demographic and clinical characteristics of propensity-score matched cohort for machine learning model development *Values reported as N (%) unless otherwise indicated.Abbreviations: PAD (peripheral artery disease).

Table 3 .
PAD-related adverse events over two years in propensity-score matched cohort for machine learning model development

Table 5 .
PAD-related adverse events over two years in prospective validation cohort *Values reported as N (%) unless otherwise indicated.Abbreviations: PAD (peripheral artery disease), MALE (major adverse limb event).

Table 4 .
Baseline demographic and clinical characteristics of prospective validation cohort *Values reported as N (%) unless otherwise indicated.Abbreviations: PAD (peripheral artery disease).