Machine learning-based risk prediction of acute kidney disease and hospital mortality in older patients

Introduction Acute kidney injury (AKI) is a prevalent complication in older people, elevating the risks of acute kidney disease (AKD) and mortality. AKD reflects the adverse events developing after AKI. We aimed to develop and validate machine learning models for predicting the occurrence of AKD, AKI and mortality in older patients. Methods We retrospectively reviewed the medical records of older patients (aged 65 years and above). To explore the trajectory of kidney dysfunction, patients were categorized into four groups: no kidney disease, AKI recovery, AKD without AKI, or AKD with AKI. We developed eight machine learning models to predict AKD, AKI, and mortality. The best-performing model was identified based on the area under the receiver operating characteristic curve (AUC) and interpreted using the Shapley additive explanations (SHAP) method. Results A total of 22,005 patients were finally included in our study. Among them, 4,434 patients (20.15%) developed AKD, 4,000 (18.18%) occurred AKI, and 866 (3.94%) patients deceased. Light gradient boosting machine (LGBM) outperformed in predicting AKD, AKI, and mortality, and the final lite models with 15 features had AUC values of 0.760, 0.767, and 0.927, respectively. The SHAP method revealed that AKI stage, albumin, lactate dehydrogenase, aspirin and coronary heart disease were the top 5 predictors of AKD. An online prediction website for AKD and mortality was developed based on the final models. Discussion The LGBM models provide a valuable tool for early prediction of AKD, AKI, and mortality in older patients, facilitating timely interventions. This study highlights the potential of machine learning in improving older adult care, with the developed online tool offering practical utility for healthcare professionals. Further research should aim at external validation and integration of these models into clinical practice.


Introduction
Acute kidney injury (AKI), a complex public health concern, prevalent in about 12% of patients (1)(2)(3) and often accompanied by multiple organ failure especially in older people, which leads to up to 1.7 million annual deaths (4)(5)(6)(7)(8).Studies reported that older AKI survivors face a considerable risk of progressing to chronic kidney disease (CKD) (9).The poor prognosis of older patients with kidney disease poses significant challenges to the healthcare system and result in a substantial economic burden on families due to multi-system damage or long-term hemodialysis treatment.
Current evidence indicates that AKI can progress to an intermediate stage called acute kidney disease (AKD), defined by the 16th Acute Disease Quality Initiative (ADQI) meeting as acute or subacute damage and/or loss of kidney function for 7-90 days after an AKI-initiating event (9).Distinguishing between AKD and AKI in clinical practice is crucial, as the management strategies and prognostic implications for these conditions differ.While AKI represents a sudden decline in kidney function, AKD encompasses a broader timeframe and includes patients who do not fully recover from an episode of AKI, presenting a poorer prognosis in older patients, with a study showing a 31.8%in-hospital mortality rate for older patients in the validation cohorts (10).Explaining this clinical distinction is vital for understanding the progression of kidney diseases and the necessity for targeted prediction models.As a transitional period, AKD may serve as a turning point for improving patients' renal function and presents significant potential for clinical research.Developing accurate prediction models for AKD has substantial clinical implications.These models can facilitate early identification of at-risk patients, enabling timely interventions that may prevent further kidney damage and improve patient outcomes.However, current studies mainly focus on AKI, with insufficient exploration of AKD's impacts and trajectories in the older adult, underscoring the importance of targeted research on AKD.
Recently, several studies have demonstrated that the superior predictive capabilities of machine learning (ML) models over traditional statistical methods in predicting AKI.For instance, in pediatric critical care, the prediction of Stage 2/3 AKI by a ML model showed an AUROC of 0.89 (11).The random forest (RF) model for predicting AKI in patients undergoing cardiac surgery achieved an AUC of 0.839 (12).Despite ML's complexity, the SHapley Additive exPlanation (SHAP) method has been developed to make these models more interpretable (13,14).Nevertheless, the application of ML and SHAP methods for the prediction of AKD in older patients remains limited.
Hence, the primary aim of this study was to investigate the incidence rates of AKD, AKI, and mortality among older patients, addressing a gap in the epidemiology of kidney injury trajectories in the older adult.Secondly, we aimed to pioneer the development of predictive ML models for AKD, AKI, and mortality.Furthermore, we integrated the SHAP approach to bolster the interpretability of prediction models.Finally, we have also developed an innovative online risk calculator rooted in ML algorithms.These may provide a critical window for early targeted interventions to improve the prognosis of the older adult, thereby alleviating pressure on healthcare systems.

Data collection
We retrospectively reviewed the medical records of 40,325 patients aged ≥65 years between October 2012 and October 2019.Patients were excluded if they met one of the following criteria: continuous dialysis, renal transplantation before AKD diagnosis, less than two serum creatinine (Scr) tests during hospitalization or missing inpatient data and the duration of hospitalization <48 h.We collected data on demographic characteristics, comorbidities, laboratory parameters, and medications from the hospital information system.Comorbidities mentioned in this study were all defined according to the International Classification of Disease (ICD) 10th Revision.The study was approved by the Institutional Review Board (IRB; QYFY WZLL 28250), ensuring patient confidentiality through anonymized data collection and adherence to privacy protocols.

Definition
The primary outcome was the occurrence of AKD, with secondary outcomes including AKI and mortality.AKI was diagnosed based on Kidney Disease: Improving Global Outcomes (KDIGO) 2012 as follows: Scr level > 26.5 mmol/L (0.3 mg/dL) within 48 h; an increase in Scr to more than 1.5-fold the baseline-confirmed value or an increase presumed to have occurred within 7 days; or urine output <0.5 mL/kg/h for more than 6 h (15).AKD was defined following the 2017 ADQI as acute or subacute damage and/or loss of kidney function for a duration of between 7 and 90 days after exposure to an AKI initiating event (9).Diagnosis and staging of AKI and AKD were determined at the first fulfillment of these criteria.
Based on the diagnostic criteria of AKI and AKD, patients were classified into the following four groups.AKI Recovery: This group included patients whose Scr levels returned to baseline within 7 days, indicating a renal impairment duration of less than 7 days or a rapid recovery within that timeframe.AKD without AKI: This group comprised patients whose Scr levels increased gradually but remained elevated for more than 7 days, indicating subacute AKD without meeting the AKI criteria.AKD with AKI: Patients in this category experienced stage ≥1 AKI that persisted for at least 7 days after the initial AKI event, indicating a continuous progression from AKI to AKD.No Kidney Disease (NKD): Patients falling into this category had an eGFR of 60 mL/ min/1.73m 2 or higher, no detectable albuminuria, and did not meet the criteria for either AKI or AKD.To thoroughly assess the influence of evolving kidney injury patterns on mortality among older patients, we integrated AKI and AKD into a unified metric termed 'dynamic' during the mortality model's construction.The 'dynamic' variable adopts values 0, 1, 2 and 3 corresponding to NKD, AKI recovery, AKD without AKI, and AKD with AKI, respectively.Baseline Scr was defined as the first Scr value measured during hospitalization.The baseline estimated glomerular filtration rate (eGFR) was calculated using the Chronic Kidney Disease Epidemiology Collaboration formula (16).

Model development
We engineered predictive models for AKD, AKI, and mortality, respectively.Scikit-learn (https://github.com/scikit-learn/scikitlearn)Frontiers in Medicine 03 frontiersin.orgpackage was used to build models including logistic regression (LR), support vector machine (SVM), random forest (RF), naïve byes (NB), k-nearest neighbor (KNN), multi-layer perceptron (MLP), gradient boosting machine (GBM) and light gradient boosting machine (LGBM).The data were divided, with 80% utilized for training and 20% for testing.Grid search method with ten-fold cross validation was used in the training set to prevent overfitting and to identify the optimal hyperparameters for each model.To address the disparity in the distribution of positive and negative samples, we implemented a strategy of class weight adjustment during the training phase of the ML model (17).

Model interpretation and evaluation
SHAP method was designed to address the "black-box" issue in prediction models by providing a means to rank the importance of input features and explain model results (14,18).This approach offers both global and local explanations, enhancing our understanding of the model's decision-making process.Globally, it provides consistent attribution values for each feature, revealing associations.Locally, it explains specific predictions for individual cases, enhancing interpretability.In our pursuit of feature optimization, we also utilized the SHAP method for feature selection in the optimal model.SHAP value-assisted feature selection was utilized to identify the top 20, 15, 10, and 5 features for model construction.This approach was to find the best balance between accuracy and complexity, leading to a final lite model.SHAP method was implemented using Python shap package (https://shap.readthedocs.io/en/latest/).
The performance of our predictive models was evaluated on the test set, focusing on their discriminative ability and clinical utility.Discrimination was quantitatively assessed using a suite of performance metrics, including area under curve (AUC) of the receiver operating characteristic (ROC) curve (19), sensitivity, specificity, recall, accuracy, F1 score, Brier score and Matthews correlation coefficient (MCC).The model demonstrating the highest AUC was designated as the optimal one.For clinical applicability, decision curve analysis (DCA) was employed, which calculated the net benefit of the final model by contrasting the predicted benefits against the expected risks associated with the outcomes (20).Furthermore, the performance of the final model was showed through precision-recall (PR) curves, Kolmogorov-Smirnov (KS) plots, and confusion matrix.

Online prediction website
We created an online web-based risk calculator utilizing the Streamlit Python framework, employing the model with the optimal number of features.Upon the values of corresponding features are provided, the website can return the probability of AKD and mortality, respectively.This tool showed the practical application of our research in a clinical setting.

Sensitivity analysis
A sensitivity analysis was performed to thoroughly examine the predictive efficacy of the models, focusing specifically on stages 2-3 of AKD.Additionally, the models' performance underwent a thorough assessment across various subgroups, with a particular emphasis on patients stratified by age brackets: 65-74 years, 75-84 years, and those aged over 85 years.

Statistical analysis
Variables with over 15% missing values were excluded, while those with less than 15% missing data were imputed using the Multivariate Imputation by Chained Equations (MICE) algorithm
The differences in characteristics between kidney injury group and NKD group are partially shown in Table 1, with a detailed comparison of all characteristics provided in Supplementary Table S1.In brief, compared to the NKD group, patients with acute/subacute kidney dysfunction were older on average (75.00 ± 13.00 vs. 73.00± 12.00, p < 0.05) with more risk factors like smoking, alcohol use, diabetes and other conditions.The baseline lab tests including eGFR, blood urea nitrogen (BUN), cystatin C (Cys), blood glucose, lipid profiles, uric acid (UA) and others were also worst in kidney dysfunction group (p < 0.05).Furthermore, the data indicated that patients with renal impairment endured longer hospital stays (18.00 ± 14.00 vs. 17.00 ± 9.00 days, p < 0.05) and encountered higher hospital mortality rates (9.6% vs. 1.5%, p < 0.05) in comparison to the NKD group.This signified that older patients with kidney dysfunction were susceptible to a worsening prognosis.

Feature selection and model performance
Eight ML models were developed to predict AKD occurrence in older patients, by utilizing all available features, with the ROC curves illustrated in Figure 1A.The LGBM model emerged as the most efficacious in predicting AKD, achieving an AUC of 0.781.The performance metrics of these eight ML models in predicting AKD were comprehensively tabulated in  S2 presented a correlation matrix heatmap, delineating the interrelationships between the predictive outcomes of the various ML models.
To identify the most significant features, we ranked the importance of LGBM features using the SHAP method in the training set.The evaluation metrics for LGBM models with different numbers of features were presented in Table 3.The model's AUC increased to 0.760 when considering the top 15 features, leading to notable improvements in accuracy and precision.However, expanding the feature set to 20 did not yield a substantial uplift in AUC, and the other performance metrics exhibited a tendency toward stabilization.Given that, we selected the top 15 critical variables as the final lite prediction model for AKD (Figure 2A).Performance of the final lite LGBM model for AKD were presented in Supplementary Figure S3.We showed a DCA demonstrating the model's substantial clinical utility.Furthermore, the confusion matrix, KS plot, and PR curve demonstrated the model exhibited satisfactory classification capabilities and maintained a favorable balance between precision and recall.
We employed the aforementioned methodology to derive features and construct models for both AKI and mortality prediction, with detailed results included in the supplementary files.The ROC curves utilizing all available features were illustrated in Figures 1B,C.The LGBM emerged as the optimal model for both AKI and mortality predictions (Supplementary Tables S2, S3), with 15 features identified as the ideal number for model performance (Supplementary Tables S4, S5).The refined model of AKI had an AUC of 0.767.In addition, it's worth emphasizing that the final lite LGBM model of mortality showed impressive predictive capabilities, achieving an AUC of 0.927, and high recall and accuracy at 0.731 and 0.933, respectively.The ROC curves and DCA of the final lite LGBM model for AKI and mortality were presented in Supplementary Figures S4, S5.

Model interpretations
The SHAP summary plot (Figure 2B) displayed the contributions of the feature to the model.The analysis revealed that the primary factors influencing the model's predictions were AKI stage, albumin (ALB), lactate dehydrogenase (LDH), the use of aspirin, and coronary heart disease (CHD).SHAP dependence plots (Figure 3) facilitated understanding how a single feature affected the output of the prediction model and showed the relationship between two features at the same time.For instance, as the value of Cys increased, so did the SHAP value and AKI stage, which implied a rising risk of developing AKD and a positive correlation between Cys and AKI stage (Figure 3A).The SHAP interaction plot (Supplementary Figure S6) revealed the interactions between all features.Furthermore, local explanation analyzed how features contributing to a particular prediction for an individual.The force plots (Figure 4) mainly presented the major factors that contributed to the final model output in a certain individual.Furthermore, the SHAP decision plots for other four patients (Supplementary Figure S7) provided a clear visualization of the decision-making paths attributed to each feature.
The SHAP method was also used for the AKI and mortality models, and detailed results were in the supplementary files.For the AKI model, Scr was the top contributing factor, as expected (Supplementary Figure S8).In the mortality model, the 'dynamic' variable ranked second in terms of significance (Supplementary Figure S9).The increasing 'dynamic' grade correlated with rising SHAP values, suggesting a higher mortality risk, highlighting the significant impact of kidney injury trajectory on older patients' survival rates.

Online prediction website
Based on the lite prediction models, we developed an online risk website to streamline external validation and assess AKD and mortality risk in older patients.https://xuly94-elderly-hospitalizedpatients-app-app-dxfrws.streamlit.app/,which can promptly generate the estimated risk for AKD and mortality offering immediate support for clinical decision-making.

Sensitivity analysis
The LGBM model demonstrated robust predictive accuracy for AKD stages 2-3, achieving an AUC of 0.843 in the test set (Supplementary Figure S10A).This indicated the model's enhanced capability in predicting more severe cases of AKD, which was crucial to improve patient outcomes.When tested across various age groups, the performance of the model also remained stable (Supplementary Figures S10B-D).Specifically, the model yielded its highest performance in the 65-74 age subgroup, with an AUC of 0.755.

Discussion
In this retrospective cohort study, we developed and validated ML algorithms to forecast AKD, AKI, and mortality among older patients.The LGBM algorithm exhibited the strongest discrimination capability across all three outcomes.Additionally, SHAP was used for individualized patient interpretations, and an online AKD and mortality risk calculator for older patients was created, aiding early prediction and intervention.To the best of our knowledge, our study is the first to establish ML models for AKD, AKI and mortality in older patients that are valuable for risk assessment and clinical decision-making.
Several investigations have been conducted to explore the epidemiology of AKD before.James et al. reported that among more than one million Canadian residents, AKD without AKI was common -the incidence per 100 of the population tested was 3.8 in   (23).In our own study cohort, we observed that 4,434 patients, accounting for 20.15% of the total, satisfied the criteria for AKD.
In recent years, ML methods have been widely employed in predicting AKI (24-27).However, there is comparatively limited research on predicting AKD, particularly in older patients.A nomogram was developed and validated to predict the transition from AKI to AKD in patients undergoing partial nephrectomy for renal masses, demonstrating good discrimination with a concordance index    (10).Unlike their study, which focused on older patients with AKD in the ICU, our research encompassed a broader spectrum, targeting the entire older patient population within hospital settings.What's more, we have crafted models for predicting not only AKD but also AKI and mortality among older patients.International consensus emphasizes the importance of early detection and prevention of AKD to mitigate its impact on patients and healthcare systems (9).Although, in theory, all older patients would benefit from comprehensive preventive measures against AKD, technical limitations often hinder early intervention.To address this issue, the ML algorithm simplifies early prediction.Furthermore, an online prediction website utilizing LGBM models can quickly identify high-risk older patients.This enables early detection and preventive interventions to enhance the prognosis for older individuals.The SHAP summary plot and force plots in Figure 2 enhanced understanding of the model's decision-making process and can further assist physicians in implementing targeted preventive interventions for AKD.
In our study, the importance of variables showed that AKI stage, ALB, LDH, the use of aspirin and CHD were the most important factors that contributed to the predicted occurrence of AKD among older patients.Numerous studies have shown that AKI is intricately linked to the development of AKD (23,(29)(30)(31).Although current studies predominantly focus on AKI, they also suggest that these factors are risk for renal function impairment, consistent with our findings.Specifically, low serum albumin levels and elevated LDH levels are both associated with AKI AND poor outcomes (32)(33)(34)(35)(36)(37)(38)(39)(40).Aspirin, a common NSAID, and CHD have also been identified as independent risk factors for AKI, particularly among older people (34, [41][42][43][44].
This study has several key clinical implications.Firstly, it represents the initial effort to compare the baseline characteristics and Thirdly, the application of the SHAP method mitigated the opacity of ML models by globally and locally identifying and elucidating the most influential features for all three outcomes.In addition, we selected an optimal number of features for our final model to ensure a balance between complexity and clinical applicability, emphasizing its practicality with features that are readily obtainable in standard clinical settings.Furthermore, our models have been designed for direct clinical use, exemplified by a web-based risk calculator that assesses the risk of AKD and mortality in older patients, thus providing physicians with a valuable tool to enhance decision-making.
Our study faced several limitations.Firstly, it had a single-center design and a lack of ethnic diversity, which may affect the generalizability of our findings.Additionally, the identification of AKD and AKI could benefit from incorporating more early diagnostic markers, such as cystatin C, to improve predictive accuracy.Besides, the retrospective nature of our data collection introduces potential recall and selection biases.To address these issues, future research should aim for nationwide, multi-center prospective trials to enhance the validation and reliability of our predictive models, ensuring their applicability across diverse populations, including testing and verifying the model among people of other ethnicities.Last but not least, this article aims to predict kidney injury in older adult patients without specifically distinguishing the etiology.Due to the complex conditions of older adult patients, including numerous underlying diseases, susceptibility to infections, use of nephrotoxic drugs, and other common causes of kidney injury, it is often the result of multiple factors combined (8).Therefore, we have established a universal, comprehensive, and representative risk prediction model.However, its effectiveness in predicting kidney injury caused by different specific factors may not be optimal.Consequently, in future research, we plan to conduct separate studies on kidney injury caused by specific factors, such as sepsis.
This study highlights the increased susceptibility of older patients to AKD.We presented LGBM models to forecast AKD, AKI, and mortality at the time of admission.Furthermore, the web tool we developed to identify high-risk AKD and mortality cases in older patients can aid in clinical decision-making.Moving forward, we will conduct nationwide, multi-center trials with diverse participation, validating our predictive models across various ethnic groups.
(21).Continuous variables were shown as mean with standard deviation, or median with interquartile range and compared by the Independent-sample T test or Wilcoxon rank-sum test.Categorial variables were expressed in quantities and percentages and compared by the Chi-square tests.All analyses were carried out with Python version 3.10.11,R version 4.3.1, and SPSS version 25.0.A 2-tailed p value of <0.05 was considered statistically significant.

FIGURE 1
FIGURE 1 Performance of eight ML models for different outcomes with all features.(A) The ROC curve of AKD.(B) The ROC curve of AKI.(C) The ROC curve of mortality.

FIGURE 2
FIGURE 2 Importance matrix plot and SHAP summary plot of the final lite LGBM model.(A) The importance ranking of the first 15 features of the LGBM model.(B) The SHAP summary plot demonstrates the general importance of each feature in LGBM model.The color bar on the right indicates the relative value of a feature in each case.Red dots indicate high values and blue dots indicate low values.The violin graph lining up on the midline is the aggregation of dots representing each case in the train set.The distance between the upper and lower margin of the violin graph represents the amount of the cases that end up with the same SHAP values offered by this feature.SHAP force plots of 4 examples of patients.Categorical features including AKI stage, CHD, Omeprazole and β-lactam antibiotics were represented by 0 and 1, while "0" means "No" and "1" means "Yes."*ALB, albumin; LDH, lactate dehydrogenase, CHD, coronary heart disease; CK, creatine kinase; Cys, cystatin C; GGT, gamma-glutamyl transferase; Scr, serum creatinine, CCB, calcium channel blocker; RBC, red blood cell count.

FIGURE 3 SHAP
FIGURE 3 SHAP dependence plots demonstrate the distribution of SHAP output value of a single feature.The colors on the dependence plot correspond to another feature that could potentially interact with the feature being analyzed.(A) The relationship between Cys and AKI stage SHAP values, with the color bar indicating various levels of AKI stage.(B) The relationship between Cys and Scr SHAP values, where the color bar represents different levels of Scr.(C) The relationship between Scr and AKI stage SHAP values, with the color bar also denoting distinct AKI stage levels.(D) The relationship between Scr and ALB SHAP values, with the color bar reflecting varying ALB levels.*ALB, albumin; Cys, cystatin C; Scr, serum creatinine.

FIGURE 4 Force
FIGURE 4 Force plots of the final lite LGBM model.(A,B) Show the examples of patients predicted to have AKD.(C,D) Show the examples of patients predicted to be non-AKD.The features shown in red represent a higher risk of AKD, while the features shown in blue represent a lower risk.The plots help physicians identify the main features in the model that have high decision power at the individual level.Categorical features including AKI stage, CHD, Omeprazole and β-lactam antibiotics were represented by 0 and 1, while "0" means "No" and "1" means "Yes."*ALB, albumin; LDH, lactate dehydrogenase, CHD, coronary heart disease; CK, creatine kinase; Cys, cystatin C; GGT, gamma-glutamyl transferase; Scr, serum creatinine, CCB, calcium channel blocker; RBC, red blood cell count.

TABLE 1
The partial baseline characteristics of the current cohort.
performance, we subsequently conducted a feature selection process specifically within the LGBM model framework.Additionally, Supplementary Figure

TABLE 2
Performance of eight ML models for predicting AKD.AUC, area under curve of the receiver operating characteristic curve.
(22)ividuals without preexisting CKD and 0.6 in individuals with pre-existing CKD(22).Su et al. reported the incidence rate of community-acquired AKD was 4.60%, while it was 28.2% for hospital-acquired AKD

TABLE 3
Performance of LGBM model for predicting AKD.
*AUC, area under curve of the receiver operating characteristic curve.