Machine learning algorithm as a prognostic tool for Epstein-Barr virus reactivation after haploidentical hematopoietic stem cell transplantation

Epstein-Barr virus (EBV) reactivation is one of the most important infections after hematopoietic stem cell transplantation (HSCT) using haplo-identical related donors (HID). We aimed to establish a comprehensive model with machine learning, which could predict EBV reactivation after HID HSCT with anti-thymocyte globulin (ATG) for graft-versus-host disease (GVHD) prophylaxis. We enrolled 470 consecutive acute leukemia patients, 60% of them (n = 282) randomly selected as a training cohort, the remaining 40% (n = 188) as a validation cohort. The equation was as follows: Probability (EBV reactivation) = 11 + exp(−Y), where Y = 0.0250 × (age) – 0.3614 × (gender) + 0.0668 × (underlying disease) – 0.6297 × (disease status before HSCT) – 0.0726 × (disease risk index) – 0.0118 × (hematopoietic cell transplantation-specific comorbidity index [HCT-CI] score) + 1.2037 × (human leukocyte antigen disparity) + 0.5347 × (EBV serostatus) + 0.1605 × (conditioning regimen) – 0.2270 × (donor/recipient gender matched) + 0.2304 × (donor/recipient relation) – 0.0170 × (mononuclear cell counts in graft) + 0.0395 × (CD34+ cell count in graft) – 2.4510. The threshold of probability was 0.4623, which separated patients into low- and high-risk groups. The 1-year cumulative incidence of EBV reactivation in the low- and high-risk groups was 11.0% versus 24.5% (P < .001), 10.7% versus 19.3% (P = .046), and 11.4% versus 31.6% (P = .001), respectively, in total, training and validation cohorts. The model could also predict relapse and survival after HID HSCT. We established a comprehensive model that could predict EBV reactivation in HID HSCT recipients using ATG for GVHD prophylaxis.


INTRODUCTION
Allogeneic hematopoietic stem cell transplantation (allo-HSCT) significantly improves the survival of patients with acute leukemia (AL). 1 Recently, with the progression of the transplant technique, human leukocyte antigen (HLA) haplo-identical donors (HIDs) are quantitatively the most important, accounting for 60% of allo-HSCT in China. 2 Although patients receiving HID HSCT can achieve longterm survival, 3,4 infection is still the most important cause of transplant-related mortality. 5 Epstein-Barr virus (EBV) reactivation is one of the most common of these infections. It has been found to be the most important risk factor for EBV-related post-transplant lymphoproliferative disorders (PTLD) [6][7][8] and can increase the risk of mortality. 9,10 Considering that the seroprevalence of EBV in the Chinese population is as high as 90% in children 8 years old or more, 11 it is important to predict the EBV reactivation after HID HSCT.
Several variables could increase the risk of EBV reactivation after allo-HSCT. Anti-thymocyte globulin (ATG) is the most critical risk factor, 6,12,13 and approximately 60%-70% of patients receiving ATG during conditioning would experience EBV reactivation after allo-HSCT. 12,14 In addition, HLA mismatch is another important risk factor for post-transplant EBV reactivation. 6,13 Thus, patients receiving HID HSCT with an ATG-based regimen would have a high risk of EBV reactivation; however, no accepted risk factors for EBV reactivation have been reported in this population, and there is no comprehensive model to predict EBV reactivation after HID HSCT.
The objective of this article is to establish a comprehensive model, with machine learning, which could predict EBV reactivation after HID HSCT with ATG to counter graft-versus-host disease (GVHD).

Study design
This study was conducted on the basis of the transplant database of Peking University, Institute of Hematology; it consisted of 470 consecutive AL patients receiving HID HSCT between January 21, 2020, and May 31, 2021. The information on acute GVHD (aGVHD) has been reported in detail, 15 and in the present study, the survivors were further followed up to March 1, 2022. The study was conducted in accordance with the Declaration of Helsinki.

Protocols for EBV monitoring and prevention
Plasma EBV copies were monitored at least weekly until day +100 with quantitative polymerase chain reaction (Q-PCR) analysis. For patients who received systemic immunosuppressive treatments, EBV monitoring was conducted regularly after day +100. If symptoms of suspected virus infection were present, additional detection was performed. The EBV reactivation was defined as more than 1 × 10 3 copies/mL EBV-DNA in plasma by Q-PCR in 1 test. 26 The protocols for infection prophylaxis other than EBV and the pre-emptive intervention for EBV reactivation are shown in the SDC, Methods (http:// links.lww.com/BS/A53).

Building machine learning models
Our method consisted of 2 steps: building the logistic regression model and ascertaining the optimal threshold ( Fig. 1; SDC, Methods, http://links.lww.com/BS/A53; and SDC, Table S1, http://links.lww.com/BS/A53). [27][28][29][30][31] Of the entire study population, 60% were randomly selected (ie, n = 282) as the training cohort; the remaining 40% were used as the validation cohort (n = 188). For the primary outcome (EBV reactivation), we performed the model-building steps in the training cohort and verified the model in the validation cohort. We also identified the sensitivity, specificity, area under the curve (AUC) score and accuracy score in both data cohorts.

Building models
We utilized logistic regression models with L2 regularization for the prediction. The model is illustrated in equation (1): In equation (1), w is the coefficient to be trained, which requires the following objective function to be minimized: During the optimization procedure, an inappropriate imbalance between the sizes of the positive and negative samples was found. We adjusted weights to each sample when conducting optimization as in equation (3): We utilized sklearn v1.0.2 with Python 3.9 to build the models based on the anaconda3 development platform. The model parameters "class_weight" and "max_iter" are set to be "balanced" and 1000, respectively. 32,33 2.4.2. Finding the optimal threshold According to equation (1), the output of the logistic regression model should be between 0 and 1. To further specify the prediction results, determining the threshold for outputting negative or positive became significant. In this article, we drew receiver operating characteristic (ROC) curves 30 and calculated the g-mean for each threshold. 31 We chose the one with the largest g-mean to be the optimal threshold.

Evaluation for model
ROC-AUC was defined as the AUC of the tpr/fpr at thresholds ranging from 0 to 1. The confusion matrix was a 2 × 2 table for summarizing the prediction results. In addition, we normalized the count values by the number of True Label (Outcome) or the number of Predicted Label (Prediction).
The detailed information for the setting of equations is shown in the SDC, Methods (http://links.lww.com/BS/A53).

Statistical methods
The primary outcome was EBV reactivation. The secondary outcomes included relapse, NRM, OS, and LFS.
We used the Mann-Whitney U test to compare continuous variables and the χ 2 and Fisher exact tests for categorical variables. The Kaplan-Meier method was used to estimate the probability of OS and LFS. We used competing risk analyses to calculate the cumulative incidence of EBV reactivation, NRM, and relapse. 36 Testing was 2-sided at the P < .05 level. Statistical analysis was performed on R software (version 4.2.0) (http:// www.r-project.org) and SPSS 26.0 software (SPSS, Chicago, Illinois). Table 1 shows the characteristics of the training and validation cohorts. The detailed information of engraftment and aGVHD have been previously reported by Shen et al. 15 A total of 438 patients (92.9%) survived until the last follow-up. The median duration of follow-up was 483 days (range, 39-770 d). The probabilities of NRM, relapse, OS and LFS at 1 year after HID HSCT were 3.9% (95% CI, 2.1%-5.7%), 8.7% (95% CI, 6.1%-11.2%), 93.9% (95% CI, 91.8%-96.1%), and 87.4% (95% CI, 84.4%-90.5%), respectively.

Predictive model for EBV reactivation
Our equation was as follows: Probability (EBV reactivation) =  Table 2). The threshold of probability was set as 0.4623, and the g-mean was 0.648; thus, the patients could be separated into low-and high-risk groups by the threshold values.
The cumulative incidence of EBV reactivation at 1 year after HID HSCT was significantly higher in the high-risk group than in the low-risk group in the patients with HCT-CI scores of 0 (SDC, Figure S1, http://links.lww.com/BS/A53). The low-risk group showed a trend to lower incidence of EBV reactivation compared with the high-risk group for patients with HCT-CI scores of ≥1 (SDC, Figure S2, http://links.lww.com/BS/A53).

Predicted value of our comprehensive model in patients without or with severe aGVHD
In patients with grade II to IV aGVHD (n = 126), the cumulative incidence of EBV reactivation at 1 year after HID HSCT allo-HSCT = allogeneic hematopoietic stem cell transplantation, CR = complete remission, EBV = Epstein-Barr virus, HCT-CI = hematopoietic cell transplantation-specific comorbidity index, HLA = human leukocyte antigen, MNC = mononuclear cell, TBI = total body irradiation.
In patients without aGVHD or with grade I aGVHD (n = 344), the cumulative incidence of EBV reactivation at 1 year after HID HSCT was 23.6% (95% CI, 17.0%-30.2%) and Table 2 Variables for building machine learning models.

Secondary outcomes after HID HSCT
The cumulative incidence of NRM, OS and LFS at 1 year after HID HSCT for patients in the high-risk group was significantly poorer than in low-risk group. The incidence of relapse was comparable between the groups (Fig. 5).

DISCUSSION
In the present study, we propose a predictive model for EBV reactivation after HID HSCT with the help of machine learning. It can categorize the patients into low-and high-risk groups for EBV reactivation. We first integrated different variables and established a comprehensive model that could effectively predict EBV reactivation in HID HSCT recipients with ATG for GVHD prophylaxis.
Several studies have already identified the risk factors for EBV reactivation after allo-HSCT. 9,10,12,13,26 However, only male patients and intensified conditioning regimens were potential risk factors besides ATG and HLA mismatched donors, and using 1 or 2 variables to predict EBV reactivations was distinctly insufficient for HID HSCT recipients on ATG-based regimens. According to machine learning theory, adding more variables can increase the capacity and performance of the upper boundary of the predictive model. 37,38 Thus, our comprehensive model included 13 demography, disease, and transplant characteristics. However, the large number of variables may induce overfitting in the training set. 39 Our strategy is to add an L2 regularization term as shown in the objective function (equation 2). By introducing a regularization term to the objective function, the weights for coefficients become more balanced, thereby reducing the risk of overfitting. 40 In addition, an imbalance problem was found between the sizes of the positive and negative samples. We adopted adjusted weights (equation 3) during the optimization procedure. 41 In this way, we enhanced the weights for the positive samples to alleviate these adverse effects. Both methods contributed to a more generalizable and robust model. Our strategy was therefore in a step-wise manner and ensured the stability of the feature-selection process. In addition, we added a penalty function of the regularization term in the model optimization process, which can decrease the risk of overfitting the training data.
Considering that not all the patients would experience particular post-transplant complications (eg, aGVHD), we only enrolled the common transplant characteristics, and the model could be used in patients without post-transplant complications. For example, we observed that this model could predict EBV reactivation in patients without CMV-DNAemia or without severe aGVHD. This may help to increase the generality of our model.
In the present study, we observed that high-risk patients showed a higher incidence of NRM and a lower probability of survival compared with low-risk patients. Some studies also reported that EBV reactivation could increase the risk of mortality after allo-HSCT. 6,9,10 This also supported the clinical significance of our predicted model.
Several studies reported that prophylactic rituximab treatment 42 or EBV-specific T-cell infusions 43 could decrease the risk of EBV reactivation. Considering that the median time from HSCT to EBV reactivation was nearly 2 months, we may have plenty of chances to conduct risk stratification-directed EBV prophylaxis after HID HSCT in high-risk patients on the basis of our predicted model, while the low-risk patients can avoid unnecessary treatment-related toxicities.
Regarding the limitations of our study, although we confirmed the model in the validation cohort successfully, this cohort was relatively small. Also, it did not enroll patients receiving unrelated-donor or identical-sibling-donor allo-HSCT with ATG for prophylaxis. Our cohort did not enroll HID HSCT with post-transplant cyclophosphamide either, although EBV reactivation is relatively rare in these patients. 44 Thus, the model should be further evaluated by independent cohorts in multicenter studies with other donor types and transplant regimens. In the present study, only 1 patient died of PTLD. It is still premature for our model to predict EBV-related deaths, and this should be investigated further. Finally, only 3 patients showed new-onset EBV reactivation after chronic GVHD (cGVHD). We could not further identify the efficacy of our model in patients with cGVHD, and this requires additional study.

CONCLUSIONS
We have established a comprehensive model that could predict EBV reactivation in HID HSCT recipients using ATG for GVHD prophylaxis with machine learning. This is the first predictive model for these patients, who have a high risk of EBV reactivation, and it can be popularized easily. In future, prospective, multicenter studies can further confirm the efficacy of our predictive model. It can also help to conduct risk stratification-directed EBV prophylaxis after HID HSCT.