Efficacy and Safety of Anticoagulation Treatment in COVID-19 Patient Subgroups Identified by Clinical-Based Stratification and Unsupervised Machine Learning: A Matched Cohort Study
Yi Bian1,2† 
Yue Le1,2† 
Han Du3 
Junfang Chen4 Ping Zhang5 
Zhigang He1,2 Ye Wang1,2 Shanshan Yu1,2 Yu Fang1,2 Gang Yu1,2 Jianmin Ling1,2 Yikuan Feng1,2 Sheng Wei6 Jiao Huang7 
Liuniu Xiao1,2 Yingfang Zheng1,2 Zhen Yu8 
Shusheng Li1,2*- 1Department of Emergency Medicine, Tongji Medical College, Tongji Hospital, Huazhong University of Science and Technology, Wuhan, China
- 2Department of Critical Care Medicine, Tongji Medical College, Tongji Hospital, Huazhong University of Science and Technology, Wuhan, China
- 3Germany Research Center for Artificial Intelligence, Saarland Informatics Campus, Saarbrücken, Germany
- 4Intelligent Medicine Research Center, Greater Bay Area Institute of Precision Medicine (Guangzhou), Fudan University, Guangzhou, China
- 5Department of Neurology, Tongji Medical College, Tongji Hospital, Huazhong University of Science and Technology, Wuhan, China
- 6Ministry of Education Key Laboratory of Environment and Health, Department of Epidemiology and Biostatistics, School of Public Health, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China
- 7Center for Evidence-Based and Translational Medicine, Zhongnan Hospital of Wuhan, Wuhan, China
- 8Department of Critical Care Medicine, Renmin Hospital of Wuhan University, Wuhan, China
Objective: To explore the efficacy of anticoagulation in improving outcomes and safety of Coronavirus disease 2019 (COVID-19) patients in subgroups identified by clinical-based stratification and unsupervised machine learning.
Methods: This single-center retrospective cohort study unselectively reviewed 2,272 patients with COVID-19 admitted to the Tongji Hospital between Jan 25 and Mar 23, 2020. The association between AC treatment and outcomes was investigated in the propensity score (PS) matched cohort and the full cohort by inverse probability of treatment weighting (IPTW) analysis. Subgroup analysis, identified by clinical-based stratification or unsupervised machine learning, was used to identify sub-phenotypes with meaningful clinical features and the target patients benefiting most from AC.
Results: AC treatment was associated with lower in-hospital death risk either in the PS matched cohort or by IPTW analysis in the full cohort. A higher incidence of clinically relevant non-major bleeding (CRNMB) was observed in the AC group, but not major bleeding. Clinical subgroup analysis showed that, at admission, severe cases of COVID-19 clinical classification, mild acute respiratory distress syndrome (ARDS) cases, and patients with a D-dimer level ≥0.5 μg/mL, may benefit from AC. During the hospital stay, critical cases and severe ARDS cases may benefit from AC. Unsupervised machine learning analysis established a four-class clustering model. Clusters 1 and 2 were non-critical cases and might not benefit from AC, while clusters 3 and 4 were critical patients. Patients in cluster 3 might benefit from AC with no increase in bleeding events. While patients in cluster 4, who were characterized by multiple organ dysfunction (neurologic, circulation, coagulation, kidney and liver dysfunction) and elevated inflammation biomarkers, did not benefit from AC.
Conclusions: AC treatment was associated with lower in-hospital death risk, especially in critically ill COVID-19 patients. Unsupervised learning analysis revealed that the most critically ill patients with multiple organ dysfunction and excessive inflammation might not benefit from AC. More attention should be paid to bleeding events (especially CRNMB) when using AC.
Introduction
Coronavirus disease 2019 (COVID-19), caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), has developed into a pandemic disease and affected nearly every country in the world. There is no comprehensive and strong clinical evidence to support the efficacy of any drugs that specifically target the SARS-CoV-2 (1). Previous research has found that coagulopathy is very common in COVID-19 patients, and includes thrombosis and coagulation abnormalities and dysfunction such as an elevated D-dimer level and prolonged prothrombin time (PT), respectively (2). Autopsy histopathologic analysis has identified widespread thrombosis and microangiopathy in small vessels and capillaries of the lung (3–5), which are different from the pathologies observed in respiratory failure caused by other diseases (3, 6–8). Some scholars have therefore proposed anticoagulation (AC) treatment as an integral part of systemic therapy in the early stage of COVID-19 (9). Generally, retrospective studies have suggested that AC may decrease mortality in COVID-19 patients (9, 10), However, these conclusions are not completely reliable nor applicable to all COVID-19 patients due to limitations in methodology such as no prospective control or matching cohort (9–11), large heterogeneity in anticoagulant therapy (9–12), and a lack of subgroup analysis (10, 11, 13). A recently completed randomized controlled trial (RCT) found that, compared with usual-care thromboprophylaxis, an initial strategy of therapeutic-dose anticoagulation did not result in a higher probability of survival in critically ill COVID-19 patients (14). The conclusion of this RCT may be inconsistent with that of previous retrospective studies. At present, the recommendations for empiric systemic AC treatment currently differ between COVID-19 management guidelines (15–17), with some recommending using anticoagulant drugs preventively for patients who have no contraindications to AC and a significantly increased D-dimer level, while others recommend that all hospitalized adults with COVID-19 should receive pharmacologic thromboprophylaxis with low molecular weight heparin (LMWH) rather than unfractionated heparin (UFH).
We conducted a retrospective cohort study using a comprehensive database of COVID-19 patients to investigate whether AC treatment was protective and safe for COVID-19 patients. Innovative analyses using propensity score matching (PSM) and inverse probability of treatment weighting (IPTW) were performed to balance baseline covariates, variates related to AC treatment assignment and variates related to the outcome between patients with or without AC treatment. Further sensitivity analyses were carried out to explore the association between outcome and duration, dosage and type of AC treatment. The second aim of the study was to identify the patients who benefited most from AC treatment using subgroup analysis that involved stratifying the data according to the severity of the acute respiratory distress syndrome (ARDS) (18), COVID-19 clinical classification (17), and D-dimer levels. Taking into account the heterogeneity of the patients, clinically relevant patient subpopulations were identified by unsupervised machine learning algorithms. The effectiveness of AC treatment was verified further in identified clusters.
Materials and Methods
Ethics and Registration
This retrospective cohort study was approved by the ethics committee of Tongji Medical College, Huazhong University of Science and Technology (No. 2020-S220). The clinical trial was registered and verified by the Chinese Clinical Trial Registry (ChiCTR2000039855).
Patient Population
This single-center retrospective cohort study was conducted in two designated branches for COVID-19 patients in Tongji Hospital, an academic hospital affiliated to Tongji Medical College, Huazhong University of Science and Technology in Wuhan, China. All patients with confirmed COVID-19 admitted consecutively to these two institutions between Jan 25 to Mar 23, 2020, were enrolled retrospectively in the study. Approval was obtained from the ethics committee at our institution that the patients did not need to provide informed consent for inclusion in the study. Patients were assigned to three groups, including one group of patients with systemic AC treatment for at least 7 days, one group of patients with systemic AC treatment for <7 days and one group of patients without AC treatment. The medications administered and clinical outcomes were followed up to June 4, 2020, when these two branches for exclusive COVID-19 treatment were closed. All COVID-19 patients were diagnosed according to the World Health Organization interim guidelines (19) and the Diagnosis and Treatment Protocol for COVID-19 Patients (Trial Version 8) (17). The exclusion criteria for the study were younger than 18 years, pregnant, length of stay <24 h, insufficient medical information, a history of severe comorbidities requiring surgical operation including, but not limited to, multiple trauma, a severe infection that required debridement, amputation or laparotomy, and patients who were classified again as COVID positive after RNA for SARS-CoV-2 was detected following their discharge from hospital.
Anticoagulation Exposure
AC treatment was defined as receiving either UFH, LMWH, Fondaparinux sodium, Argatroban, or direct-acting oral anticoagulants (DOACs) (mainly Rivaroxaban). The initiation of AC treatment was decided by the bedside physicians. Possible reasons for AC treatment were extracted from electronic case files. Immortal time is a gap period between exposure and initiation of follow-up (20). We carried out a Cox proportional hazards model with a time-dependent manner for the drug exposure in this study.
Outcomes, Definitions, and Data Collection
The primary outcome of this study was in-hospital mortality. The safety endpoints included bleeding events and thrombocytopenia. Major bleeding was defined according to the International Society on Thrombosis and Haemostasis (ISTH) statement (21) as those that resulted in death, were life-threatening, caused chronic sequelae, or consumed major healthcare resources. Hemorrhage that did not fit the criteria for the ISTH definition of major bleeding but required medical intervention was classified as clinically relevant non-major bleeding (CRNMB) (22). Other bleeding events which did not meet the criteria of either major bleeding or CRNMB, including bloody sputum, positive fecal occult blood test/gastric occult blood test and microscopic hematuria, were reported separately. Thrombocytopenia was defined as a platelet count <100 × 109/L (23).
The CURB-65 score (21, 24), ARDS (18, 25, 26), and quick sequential organ failure assessment (qSOFA) (27) were defined according to the literature, while the COVID-19 clinical classification was made according to the Diagnosis and Treatment Protocol for COVID-19 Patients (Trial Version 8) (17). The detailed definition of ARDS and COVID-19 clinical classification is shown in Supplementary Table 1.
All the characteristics and clinical information of the patients were obtained from electronic medical and nursing record systems. This data included age, gender, current smoking history, comorbidities, laboratory results at admission, CURB-65 score and qSOFA score at admission, ARDS classification (18) and COVID-19 clinical classification at admission and during the hospital stay, antiviral therapies and other treatments during hospitalization, the level of oxygen therapy at admission, and the most intense level of oxygen therapy during hospitalization. Variables with missing data >20% were excluded from this analysis. Multiple imputations were conducted to address the presence of missing values.
Unsupervised Clustering
For this work, we used the K-Medoids clustering algorithm to partition our data into subclasses in an unsupervised manner. The K-Medoids algorithm randomly selects K samples in the training data as the medoids. The remaining samples are assigned to each subclass based on the pairwise dissimilarities. The sample, which is more similar to the medoid, is assigned to the corresponding subclass. Next, the medoids are updated based on the new results of subgrouping. These two steps are iterated multiple times until there is no change in the assignments. In particular, we used the Partitioning Around Medoids (PAM), which is the most common implementation of K-Medoids. For the dissimilarity measure, we adopted the Manhattan distance because it has better performance than the Euclidean distance for the data containing both binary and category variables (28).
A total of 25 variables representing the patients' clinical characteristics were used as input features in the unsupervised learning method, which included demographic features, comorbidities, vital signs, biomarkers, and oxygen therapy types at admission and during the hospital stay. Each patient in our database was presented as a vector with 25 dimensions. To prepare each patient's case as a vector for modeling training, we converted the binary variable as (0, 1), with the category variable represented by the corresponding categorical index. Normalization of the numerical variables was performed. After the normalization, we made sure that each numerical variable had a normal distribution.
Two prominent probabilistic model selection methods: Bayesian information criterion (BIC) and Akaike information criterion (AIC) were used to determine the optimal number of clusters in this work. In general, the measurement of AIC and BIC scores are similar. BIC penalizes the complexity of the model more than AIC (29).
Statistical Analysis
To minimize bias caused by the non-random allocation of potentially confounding covariates between the AC and non-AC groups, we adopted PSM methods (30). Propensity score (PS) was calculated using a logistic regression model, adjusted for the following covariates: level of oxygen therapy, clinical classification, high-sensitivity C reactive protein (hs-CRP), D-dimer levels, platelets count, CURB-65 score for the severity of pneumonia (31) at hospital admission, and the highest level of oxygen therapy during hospitalization. The match ratio was set at 1 to 3 and the maximum allowable distance (caliper) at 0.1 (32). To detect the selective bias potentially caused by this PSM, inverse probability of treatment weighting analysis (IPTW) was carried out based on the same variates used in PSM modeling (33).
Continuous variables were expressed as medians and interquartile ranges (IQR) and compared using the Mann-Whitney U test or Kruskal-Wallis H test. Categorical variables were compared using the Pearson χ2 test, continuity correction, or Fisher's exact test, as appropriate. Differences between clusters identified by unsupervised machine learning were judged by two-tailed Bonferroni correction post hoc tests, with a P-value < (0.05/6) considered statistically significant. A Kaplan-Meier curve was used to analyze survival during hospitalization, with the data stratified according to AC treatment and PAM clustering subphenotypes.
Univariate and multivariate Cox proportional hazards regression was used to determine the risk factors for in-hospital mortality in the PS matched cohort. Residual imbalanced variates were included in the multivariate Cox proportional hazards regression for the PSM cohort. Cox regression analysis with IPTW adjusted covariates of important demographic characteristics as well as variates associated with outcomes either reported previously (34–36) or by general clinic consensus, which included age, gender, platelets count, PT, D-dimer, total bilirubin, lactate dehydrogenase, urea and hs-CRP. Competing risk model analyses were carried out using Fine-Gray tests, which considered death as a competing event for the safety endpoints, including bleeding events and thrombocytopenia. Sensitivity analyses were carried out according to the AC exposure duration, type and dosage in the full cohort. In subgroups analysis among PS matched cohort, univariate and multivariate logistic regression analysis was used to explore the association of outcomes and AC treatment. Residual imbalanced variates were included in the multivariate logistic regression here. The interaction effect between AC treatment and subgroups was also analyzed by logistic regression.
SPSS version 26.0 software (IBM Corp., Armonk, New York, USA) and SAS version 9.4 (SAS Institute Inc. Cary, NC, USA) were used for the statistical analyses and PS matching. The Kaplan-Meier survival plot and forest plot were constructed using GraphPad Prism version 4.0 software (GraphPad Software Inc., La Jolla, CA, USA). All tests were two-tailed, with a P < 0.05 considered statistically significant.
Results
Clinical Characteristics of the Patients at Presentation
Two thousand four hundred and sixty nine confirmed COVID-19 patients admitted to Tongji Hospital between Jan 25 and Mar 23, 2020, were consecutively and unselectively reviewed. After excluding 197 patients who did meet study exclusion criteria, a total of 2,272 patients were identified for IPTW analysis and sensitivity analyses. In PSM modeling, 78 patients who received AC treatment for <7 days were not included for matching. Finally, PS matching yielded 165 patients in the AC group (patients who received AC for 7 days or longer) and 393 in the non-AC group (patients who did not receive AC) (Figure 1). Detailed AC treatment type, dosage, duration, time of initiation from admission and possible reasons for AC treatment were shown in Supplementary Figure 1. In the PS matched cohort, compared to the non-AC group, patients in the AC group were older (69 years, interquartile range [IQR] 60–78 vs. 65 years, IQR 53–71 years, P < 0.001) and had more comorbidities at admission (75.2% vs. 58.5%, P < 0.001, Table 1).
Figure 1. Flow diagram of the study. COVID-19, Coronavirus Disease 2019; AC, anticoagulation; hs-CRP, high sensitivity C-reactive protein; PS, propensity score.
Table 1. Patients Baseline Characteristics and Treatments in propensity score matched cohort.
Primary and Secondary Outcomes
In the PS matched cohort, univariate Cox proportional hazard regression analysis showed a significantly lower probability of in-hospital death in the AC treatment group compared to that in the non-AC treatment group (hazard ratio [HR] = 0.450; 95% confidence interval [CI], 0.278 to 0.727; P < 0.001). Since there were still residual imbalances between AC and non-AC groups, multivariate Cox proportional hazard regression was carried out by adjusting imbalance covariates, including age, smoking, comorbidities, white blood cells, lactate dehydrogenase, urea, D-dimer, antiviral therapy, intravenous immunoglobulin, and oxygen therapy during hospitalization. Multivariate Cox analysis showed that AC treatment was associated with a lower probability of in-hospital mortality (adjusted HR = 0.249, 95% CI 0.143 to 0.436, P < 0.001, Table 2). To explore the immortal time bias, a further Cox proportional hazards model with a time-dependent manner for AC exposure was carried out in this study. This showed that the AC treatment was still associated with a lower probability of in-hospital death risk in the PS matched cohort (adjusted HR = 0.531, 95% CI 0.301 to 0.935, P = 0.028, Figure 2).
Table 2. Primary and secondary outcomes of PS matched cohort.
Figure 2. Cumulative probability of death in COVID-19 patients with and without AC treatment. Patients with AC treatment had a lower probability of in-hospital mortality than those without AC treatment (adjusted* HR = 0.531, 95% CI: 0.301–0.935, P = 0.0283). *Adjusted for time-dependent AC exposure and baseline covariates including age, smoking, comorbidities, white blood cells, D-dimer, lactate dehydrogenase, urea, antiviral therapy, intravenous immunoglobulin, convalescent plasma therapy, and oxygen therapy in hospitalization. HR, hazard ratio; CI, confidence interval; AC, anticoagulation.
To detect the selective bias caused by PSM, IPTW analysis was performed in the full cohort including 2,272 patients. Multivariate Cox analysis again showed that AC treatment was associated with a lower probability of in-hospital mortality by comparing patients without AC treatment and patients with AC treatment for 7 days or longer (adjusted HR = 0.164; 95% CI: 0.104 to 0.260, P < 0.001). Sensitivity analyses were carried out according to the AC exposure duration, type and dosage in the full cohort. Duration of AC therapy <7 days was not associated with lower in-hospital mortality (adjusted HR = 1.018; 95% CI: 0.742 to 1.399, P = 0.910). In addition, AC treatment in all various dosages and types remained consistently associated with lower mortality (Table 3).
Table 3. Hazard ratio for in-hospital mortality in the full cohort by AC treatment duration, dosage and type in sensitivity analyses.
Secondary outcomes included bleeding events and thrombocytopenia. By adjusting baseline covariates and time-dependent AC exposure, it was revealed that AC treatment was associated with higher risk of total bleeding events (adjusted HR = 3.187, 95% CI, 1.846 to 5.504, P < 0.001), CRNMB (adjusted HR = 3.713 95% CI, 1.446 to 9.532, P = 0.006) and microscopic hematuria (adjusted HR = 2.624, 95% CI, 1.280 to 5.380, P = 0.008), but not associated with major bleeding (P = 1.000) and thrombocytopenia (adjusted HR = 2.167, 95% CI, 0.750 to 6.263, P = 0.153) (Table 2). Considering death as a competing event, a competing risk model analysis using the Fine-Gray test was conducted to explore the association between AC and bleeding events or thrombocytopenia. Finally, the AC group had no significantly higher risk of bleeding events (Fine-Gray test, P = 0.500) or thrombocytopenia (Fine-Gray test, P = 0.911) than the non-AC group when using death as a competing event in the model.
Clinical Subgroup Stratification
In-hospital mortality between the AC and the non-AC groups was compared in individuals stratified according to ARDS classification, COVID-19 clinical classification, and D-dimer levels at both hospital admission and during hospitalization. At hospital admission, AC treatment was associated with lower in-hospital mortality in subgroups of mild ARDS (adjusted odds ratio [OR] = 0.005, 95% CI, 0.000–0.174, P = 0.004), severe COVID-19 cases (adjusted OR = 0.076, 95% CI, 0.024–0.236, P < 0.001) and patients with a D-dimer level ≥0.5 μg/mL (adjusted OR = 0.042, 95% CI, 0.003–0.603, P = 0.020). During the hospital stay, AC treatment was associated with lower in-hospital mortality among patients who developed to severe ARDS (adjusted OR = 0.046, 95% CI, 0.013–0.157, P < 0.001) or critical COVID-19 (adjusted OR = 0.095, 95% CI, 0.034–0.266, P < 0.001) (Figure 3).
Figure 3. Subgroup analysis based on clinical stratification of in-hospital mortality between AC treatment and non-AC treatment patients. The multivariate logistic regression analysis adjusted for baseline covariates including age, smoking, comorbidities, white blood cells, D-dimer, lactate dehydrogenase, urea, antiviral therapy, intravenous immunoglobulin, convalescent plasma therapy, and oxygen therapy in hospitalization. ARDS, acute respiratory distress syndrome; AC, anticoagulation.
The interaction effect between AC treatment and the subgroups was also analyzed by logistic regression (Supplementary Table 2). An interaction effect was observed between the subgroups identified by ARDS at admission (P < 0.001) or clinical classification at admission (P < 0.001) and AC treatment.
PAM Clustering Analysis
The Optimal Number of Clustering Determination
BIC and AIC scores were used to automatically select the best number of clusters K for our model. As shown in Figure 4, we tried to find the optimal K by conducting an exhaustive search of the possible K values. By and large, the results from AIC and BIC are proportional. BIC score suggests that four clusters are optimal. For the AIC score, a shape “elbow point” is also indicated at four, after four the decrease is becoming notably smaller. Even though a better fitness of the data might be achieved by increasing the number of clusters, an additional cost will be needed including the over-fitting issue and the complexity of interpreting clinically plausible subgroups. Therefore, four clusters were selected to be optimal in this work.
Figure 4. Model selection and cluster visualization. The selection of the best number of clusters K is based on BIC and AIC score. K = 4 was chosen after comparing the BIC score of models with different number of clusters by unsupervised clustering analysis. For AIC score, K = 4 is also a good choice for the trade-off between model complexity and the fitting of the data. (A) Unsupervised clustering analysis for choosing the best number of clusters. BIC, Bayesian information criteria; AIC, Akaike information criterion. (B) Three-dimensional visualization of clustering results. We visualize our clustering results in three-dimensional space. The high dimensional training data was projected into three-dimensional space by using principal component analysis. PC, Principal component.
Clinical Features of Patients' Subgroups
The clinical features of the four clusters are shown in Tables 4, 5 and Figure 5. Patients in clusters 1 and 2 were non-critical cases. Patients in clusters 1 and 2 had no ARDS at admission then developed mild ARDS during the hospital stay. They mainly needed nasal cannula oxygenation at admission as well as during hospitalization. Within these non-critical clusters, cluster 1 was characterized by the youngest age (51 years, IQR 39–63) and had the least number of comorbidities (0, IQR 0–1). Compared to clusters 1 and 2, clusters 3 and 4 were critical cases. In laboratory testing results at admission, both clusters 3 and 4 had significantly higher neutrophils count, lower lymphocytes and platelets count, higher lactate dehydrogenase and hs-CRP, higher urea and creatinine, elevated prothrombin time and D-dimer. In vital signs, clusters 3 and 4 had significantly lower peripheral blood oxygen saturation and respiratory rate at admission. Either at admission or during the hospital stay, both clusters 3 and 4 had significantly more severe levels of ARDS and need a higher level of oxygen therapy. Notably, cluster 4 exhibited as the most critical sub-phenotype. Compared to cluster 3, patients in cluster 4 had significantly excessive inflammation (elevated white blood cells, neutrophils and lactate dehydrogenase), organ dysfunction (higher total bilirubin and urea), severe coagulopathy (elevated prothrombin time and D-dimer), unstable hemodynamics (higher rate of vasopressor use) and neurologic dysfunction (disturbance of consciousness). Compared to the other three clusters, cluster 4 had severe ARDS either at admission or during the hospital stay. As a result, cluster 4 also needed the highest level of oxygen therapy accordingly.
Table 4. Variables included in the PAM-based clustering model.
Table 5. Outcome and variables not included in the PAM-based clustering model.
Figure 5. Clinical characteristics and probability of death of the four clusters. Clusters 1 and 2 were non-critical cases from admission to discharge and they showed lower probability of death. Cluster 3 had significant higher in hospital mortality and probability of death than clusters 1 and 2, but lower than that in cluster 4. Cluster 4 had the highest in hospital mortality and probability of death, and there were significant differences with the other three clusters. (A) Clinical characteristics of four clusters. Some of the significant clinical features are plotted for each cluster. The features are displayed by color-coded heatmap with normalized values. (B) Comparison of probability of death between four clusters. There was statistically significant difference in the survival distribution between any two groups (P < 0.001), except that there was no statistically significant difference in the survival distribution of cluster 1 and cluster 2 (Log-Rank test, P = 0.512 > 0.008).
AC Treatment and Outcomes in Different Patients Clusters
In the non-critical cluster (clusters 1 and 2), we found no significantly lower in-hospital death risk associated with AC treatment (Figure 6). In cluster 3, patients who received AC treatment had a significantly lower in-hospital mortality than those who did not receive this treatment (adjusted OR = 0.027, 95% CI, 0.005 to 0.134, P < 0.001). However, patients in cluster 4, who had elevated inflammation biomarkers and even severe multi-organ dysfunction, did not benefit from AC treatment. For safety endpoint, AC treatment was not associated with increasing bleeding events (Figure 6). In addition, the interaction effect was found between the AC treatment and sub-phenotypes identified by PAM clustering (P < 0.001, Supplementary Table 2).
Figure 6. Comparing of in-hospital mortality and bleeding events between AC treatment and non-AC treatment patients based on unsupervised machine learning. AC, anticoagulation.
Discussion
COVID-19 infections have affected patients globally. The ISTH pointed out that COVID-19 patients develop a clinically significant coagulopathy, characterized by thrombocytopenia, mildly prolonged prothrombin time, and elevated serum D-dimer levels (27). Recent research indicates that coagulopathy is not only common in COVID-19 patients but is also associated with increased mortality (9). The potential mechanism for the development of coagulopathy in COVID-19 patients may be related to endothelial cell dysfunction (37) and hypoxia-induced thrombosis (38) following a SARS-CoV-2 infection. Because the endothelium plays an important role in regulating hemostasis, fibrinolysis, and vessel wall permeability, endothelial dysfunction in pulmonary microvessels may act as a trigger for immunothrombosis, resulting in coagulopathy. Histological analysis of pulmonary vessels in COVID-19 patients shows more widespread thrombosis with microangiopathy compared to that observed in patients with influenza. Based on this preliminary evidence, AC treatment may be beneficial for COVID-19 patients by inhibiting thrombin generation and thereby reducing mortality. The ISTH suggests that a prophylactic dose of LMWH should be considered in all patients without contraindications. Moreover, the Chinese Diagnosis and Treatment Protocol for COVID-19 Patients (Version 8.0) also suggested using AC treatment in selected patients. However, these recommendations require additional clinical evidence to determine the association between AC treatment and the outcome of COVID-19 patients, and also to clarify the indications, contradictions and optimal duration, dose, and time to use AC. We conducted this matched cohort study using a comprehensive source of COVID-19 patients. In general, results showed that receiving AC treatment was associated with a decreased in-hospital mortality in these patients. Although patients who received AC treatment exhibited a significant increase in CRNMB and microscopic hematuria, they had no increase in the incidence of major bleeding. A clinical subgroup analysis was also carried out to identify patient subgroups who receive greater benefit from AC treatment. At hospital admission, patients of severe COVID-19 clinical cases, patients with mild ARDS or patients who had a D-dimer level ≥0.5 μg/mL were more likely to benefit from AC therapy. During hospitalization, patients who developed severe ARDS or critical COVID-19 cases were more likely to benefit from AC therapy (Figure 3). Results of clusters identified by unsupervised machine learning revealed similar results as of clinical subgroups. Critical patients of cluster 3 could benefit from AC treatment whereas non-critical patients in clusters 1 and 2 did not. Nevertheless, a sub-phenotype (cluster 4) exhibited even severe multiple organ dysfunction and excessive inflammation might not benefit from AC therapy.
To date, several research works have investigated systemic AC therapy in COVID-19 patients (9–13, 39). Although the results generally suggested that AC treatment was associated with lower mortality of COVID-19 patients, a constant instruction for clinic application was not easy to conclude. Several possible reasons are worth to be noted. As a retrospective cohort study, imbalance of baseline covariates, covariates related to outcome and covariates related to exposure assignment might lead to biased results. Among the existing studies, some studies used propensity score methods for reducing the effects of confounding (12, 39), some did not (9, 10). In our study, we applied PSM which yield a relatively balanced cohort. We also did IPTW analysis, another propensity score method, to detect the selective bias potentially caused by PSM in the full cohort. The results from PS matched cohort and IPTW analysis in the full cohort both revealed that AC treatment was associated with lower death risk. We also noted that, without the PS method, either the crude results or baseline-adjusted results will lead to an adverse conclusion. Immortal time is a gap period between exposure (usually the span after cohort follow-up) and initiation of follow-up (20). This might cause potential immortal time bias and exaggerate the association between the exposure and outcome. As a result, we carried out a Cox proportional hazards model with a time-dependent manner for the drug exposure in this study.
Another key question concerns the confounders involving various durations, dosages and types of AC treatment. In a retrospective cohort study from the Mount Sinai Health System (11), the duration of hospitalization (median 5 days, IQR 3–8 days) and the course of AC treatment (median 3 days, IQR 2–7 days) were relatively short. Within the current consensus on anticoagulant therapy for venous thromboembolism, it is generally considered that patients with confirmed deep vein thrombosis or pulmonary embolism need LMWH treatment for at least 5 days followed by Dabigatran or Edoxaban (40). As a new disease without comprehensive study until now, to determine the optimal duration, type and dosage of AC treatment need more evidence. In our study, we conducted a series of sensitivity analyses to investigate the relationship between outcome and AC treatment duration, dosage and type. We found that AC treatment for 7 days or longer was associated with a lower death risk while AC treatment for <7 days was not. Low dose thromboprophylaxis, intermediate dose thromboprophylaxis and therapeutic dose anticoagulation were all associated with lower death risk. Although we recorded detailed AC treatment type, the majority was LMWH (Supplementary Figure 1), we only investigated LMWH and non-LMWH for sensitivity analyses here. It is revealed that both LMWH and non-LMWH treatment were associated with a lower death rate.
The heterogeneity of the research population is another vital confounder that influences the results. In our study, we analyzed a full cohort of unselected patients from two designated hospitals including mild to critical cases. In general, we found that AC treatment was associated with low mortality, which had a constant result with the previous studies. Furtherly, we investigated who might benefit from AC treatment in subgroup analyses. The stratification criteria used in our study included the most frequently used classification of clinical severity of the COVID-19 patients. It is well-known that hypoxia is a core clinical manifestation and major pathophysiology change that contributes to the death of COVID-19 patients (41–43). The classification of both ARDS (18, 29, 44) and COVID-19 clinical severity classification (17, 45) indicates the severity of hypoxia and accordingly, they are used frequently by clinicians to evaluate and triage patients and to decide major treatments (e.g., levels of oxygen therapy). As a result, we stratified patients according to ARDS classification, COVID-19 clinical classification, and D-dimer levels at both hospital admission and during hospitalization. By this strategy, we found that, at admission, severe cases of COVID-19 clinical classification, mild ARDS cases and patients with a D-dimer level ≥0.5 μg/mL may benefit from AC. While during the hospital stay, critical cases and severe ARDS cases may benefit from AC. These results were in constant with Sun et al.' study (9) with severe cases and subgroup analysis from Shen et al. (39) and CORIST Studies (12).
Clustering the study population may help minimize the influence of heterogeneity on the results. The traditional way to categorize patients is based on pre-defined standards. The standards are usually defined by a group of experienced experts with a strong background and prior knowledge in the medical area. Therefore, the procedure for generating the standards alone takes considerable effort and time. In addition, these standards cannot easily be quickly established or updated for a new situation in a short period, which was apparent when we faced this new pandemic, COVID-19. Unsupervised clustering algorithms in machine learning offer another perspective to perform the identification of data subclasses. Unsupervised clustering approaches can achieve more stable and robust clustering results without any prior knowledge of the meaning of each variable in the data. In addition, it may also identify some intrinsic correlations between the variables which sometimes cannot be easily noticed by human experts. Considering the heterogeneity of COVID-19 patients, an innovative strategy was carried out to identify subphenotype of patients who exhibited distinct clinical characteristics and respond to certain treatment using unsupervised learning approach (46). To this end, a four-class PAM-based clustering model was established, representing four distinct COVID-19 patient subphenotypes with different clinical characteristics. In particular, clusters 1 and 2 were non-critical cases with significantly lower mortality. Patients in these two clusters did not benefit from AC treatment. Clusters 3 and 4 were critical cases both exhibited significant abnormal laboratory testing results at admission and unstable vital sign. Cluster 3 had mild to moderate ARDS at admission and progressed to severe ARDS during the hospital stay. Patients in cluster 3 can benefit from AC treatment and had no significant increase in bleeding events. Compared to cluster 3, cluster 4 was the most critical cases and has the highest mortality. A novel result by the clustering approach was that, among these most critical patients (clusters 4), who had moderate or severe ARDS at admission and developed severe ARDS during the hospital stay, AC treatment was not associated with a lower death risk. Further characteristic analysis of these clusters revealed cluster 4 was characterized by multiple organ dysfunction and excessive inflammation. This led us to conclude that critical COVID-19 patients with these features cannot benefit from AC treatment. Recently, an open-label, adaptive, multiplatform, randomized control trial was published (14), with the researchers noting that the initial strategy of therapeutic-dose anticoagulation did not result in a greater probability of survival in critically ill COVID-19 patients (defined as COVID-19 that led to the receipt of ICU-level respiratory or cardiovascular organ support in an ICU) compared to usual-care thromboprophylaxis. This result was different from our clinical subgroup analysis but similar to the phenotypes of clusters 4 in our unsupervised clustering analysis.
Analysis of safety endpoints showed that although the risk of bleeding events, including CRNMB and microscopic hematuria, were higher in the AC group compared to the non-AC group, there was no significant difference in the risk of major bleeding events or thrombocytopenia between the two groups. In brief, the above findings suggested that the use of AC treatment for 7 days or longer in hospitalized COVID-19 patients was associated with increased CRNMB and microscopic hematuria but not with other bleeding events, especially major bleeding. These key observations are consistent with those reported in recent studies (11, 39). Although the competing risk model analysis in this study revealed that there was no significant difference in bleeding risk between the AC and non-AC groups when considering death as a competitive event, the increase of CRNMB still reminds clinicians should be more cautious when using anticoagulation treatment.
This study had several limitations. Firstly, as a retrospective cohort study, imbalanced confounders and selective bias may exist. Large-scale, multicenter, randomized, controlled trials are urgently needed to fully assess the efficacy of AC in patients with COVID-19. Besides, our unsupervised clustering model did not take the importance of each variable into consideration, as it treated all the variables equally as numerical values and measured the similarity between patients based on geometric distance. However, the variables could have completely different semantic meanings. Therefore, it is still necessary for human experts to inspect the clustering results to make sure that the results are explainable. Future work could be to intrinsically integrate the importance of clinic variables into the similarity measurements of unsupervised cluster models.
Conclusion
COVID-19 patients who received AC treatment for 7 days or longer had a significantly lower in-hospital death risk but not higher risk of major bleeding. Through the clinical subgroup analysis, critically ill patients were more likely to benefit from AC treatment. Specifically, the unsupervised machine learning model revealed that, within critically ill COVID-19 patients, patients characterized by multiple organ dysfunction (neurologic, circulation, coagulation, kidney and liver dysfunction) and excessive inflammation may not benefit from AC treatment.
Data Availability Statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Ethics Statement
This retrospective cohort study was approved by the ethics committee of Tongji Medical College, Huazhong University of Science and Technology (No. 2020-S220). The clinical trial was registered and verified by the Chinese Clinical Trial Registry (ChiCTR2000039855).
Author Contributions
YB, YL, SW, and SL conceptualized the paper. YB, ZH, and SL had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. HD and JC performed the unsupervised clustering analysis by machine learning and established the clustering model. YB, YL, ZH, YFa, GY, SY, YW, JL, YFe, LX, YZ, and ZY contributed to the acquisition, analysis, and interpretation of data. YB and YL drafted the manuscript. HD and JC drafted the clustering model part of the manuscript. PZ and SL helped revise the manuscript. YB, SW, and JH did the statistical analysis. SL and YB obtained the fundings. SL attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted. YB and SL supervised the study and are the guarantors. All authors contributed to the critical revision of the manuscript for important intellectual content and gave final approval of the version to be published.
Funding
This study was supported by the COVID-19 Rapid Response Research Project of Huazhong University of Science and Technology (Grant 2020kfyXGYJ049, to SL), and the Natural Science Foundation of Hubei Province (Grant No. 2021CFB376, to YB).
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher's Note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Acknowledgments
The authors thank all the included patients and their families, physicians, nurses, dieticians, and all staff.
Supplementary Material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fmed.2021.786414/full#supplementary-material
References
1. Asselah T, Durantel D, Pasmant E, Lau G, Schinazi RF. COVID-19: discovery, diagnostics and drug development. J Hepatol. (2021) 74:168–84. doi: 10.1016/j.jhep.2020.09.031
2. Huang C, Wang Y, Li X, Ren L, Zhao J, Hu Y, et al. Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China. Lancet. (2020) 395:497–506. doi: 10.1016/S0140-6736(20)30183-5
3. Ackermann M, Verleden SE, Kuehnel M, Haverich A, Welte T, Laenger F, et al. Pulmonary vascular endothelialitis, thrombosis, and angiogenesis in Covid-19. N Engl J Med. (2020) 383:120–8. doi: 10.1056/NEJMoa2015432
4. Fox SE, Akmatbekov A, Harbert JL Li G, Quincy Brown J, Vander Heide RS. Pulmonary and cardiac pathology in African American patients with COVID-19: an autopsy series from New Orleans. Lancet Respir Med. (2020) 8:681–6. doi: 10.1016/S2213-2600(20)30243-5
5. Magro C, Mulvey JJ, Berlin D, Nuovo G, Salvatore S, Harp J, et al. Complement associated microvascular injury and thrombosis in the pathogenesis of severe COVID-19 infection: a report of five cases. Transl Res. (2020) 220:1–13. doi: 10.1016/j.trsl.2020.04.007
6. Ding Y, Wang H, Shen H, Li Z, Geng J, Han H, et al. The clinical pathology of severe acute respiratory syndrome (SARS): a report from China. J Pathol. (2003) 200:282–9. doi: 10.1002/path.1440
7. Nicholls JM, Poon LLM, Lee KC, Ng WF, Lai ST, Leung CY, et al. Lung pathology of fatal severe acute respiratory syndrome. Lancet. (2003) 361:1773–8. doi: 10.1016/S0140-6736(03)13413-7
8. Ng DL, Al Hosani F, Keating MK, Gerber SI, Jones TL, Metcalfe MG, et al. Clinicopathologic, immunohistochemical, and ultrastructural findings of a fatal case of middle east respiratory syndrome coronavirus infection in the United Arab Emirates, April 2014. Am J Pathol. (2016) 186:652–8. doi: 10.1016/j.ajpath.2015.10.024
9. Tang N, Bai H, Chen X, Gong J, Li D, Sun Z. Anticoagulant treatment is associated with decreased mortality in severe coronavirus disease 2019 patients with coagulopathy. J Thromb Haemost. (2020) 18:1094–9. doi: 10.1111/jth.14817
10. Ayerbe L, Risco C, Ayis S. The association between treatment with heparin and survival in patients with Covid-19. J Thromb Thrombolysis. (2020) 50:298–301. doi: 10.1007/s11239-020-02162-z
11. Paranjpe I, Fuster V, Lala A, Russak AJ, Glicksberg BS, Levin MA, et al. Association of treatment dose anticoagulation with in-hospital survival among hospitalized patients with COVID-19. J Am Coll Cardiol. (2020) 76:122–4. doi: 10.1016/j.jacc.2020.05.001
12. Di Castelnuovo AF, Costanzo S, Iacoviello L. Heparin in COVID-19 patients is associated with reduced in-hospital mortality: the multicentre Italian CORIST Study. Thromb Haemost. (2021) 121:1054–65. doi: 10.1055/a-1347-6070
13. Evans EL, Povstyan OV, De Vecchis D, Macrae F, Lichtenstein L, Futers TS, et al. Impact of anticoagulation prior to COVID-19 infection: a propensity score–matched cohort study. Blood. (2020) 136:140–4. doi: 10.1182/blood.2019004174
14. Investigators R-C, Investigators ACA, Investigators A, Goligher EC, Bradbury CA, McVerry BJ, et al. Therapeutic Anticoagulation with Heparin in Critically Ill Patients with Covid-19. N Engl J Med. (2021) 385:777–89. doi: 10.1056/NEJMoa2103417
15. American Society of Hematology. COVID-19 and VTE/Anticoagulation: Frequently Asked Questions. (2021). Available online at: https://www.hematology.org/%20covid-19/covid-19-and-vte-anticoagulation (accessed April 12, 2021).
16. National Institutes of Health. COVID-19 Treatment Guidelines. (2021). Available online at: https://www.covid19treatmentguidelines.nih.gov./antithrombotic-therapy/ (accessed April 12, 2021).
17. National Health Commission of the People's Republic of China. Diagnosis and Treatment Protocol for COVID-19 Patients (Trial Version 8). (2020). Available online at: http://www.nhc.gov.cn/yzygj/s7653p/202008/0a7bdf12bd4b46e5bd28ca7f9a7f5e5a.shtml (accessed August 19, 2020).
18. Force ADT, Ranieri VM, Rubenfeld GD, Thompson BT, Ferguson ND, Caldwell E, et al. Acute respiratory distress syndrome: the Berlin Definition. JAMA. (2012) 307:2526–33. doi: 10.1001/jama.2012.5669
19. World Health Organization. Clinical Management of Severe Acute Respiratory Infection When Novel Coronavirus (2019-nCoV) Infection Is Suspected: Interim Guidance. (2020). Available online at: https://www.who.int/publications/i/item/10665-332299 (accessed April 12, 2021).
20. Suissa S. Immortal time bias in pharmaco-epidemiology. Am J Epidemiol. (2008) 167:492–9. doi: 10.1093/aje/kwm324
21. Schulman S Kearon Kearon C; Subcommittee on Control of Anticoagulation of the Scientific and Standardization Committee of the International Society on Thrombosis and Haemostasis. Definition of major bleeding in clinical investigations of antihemostatic medicinal products in non-surgical patients. J Thromb Haemost. (2004) 3:692–4. doi: 10.1111/j.1538-7836.2005.01204.x
22. Kaatz S, Ahmad D, Spyropoulos AC, Schulman S. Subcommittee on Control of Anticoagulation. Definition of clinically relevant non-major bleeding in studies of anticoagulants in atrial fibrillation and venous thromboembolic disease in non-surgical patients: communication from the SSC of the ISTH. J Thromb Haemost. (2015) 13:2119–26. doi: 10.1111/jth.13140
23. Thachil J, Tang N, Gando S, Falanga A, Cattaneo M, Levi M, et al. ISTH interim guidance on recognition and management of coagulopathy in COVID-19. J Thromb Haemost. (2020) 18:1023–6. doi: 10.1111/jth.14810
24. Neill AM, Martin IR, Weir R, Anderson R, Chereshsky A, Epton MJ, et al. Community acquired pneumonia: aetiologyand usefulness ofseveritycriteriaon admission. Thorax. (1996) 51:1010–6. doi: 10.1136/thx.51.10.1010
25. Ware LB, Matthay MA. The acute respiratory distress syndrome. New Engl J Med. (2000) 342:1334–49. doi: 10.1056/NEJM200005043421806
26. Riviello ED, Kiviri W, Twagirumugabe T, Mueller A, Banner-Goodspeed VM, Officer L, et al. Hospital incidence and outcomes of ARDS using the Kigali modification of the Berlin definition. Am J Respir Crit Care Med. (2016) 193:52–9. doi: 10.1164/rccm.201503-0584OC
27. Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, et al. The Third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA. (2016) 315:801–10. doi: 10.1001/jama.2016.0287
28. Aggarwal CC, Hinneburg A, Keim DA. On the surprising behavior of distance metrics in high dimensional space. In: International Conference on Database Theory. Berlin: Springer (2001). p. 420–34.
29. Weakliem David L. A critique of the Bayesian information criterion for model selection. Sociol Methods Res. (1999) 27:359–97. doi: 10.1177/0049124199027003002
30. Austin PC. An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivariate Behav Res. (2011) 46:399–424. doi: 10.1080/00273171.2011.568786
31. Lim WS, van der Eerden MM, Laing R, Boersma WG, Karalus N, Town GI, et al. Defining community acquired pneumonia severity on presentation to hospital: an international derivation and validation study. Thorax. (2003) 58:377–82. doi: 10.1136/thorax.58.5.377
32. Kuss O, Blettner M, Borgermann J. Propensity score: an alternative method of analyzing treatment effects. Dtsch Arztebl Int. (2016) 113:597–603. doi: 10.3238/arztebl.2016.0597
33. Geleris J, Sun Y, Platt J, Zucker J, Baldwin M, Hripcsak G, et al. Observational study of hydroxychloroquine in hospitalized patients with Covid-19. N Engl J Med. (2020) 382:2411–8. doi: 10.1056/NEJMoa2012410
34. Chen C, Chen C, Yan JT, Zhou N, Zhao JP, Wang DW. Analysis of myocardial injury in patients with COVID-19 and association between concomitant cardiovascular diseases and severity of COVID-19. Zhonghua Xin Xue Guan Bing Za Zhi. (2020) 48:567–71. doi: 10.3760/cma.j.cn112148-20200225-00123
35. Liang W, Liang H, Ou L, Chen B, Chen A, Li C, et al. Development and validation of a clinical risk score to predict the occurrence of critical illness in hospitalized patients with COVID-19. JAMA Intern Med. (2020) 180:1081–9. doi: 10.1001/jamainternmed.2020.2033
36. Wu C, Chen X, Cai Y, Xia J, Zhou X, Xu S, et al. Risk factors associated with acute respiratory distress syndrome and death in patients with coronavirus disease 2019 pneumonia in Wuhan, China. JAMA Intern Med. (2020) 180:934–43. doi: 10.1001/jamainternmed.2020.0994
37. Gando S, Levi M, Toh CH. Disseminated intravascular coagulation. Nat Rev Dis Primers. (2016) 2:16037. doi: 10.1038/nrdp.2016.37
38. Gupta N, Zhao YY, Evans CE. The stimulation of thrombosis by hypoxia. Thromb Res. (2019) 181:77–83. doi: 10.1016/j.thromres.2019.07.013
39. Shen L, Qiu L, Liu D, Wang L, Huang H, Ge H, et al. The association of low molecular weight heparin use and in-hospital mortality among patients hospitalized with COVID-19. Cardiovasc Drugs Ther. (2021) 4:1–8. doi: 10.1007/s10557-020-07133-3
40. Schulman S, Konstantinides S, Hu Y, Tang LV. Venous thromboembolic diseases: diagnosis, management and thrombophilia testing: observations on NICE guideline [NG158]. Thromb Haemost. (2020) 120:1143–6. doi: 10.1055/s-0040-1712913
41. Somers VK, Kara T, Xie J. Progressive hypoxia: a pivotal pathophysiologic mechanism of COVID-19 pneumonia. Mayo Clin Proc. (2020) 95:2339–42. doi: 10.1016/j.mayocp.2020.09.015
42. Grasselli G, Zangrillo A, Zanella A, Antonelli M, Cabrini L, Castelli A, et al. Baseline characteristics and outcomes of 1591 patients infected with SARS-CoV-2 admitted to ICUs of the Lombardy Region, Italy. JAMA. (2020) 323:1574–81. doi: 10.1001/jama.2020.5394
43. Xie J, Covassin N, Fan Z, Singh P, Gao W, Li G, et al. Association between hypoxemia and mortality in patients with COVID-19. Mayo Clin Proc. (2020) 95:1138–47. doi: 10.1016/j.mayocp.2020.04.006
44. Ferrando C, Suarez-Sipmann F, Mellado-Artigas R, Hernandez M, Gea A, Arruti E, et al. Clinical features, ventilatory management, and outcome of ARDS caused by COVID-19 are similar to other causes of ARDS. Intensive Care Med. (2020) 46:2200–11. doi: 10.1007/s00134-020-06251-8
45. Nicastri E, Petrosillo N, Ascoli Bartoli T, Lepore L, Mondi A, Palmieri F, et al. National Institute for the Infectious Diseases "L. Spallanzani”, IRCCS Recommendations for COVID-19 clinical management. Infect Dis Rep. (2020) 12:8543. doi: 10.4081/idr.2020.8543
Keywords: COVID-19, anticoagulation, outcomes, mortality, bleeding events, unsupervised machine learning
Citation: Bian Y, Le Y, Du H, Chen J, Zhang P, He Z, Wang Y, Yu S, Fang Y, Yu G, Ling J, Feng Y, Wei S, Huang J, Xiao L, Zheng Y, Yu Z and Li S (2021) Efficacy and Safety of Anticoagulation Treatment in COVID-19 Patient Subgroups Identified by Clinical-Based Stratification and Unsupervised Machine Learning: A Matched Cohort Study. Front. Med. 8:786414. doi: 10.3389/fmed.2021.786414
Received: 30 September 2021; Accepted: 26 November 2021;
Published: 24 December 2021.
Edited by:
Zhongheng Zhang, Sir Run Run Shaw Hospital, ChinaReviewed by:
Qilin Yang, The Second Affiliated Hospital of Guangzhou Medical University, ChinaChen Hui, The First Affiliated Hospital of Soochow University, China
Yanfei Shen, Zhejiang Hospital, China
Copyright © 2021 Bian, Le, Du, Chen, Zhang, He, Wang, Yu, Fang, Yu, Ling, Feng, Wei, Huang, Xiao, Zheng, Yu and Li. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Shusheng Li, lishusheng@hust.edu.cn
†These authors have contributed equally to this work and share first authorship