Development and Validation of a Radiomics Nomogram Based on 18F-Fluorodeoxyglucose Positron Emission Tomography/Computed Tomography and Clinicopathological Factors to Predict the Survival Outcomes of Patients With Non-Small Cell Lung Cancer

Purpose: In this study, we developed and validated a radiomics nomogram by combining the radiomic features extracted from ¹⁸F-fluorodeoxyglucose positron emission tomography/computed tomography (¹⁸F-FDG PET/CT) images and clinicopathological factors to evaluate the overall survival (OS) of patients with non-small cell lung cancer (NSCLC).

Patients and Methods: A total of 315 consecutive patients with NSCLC (221 in the training cohort and 94 in the validation cohort) were enrolled in this study. A total of 840 radiomic features were extracted from the CT and PET images. Three radiomic scores (rad-scores) were calculated using the least absolute shrinkage and selection operator (LASSO) Cox regression based on subsets of CT, PET, and PET/CT radiomic features. A multivariate Cox regression analysis was performed for each rad-score combined with clinicopathological factors to determine the independent risk factors. The OS nomogram was constructed based on the PET/CT rad-score and independent clinicopathological factors. Validation and calibration were conducted to evaluate the performance of the model in the training and validation cohorts, respectively.

Results: A total of 144 (45.71%) women and 171 (54.29%) men with NSCLC were enrolled in this study. The PET/CT rad-score combined with the clinical model had the best C-index (0.776 and 0.789 for the training and validation cohorts, respectively). Distant metastasis, stage, carcinoembryonic antigen (CEA), and targeted therapy were independent risk factors for patients with NSCLC. The validation curve showed that the OS nomogram had a strong predictive power in patients' survival. The calibration curve showed that the predicted survival time was significantly close to the observed one.

Conclusion: A radiomic nomogram based on ¹⁸F-FDG PET/CT rad-score and clinicopathological factors had good predictive performance for the survival outcome, offering feasible, and practical guidance for individualized management of patients with NSCLC.

Introduction

Lung cancer is a malignant tumor with the highest morbidity and mortality worldwide (1). Non-small cell lung cancer (NSCLC) is the most common pathological type of lung cancer, accounting for ~85% of all patients with lung cancer (1, 2). Considering that early signs and symptoms of NSCLC do not manifest in some patients, ~70% of patients have developed metastasis at the time of diagnosis and thus have lost the opportunity for surgical treatment (3, 4). The tumor-node-metastasis (TNM) staging system is currently the most commonly used tumor staging system worldwide and is considered to be the most valuable method for assessing the prognosis of malignant tumors (5–7). However, the TNM staging system still has several limitations when used to evaluate lung cancer prognosis in clinical practice. There are notable differences in the prognosis of tumors in the same stage, indicating that the TNM staging system cannot be used alone to fully evaluate the prognosis of patients with NSCLC. Thus, a comprehensive analysis of the TNM staging system in combination with other tumor biological characteristics that affect the prognosis of patients with NSCLC should be performed (8, 9). Therefore, determining additional effective prognostic indicators other than the TNM staging system, evaluating patients' responses to treatment at an early stage, and predicting the overall survival (OS) of patients are considered important to achieve individualized medical treatments.

With the development of genomic biology and technology, survival-related genomic characteristics have been included in the prognostic evaluation of several diseases, thereby improving the accuracy of the prognostic evaluation of several patients. However, the main limitation of these invasive technologies is that they cannot capture comprehensive information on the spatiotemporal heterogeneity of tumors (10–13). Therefore, an effective method is urgently required to comprehensively quantify the spatiotemporal heterogeneity of tumors and to evaluate the prognosis of several diseases. ¹⁸F-fluorodeoxyglucose positron emission tomography/computed tomography (¹⁸F-FDG PET/CT) is an important imaging method widely used for functional metabolic and anatomical/morphological imaging of various types of malignant tumors and metastatic lesions. ¹⁸F-FDG PET/CT provides not only intuitive imaging differences through image comparisons, but also several metabolic parameters to distinguish metabolically active or inactive tumor tissues. In particular, PET/CT has been widely used in clinical practice for the establishment of diagnosis, staging, efficacy monitoring, and prognostic evaluation of NSCLC (14–16). Several studies have confirmed that the FDG uptake of primary tumors is an independent risk factor for patients with early NSCLC (17, 18), but its application value in the prognostic evaluation of NSCLC is still controversial (19, 20). As an emerging and promising image analysis tool, radiomics is a non-invasive quantitative research method that can be used to convert medical images into mineable data for the identification of tumor heterogeneity. The integration of genetic pathology and imaging multimodality could improve the non-invasive quantitative analysis of tumor spatiotemporal heterogeneity and microenvironment (21, 22). Studies have shown that radiomics may have good predictive prognostic performance and decision support in oncology (23, 24). In previous studies, the texture characteristics or radiomics based on ¹⁸F-FDG PET/CT have been used to predict the EGFR and KRAS mutation status in patients with NSCLC, to evaluate NSCLC radiation tumor response, to predict the prognosis of patients with NSCLC after stereotactic body radiotherapy, and to stratify the risk of patients with poor prognosis. The results of these studies showed that the PET/CT-based texture characteristics or radiomics had good classification or predictive prognostic performance. Radiomics based on PET/CT may provide complementary information for predicting survival in patients with lung cancer (25–29). A nomogram is based on multivariate regression analysis and includes important influencing factors related to tumor prognosis. By constructing an intuitive graph using a statistical predictive model, the nomogram provides the numerical probability of a clinical event. The nomogram has become the focus of interest in cancer research in recent years and is considered a useful tool for quantifying risk (30–32).

Therefore, this study primarily aimed to construct a predictive model of the OS nomogram based on the radiomic features of PET/CT combined with the clinicopathological factors to predict prognosis and risk stratification as well as to determine the role of radiomic features in predicting the prognosis of NSCLC. To improve the prognostic assessment of patients with NSCLC, advancements in the areas of individualized treatment and precision medicine are necessary.

Patients and Methods

Patients and Clinicopathological Data

The institutional review board of Jinling Hospital, Medical School of Nanjing University approved this retrospective study and waived the need to obtain informed consent from the patients. This was a retrospective study, and the medical records of patients between October 2007 to August 2016 were reviewed. The medical records were searched consecutively, and 343 patients who had a lung tumor as assessed by histopathological analysis were identified. Patients with the following characteristics were included in the study: (a) patients undergoing PET/CT examination within 1 month before surgery or biopsy, (b) patients who did not receive antitumor treatment before PET/CT examination, and (c) patients with histologically confirmed NSCLC through surgery or biopsy. However, patients with the following characteristics were excluded: (a) patients with partial loss of PET or CT images (n = 15); (b) patients with diseases not related to NSCLC (n = 2), (c) patients with unclear tumor boundaries that could not be accurately delineated (n = 9), and (d) patients with metastases in the lung (n = 2). The final cohort included 315 patients (Figure 1). We randomly divided the patients into the training cohort (n = 221) and the validation cohort (n = 94) with a 7:3 ratio. Clinicopathological data were obtained from the patients' medical records, which included age, sex, family history, smoking history, histological grade, lymph node metastasis, distant metastasis, and TNM stage (defined according to the eighth edition of the TNM classification and staging system by the American Joint Committee on Cancer), histologic type (adenocarcinoma, squamous cell carcinoma, or not otherwise specified [nos]), treatment methods (surgery, chemotherapy, targeted therapy, and radiotherapy), thyroid transcription factor-1 (TTF-1) level, carcinoembryonic antigen (CEA) level, tumor location, and PET/CT metabolic parameters were obtained (Table 1). The survival information of these patients was obtained through telephone calls. Follow-up data were collected from October 2007 to January 2019. The mean and median follow-up periods were 37.99 (95% confidence interval [CI], 35.464–40.522) and 36.00 (range, 20.00–52.00) months, respectively. The endpoint of this study was OS, which was defined as the period from the date of ¹⁸F-FDG PET/CT examination to the date of telephone follow-up or the date of the patient's death.

FIGURE 1

Figure 1. Flowchart of patient enrollment, eligibility, and exclusion criteria of the dataset.

TABLE 1

Table 1. Clinicopathological factors of patients in the training and validation cohorts.

PET/CT Image Acquisition and Analysis

Patients underwent PET/CT imaging (Biography 16, Siemens, Erlangen, Germany) using ¹⁸F-FDG synthesized by the Canadian EBCO TR19 medical cyclotron and chemical synthesis system. All PET/CT acquisitions were carried out in free breathing mode, and no steps were taken to correct for motion. The radiochemical purity was >95%. All acquisitions were carried out in a free-breathing mode. The patients fasted for 6–8 h before undergoing the scan. Patients were intravenously injected with ¹⁸F-FDG (5.55 MBq/kg) and underwent a whole-body PET/CT scan of the skull base to the upper part of the thigh, and the data included CT and PET scans. The CT scanning parameters were as follows: power, 120 kV; current, 140 mAs; slice thickness and spacing, 5 mm; matrix, 512 × 512; and tube rotation speed, 0.8 s/r. The PET acquisition parameters were as follows: three-dimensional at 3 min/bed; iterative algorithm; iterations, 4; subset, 8; resolution, 4.1 mm lateral, 4.6 mm axial; matrix, 128 × 128; voxel size, 5.3 × 5.3 × 5.3 mm³. These settings were the same for all included patients. Images were reconstructed using an iterative reconstruction method resulting in CT, PET, and PET/CT fusion images that were transferred to a post-processing workstation. We used Microsoft Viewer software (version VB10, Siemens) to calculate the metabolic parameters on the PET images. PET images were first converted to SUV images in the software without other processing methods. Then, the 3-dimensional region of interest (ROI) was manually delineated by a radiologist (W.Q.G.) to calculate the maximum standard uptake value (SUVmax, with a threshold set to 40%), mean standard uptake value (SUVmean), and metabolic tumor volume (MTV). Subsequently, the total lesion glycolysis (TLG) (TLG=SUVmean × MTV) was calculated.

Tumor Segmentation

Our study followed and adhered to the Image Biomarker Standardization Initiative (IBSI) guidelines (33), and the software (Radiomics, Frontier, Siemens) used was IBSI-compliant. A volume of interest(VOI)was drawn semiautomatically around the tumor by a chest radiologist (Y.B., 9 years of experience) in the lung diagnosis using the radiomics prototype (Radiomics, Frontier, Siemens) and confirmed by another chest radiologist (W.Q.G., 5 years of experience). Both radiologists were blinded to the patients' clinical information. Firstly, we import CT images into radiomics prototype software (Radiomics, Frontier, Siemens). In the segmentation module, a few segmentation tools were available for semiautomatic delineation of the tumor in three dimensions. The segmentation was semiautomatically produced by drawing a line across the boundary of the tumor, then, the tool automatically find the neighboring voxels in 3D space with the same gray level through an automatic algorithm, and this is a Random Walker-based lesion segmentation for solid and subsolid lung lesions (34). The first step is to obtain a superset of voxels that may be part of the lesion. This can be implemented efficiently as a 3 D region growing starting from the center of the ROI. Then the thresholds can be fixed for lesions or determined adaptively from an analysis of the density distribution in the ROI. The region growing results in complete lesion and additionally parts of the attached vasculature. A morphological opening operation is applied to remove the vessels finally (35). If the segmentation wasn't right, the operators could correct it manually in the 3D domain using the radiomics prototype. The algorithm aimed at K-way image segmentation with given seeds indicating regions of the image belonging to the K objects(the objects to be segmented). Each seed specifies a location with a user-defined label. The algorithm labels an unseeded pixel by resolving the question: Given a random walker starting at this location, what is the probability that it first reaches each of the K seed points? It will be shown that this calculation may be performed exactly without the simulation of a random walk. By performing this calculation, we assign a K-tuple vector to each pixel that specifies the probability that a random walker starting from each un-seeded pixel will first reach each of the K seed points. A final segmentation may be derived from these K-tuples by selecting for each pixel the most probable seed destination for a random walker. By biasing the random walker to avoid crossing sharp intensity gradients, a quality segmentation is obtained that respects object boundaries (including weak boundaries) (36). And then in the radiomics module to click the computer features tool to calculate the CT radiomic features, and export the CT Masks+STL at the same time. Then we import the PET image into the software. If the tumor on the PET image is not at the same slice as the CT, we manually adjust the slice of PET image. Then, the CT Masks+STL will be imported into the software to cover the tumor on the PET image. If the CT Masks+STL does not cover the tumor, two radiologists(Y.B; W.QG) manually adjusted the CT Masks+STL through edit tools and reached a consensus to ensure that the CT Masks+STL completely covered the tumor lesions on the PET image as much as possible, and then use the same method to extract PET radiomic features. So, the 3D ROI (VOI) was delineated on CT image, and could be used by the PET image when the PET image were transformed to the CT image space using the transformation matrix obtained in PET-CT fusion.

Radiomic Feature Extraction

The Radiomic features from volumes of interest were then computed with both CT and PET images on a prototype that interfaces with the PyRadiomics library in manner similar to the 3D slicer's Radiomics plugin (34). The PyRadiomics library provides a variety of options to customize image pre-processing before feature extraction. Laplacians of Gaussian filtering, wavelet filtering, and non-linear intensity transforms were selected for image pre-processing. The feature classes contain 162 first-order features, 12 shape features, and 666 texture features. We also extracted numerous features (e.g., wavelets) that have not yet been standardized or validated by the IBSI. As a result, a total of 840 radiomic features were extracted from the CT and PET images using the software (Figure 2). The IBSI guidelines for reporting all necessary details are provided in the Supplemental Material.

FIGURE 2

Figure 2. The workflow in developing a radiomic overall survival nomogram. Computed tomography (CT) and positron emission tomography (PET) images were segmented semiautomatically using the Siemens radiomics prototype. Features including histogram features, shape features, texture features, and wavelet features were extracted from CT and PET images using the software. Three rad-scores were calculated using the least absolute shrinkage and selection operator Cox regression based on subsets of CT, PET, and PET/CT radiomic features. The predictive ability of CT, PET, and PET/CT rad-scores on overall patient survival was evaluated. The overall survival nomogram was constructed based on the PET/CT rad-score and clinicopathological factors.

Feature Selection and Radiomics Signature Construction

Considering the redundancy of the features and to reduce model overfitting, feature engineering was performed using two methods, Spearman correlation test and the least absolute shrinkage and selection operator (LASSO) Cox regression analysis. The Spearman correlation test was initially used to reduce feature redundancy, and a cutoff value of 0.9 was adopted. A ten-fold cross-validation LASSO Cox regression method, which is suitable for the regression of high-dimensional data in survival analysis, was conducted to select the most useful predictive features from the training cohort. The specified step of the LASSO Cox analysis included determining the optimized hyperparameter λ, which ensured that the model had the least deviance. Features with non-zero coefficients were preserved. The rad-score was calculated via a linear combination of selected features weighted by their respective coefficients. Three rad-scores including the CT, PET, and PET/CT rad-scores for each patient were calculated using PET, CT, and PET/CT features, respectively (Figure 2).

Clinicopathological Factor Analysis

Clinicopathological factors including PET/CT metabolic parameters were analyzed using a univariate Cox proportional hazards regression analysis. Factors with p < 0.05 were analyzed using the Kaplan–Meier curve and log-rank test. These significant factors were combined into a multivariate Cox proportional hazards regression analysis to identify independent risk factors.

Construction and Validation of OS Nomogram

Before constructing the OS nomogram, the performance of each rad-score was evaluated using the concordance index (C-index). The largest rad-score integrated with the independent factors was used to construct the nomogram. The prognostic ability of the nomogram was evaluated in the training cohort and validated in the validation cohort. The discrimination performance of the nomogram was assessed using Harrell's C-index. The C-index ranges between 0.5 and 1.0, with 0.5 indicating a random distribution of data and 1.0 indicating the outcome of the model perfectly predicting the observed survival information. The calibration curves of the nomogram were subsequently drawn for the patients' 5-year OS. The calibration curves were used to determine the independent risk factors and also illustrated both survival probabilities predicted by the nomogram and the observed probabilities.

Statistical Analysis

R software (version 3.5.0, www.Rproject.org) was used for statistical analysis in this study. LASSO was conducted using the “glmnet” package, while the “hdnom” package was used for the survival analysis. All statistical tests were two-sided, with a significance level of 0.05. Finally, a decision curve analysis was conducted using the “rmda” package to determine the clinical usefulness of radiomics nomogram by quantifying the net benefits at different threshold probabilities (37).

Results

Clinical Characteristics of Patients

The study patients were divided into two groups: the training cohort with 221 patients (120 men and 101 women) and a validation cohort with 94 patients (51 men and 43 women). There were no significant differences in sex, family history, smoking status, histological grade, lymph node metastasis, distant metastasis, TNM stage, and TTF-1 level (p = 0.054–1.000) between the training and validation cohorts. CEA levels were significantly different between the training and validation cohorts (p = 0.023). Other clinicopathological characteristics are shown in Table 1.

Establishment of Multivariate Cox Proportional Hazards Model

Before constructing the final model, we used a multivariate Cox regression analysis to test the hazard ratio (HR) of each parameter and to determine its significance in the probability of death. The results were as follows: distant metastasis (HR, 1.94 [95% CI, 1.17–3.21]), (HR, 1.71 [95% CI, 0.81–3.61]); stage (HR, 3.24 [95% CI, 1.74–6.02]), (HR, 8.34 [95% CI, 2.28–30.56]); CEA (HR, 1.12 [95% CI, 1.03–1.21]), (HR, 1.18 [95% CI, 1.01–1.37]) and targeted therapy (HR, 0.35 [95% CI, 0.22–0.56]), (HR, 0.41 [95% CI, 0.19–0.89]) were the independent risk factors in the training and validation cohorts, respectively (Table 2).

TABLE 2

Table 2. HR analysis for the different independent clinicopathological factors for clinical model.

Important Radiomic Features Selection and Calculation of the Rad-Score: Model Construction and Comparison

We performed a selection using the LASSO regression model on the PET/CT features, as shown in Figures 3A,B. To calculate the rad-score, the following six important features were selected from the 840 radiomic features, as shown in Figure 3C: PET_wavelet_HLH_glcm_Inverse Variance, CT_wavelet_LLL_glrlm_Long Run Low Gray Level Emphasis,PET_wavelet_LHL_firstorder_Maximum,CT_wavelet_LHL_firstorder_Mean, PET_wavelet_HLL_firstorder_Kurtosis, and CT_original_glszm_Small Area High Gray Level Emphasis. Subsequently, the rad-scores were calculated. The PET/CT rad-score was determined using the following formula: Rad-score=0.19 × CT_original_glszm_SmallAreaHighGrayLevelEmphasis+0.07 × CT_wavelet_LHL_firstorder_Mean+0.017 × CT_wavelet_LLL_glrlm_LongRunLowGrayLevelEmphasis+0.028 × PET_wavelet_HLH_glcm_InverseVariance+0.104 × PET_wavelet_HLL_firstorder_Kurtosis+0.05 × PET_wavelet_LHL_firstorder_Maximum−0.019.

FIGURE 3

Figure 3. (A,B) Radiomic features were selected using the ten-fold cross-validation least absolute shrinkage and selection operator Cox regression model in the training cohort (number of patients: 221). The following two steps were included: determining the hyperparameter/lambda with a partial likelihood deviance as the criterion (top row) and using the optimized/lambda (the vertical dashed line) to select features with nonzero coefficients (bottom row). (C) A total of six important radiomic features were selected.

We constructed three rad-scores based on CT features, PET features, and PET/CT combined features. The C-index of the rad-scores is shown in Table 3. Among these three rad-scores, the CT rad-scores were 0.685 and 0.658 in the training and validation cohorts, respectively. The PET rad-score had a lower C-index (0.662 and 0.611 for the training and validation cohorts, respectively) than the CT rad-score. The PET/CT rad-score had the best C-index (0.706 and 0.661 for the training and validation cohorts, respectively). The C-index values of the clinical model with clinicopathological factors were 0.730 and 0.774 in the training and validation cohorts, respectively (Table 3). The C-index values of the TNM stage and tumor volume (0.618 and 0.635 for the training and validation cohorts, respectively) were significantly higher than that of the TNM stage (0.552 and 0.531, respectively) or tumor volume (0.607 and 0.644, respectively) alone (Table 3). A rad-score was combined with the clinicopathological factors to construct a nomogram based on LASSO, as shown in Figure 4A. The C-index (0.776 and 0.789 for the training and validation cohorts, respectively) of the PET/CT rad-score combined with the clinical model was higher than that of the clinical model without the rad-score (Table 3). The validation of the nomogram showed that it had good predictive performance, as shown in Figures 4B,C. The calibration curve showed that the predicted probability was significantly close to the actual survival time of patients, as shown in Figures 4D,E. We also analyzed the association of PET/CT rad-score, OS nomogram, tumor volume, stage, and clinical model with the survival time of patients with NSCLC using a Kaplan-Meier analysis. Figures 5A–E shows the survival probability of the patients in the high-risk or low-risk cohorts. The results of the log-rank test indicate significant discrimination between the two groups.

TABLE 3

Table 3. Harrell's concordance index of different modalities.

FIGURE 4

Figure 4. (A) Establishment of a comprehensive nomogram by combining the positron emission tomography/computed tomography (PET/CT) rad-score and clinicopathological factors for predicting the 5-year overall survival of patients with non-small cell lung cancer. (B,C) A validation analysis of the nomogram showed that the area under the curve (AUC) at five time points was obtained on the training and validation cohorts. The AUC for predicting the prognosis from 1–5 years was >0.7. With the extension of follow-up time, the predicted AUC gradually increased, indicating that the nomogram has a good performance in predicting prognosis. (D,E) The calibration curve was used to estimate the 5-year overall survival predicted using a nomogram. The diagonal gray line represents an ideal evaluation, and the blue line represents the performance of the nomogram. The calibration curves for the training and validation cohorts showed the calibration of two cohorts in terms of the agreement between the estimated and observed 5-year outcomes.

FIGURE 5

Figure 5. (A–E). Predictive performance of the PET/CT rad-score, overall survival nomogram, tumor volume, stage, and clinical model Kaplan-Meier survival analysis of the patients in the high- and low-risk groups in the training cohort. Kaplan-Meier analysis for the PET/CT rad-score (A), overall survival (OS) nomogram (B), tumor volume (C), stage (D), and clinical model (E). The patients were stratified into high- and low-risk groups based on PET/CT rad-score (A, p < 0.0001, log-rank test), OS nomogram (B, p < 0.0001, log-rank test), tumor volume (C, p < 0.0001, log-rank test), stage (D, p < 0.0001, log-rank test), and clinical model (E, p < 0.0001, log-rank test).

To determine the clinical usefulness of the radiomics nomogram model, a decision curve analysis was performed. The decision curve analysis showed that the radiomics nomogram had a higher overall net benefit than 4 other clinical models (tumor volume, TNM stage, TNM stage, and tumor volume, and clinical model) across the majority of the range of reasonable threshold probabilities as shown in Figure 6.

FIGURE 6

Figure 6. Decision curve analysis for each model. The y-axis denotes the net benefit, which was calculated using true-positive and false-positive results. The radiomics nomogram model has the highest net benefit at the threshold from 0.1 to 0.9 among all positive predictions (line labeled “All”), all negative prediction (line labeled “None”), and another 4 clinical models (line labeled “Tumor volume, tumor-node-metastasis [TNM] stage, TNM stage and tumor volume, and clinical model”).

Discussion

In this study, we used ¹⁸F-FDG PET/CT radiomics to investigate the prognosis of patients with NSCLC. We extracted radiomic features from CT and PET, constructed a radiomics signature, and calculated the rad-score. Subsequently, we compared the predictive performance of CT, PET, and PET/CT rad-scores to determine the prognosis of patients with NSCLC. Considering that PET/CT has the best predictive performance among the three modalities, we further combined the PET/CT rad-score with the clinicopathological factors to predict the prognosis of patients with NSCLC. In addition, we performed Cox proportional hazards regression analysis on the clinicopathological risk factors and selected the independent risk factors related to the patient's prognosis. Finally, we constructed two prediction models based on LASSO: clinical models with and without the rad-score. In addition, 3 other clinical models were established (TNM stage, tumor volume, and tumor volume) to predict the prognosis of patients with NSCLC. Our results showed that the OS nomogram had good predictive performance for prognosis and could successfully stratify patients into high-risk and low-risk groups.

We extracted 840 radiomic features from CT, PET, and PET/CT images. To avoid redundancy and overfitting caused by the small sample size and additional radiomic features, we used the LASSO method to select important radiomic features. LASSO can be used to select biomarkers from high-dimensional radiomic features to overcome the problem of a small sample size and to select features that are most relevant to survival time (38). In addition, the LASSO method with cross-validation, as presented in this study, can be used to elegantly address issues of overfitting, collinearity, and multiple-hypothesis testing in feature selection. The LASSO method was also used to select radiomic features related to prognosis that were consistent with previous reports (39, 40). Furthermore, our prediction performance after using the LASSO method was better than the prediction performance of previous studies (38, 41, 42). Finally, we selected a total of six important radiomic features to construct CT, PET, and PET/CT radiomics signatures. The rad-scores were subsequently calculated for the three modalities to compare their predictive performances, revealing that PET/CT had the best predictive performance. Hence, we further studied the PET/CT modality, and combined the PET/CT rad-score with the clinicopathological factors that acquired good predictive performance with the C-index (0.776 and 0.789 for the training and validation cohorts, respectively). Our results showed that TNM staging was inconsistent with prognostic assessment; therefore, the prognosis cannot be predicted well. Radiomics can be used to comprehensively and quantitatively assess the spatiotemporal heterogeneity of tumors, and when combined with clinicopathological factors, the predictive performance of prognosis may be improved. According to Kirienko et al. (43), the Cox proportional hazards regression model was established based on CT, PET, and PET/CT radiomic signatures to predict the disease-free survival of patients with NSCLC. The results showed that the Cox proportional hazards regression models including radiomic features for the CT, PET, and PET/CT images had areas under the curve (AUCs) of 0.75, 0.68, and 0.68, respectively. The addition of clinicopathological risk factors to the Cox proportional hazards regression models resulted in AUCs of 0.61, 0.64, and 0.65 for the CT, PET, and PET/CT images, respectively. Mattonen et al. (44) constructed a Cox proportional hazards model that included stage and an MTV plus penumbra texture feature to predict recurrence/progression in NSCLC based on the LASSO method. The results showed that the C-index of the training and validation sets of this multivariate model were both 0.74. Wang et al. (45) used the consensus clustering method to automatically select the stable and prognostic radiomic features and subsequently constructed a multivariate Cox proportional hazards model that incorporated CT radiomic, clinical, and hematological features. These were found to be more predictive with a C-index of 0.792 and retained a C-index of 0.743 in the cross-validation analysis, therefore outperforming the radiomic, clinical, or hematological models. In addition, Dissaux et al. (27) and Oikonomou et al. (28) used PET/CT-based radiomics to predict the prognosis of patients with lung cancer who were treated with stereotactic body radiotherapy (SBRT). The results showed that radiomic features derived from PET/CT were associated with local control in patients with NSCLC undergoing SBRT, and can be used as predictors of OS, disease-specific survival, and regional control. Radiomics on PET/CT provided complementary information for the prediction of control and survival in patients with SBRT-treated lung cancer and could be helpful in clinical decision-making. The above studies showed that radiomic features were related to prognosis and had good prognostic predictive performance. Although some of the studies were multi-center studies, the sample sizes were generally small; therefore, the prediction model may have been overfitted. Our prediction model has better predictive performance and our sample size was much larger compared to previous studies. Our results indicated that the rad-score from the CT, PET, or PET/CT group has a favorable predictive power for survival. Moreover, the PET/CT rad-score had the best performance among the three rad-scores and could improve the predictive performance of the PET/CT models when combined with the clinicopathological factors. Hence, we believe that more tumor details are contained in the PET/CT entity model compared to an individual CT or PET entity model, a finding consistent with the findings of previous studies.

Additionally, we performed univariate Cox regression analysis on the clinicopathological factors to test the HR of each parameter and to determine its significance in the probability of death. The results showed that distant metastasis, stage, CEA, and targeted therapy were independent risk factors; We subsequently constructed a nomogram by combining the PET/CT rad-score and clinicopathological factors. The calibration curve showed that the predicted probability was significantly close to the actual survival time of patients. The validation of the OS nomogram showed that with the extension of follow-up time, the AUC for predicting prognosis gradually increased, and our results indicated that the OS nomogram had good predictive performance. We also evaluated the reliability of the PET/CT rad-score, OS nomogram, tumor volume, stage, and clinical model in predicting patient survival using a Kaplan–Meier analysis. The results of the Kaplan–Meier analysis demonstrated that the OS nomogram can clearly divide the patients into high-risk and low-risk groups, indicating that our nomogram had a strong predictive power in patients with high and low risks. Thus, it is considered significantly robust and reliable, and can be used as evidence for additional treatment and close follow-up in patients with poor prognosis, which is consistent with the research results of Dessroit et al. (29). In addition, the decision curve analysis demonstrated that the radiomics nomogram was superior to 4 other clinical models (tumor volume, TNM stage, TNM stage and tumor volume, and clinical model) across the majority of the range of reasonable threshold probabilities, which indicated that the radiomics nomogram added incremental value to the traditional staging system and other clinicalpathologic factors for individualized estimations. We believe that with the combination of the rad-score and clinicopathological factors to construct an OS nomogram, the predictive performance was largely improved, suggesting that the rad-score played an important role in the predictive accuracy of the OS of patients with NSCLC, a result that was consistent with the results of previous studies (46, 47). It is worth nothing that because of variations in technical parameters or inconsistent imaging parameters, a limited sample size, and heterogeneous patient characteristics, radiomic features may be insignificant in predicting prognosis in certain situations. Therefore, cohorts and validation datasets need to be evaluated, methodologies need to be standardized, and data on studies that evaluate radiomic features need to be harmonized in future studies, especially those with retrospective multi-centric datasets (48, 49).

Our study has some limitations, including the relatively small sample size and single-center cohort, the retrospective nature of the data, and the lack of external validation, which may have introduced selection bias, thereby resulting in poor model generalization and capacity. However, we plan to rapidly expand the sample size, and multi-center cohorts should be recruited for validation in the near future. Secondly, in this study, all texture matrices using 26-connectivity to find the neighboring voxels with distance 1 and 13 angles. Finally, the value of a feature is calculated for each angle separately, after which the mean of these values is used. However, it has been shown this strategy leads to less informative features compared to extracting the feature from as single matrix implementing all 13 directions, so, strategy should be implemented to merge the angle specific features into the texture matrix in future studies. Thirdly, We measured four metabolic parameters in a different way (different volume, different software) than the other radiomic features. It may lead to a bias in the comparison of the potential value of these four metrics with respect to all other features calculated by pyradiomics. And the stability and repeatability of the MTV and TLG values derived from a volume determined through a fixed threshold at 40% of maximum intensity are still controversial. So, we will try to use a consensus of several manual delineation instead of fixed thresholding to calculate tumor metabolic volume in the near future study (50, 51). In addition, all PET/CT acquisitions were carried out in free breathing mode, and no steps were taken to correct for motion may lead to extraction of features might have been suboptimal in the case of small lesions affected by motion, but also in some larger heterogeneous uptakes affected by motion blur.

In conclusion, the identified radiomic signature based on PET/CT can be potentially used as a biomarker for risk stratification of the OS in patients with NSCLC. The OS nomogram combining radiomics and clinicopathological factors for individualized OS estimation may provide more precise guidance for the accurate diagnosis and treatment of NSCLC in clinical practice.

Data Availability Statement

The raw data supporting the conclusions of this manuscript will be made available by the authors, without undue reservation, to any qualified researcher. Requests to access the datasets should be directed to Hong Zhu, zh_zy@163.com.

Ethics Statement

The Institutional Review Board of Jinling Hospital, Medical School of Nanjing University approved this retrospective study and waived the need to obtain informed consent from the patients.

Author Contributions

BY conceived the idea of the study. BY, JiaZ, JinZ, LM, AL, HJ, CZhou, and QW collected the data. HZ and GL performed image analysis. BY wrote the manuscript. SD performed the statistical analysis. SD, LZ, CZhu, JT, FW, and GL edited and reviewed the manuscript. All authors contributed to the article and approved the submitted version.

Funding

This work was supported by the National Key Research and Development Program of China (2017YFC0113400 for LZ) and Natural Science Foundation of Jiangsu Province (BK2011665).

Conflict of Interest

SD was employed by GE Healthcare China.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We would like to thank all the personnel from the Department of Nuclear Medicine and Department of Medical Imaging, Jinling Hospital, Medical School of Nanjing University for their collective efforts on the image post-processing, clinical management, and follow-up. Additionally, we would like to thank Dr. Yingqian Ge and Dr. Chengyan Wang for their guidance in our project.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fonc.2020.01042/full#supplementary-material

Abbreviations

C-index, Harrell's concordance index; CEA, carcinoembryonic antigen; HR, hazard ratio; MTV, metabolic tumor volume; NSCLC, non-small cell lung cancer; NOS, not otherwise specified; PET/CT, positron emission tomography/computed tomography; rad-score, radiomic score; SUVmax, maximal standard uptake value; SUVmean, mean standard uptake value; TLG, total lesion glycolysis; TNM, tumor-node-metastasis.

References

1. Siegel RL, Miller KD, Jemal A. Cancer statistics, 2018. Cancer J Clin. (2018) 68:7–30. doi: 10.3322/caac.21442

CrossRef Full Text | Google Scholar

2. Addario BJ. Lung cancer is a global epidemic and requires a global effort. Ann Transl Med. (2015) 3:26. doi: 10.3978/j.issn.2305-5839.2015.01.20

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Curran WJ, Paulus R, Langer CJ, Komaki R, Lee JS, Hauser S, et al. Sequential vs. concurrent chemoradiation for stage III non-small cell lung cancer: randomized phase III trial RTOG 9410. J Natl Cancer Inst. (2011) 103:1452–60. doi: 10.1093/jnci/djr325

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Molina JR, Yang P, Cassivi SD, Schild SE, Adjei AA. Non-small cell lung cancer: epidemiology, risk factors, treatment, and survivorship. Mayo Clin Proc. (2008) 83:584–94. doi: 10.4065/83.5.584

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Goldstraw P, Chansky K, Crowley J, Rami-Porta R, Asamura H, Eberhardt WE, et al. The IASLC lung cancer staging project: proposals for revision of the TNM stage groupings in the forthcoming (Eighth) Edition of the TNM classification for lung cancer. J Thorac Oncol. (2016) 11:39–51. doi: 10.1016/j.jtho.2015.09.009

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Chansky K, Sculier JP, Crowley JJ, Giroux D, Van Meerbeeck J, Goldstraw P, et al. The International Association for the Study of Lung Cancer Staging Project: prognostic factors and pathologic TNM stage in surgically managed non-small cell lung cancer. J Thorac Oncol. (2009) 4:792–801. doi: 10.1097/JTO.0b013e3181a7716e

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Kawaguchi T, Takada M, Kubo A, Matsumura A, Fukai S, Tamura A, et al. Performance status and smoking status are independent favorable prognostic factors for survival in non-small cell lung cancer: a comprehensive analysis of 26, 957 patients with NSCLC. J Thorac Oncol. (2010) 5:620–30. doi: 10.1097/JTO.0b013e3181d2dcd9

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Higashi K, Ueda Y, Arisaka Y, Sakuma T, Nambu Y, Oguchi M, et al. 18F-FDG uptake as a biologic prognostic factor for recurrence in patients with surgically resected non-small cell lung cancer. J Nucl Med. (2002) 43:39–45. doi: 10.1587/transcom.E96.B.802

PubMed Abstract | CrossRef Full Text | Google Scholar

9. William WN, Lin HY, Lee JJ, Lippman SM, Roth JA, Kim ES. Revisiting stage IIIB and IV non-small cell lung cancer: analysis of the surveillance, epidemiology, and end results data. Chest. (2009) 136:701–9. doi: 10.1378/chest.08-2968

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Halabi S, Lin CY, Kelly WK, Fizazi KS, Moul JW, Kaplan EB, et al. Updated prognostic model for predicting overall survival in first-line chemotherapy for patients with metastatic castration-resistant prostate cancer. J Clin Oncol. (2014) 32:671–7. doi: 10.1200/JCO.2013.52.3696

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Zhang JX, Song W, Chen ZH, Wei JH, Liao YJ, Lei J, et al. Prognostic and predictive value of a microRNA signature in stage II colon cancer: a microRNA expression analysis. Lancet Oncol. (2013) 14:1295–306. doi: 10.1016/S1470-2045(13)70491-1

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Tran B, Dancey JE, Kamel-Reid S, McPherson, JD, Bedard PL, et al. Cancer genomics: technology, discovery, and translation. J Clin Oncol. (2012) 30:647–60. doi: 10.1200/JCO.2011.39.2316

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Hofman V, Ilie M, Long E, Lassalle S, Butori C, Bence C, et al. Immunohistochemistry and personalised medicine in lung oncology: advantages and limitations. Bull Cancer. (2014) 101:958–65. doi: 10.1684/bdc.2014.2041

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Grootjans W, de Geus-Oei LF, Troost EG, Visser EP, Oyen WJ, Bussink J. PET in the management of locally advanced and metastatic NSCLC. Nat Rev Clin Oncol. (2015) 12:395–407. doi: 10.1038/nrclinonc.2015.75

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Kirienko M, Gallivanone F, Sollini M, Veronesi G, Voulaz E, Antunovic L, et al. FDG PET/CT as theranostic imaging in diagnosis of non-small cell lung cancer. Front Biosci. (2017) 22:1713–23. doi: 10.2741/4567

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Hatt M, Majdoub M, Vallières M, Tixier F, Le Rest CC, Groheux D, et al. 18F-FDG PET uptake characterization through texture analysis: investigating the complementary nature of heterogeneity and functional tumor volume in a multi-cancer site patient cohort. J Nucl Med. (2015) 56:38-44. doi: 10.2967/jnumed.114.144055

CrossRef Full Text | Google Scholar

17. Goodgame B, Pillot GA, Yang Z, Shriki J, Meyers BF, Zoole J, et al. Prognostic value of preoperative positron emission tomography in resected stage I non-small cell lung cancer. J Thorac Oncol. (2008) 3:130–4. doi: 10.1097/JTO.0b013e318160c122

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Kim HR, Kim DJ, Lee WW, Jheon S, Sung SW. The significance of maximum standardized uptake values in patients with stage I pulmonary adenocarcinoma. Eur J Cardiothorac Surg. (2009) 35:712–6. doi: 10.1016/j.ejcts.2008.12.030

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Hoang JK, Hoagland LF, Coleman RE, Coan AD, Herndon JE, Patz EF Jr. Prognostic value of fluorine-18 fluorodeoxyglucose positron emission tomography imaging in patients with advanced-stage non-small-cell lung carcinoma. J Clin Oncol. (2008) 26:1459–64. doi: 10.1200/JCO.2007.14.3628

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Inal A, Kucukoner M, Kaplan MA, Urakci Z, Karakus A, Komek H, et al. Prognostic value of fluorine-18 fluorodeoxyglucose positron emission tomography in patients with advanced non-small cell lung cancer: single center experience. J BUON. (2012) 17:724–8. doi: 10.1158/1078-0432.CCR-12-1202

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Lambin P, Rios-Velazquez E, Leijenaar R, Carvalho S, van Stiphout RG, Granton P, et al. Radiomics: extracting more information from medical images using advanced feature analysis. Eur J Cancer. (2012) 48:441–6. doi: 10.1016/j.ejca.2011.11.036

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Aerts HJ, Velazquez ER, Leijenaar RT, Parmar C, Grossmann P, Carvalho S, et al. Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach. Nat Commun. (2014) 5:4006. doi: 10.1038/ncomms5644

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Zhang B, Tian J, Dong D, Gu D, Dong Y, Zhang L, et al. Radiomics features of multiparametric MRI as novel prognostic factors in advanced nasopharyngeal carcinoma. Clin Cancer Res. (2017) 23:4259–69. doi: 10.1158/1078-0432.CCR-16-2910

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Gillies RJ, Kinahan PE, Hricack H. Radiomics: images are more than pictures, they are data. Radiology. (2016) 278:563–77. doi: 10.1148/radiol.2015151169

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Shiri I, Maleki H, Hajianfar G, Abdollahi H, Ashrafinia S, Hatt M, et al. Next-generation radiogenomics sequencing for prediction of EGFR and KRAS mutation status in NSCLC patients using multimodal imaging and machine learning algorithms. Mol Imaging Biol. (2020). doi: 10.1007/s11307-020-01487-8

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Vaidya M, Creach KM, Frye J, Dehdashti F, Bradley JD, El Naqa I. Combined PET/CT image characteristics for radiotherapy tumor response in lung cancer. Radiother Oncol. (2012) 102:239–45. doi: 10.1016/j.radonc.2011.10.014

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Dissaux G, Visvikis D, Da-Ano R, Pradier O, Chajon E, Barillot I, et al. Pre-treatment F-FDG PET/CT Radiomics predict local recurrence in patients treated with stereotactic radiotherapy for early-stage non-small cell lung cancer: a multicentric study. J Nucl Med. (2019) 61:814–20. doi: 10.2967/jnumed.119.228106

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Oikonomou A, Khalvati F, Tyrrell PN, Haider MA, Tarique U, Jimenez-Juan L, et al. Radiomics analysis at PET/CT contributes to prognosis of recurrence and survival in lung cancer treated with stereotactic body radiotherapy. Sci Rep. (2018) 8:4003. doi: 10.1038/s41598-018-22357-y

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Desseroit MC, Visvikis D, Tixier F, Majdoub M, Perdrisot R, Guillevin R, et al. Development of a nomogram combining clinical staging with (18)F-FDG PET/CT image features in non-small-cell lung cancer stage I-III. Eur J Nucl Med Mol Imaging. (2016) 43:1477–85. doi: 10.1007/s00259-016-3325-5

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Han DS, Suh YS, Kong SH, Lee HJ, Choi Y, Aikou S, et al. Nomogram predicting long-term survival after d2 gastrectomy for gastric cancer. J Clin Oncol. (2012) 30:3834–40. doi: 10.1200/JCO.2012.41.8343

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Tang XR, Li YQ, Liang SB, Jiang W, Liu F, Ge WX, et al. Development and validation of a gene expression-based signature to predict distant metastasis in locoregionally advanced nasopharyngeal carcinoma: a retrospective, multicentre, cohort study. Lancet Oncol. (2018) 19:382–93. doi: 10.1016/S1470-2045(18)30080-9

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Liang W, Zhang L, Jiang G, Wang Q, Liu L, Liu D, et al. Development and validation of a nomogram for predicting survival in patients with resected non-small-cell lung cancer. J Clin Oncol. (2015) 33:861–9. doi: 10.1200/JCO.2014.56.6661

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Zwanenburg A, Vallières M, Abdalah MA, Aerts HJWL, Andrearczyk V, Apte A, et al. The image biomarker standardization initiative: standardized quantitative radiomics for high-throughput image-based phenotyping. Radiology. (2020) 295:328–38. doi: 10.1148/radiol.2020191145

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Wels MG, Félix Lades, Muehlberg A, Suehling M. General purpose radiomics for multi-modal clinical research. Computer-Aided Diagnosis. (2019). doi: 10.1117/12.2511856

CrossRef Full Text | Google Scholar

35. Moltz J H, Bornemann L, Kuhnigk JM, Dicken V, Peitgen E, Meier S, et al. Advanced segmentation techniques for lung nodules, liver metastases, and enlarged lymph nodes in ct scans. IEEE J Select Top Sig Process. (2009) 3:122–34. doi: 10.1109/JSTSP.2008.2011107

CrossRef Full Text | Google Scholar

36. Grady L. Random walks for image segmentation. IEEE Trans Pattern Anal Mach Intell. (2006) 28:1768–83. doi: 10.1109/TPAMI.2006.233

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making. (2006) 26:565–74. doi: 10.1177/0272989X06295361

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Parmar C, Grossmann P, Bussink J, Lambin P, Aerts HJWL. Machine learning methods for quantitative radiomic biomarkers. Sci Rep. (2015) 5:13087. doi: 10.3389/fonc.2015.00272

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Ohri N, Duan F, Snyder BS, Wei B, Machtay M, Alavi A, et al. Pretreatment 18F-FDG PET textural features in locally advanced non-small cell lung cancer: secondary analysis of ACRIN 6668/RTOG 0235. J Nucl Med. (2016) 57:842–8. doi: 10.2967/jnumed.115.166934

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Wu J, Aguilera T, Shultz D, Gudur M, Rubin DL, Loo BW Jr, et al. Early-stage non-small cell lung cancer: quantitative imaging characteristics of (18)f fluorodeoxyglucose PET/CT allow prediction of distant metastasis. Radiology. (2016) 281:270–8. doi: 10.1148/radiol.2016151829

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Leger S, Zwanenburg A, Pilz K, Lohaus F, Linge A, Zöphel K, et al. A comparative study of machine learning methods for time-to-event survival data for radiomics risk modelling. Sci Rep. (2017) 7:13206. doi: 10.1038/s41598-017-13448-3

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Deist TM, Dankers FJWM, Valdes G, Wijsman R, Hsu IC, Oberije C, et al. Machine learning algorithms for outcome prediction in (chemo)radiotherapy: An empirical comparison of classifiers. Med Phys. (2018) 45:3449–59. doi: 10.1002/mp.12967

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Kirienko M, Cozzi L, Antunovic L, Lozza L, Fogliata A, Voulaz E, et al. Prediction of disease-free survival by the PET/CT radiomic signature in non-small cell lung cancer patients undergoing surgery. Eur J Nucl Med Mol Imaging. (2018) 45:207–17. doi: 10.1007/s00259-017-3837-7

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Mattonen SA, Davidzon GA, Bakr S, Echegaray S, Leung ANC, Vasanawala M, et al. [18F] FDG positron emission tomography (PET) tumor and penumbra imaging features predict recurrence in non-small cell lung cancer. Tomography. (2019) 5:145–53. doi: 10.18383/j.tom.2018.00026

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Wang L, Dong T, Xin B, Xu C, Guo M, Zhang H, et al. Integrative nomogram of CT imaging, clinical, and hematological features for survival prediction of patients with locally advanced non-small cell lung cancer. Eur Radiol. (2019) 29:2958–67. doi: 10.1007/s00330-018-5949-2

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Yang L, Yang J, Zhou X, Huang L, Zhao W, Wang T, et al. Development of a radiomics nomogram based on the 2D and 3D CT features to predict the survival of non-small cell lung cancer patients. Eur Radiol. (2019) 29:2196–206. doi: 10.1007/s00330-018-5770-y

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Huang Y, Liu Z, He L, Chen X, Pan D, Ma Z, et al. Radiomics signature: a potential biomarker for the prediction of disease-free survival in early-stage (I or II) non-small cell lung cancer. Radiology. (2016) 281:947–57. doi: 10.1148/radiol.2016152234

PubMed Abstract | CrossRef Full Text | Google Scholar

48. van Timmeren JE, Carvalho S, Leijenaar RTH, Troost EGC, van Elmpt W, de Ruysscher D, et al. Challenges and caveats of a multi-center retrospective radiomics study: an example of early treatment response assessment for NSCLC patients using FDG-PET/CT radiomics. PLoS ONE. (2019) 14:e0217536. doi: 10.1371/journal.pone.0217536

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Krarup MMK, Nygård L, Vogelius IR, Andersen FL, Cook G, Goh V, et al. Heterogeneity in tumours: Validating the use of radiomic features on F-FDG PET/CT scans of lung cancer patients as a prognostic tool. Radiother Oncol. (2020) 144:72–8. doi: 10.1016/j.radonc.2019.10.012

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Hatt M, Laurent B, Ouahabi A, Fayad H, Tan S, Li L, et al. The first MICCAI challenge on PET tumor segmentation. Med Image Anal. (2018) 44:177–95. doi: 10.1016/j.media.2017.12.007

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Berthon B, Spezi E, Galavis P, Shepherd T, Apte A, Hatt M, et al. Toward a standard for the evaluation of PET-Auto-Segmentation methods following the recommendations of AAPM task group No.211: requirements and implementation. Med Phys. (2017) 44:4098–111. doi: 10.1002/mp.12312

PubMed Abstract | CrossRef Full Text | Google Scholar