Predicting successful clinical candidates for fiducial‐free lung tumor tracking with a deep learning binary classification model

Abstract Objectives The CyberKnife system is a robotic radiosurgery platform that allows the delivery of lung SBRT treatments using fiducial‐free soft‐tissue tracking. However, not all lung cancer patients are eligible for lung tumor tracking. Tumor size, density, and location impact the ability to successfully detect and track a lung lesion in 2D orthogonal X‐ray images. The standard workflow to identify successful candidates for lung tumor tracking is called Lung Optimized Treatment (LOT) simulation, and involves multiple steps from CT acquisition to the execution of the simulation plan on CyberKnife. The aim of the study is to develop a deep learning classification model to predict which patients can be successfully treated with lung tumor tracking, thus circumventing the LOT simulation process. Methods Target tracking is achieved by matching orthogonal X‐ray images with a library of digital radiographs reconstructed from the simulation CT scan (DRRs). We developed a deep learning model to create a binary classification of lung lesions as being trackable or untrackable based on tumor template DRR extracted from the CyberKnife system, and tested five different network architectures. The study included a total of 271 images (230 trackable, 41 untrackable) from 129 patients with one or multiple lung lesions. Eighty percent of the images were used for training, 10% for validation, and the remaining 10% for testing. Results For all five convolutional neural networks, the binary classification accuracy reached 100% after training, both in the validation and the test set, without any false classifications. Conclusions A deep learning model can distinguish features of trackable and untrackable lesions in DRR images, and can predict successful candidates for fiducial‐free lung tumor tracking.

The goal of this study was to develop an automation tool for determining a priory the optimal lung tumor tracking method.
Leveraging the computing power of

II.B. Deep-Learning Parameter Optimization
The fine-tuning of a network with transfer learning is easier and faster than training a network from the ground up with randomly

III. Results
The training time on a single 1.6GHz Intel i5 processor (no GPU) and 16GB of RAM The features extracted from the strongest activation channel in the middle layer of the Inception-ResNet-v2 network for a trackable tumor is shown in figure 6, and for a non-trackable tumor in figure 7.
These results show that this specific task of classifying the trackability of lung lesions

IV. Discussion
In this study, we presented a novel AI

2 used for alignment 4 .
figure 1c shows the tumor template DRR, with the background removed.The blue contour around the lesion represents the target tracking volume (TTV).The TTV is projected on the DRR to perform the registration based on the tumor intensity, and it is used to visually assess proper target identification.The tumor template DRR is generated by limiting the ray-tracing projections through a volume of the planning CT around the TTV, thus enhancing the visibility of the tumor, and the ability of the tracking algorithm to identify its location.Registration between tumor template DRR and X-ray image is performed separately for each view, therefore image A and image B can be analyzed independently.Figure 2 shows, for the same patient, the DRR visible at the treatment console (a) and the overlay between the DRR and the X-ray images (b).

Figure 1 :
Figure 1: Example of a lesion that can be tracked with 2-views lung tumor tracking: a) axial, coronal and sagittal CT slices showing the lesion; b) full content DRR; c) Tumor template DRR.

Figure 2 :
Figure 2: Treatment images for the same lesion displayed in figure 1: a) DRR; b) DRR overlay with 135 kV X-ray image.scan, or inhale and exhale breath-hold scans to avoid motion artifacts and target figure 3, and an untrackable tumor is shown in figure 4.

Figure 3 :
Figure 3: Enhanced DRR image of a trackable lung tumor.

Figure 4 :
Figure 4: Enhanced DRR image of a nontrackable lung tumor.initialized weights.In order to perform a new classification task, the pretrained network has to be retrained, which requires to replace the last learnable layer and the final classification layer with new layers that are adapted to the new data set.The last fully connected layer of the pre-trained network was replaced with a new fully connected layer that has the same number of outputs as there are classes in the new data set.As this study concerns binary classification, there are 2 classes, trackable and untrackable tumors.The newly replaced last fully connected layer with learnable weights had a weight learning rate factor of 1e8, and a bias learning rate factor of 1e7.

Figure 5 :
Figure 5: The training progress and the final classification accuracy for the Inception-ResNet-v2 deep-learning network which results in 100% validation accuracy in determining the feasibility to track the tumor.
and the final classification accuracy for the Inception-ResNet-v2 deep-learning network is shown in figure 5.The deep-learning based binary classification accuracy reached 100% after training, both in the validation and the testing phase, for all 5 convolutional neural network architectures.There were no false classifications of trackable or untrackable lung lesions.The training results are shown in table 1.
enough to be handled by an adequately tuned deep-learning model.The novel deep-learning based binary classification technique presented in this work can distinguish between trackable and untrackable lung lesions, and thus determine at the time of CT acquisition whether a patient could be treated with lung tumor tracking on CyberKnife, without ever having to perform an additional LOT simulation again.

Figure 6 :
Figure 6: Features extracted from the strongest activation channel in the middle layer of the Inception-ResNet-v2 network for a trackable lung tumor.Three full arrows are pointing to several bright white disks throughout the image, located at the corners of a blurry squared shape at the center of the image.These features are associated with a trackable lung tumor.

Figure 7 :
Figure 7: Features extracted from the strongest activation channel in the middle layer of the Inception-ResNet-v2 network for a non-trackable lung tumor.One full arrow is pointing to a bright white disk on the top left corner of the image, whereas three dashed arrows are pointing to dark black disks which are located at the corners of a blurry squared shape at the center of the image.These features are associated with a non-trackable lung tumor.
method leveraging a pretrained deep-learning convolutional neural network to distinguish features of trackable and untrackable lung lesions visible on DRR images.The trained network can be saved and used to classify the trackability of new DRR images generated from the CT scan at simulation time.This novel tool allows to automatically select the optimal tumor tracking method for patient undergoing lung SBRT on CyberKnife and forgo the time-consuming LOT simulation process, thus improving the workflow efficiency, saving the cost associated with resource utilization, and helping physicians in deciding whether a patient should undergo fiducial implantation or be treated on another machine.To our knowledge, this is the first study employing AI to predict successful lung tumor tracking candidates.Bahig et al. 3 investigated predictive factors for the applicability of lung tumor tracking in a cohort of 133 lung SBRT patients, pre-selected based on manufacturer recommendations and staff experience.They concluded that tumor size, volume, and density are most predictive of lung tumor tracking success.Lesions larger than Binary classification of lung tumor trackability page 11 3.5 cm had a success probability of 80%, but such probability decreased with decreasing tumor size.We chose to perform this study with tumor template DRRs, generated by the CyberKnife tracking system, to remove potential confounders in proving that determination of tumor trackability is feasible.Tumor template DRRs enhance the ability of the tracking algorithm to localize lesions, increasing the percentages of patients that can be treated with fiducial-free lung tumor tracking 9 .In the latest generation of lung tumor tracking algorithm developed in 2011, the registration between the live X-ray image and the tumor template DRR is performed by dividing the template in small square patches of a few millimeters, and by registering each patch independently.The registration for the whole template is calculated as the weighted sum of the individual patch registration, including only patches with a strong correlation with their location in the full content DRR 1 .Patches that include clearly identifiable features are given more importance.It is interesting to note how the strongest activation channel in the middle layer of the Inception-ResNet-v2 network extracts clearly identifiable features.As shown in figure 6 and figure 7, the detected patterns differ between trackable and untrackable images.In general for trackable lung tumor images (figure 6), there are three bright white circular regions throughout the image, located in the corners of a blurry squared shape at the center of the image.For most untrackable lung tumor images (figure 7), there is a bright white circular region that is located in the top left corner, and three dark black disks (instead of bright white disks) located in the corners of a blurry squared shape at the center of the image.It seems as if the algorithm can classify the images based on these features.Although these are typical patterns for most images, we do not have a definitive explanation as to the meaning of these patterns extracted by the algorithm which differ depending on lung tumor trackability.This highlights the black box nature of deep learning algorithms, but is potentially a step forward in trying to interpret the results.Machine learning and deep learning interpretability is an emerging field, and various interpretation tools have been proposed to understand how models make predictions 10 .Trying to understand the specific features found by our algorithm would require a separate study and is out of the scope of the present work.The results of this study would be implemented clinically by using one of our trained networks to feed it a pair of DRRs from a given patient, so that it can page 12 determine which, if any, of the images can be tracked.Although the analysis is performed independently for each image, the results are considered together in order to select the appropriate tracking method.If both images are deemed trackable, then we would proceed with 2-views tracking; if the algorithm deems that only one of the two images can be used for tracking, then we would proceed with 1-view tracking based on either image A or image B; if neither image can be used for tumor tracking, then spine tracking (0-view) or another treatment modality must be employed.Once a specific treatment pathway is chosen based on the algorithm's findings, the remaining aspects of the treatment process follow the normal clinical workflow.Although we have demonstrated the feasibility of using AI to predict the optimal tracking method, further work is needed prior to clinical implementation.This study has indeed three limitations: 1) small sample size -due to the time consuming process of extracting DRRs from the CyberKnife system, our study included a limited number of images (n=230 trackable lesions, n=41 untrackable lesions).2) DRR generation process -as described above, we elected to use tumor template DRR generated by the CyberKnife system.While tumor template DRRs enhance the tracking ability, full content DRRs are used to weight the importance of identifiable features, and for final visual evaluation of tracking results (the full content DRR is displayed at the control console and visually compared to the X-ray image).Although we were able to achieve 100% classification accuracy, it is possible that including the full content DRR as a validation step could further improve the robustness of the method.Faster methods of DRR generation should also be considered.3) Absence of internal and external cross-validation.While we have prevented bias by keeping the images in each set completely separated (i.e., we have used the holdout method by using an independent "testing" data set to perform an unbiased evaluation of our model, with the independent validation data set being used to tune our model and its hyperparameters), we have not employed an internal cross validation method (such as K-fold, leave-one-out, leave-p-out, etc.).Finally, our work represents a single institution retrospective study.An external validation using data from another institution, or prospectively analyzing images from patients undergoing LOT simulation, should be carried out to validate our findings.The method proposed in our study could be easily extended to lung cancer patients treated with other radiotherapy devices.Binary classification of lung tumor trackability page 13 Our deep learning algorithm should be able to classify any lung tumor as seen on a DRR as being trackable or untrackable, regardless of the specific technology used to deliver radiation, such as C-arm LINACs from various vendors and the Radixact system (Accuray Inc., CA, USA).V. Conclusion In this work, we have demonstrated a new technique for leveraging AI and deploying a transfer learning approach for binary classification of tumor trackability for lung SBRT treatments on CyberKnife.We have evaluated 5 different convolutional neural network architectures to implement binary classification of trackable and untrackable lung tumors enabling an automatic selection of the optimal tumor tracking method for patient undergoing robotic radiosurgery.These 5 different deep-learning convolutional neural networks can distinguish features of trackable and untrackable DRR images generated from a CT scan at simulation time with 100% accuracy in both the validation and testing phases, without false classifications.The chosen pretrained network can be saved and used to classify new DRR images generated from the CT scan at simulation time, thus enabling SBRT lung treatments to be performed on CyberKnife without ever having to complete a time-consuming LOT simulation again.This is an important tool for improving the efficiency of radiation therapy workflows on CyberKnife and saving precious resources and time in the clinic, both for patients and staff.

Table 1 :
Deep learning model training results.
based on tumor template DRRs is simple