Abstract
Detection of Benign and malignant pulmonary nodules is a significant help for early lung cancer diagnosis. Owing to the superior performance of the transformer based deep learning methods in different computer vision tasks, this study attempts to introduce it into the CT image classification task of pulmonary nodules. However, the problems of rare samples and harrowing local feature extraction in this field still need to solve. To this end, we introduce a CT image-based transformer for pulmonary nodule diagnosis (TransPND). Specifically, firstly, we introduce a 2D Panning Sliding Window (2DPSW) for data enhancement, making it more focused on local features. Secondly, unlike the encoder of the traditional transformer, we divide the encoder part of TransPND into two parts: Self Attention Encoder (SA) and Directive Class Attention Encoder (DCA). SA is similar to the traditional self-attention mechanism, except that we introduce Local Diagonal Masking (LDM) to select the attention location and focus on the correlation between tokens rather than itself score. Meanwhile, based on SA, we improve it and propose DCA to guide attention to focus more on local features and reduce computational effort. Finally, to solve the model overfitting problem caused by the increasing depth, we choose the Weight Learning Diagonal Matrix (WLDM) to gate each residual connection in both the SA and DCA stages. We conducted extensive experiments on the LIDC-IDRI dataset. The experimental results show that our model achieves an accuracy of 93.33\(\%\) compared to other studies using this dataset for lung nodule classification. To the best of our knowledge, TransPND is the first research on the classification of lung nodule CT images based on pure transformer architecture.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Siegel, R.L., Miller, K.D., Fuchs, H.E., Jemal, A.: Cancer statistics, 2022. CA: A Cancer J. Clin. 72 (2022)
Xia, C., et al.: Cancer statistics in china and united states, 2022: profiles, trends, and determinants. Chin. Med. J. 135, 584–590 (2022)
Tang, H., Liu, W., Huang, K.: Tereotactic ablative radiotherapy for inoperable t1–2n0m0 small-cell lung cancer. Thoracic Cancer 13, 1100–1101 (2022)
Kane, G.C., Barta, J.A., Shusted, C.S., Evans, N.R.: Now is the time to make screening for lung cancer reportable. Ann. Internal Med. (2022)
Kumar, D., Wong, A., Clausi, D.A.: Lung nodule classification using deep features in CT images. In: 2015 12th Conference on Computer and Robot Vision, pp. 133–138 (2015)
Shen, W., Zhou, M., Yang, F., Yang, C., Tian, J.: Multi-scale convolutional neural networks for lung nodule classification. In: Ourselin, S., Alexander, D.C., Westin, C.-F., Cardoso, M.J. (eds.) IPMI 2015. LNCS, vol. 9123, pp. 588–599. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-19992-4_46
Shen, S., Han, S.X., Aberle, D.R., Bui, A.A.T., Hsu, W.: An interpretable deep hierarchical semantic convolutional neural network for lung nodule malignancy classification. Expert Syst. Appl. 128, 84–95 (2019)
Liu, H., et al.: Multi-model ensemble learning architecture based on 3D CNN for lung nodule malignancy suspiciousness classification. J. Digital Imaging 1–15 (2020). https://doi.org/10.1007/s10278-020-00372-8
Zhu, W., Liu, C., Fan, W., Xie, X.: DeepLung: deep 3D dual path nets for automated pulmonary nodule detection and classification. In: 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 673–681 (2018)
Nibali, A., He, Z., Wollersheim, D.: Pulmonary nodule classification with deep residual networks. Int. J. Comput. Assist. Radiol. Surg. 12, 1799–1808 (2017)
Zhai, P., Tao, Y., Chen, H., Cai, T., Li, J.: Multi-task learning for lung nodule classification on chest CT. IEEE Access 8, 180317–180327 (2020)
Xie, Y., Zhang, J., Xia, Y.: Semi-supervised adversarial model for benign-malignant lung nodule classification on chest CT. Med. Image Anal. 57, 237–248 (2019)
Dosovitskiy, A., et al.: An image is worth 16 \(\times \) 16 words: transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016)
Sun, C., Shrivastava, A., Singh, S., Gupta, A.K.: Revisiting unreasonable effectiveness of data in deep learning era. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 843–852 (2017)
Ali, I., Muzammil, M., ul Haq, I., Khaliq, A.A., Abdullah, S.: Efficient lung nodule classification using transferable texture convolutional neural network. IEEE Access 8, 175859–175870 (2020)
Heo, B., Yun, S., Han, D., Chun, S., Choe, J., Oh, S.J.: Rethinking spatial dimensions of vision transformers. In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 11916–11925 (2021)
Liu, Z., et al.: Swin transformer: hierarchical vision transformer using shifted windows. In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 9992–10002 (2021)
Al-Shabi, M., Lan, B.L., Chan, W.Y., Ng, K.H., Tan, M.: Lung nodule classification using deep local-global networks. Int. J. Comput. Assist. Radiol. Surg. 14, 1815–1819 (2019)
Sun, W., Zheng, B., Qian, W.: Automatic feature learning using multichannel ROI based on deep structured algorithms for computerized lung cancer diagnosis. Comput. Biol. Med. 89, 530–539 (2017)
Xia, K.J., Chi, J., Gao, Y., Jiang, Y., Wu, C.: Adaptive aggregated attention network for pulmonary nodule classification. Appl. Sci. 11, 610 (2021)
Brown, T., et al.: Language models are few-shot learners. Adv. Neural. Inf. Process. Syst. 33, 1877–1901 (2020)
Mehta, S., Ghazvininejad, M., Iyer, S., Zettlemoyer, L., Hajishirzi, H.: Delight: Very deep and light-weight transformer. arXiv preprint arXiv:2008.00623 (2020)
Child, R., Gray, S., Radford, A., Sutskever, I.: Generating long sequences with sparse transformers. arXiv preprint arXiv:1904.10509 (2019)
Tay, Y., Bahri, D., Metzler, D., Juan, D.C., Zhao, Z., Zheng, C.: Synthesizer: rethinking self-attention in transformer models. In: ICML (2021)
Bachlechner, T.C., Majumder, B.P., Mao, H.H., Cottrell, G., McAuley, J.: Rezero is all you need: Fast convergence at large depth. In: UAI (2021)
Armato III, S.G., et al.: The lung image database consortium (LIDC) and image database resource initiative (IDRI): a completed reference database of lung nodules on CT scans. Med. Phys. 38(2), 915–931 (2011)
Al-Shabi, M., Lee, H.K., Tan, M.: Gated-dilated networks for lung nodule classification in CT scans. IEEE Access 7, 178827–178838 (2019)
Shen, W., et al.: Multi-crop convolutional neural networks for lung nodule malignancy suspiciousness classification. Pattern Recognit. 61, 663–673 (2017)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Zhang, H., Cisse, M., Dauphin, Y.N., Lopez-Paz, D.: mixup: beyond empirical risk minimization. arXiv preprint arXiv:1710.09412 (2017)
Yan, X., et al.: Classification of lung nodule malignancy risk on computed tomography images using convolutional neural network: a comparison between 2D and 3D strategies. In: ACCV Workshops (2016)
Liu, Y., Hao, P., Zhang, P., Xu, X., Wu, J., Chen, W.: Dense convolutional binary-tree networks for lung nodule classification. IEEE Access 6, 49080–49088 (2018)
Jiang, H., Shen, F., Gao, F., Han, W.: Learning efficient, explainable and discriminative representations for pulmonary nodules classification. Pattern Recognit. 113, 107825 (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, R., Zhang, Y., Yang, J. (2022). TransPND: A Transformer Based Pulmonary Nodule Diagnosis Method on CT Image. In: Yu, S., et al. Pattern Recognition and Computer Vision. PRCV 2022. Lecture Notes in Computer Science, vol 13535. Springer, Cham. https://doi.org/10.1007/978-3-031-18910-4_29
Download citation
DOI: https://doi.org/10.1007/978-3-031-18910-4_29
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-18909-8
Online ISBN: 978-3-031-18910-4
eBook Packages: Computer ScienceComputer Science (R0)