Abstract
Thyroid nodule (TYN) is a life-threatening disease that is commonly observed among adults globally. The applications of deep learning in computer-aided diagnosis systems (CADs) for diagnosing thyroid nodules have attracted attention among clinical professionals due to their significantly potential role in reducing the occurrence of missed diagnoses. However, most techniques for segmenting thyroid nodules rely on U-Net structures or deep convolutional neural networks, which have limitations in obtaining different context information due to the diversities in the shapes and sizes, ambiguous boundaries, and heterostructure of thyroid nodules. To resolve these challenges, we present an encoder-decoder-based architecture (referred to as CIL-Net) for boosting TYN segmentation. There are three contributions in the proposed CIL-Net. First, the encoder is established using dense connectivity for efficient feature extraction and the triplet attention block (TAB) for highlighting essential feature maps. Second, we design a feature improvement block (FIB) using dilated convolutions and attention mechanisms to capture the global context information and also build up robust feature maps between the encoder-decoder branches. Third, we introduce the residual context block (RCB), which leverages residual units (ResUnits) to accumulate the context information from the multiple blocks of decoders in the decoder branch. We assess the segmentation quality of our proposed method using six different evaluation metrics on two standard datasets (DDTI and TN3K) of TYN and demonstrate competitive performance against advanced state-of-the-art methods. We consider that the proposed method advances the performance of TYN region localization and segmentation, which heavily rely on an accurate assessment of different context information. This advancement is primarily attributed to the comprehensive incorporation of dense connectivity, TAB, FIB, and RCB, which effectively capture both extensive and intricate contextual details. We anticipate that this approach reliability and visual explainability make it a valuable tool that holds the potential to significantly enhance clinical practices by offering reliable predictions to facilitate cognitive and healthcare decision-making.
Similar content being viewed by others
Data Availability
In this study, publicly accessible datasets have been examined. This data can be found here: https://drive.google.com/file/d/1reHyY5eTZ5uePXMVMzFOq5j3eFOSp50F/view and https://drive.google.com/file/d/1wwlsEhwfSyvQsJBRjeDLhUjqZh8eaH2R/view.
References
Tasnimi M, Ghaffari HR. Diagnosis of anomalies based on hybrid features extraction in thyroid images. Multimedia Tools and Applications. 2023;82(3):3859–77.
Huang J, Ngai CH, Deng Y, Pun CN, Lok V, Zhang L, Xu Q, Lucero-Prisno DE, Xu W, Zheng Z-J, et al. Incidence and mortality of thyroid cancer in 50 countries: a joinpoint regression analysis of global trends. Endocrine. 2023;80(2):355–65.
Yu Z, Liu S, Liu P, Liu Y. Automatic detection and diagnosis of thyroid ultrasound images based on attention mechanism. Comput Biol Med. 2023;155:106468.
Siegel RL, Giaquinto AN, Jemal A. Cancer statistics, 2024. CA Cancer J Clin. 2024;74(1):12–49.
Cancer Statistics. https://www.cancer.org/cancer/types/thyroid-cancer/about/key-statistics.html Accessed 29 March 2024.
Kong M, Guo Q, Zhou S, Li M, Kuang K, Huang Z, Wu F, Chen X, Zhu Q. Attribute-aware interpretation learning for thyroid ultrasound diagnosis. Artif Intell Med. 2022;131:102344.
Giovanella L, Avram AM, Ovčariček PP, Clerc J. Thyroid functional and molecular imaging. La Presse Médicale. 2022;51(2):104116.
Imperiale A, Berti V, Burgy M, Cazzato RL, Piccardo A, Treglia G. Molecular imaging and related therapeutic options for medullary thyroid carcinoma: state of the art and future opportunities. Rev Endocr Metab Disord. 2024;25(1):187–202.
Trimboli P, Mian C, Piccardo A, Treglia G. Diagnostic tests for medullary thyroid carcinoma: an umbrella review. Endocrine. 2023;1–11.
Ren J-Y, Lv W-Z, Wang L, Zhang W, Ma Y-Y, Huang Y-Z, Peng Y-X, Lin J-J, Cui X-W. Dual-modal radiomics nomogram based on contrast-enhanced ultrasound to improve differential diagnostic accuracy and reduce unnecessary biopsy rate in ACR TI-RADS 4–5 thyroid nodules. Cancer Imaging. 2024;24(1):17.
Wong CM, Kezlarian BE, Lin O. Current status of machine learning in thyroid cytopathology. J Pathol Inform. 2023;100309.
Huérfano-Maldonado Y, Mora M, Vilches K, Hernández-García R, Gutiérrez R, Vera M. A comprehensive review of extreme learning machine on medical imaging. Neurocomputing. 2023;126618.
Resta IT, Gubbiotti M, Montone K, Livolsi V, Baloch Z. An investigation into noninvasive follicular thyroid neoplasms with papillary-like nuclear features: does the initial proposal on noninvasive follicular thyroid neoplasms with papillary-like nuclear features behavior hold true? Hum Pathol. 2023;141:139–48.
Guetari R, Ayari H, Sakly H. Computer-aided diagnosis systems: a comparative study of classical machine learning versus deep learning-based approaches. Knowl Inf Syst. 2023;1–41.
Xie F, Luo Y-K, Lan Y, Tian X-Q, Zhu Y-Q, Jin Z, Zhang Y, Zhang M-B, Song Q, Zhang Y. Differential diagnosis and feature visualization for thyroid nodules using computer-aided ultrasonic diagnosis system: initial clinical assessment. BMC Med Imaging. 2022;22(1):153.
Prochazka A, Gulati S, Holinka S, Smutek D. Patch-based classification of thyroid nodules in ultrasound images using direction independent features extracted by two-threshold binary decomposition. Comput Med Imaging Graph. 2019;71:9–18.
Guetari R, Ayari H, Sakly H. Computer-aided diagnosis systems: a comparative study of classical machine learning versus deep learning-based approaches. Knowl Inf Syst. 2023;1–41.
Selvachandran G, Quek SG, Paramesran R, Ding W, Son LH. Developments in the detection of diabetic retinopathy: a state-of-the-art review of computer-aided diagnosis and machine learning methods. Artif Intell Rev. 2023;56(2):915–64.
Michel A, Ro V, McGuinness JE, Mutasa S, Terry MB, Tehranifar P, May B, Ha R, Crew KD. Breast cancer risk prediction combining a convolutional neural network-based mammographic evaluation with clinical factors. Breast Cancer Res Treat. 2023;1–9.
Houssein EH, Emam MM, Ali AA, Suganthan PN. Deep and machine learning techniques for medical imaging-based breast cancer: a comprehensive review. Expert Syst Appl. 2021;167:114161.
Radak M, Lafta HY, Fallahi H. Machine learning and deep learning techniques for breast cancer diagnosis and classification: a comprehensive review of medical imaging studies. J Cancer Res Clin Oncol. 2023;1–19.
Xing G, Miao Z, Zheng Y, Zhao M. A multi-task model for reliable classification of thyroid nodules in ultrasound images. Biomed Eng Lett. 2023;1–11.
Kim Y-J, Choi Y, Hur S-J, Park K-S, Kim H-J, Seo M, Lee MK, Jung S-L, Jung CK. Deep convolutional neural network for classification of thyroid nodules on ultrasound: comparison of the diagnostic performance with that of radiologists. Eur J Radiol. 2022;152:110335.
Yang H, Yang D. CSwin-PNet: a CNN-Swin transformer combined pyramid network for breast lesion segmentation in ultrasound images. Expert Syst Appl. 2023;213:119024.
Liu Z, Tong L, Chen L, Jiang Z, Zhou F, Zhang Q, Zhang X, Jin Y, Zhou H. Deep learning based brain tumor segmentation: a survey. Complex & Intelligent Systems. 2023;9(1):1001–26.
Ahamed MF, Hossain MM, Nahiduzzaman M, Islam MR, Islam MR, Ahsan M, Haider J. A review on brain tumor segmentation based on deep learning methods with federated learning techniques. Comput Med Imaging Graph. 2023;102313.
Ranjbarzadeh R, Caputo A, Tirkolaee EB, Ghoushchi SJ, Bendechache M. Brain tumor segmentation of MRI images: a comprehensive review on the application of artificial intelligence tools. Comput Biol Med. 2023;152:106405.
Kumar V, Webb J, Gregory A, Meixner DD, Knudsen JM, Callstrom M, Fatemi M, Alizad A. Automated segmentation of thyroid nodule, gland, and cystic components from ultrasound images using deep learning. IEEE Access. 2020;8:63482–96.
Zheng T, Qin H, Cui Y, Wang R, Zhao W, Zhang S, Geng S, Zhao L. Segmentation of thyroid glands and nodules in ultrasound images using the improved u-net architecture. BMC Med Imaging. 2023;23(1):56.
Abdolali F, Kapur J, Jaremko JL, Noga M, Hareendranathan AR, Punithakumar K. Automated thyroid nodule detection from ultrasound imaging using deep convolutional neural networks. Comput Biol Med. 2020;122:103871.
Kang Q, Lao Q, Li Y, Jiang Z, Qiu Y, Zhang S, Li K. Thyroid nodule segmentation and classification in ultrasound images through intra-and inter-task consistent learning. Med Image Anal. 2022;79:102443.
Liu R, Zhou S, Guo Y, Wang Y, Chang C. U2F-GAN: weakly supervised super-pixel segmentation in thyroid ultrasound images. Cogn Comput. 2021;13:1099–113.
Fei X, Shen L, Ying S, Cai Y, Zhang Q, Kong W, Zhou W, Shi J. Parameter transfer deep neural network for single-modal B-mode ultrasound-based computer-aided diagnosis. Cogn Comput. 2020;12:1252–64.
Chu C, Zheng J, Zhou Y. Ultrasonic thyroid nodule detection method based on U-Net network. Comput Methods Programs Biomed. 2021;199:105906.
Dang T-V, Bui N-T. Multi-scale fully convolutional network-based semantic segmentation for mobile robot navigation. Electronics. 2023;12(3):533.
Zhou Z, Rahman Siddiquee MM, Tajbakhsh N, Liang J. Unet++: a nested u-net architecture for medical image segmentation. In: Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. 2018. pp. 3–11. Springer.
Alom, M.Z., Hasan, M., Yakopcic, C., Taha, T.M., Asari, V.K.: Recurrent residual convolutional neural network based on u-net (r2u-net) for medical image segmentation. 2018. arXiv preprint arXiv:1802.06955
Diakogiannis FI, Waldner F, Caccetta P, Wu C. ResUNet-a: a deep learning framework for semantic segmentation of remotely sensed data. ISPRS J Photogramm Remote Sens. 2020;162:94–114.
Guo M-H, Xu T-X, Liu J-J, Liu Z-N, Jiang P-T, Mu T-J, Zhang S-H, Martin RR, Cheng M-M, Hu S-M. Attention mechanisms in computer vision: a survey. Computational visual media. 2022;8(3):331–68.
Parvaiz A, Khalid MA, Zafar R, Ameer H, Ali M, Fraz MM. Vision transformers in medical computer vision-a contemplative retrospection. Eng Appl Artif Intell. 2023;122:106126.
Bakasa W, Viriri S. Intelligent automated pancreas segmentation using U-Net model variants. In: International Conference on Computational Collective Intelligence. 2023. pp. 606–618. Springer.
Zheng T, Qin H, Cui Y, Wang R, Zhao W, Zhang S, Geng S, Zhao L. Segmentation of thyroid glands and nodules in ultrasound images using the improved U-Net architecture. BMC Med Imaging. 2023;23(1):56.
Gan J, Zhang R. Ultrasound image segmentation algorithm of thyroid nodules based on improved U-Net network. In: Proceedings of the 2022 3rd International Conference on Control, Robotics and Intelligent System. 2022. pp. 61–66.
Ajilisa O, Jagathy Raj V, Sabu M. Segmentation of thyroid nodules from ultrasound images using convolutional neural network architectures. Journal of Intelligent & Fuzzy Systems. 2022;43(1):687–705.
Chi J, Li Z, Sun Z, Yu X, Wang H. Hybrid transformer UNet for thyroid segmentation from ultrasound scans. Comput Biol Med. 2023;153:106453.
Badrinarayanan V, Kendall A, Cipolla R. Segnet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell. 2017;39(12):2481–95.
Jégou S, Drozdzal M, Vazquez D, Romero A, Bengio Y. The one hundred layers tiramisu: fully convolutional densenets for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2017. pp. 11–19.
Zou W, Jiang M, Zhang Y, Chen L, Lu Z, Wu Y. Sdwnet: a straight dilated network with wavelet transformation for image deblurring. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021. pp. 1895–1904.
Acar V, Eksioglu EM. Densely connected dilated residual network for image denoising: Ddr-net. Neural Process Lett. 2023;55(5):5567–81.
Li Z, Jiang J, Chen X, Laganière R, Li Q, Liu M, Qi H, Wang Y, Zhang M. Dense-scale dynamic network with filter-varying atrous convolution for semantic segmentation. Appl Intell. 2023;53(22):26810–26.
Chen LC, Zhu Y, Papandreou G, Schroff F, Adam H. Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the European Conference on Computer Vision (ECCV). 2018. pp. 801–818.
Gu Z, Cheng J, Fu H, Zhou K, Hao H, Zhao Y, Zhang T, Gao S, Liu J. Ce-net: context encoder network for 2d medical image segmentation. IEEE Trans Med Imaging. 2019;38(10):2281–92.
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I. Attention is all you need. Adv Neural Inf Proces Syst 2017;30.
Chen J, Lu Y, Yu Q, Luo X, Adeli E, Wang Y, Lu L, Yuille AL, Zhou Y. Transunet: transformers make strong encoders for medical image segmentation. 2021. arXiv preprint arXiv:2102.04306.
Wu H, Chen S, Chen G, Wang W, Lei B, Wen Z. FAT-Net: feature adaptive transformers for automated skin lesion segmentation. Med Image Anal. 2022;76:102327.
Yang D, Li Y, Yu J. Multi-task thyroid tumor segmentation based on the joint loss function. Biomed Signal Process Control. 2023;79:104249.
Ma X, Sun B, Liu W, Sui D, Shan S, Chen J, Tian Z. Tnseg: adversarial networks with multi-scale joint loss for thyroid nodule segmentation. J Supercomput. 2024;80(5):6093–118.
Srivastava R, Kumar P. Optimizing CNN based model for thyroid nodule classification using data augmentation, segmentation and boundary detection techniques. Multimedia Tools and Applications. 2023;82(26):41037–72.
Shao J, Pan T, Fan L, Li Z, Yang J, Zhang S, Zhang J, Chen D, Zhu X, Chen H, et al. FCG-Net: an innovative full-scale connected network for thyroid nodule segmentation in ultrasound images. Biomed Signal Process Control. 2023;86:105048.
Inan NG, Kocadağlı O, Yıldırım D, Meşe İ, Kovan Ö. Multi-class classification of thyroid nodules from automatic segmented ultrasound images: hybrid ReNnet based UNet convolutional neural network approach. Comput Methods Programs Biomed. 2024;243:107921.
Dai H, Xie W, Xia E. SK-Unet++: an improved Unet++ network with adaptive receptive fields for automatic segmentation of ultrasound thyroid nodule images. Med Phys; 2023.
Wang S, Li Z, Liao L, Zhang C, Zhao J, Sang L, Qian W, Pan G, Huang L, Ma H. DPAM-PSPNet: ultrasonic image segmentation of thyroid nodule based on dual-path attention mechanism. Physics in Medicine & Biology. 2023;68(16):165002.
Xie Y, Yang Z, Yang Q, Liu D, Tang S, Yang L, Duan X, Hu C, Lu Y-J, Wang J. Identification method of thyroid nodule ultrasonography based on self-supervised learning dual-branch attention learning framework. Health Information Science and Systems. 2024;12(1):1–13.
Chen Y, Li D, Zhang X, Liu P, Meng F, Jin J, Shen Y. A devised thyroid segmentation with multi-stage modification based on Super-pixel U-Net under insufficient data. Ultrasound in Medicine & Biology. 2023;49(8):1728–41.
Yadav N, Dass R, Virmani J. Objective assessment of segmentation models for thyroid ultrasound images. J Ultrasound. 2023;26(3):673–85.
Liu Y, Chen C, Wang K, Zhang M, Yan Y, Sui L, Yao J, Zhu X, Wang H, Pan Q, et al. The auxiliary diagnosis of thyroid echogenic foci based on a deep learning segmentation model: a two-center study. Eur J Radiol. 2023;167:111033.
Mi J, Wang R, Feng Q, Han L, Zhuang Y, Chen K, Chen Z, Hua Z, Luo Y, Lin J. Three-dimensional visualization of thyroid ultrasound images based on multi-scale features fusion and hierarchical attention. Biomed Eng Online. 2024;23(1):31.
Chen Z, Zhu H, Liu Y, Gao X. MSCA-UNet: multi-scale channel attention-based UNet for segmentation of medical ultrasound images. Clust Comput. 2024;1–18.
Al Qurri A, Almekkawy M. Improved UNet with attention for medical image segmentation. Sensors. 2023;23(20):8589.
Wang L, Cao M, Yuan X. Efficientsci: densely connected network with space-time factorization for large-scale video snapshot compressive imaging. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023. 18477–18486.
Zhang Y, Luo L, Dou Q, Heng P-A. Triplet attention and dual-pool contrastive learning for clinic-driven multi-label medical image classification. Med Image Anal. 2023;86:102772.
Gangrade S, Sharma PC, Sharma AK. Colonoscopy polyp segmentation using deep residual U-Net with bottleneck attention module. In: 2023 Fifth International Conference on Electrical, Computer and Communication Technologies (ICECCT). 2023. pp. 1–6. IEEE.
Zhang Z, Sabuncu M. Generalized cross entropy loss for training deep neural networks with noisy labels. Adv Neural Inf Proces Syst. 2018;31.
Celard P, Iglesias E, Sorribes-Fdez J, Romero R, Vieira AS, Borrajo L. A survey on deep learning applied to medical images: from simple artificial neural networks to generative models. Neural Comput Appl. 2023;35(3):2291–323.
Ming Z, Zhu M, Wang X, Zhu J, Cheng J, Gao C, Yang Y, Wei X. Deep learning-based person re-identification methods: a survey and outlook of recent works. Image Vis Comput. 2022;119:104394.
Sudre CH, Li W, Vercauteren T, Ourselin S, Jorge Cardoso M. Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations. In: Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. 2017. pp. 240–248. Springer.
Wang M, Yuan C, Wu D, Zeng Y, Zhong S, Qiu W. Automatic segmentation and classification of thyroid nodules in ultrasound images with convolutional neural networks. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. 2021. pp. 109–115. Springer.
Gong H, Chen G, Wang R, Xie X, Mao M, Yu Y, Chen F, Li G. Multi-task learning for thyroid nodule segmentation with thyroid region prior. In: 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI). 2021. pp. 257–261. IEEE.
Naylor P, Laé M, Reyal F, Walter T. Segmentation of nuclei in histopathology images by deep regression of the distance map. IEEE Trans Med Imaging. 2018;38(2):448–59.
Sirinukunwattana K, Pluim JP, Chen H, Qi X, Heng P-A, Guo YB, Wang LY, Matuszewski BJ, Bruni E, Sanchez U, et al. Gland segmentation in colon histology images: the Glas challenge contest. Med Image Anal. 2017;35:489–502.
Deb SD, Jha RK. Modified double u-net architecture for medical image segmentation. IEEE Transactions on Radiation and Plasma Medical Sciences. 2022;7(2):151–62.
Xiang T, Zhang C, Liu D, Song Y, Huang H, Cai W. Bio-net: learning recurrent bi-directional connections for encoder-decoder architecture. In: International Conference on Medical Image Computing and Computer-assisted Intervention. 2020. pp. 74–84. Springer.
Ibrahem H, Salem A, Kang H-S. SDDS-Net: space and depth encoder-decoder convolutional neural networks for real-time semantic segmentation. IEEE Access. 2023;11:119362–72.
Funding
This work is supported in part by the National Natural Science Foundation of China under Grant No. of 62076159, 12031010, 61673251 and is also supported by the Fundamental Research Funds for the Central Universities under Grant No. of GK202105003, GK202304044.
Author information
Authors and Affiliations
Contributions
Haider Ali: conceptualization, methodology, writing—original draft, writing—review and editing, visualization. Mingzhao Wang: investigation. Juanying Xie: supervision, investigation, writing—review and editing.
Corresponding author
Ethics declarations
Ethics Approval
This article does not contain any studies with human participants or animals performed by any of the authors.
Conflict of interest
The authors declare no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Ali, H., Wang, M. & Xie, J. CIL-Net: Densely Connected Context Information Learning Network for Boosting Thyroid Nodule Segmentation Using Ultrasound Images. Cogn Comput 16, 1176–1197 (2024). https://doi.org/10.1007/s12559-024-10289-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12559-024-10289-x