Skip to main content
SpringerLink
Log in

A segmentation-based algorithm for classification of benign and malignancy Thyroid nodules with multi-feature information

  • Original Article
  • Published:
Biomedical Engineering Letters Aims and scope Submit manuscript

Abstract

The aim of this study is to propose a new diagnostic model based on "segmentation + classification" to improve the routine screening of Thyroid nodule ultrasonography by utilizing the key domain knowledge of medical diagnostic tasks. A Multi-scale segmentation network based on a pyramidal pooling structure of multi-parallel void spaces is proposed. First, in the segmentation network, the exact information of the underlying feature space is obtained by an Attention Gate. Second, the inflated convolutional part of Atrous Spatial Pyramid Pooling (ASPP) is cascaded for multiple downsampling. Finally, a three-branch classification network combined with expert knowledge is designed, drawing on doctors' clinical diagnosis experience, to extract features from the original image of the nodule, the regional image of the nodule, and the edge image of the nodule, respectively, and to improve the classification accuracy of the model by utilizing the Coordinate attention (CA) mechanism and cross-level feature fusion. The Multi-scale segmentation network achieves 94.27%, 93.90% and 88.85% of mean precision (mPA), Dice value (Dice) and mean joint intersection (MIoU), respectively, and the accuracy, specificity and sensitivity of the classification network reaches 86.07%, 81.34% and 90.19%, respectively. Comparison tests show that this method outperforms the U-Net, AGU-Net and DeepLab V3+ classical models as well as the nnU-Net, Swin UNetr and MedFormer models that have emerged in recent years. This algorithm, as an auxiliary diagnostic tool, can help physicians more accurately assess the benign or malignant nature of Thyroid nodules. It can provide objective quantitative indicators, reduce the bias of subjective judgment, and improve the consistency and accuracy of diagnosis. Codes and models are available at https://github.com/enheliang/Thyroid-Segmentation-Network.git

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Mallick UK. The Revised American Thyroid Association Management Guidelines 2009 for patients with differentiated thyroid cancer: an evidence based risk adapted approach. Clin Oncol. 2010;22(06):472–4.

    Article  Google Scholar 

  2. Véronique Terrasse. Global cancer burden growing, amidst mounting need for services. The International Agency for Research on Cancer (IARC), 1 February 2024, PRESS RELEASE No. 345.

  3. Schlumberger M, Tahara M, Wirth LJ, Robinson B, Brose MS, Elisei R, Habra MA, Newbold K, Shah MH, Hoff AO, et al. Lenvatinib versus placebo in radioiodine-refractory Thyroid cancer. N Engl J Med. 2015;372(7):621–30.

    Article  Google Scholar 

  4. Yu X, Song X, Sun W, et al. Independent risk factors predicting central lymph node metastasis in papillary Thyroid microcarcinoma. Horm Metab Res. 2017;49(3):201–7.

    Article  Google Scholar 

  5. Wang J, Wei W, Guo R. Ultrasonic elastography and conventional ultrasound in the diagnosis of Thyroid micro-nodules. Pak J Med Sci. 2019;35(6):1526.

    Article  Google Scholar 

  6. Lian C, Liu M, Zhang J, et al. Hierarchical fully convolutional network for joint atrophy localization and Alzheimer’s disease diagnosis using structural MRI. IEEE Trans Pattern Anal Mach Intell. 2018;42(4):880–93.

    Article  Google Scholar 

  7. Zhang J, Zhang Z, Liu H, et al. SaTransformer: semantic-aware transformer for breast cancer classification and segmentation. IET Image Proc. 2023;17(13):3789–800.

    Article  Google Scholar 

  8. Ji Z, Zhao Z, Zeng X, et al. ResDSda_U-Net: A novel U-Net based residual network for segmentation of pulmonary nodules in lung CT images. IEEE Access. 2023;11:87775–87789.

  9. Wang J, Zhang R, Wei X, et al. An attention-based semi-supervised neural network for Thyroid nodules segmentation. In: 2019 IEEE International conference on bioinformatics and biomedicine (BIBM). IEEE; 2019. pp. 871–876.

  10. Ding J, Huang Z, Shi M, et al. Automatic Thyroid ultrasound image segmentation based on u-shaped network. In: 2019 12th International congress on image and signal processing, BioMedical Engineering and Informatics (CISP-BMEI). IEEE; 2019. pp. 1–5.

  11. Nandamuri S, China D, Mitra P, et al. Sumnet: fully convolutional model for fast segmentation of anatomical structures in ultrasound volumes. In: 2019 IEEE 16th international symposium on biomedical imaging (ISBI 2019). IEEE; 2019. pp. 1729–1732.

  12. Song R, Zhang L, Zhu C, et al. Thyroid nodule ultrasound image classification through hybrid feature cropping network. IEEE Access. 2020;8:64064–74.

    Article  Google Scholar 

  13. Nguyen DT, Pham TD, Batchuluun G, et al. Artificial intelligence-based Thyroid nodule classification using information from spatial and frequency domains. J Clin Med. 2019;8(11):1976.

    Article  Google Scholar 

  14. Misra S, Yoon C, Kim KJ, et al. Deep learning-based multimodal fusion network for segmentation and classification of breast cancers using B-mode and elastography ultrasound images. Bioeng Transl Med. 2022;8:e10480.

    Article  Google Scholar 

  15. Misra S, Jeon S, Managuli R, et al. Bi-modal transfer learning for classifying breast cancers via combined b-mode and ultrasound strain imaging. IEEE Trans Ultrason Ferroelectr Freq Control. 2021;69(1):222–32.

    Article  Google Scholar 

  16. Oktay O, Schlemper J, Folgoc LL, et al. Attention u-net: learning where to look for the pancreas. arXiv preprint arXiv:1804.03999, 2018.

  17. Wang P, Chen P, Yuan Y, et al. Understanding convolution for semantic segmentation. In: 2018 IEEE winter conference on applications of computer vision (WACV). IEEE; 2018. pp. 1451–1460.

  18. Chen LC, Papandreou G, Schroff F, et al. Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXiv:1706.05587, 2017.

  19. Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation. In: Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5–9, 2015, Proceedings, Part III 18. Springer International Publishing; 2015. pp. 234–241.

  20. Chen LC, Zhu Y, Papandreou G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the European conference on computer vision (ECCV); 2018. pp. 801–818.

  21. He K, Zhang X, Ren S, et al. Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2016. pp. 770–778.

  22. Ho Q, Zhou D, Feng J. Coordinate attention for efficient mobile network design. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition; 2021. pp. 13713–13722.

  23. Tang Y, Yang D, Li W, et al. Self-supervised pre-training of swin transformers for 3d medical image analysis. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022. pp. 20730–20740.

  24. Isensee F, Petersen J, Klein A, et al. nnu-net: Self-adapting framework for u-net-based medical image segmentation. arXiv preprint arXiv:1809.10486, 2018.

  25. Gao Y, Zhou M, Liu D, et al. A data-scalable transformer for medical image segmentation: architecture, model efficiency, and benchmark. arXiv preprint arXiv:2203.00131, 2022.

  26. Gong H, Chen J, Chen G, et al. Thyroid region prior guided attention for ultrasound segmentation of thyroid nodules. Comput Biol Med. 2023;155:106389.

    Article  Google Scholar 

  27. Pedraza L, Vargas C, Narváez F, et al. An open access thyroid ultrasound image database. In: 10th International symposium on medical information processing and analysis. SPIE, 2015, 9287. pp. 188–193.

  28. Wunderling T, Golla B, Poudel P, et al. Comparison of thyroid segmentation techniques for 3D ultrasound. In: Medical imaging 2017: image processing. SPIE, 2017;10133:346–352.

  29. Gong H, Chen G, Wang R, et al. Multi-task learning for thyroid nodule segmentation with thyroid region prior. In: 2021 IEEE 18th international symposium on biomedical imaging (ISBI). IEEE; 2021. pp. 257–261.

  30. Feng S, Zhao H, Shi F, et al. CPFNet: Context pyramid fusion network for medical image segmentation. IEEE Trans Med Imaging. 2020;39(10):3008–18.

    Article  Google Scholar 

  31. Wang S, Li Z, Liao L, et al. DPAM-PSPNet: ultrasonic image segmentation of thyroid nodule based on dual-path attention mechanism. Phys Med Biol. 2023;68(16): 165002.

    Article  Google Scholar 

  32. Huang G, Liu Z, Van Der Maaten L, et al. Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2017. pp. 4700–4708.

  33. Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16×16 words: transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.

  34. Szegedy C, Vanhoucke V, Ioffe S, et al. Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2016. pp. 2818–2826.

  35. Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.

  36. Zhang Y, Lai H, Yang W. Cascade UNet and CH-UNet for Thyroid nodule segmentation and benign and malignancy classification. In: International conference on medical image computing and computer-assisted intervention. Springer, Cham; 2020. pp. 129–134.

Download references

Acknowledgements

Sponsors: The Natural Science Foundation of Inner Mongolia Autonomous Region (No.2020MS06015, No.2020MS08042) and in part by the National Natural Science Foundation of China (No. 61966026).

Author information

Authors and Affiliations

  1. School of Electronic Information Engineering, Inner Mongolia University, 235 Daxue West Road, Saihan District, Hohhot, 010021, Inner Mongolia, China

    Zhiqiang Zheng, Enhe Liang, Yujie Zhang, Zhi Weng & Tianyi Su

  2. Imaging Medicine Department, Inner Mongolia People’s Hospital, No. 20, Zhaowuda Road, Saihan District, Hohhot, 737399, Inner Mongolia, China

    Jun Chai, Wenjin Bu & Jinjin Xu

Authors
  1. Zhiqiang Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Enhe Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yujie Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhi Weng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jun Chai
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Wenjin Bu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jinjin Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Tianyi Su
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

As the main executor of this research, Zheng formulated the research plan according to the existing difficulties, was responsible for theoretical innovation and feasibility analysis, and also completed the manuscript writing. As participants of this study, Liang, Zhang and Su made a complete data set, executed and completed a number of experiments, analyzed the experimental results, and gave feedback to the team. Weng is the main person in charge of the team and the corresponding author of this paper. In this research, he is mainly responsible for the formulation of experimental plans, feasibility analysis, and verification of experimental results. Chai, Bu, and Xu are doctors, and they are mainly responsible for the guidance of data labeling in the team. At the same time, they provide a lot of clinical knowledge for this research topic, which provides a good reference for this research.

Corresponding authors

Correspondence to Zhi Weng or Jun Chai.

Ethics declarations

Competing interests

The authors declare no competing interest.

Ethical statement

This study was done in collaboration with the Imaging Department of the Inner Mongolia People’s Hospital. The ultrasound data used in the experiment contains 4021 images of Thyroid nodules diagnosed in the Inner Mongolia People’s Hospital from October 2017 to December 2020, including 1844 images of benign nodules and 2177 images of malignant nodules. The data were acquired using a GE LOGIQ E9 device, and the images were manually cropped by a physician. All ultrasonic images of nodules were labeled with benign and malignancy categories by senior physicians, and the category labels and external rectangular coordinates of nodules were obtained. The experimental protocol were approved by ethics committee of Inner Mongolia People’s Hospital, and informed consent from patients was obtained prior to this study. These data were desensitized by doctors and do not contain the patient's private information, only the ultrasound area of the ultrasound image. The ethical research content involved in this research will be managed and engaged in scientific research in strict accordance with relevant national laws, regulations and international practices. The incidence of Thyroid nodules in the population rises year by year, and ultrasound is an important methodology that is currently used in the diagnosis of Thyroid cancer because it is noninvasive, safe, and economical. However, ultrasonic images have disadvantages, such as low contrast, low resolution and ease of being polluted by noise, and the rates of missed diagnosis and misdiagnosis by doctors are higher. Deep learning carries out big data training through the construction of a deep convolutional neural network, and the network learns autonomously and is robust. If the project can be implemented in the future, it can realize automatic detection and classification of benign and malignancy Thyroid nodules, and improve the diagnostic accuracy of doctors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zheng, Z., Liang, E., Zhang, Y. et al. A segmentation-based algorithm for classification of benign and malignancy Thyroid nodules with multi-feature information. Biomed. Eng. Lett. (2024). https://doi.org/10.1007/s13534-024-00375-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s13534-024-00375-2

Keywords

Search

Navigation