Skip to main content

Advertisement

SpringerLink
Log in

Nodule Localization in Thyroid Ultrasound Images with a Joint-Training Convolutional Neural Network

  • Published:
Journal of Digital Imaging Aims and scope Submit manuscript

Abstract

The accurate localization of nodules in ultrasound images can convey crucial information to support a reliable diagnosis. However, this is usually challenging due to low contrast and image artifacts, especially in thyroid ultrasound images where nodules are relatively small in most cases. To address these problems, in this paper, we propose a joint-training convolutional neural network (CNN) for thyroid nodule localization in ultrasound images. Considering the advantage of the faster region-based CNN (Faster R-CNN) in detecting natural targets, we adopt it as the basic framework. To boost the representative power and noise suppression capability of the network, the attention mechanism module is embedded for adaptive feature refinement along the channel and spatial dimensions. Furthermore, in the training process, we annotate the training set in a novel way, called joint-training annotation, by exploiting the fake foreground (FFG) area around the nodule as a spatial prior constraint to improve the sensitivity to small nodules. Ablation experiments are conducted to verify the effectiveness of our proposed method. The experimental results show that our method outperforms others by a mean average precision (mAP) of 0.93 and achieves an intersection over union (IoU) of 0.9, indicating that the localization results agree well with the ground truth. Furthermore, extended experiments on breast nodule datasets are also conducted to verify the generalizability of the proposed approach. Above all, the proposed algorithm is of considerable significance for accurate thyroid nodule localization in ultrasound images and can be generalized to other types of nodules, thereby providing trustworthy assistance for clinical diagnosis.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Haugen BR, Alexander EK, Bible KC: 2015 American Thyroid Association management guidelines for adult patients with thyroid nodules and differentiated thyroid cancer: the American Thyroid Association Guidelines Task Force on thyroid nodules and differentiated thyroid cancer. Thyroid 26(1):1–133, 2016

    Article  PubMed  PubMed Central  Google Scholar 

  2. Gharib H, Papini E, Paschke R: Medical guidelines for clinical practice for the diagnosis and management of thyroid nodules. Endocr Pract 12(1):63-102, 2006

    Article  PubMed  Google Scholar 

  3. Kloos RT , Eng C , Evans DB: Medullary thyroid cancer: management guidelines of the American Thyroid Association. Thyroid 19(6):565-612, 2009

    Article  PubMed  Google Scholar 

  4. Haugen BR, Alexander EK, Bible KC: The American Thyroid Association (ATA) guidelines task force on thyroid nodules and differentiated thyroid cancer. Thyroid 26(1):1-133, 2016

    Article  PubMed  PubMed Central  Google Scholar 

  5. Gautherie, Michel: Thermobiological assessment of benign and malignant breast diseases. Am J Obstet Gynecol 147(8):861-869, 1983

    Article  CAS  PubMed  Google Scholar 

  6. Gharib H, Papini E, Paschke R: American Association of Clinical Endocrinologists, Associazione Medici Endocrinologi, and European Thyroid Association medical guidelines for clinical practice for the diagnosis and management of thyroid nodules: executive summary of recommendations. J Endocrinol Investig 33:287, 2010

    Article  CAS  Google Scholar 

  7. Yap MH, Edirisinghe E, Bez H: Processed images in human perception: a case study in ultrasound breast imaging. Eur J Radiol 73(3): 682–687, 2010

    Article  PubMed  Google Scholar 

  8. Calas MJG, Almeida RMVR, Gutfilen B, Pereira WCA: Intraobserver interpretation of breast ultrasonography following the bi-rads classification. Eur J Radiol 74(4):525-528, 2010

    Article  CAS  PubMed  Google Scholar 

  9. Yap, Moi Hoon, Edirisinghe E, Bez H: Processed images in human perception: a case study in ultrasound breast imaging. Eur J Radiol 73(11):682–687, 2010

    Article  PubMed  Google Scholar 

  10. Chang RF, Wu WJ, Moon WK, Chen DR: Improvement in breast nodule discrimination by support vector machines and speckle-emphasis texture analysis. Ultrasound Med Biol 29(5):679-686, 2003

    Article  PubMed  Google Scholar 

  11. Noble JA, Boukerroui D: Ultrasound image segmentation: a survey. IEEE Trans Med Imaging 25(8): 987-1010, 2006

    Article  PubMed  Google Scholar 

  12. Lee H, Chen YPP: Image based computer aided diagnosis system for cancer detection. Expert Syst Appl 42(2):5356-5365. 2015

    Article  Google Scholar 

  13. LeCun Y, Bengio Y, Hinton G: Deep Learning. Nature 521(5): 436-444, 2015

    Article  CAS  PubMed  Google Scholar 

  14. Shin HC, Roth HR, Gao M, Lu L, Xu Z, Nogues I: Deep convolutional neural networks for computer-aided detection: cnn architectures, dataset characteristics and transfer learning. IEEE Trans Med Imaging 35(5):1285–1298, 2016

    Article  PubMed  Google Scholar 

  15. Litjens G, Kooi T, Bejnordi BE, Setio A, Ciompi F, Ghafoorian M: A survey on deep learning in medical image analysis. Med Image Anal 42(12):60-88, 2017

    Article  PubMed  Google Scholar 

  16. Hussain MA, Amir-Khalili A, Hamarneh G, Abugharbieh R: Segmentation-free kidney localization and volume estimation using aggregated orthogonal decision cnns. In: International Conference on Medical Image Computing & Computer-assisted Intervention. Springer, Cham, 2017, pp 612–620

  17. Chen H, Ni D, Qin J, Li SL, Yang X, Wang TF: Standard plane localization in fetal ultrasound via domain transferred deep neural networks. IEEE J Biomed Health Inform 19(4):1627–1636, 2015

    Article  PubMed  Google Scholar 

  18. Ren S, He K, Girshick R, Sun J: Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6): 1137–1149, 2017

    Article  PubMed  Google Scholar 

  19. Redmon J, Divvala S, Girshick R, Farhadi A: You Only Look Once: Unified, Real-Time Object Detection. In: 2016 IEEE International Conference on Computer Vision and Pattern Recognition (CVPR). 2016, pp 779-788

  20. Fang J, Cheng J: Target detection and recognition based on improved Faster R-CNN. J Image Signal Process 8(1):43-50, 2019

    Article  Google Scholar 

  21. Szegedy C, Ioffe S, Vanhoucke V, Alemi A: Inception-v4, inception-resnet and the impact of residual connections on learning. arXiv:1602.07261, 2016

  22. Wang X, Girshick R, Mulam H, He K: Non-Local Neural Networks. arXiv:1711.07971, 2017

  23. Wei SE, Ramakrishna V, Kanade T, Sheikh Y: Convolutional Pose Machines. arXiv:1602.00134, 2016

  24. Neubeck A, Van Gool L: Efficient non-maximum suppression. In: 18th International Conference on Pattern Recognition (ICPR). 2006, pp 850-855

  25. Girshic R, Donahue J, Darrell T: Rich feature hierarchies for accurate object detection and semantic segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2014, pp 580-587

  26. Ribli D, Horváth A, Unger Z, Pollner P, Péter, Csabai I: Detecting and classifying lesions in mammograms with deep learning. Sci Rep 8:4165, 2018

  27. Chi J, Walia E, Babyn P, Wang J, Eramian M: Thyroid nodule classification in ultrasound images by fine-tuning deep convolutional neural network. J Digit Imaging 30(3):477-486, 2017

    Article  PubMed  PubMed Central  Google Scholar 

  28. Ma J, Wu F, Jiang T, Zhu J, Kong D: Cascade convolutional neural networks for automatic detection of thyroid nodules in ultrasound images. Med Phys 44(5):1678-1691, 2017

    Article  PubMed  Google Scholar 

  29. Woo S, Park J, Lee JY: CBAM: Convolutional Block Attention Module. arXiv:1807.06521, 2018

  30. Pizer SM, Amburn EP, Austin JD, Cromartie R, Zuiderveld K: Adaptive histogram equalization and its variations. Comput Vis Graphics Image Process 39(9):355-368, 1987

    Article  Google Scholar 

  31. Ploquin M, Basarab A, Kouamé D: Resolution enhancement in medical ultrasound imaging. J Med Imaging 2(1):017001, 2015

    Article  Google Scholar 

  32. He K, Zhang X, Ren S, Sun J: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2016, pp 770-778

  33. Kingma D, Ba J: Adam: A method for stochastic optimization. arXiv:1412.6980, 2015

  34. Abadi M, Barham P, Chen J, Chen Z, Davis A, Dean J: TensorFlow: a system for large-scale machine learning. In: Conference on Operating Systems Design and Implementation. 2016, pp 265-283

Download references

Funding

This work was funded by the National Natural Science Foundation of China (61871135, 81830058, 81627804) and the Science and Technology Commission of Shanghai Municipality (18511102904, 17411953400).

Author information

Author notes

  1. Ruoyun Liu and Shichong Zhou contributed equally to this work.

Authors and Affiliations

  1. Department of Electronic Engineering, Fudan University, No. 220, Handan Road, Yangpu District, Shanghai, 200433, China

    Ruoyun Liu, Yi Guo & Yuanyuan Wang

  2. Key Laboratory of Medical Imaging Computing and Computer Assisted Intervention (MICCAI) of Shanghai, Shanghai, 200032, China

    Ruoyun Liu, Yi Guo & Yuanyuan Wang

  3. Department of Ultrasound, Fudan University Shanghai Cancer Center, Shanghai, China

    Shichong Zhou & Cai Chang

  4. Department of Oncology, Shanghai Medical College, Fudan University, Shanghai, China

    Shichong Zhou & Cai Chang

Authors
  1. Ruoyun Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shichong Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yi Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yuanyuan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Cai Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Yi Guo or Yuanyuan Wang.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflict of interest.

Informed Consent

Informed consent was obtained from all individual participants included in the study.

Ethical Approval

This article does not contain any studies with human participants performed by any of the authors.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, R., Zhou, S., Guo, Y. et al. Nodule Localization in Thyroid Ultrasound Images with a Joint-Training Convolutional Neural Network. J Digit Imaging 33, 1266–1279 (2020). https://doi.org/10.1007/s10278-020-00366-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10278-020-00366-6

Keywords

Search

Navigation