Skip to main content
SpringerLink
Log in

Context fusion network with multi-scale-aware skip connection and twin-split attention for liver tumor segmentation

  • Original Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

Manually annotating liver tumor contours is a time-consuming and labor-intensive task for clinicians. Therefore, automated segmentation is urgently needed in clinical diagnosis. However, automatic segmentation methods face certain challenges due to heterogeneity, fuzzy boundaries, and irregularity of tumor tissue. In this paper, a novel deep learning-based approach with multi-scale-aware (MSA) module and twin-split attention (TSA) module is proposed for tumor segmentation. The MSA module can bridge the semantic gap and reduce the loss of detailed information. The TSA module can recalibrate the channel response of the feature map. Eventually, we can count tumors based on the segmentation results from a 3D perspective for cancer grading. Extensive experiments conducted on the LiTS2017 dataset show the effectiveness of the proposed method by achieving a Dice index of 85.97% and a Jaccard index of 81.56% over the state of the art. In addition, the proposed method also achieved a Dice index of 83.67% and a Jaccard index of 80.11% in 3Dircadb dataset verification, which further reflects its robustness and generalization ability.

Graphical abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Bray F, Ferlay J, Soerjomataram I, Siegel RL, Torre LA, Jemal A (2018) Global cancer statistics 2018: Globocan estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin 68(6):394–424

    Article  PubMed  Google Scholar 

  2. Abdulla AA (2020) Efficient computer-aided diagnosis technique for leukaemia cancer detection. IET Image Process 14(17):4435–4440

    Article  Google Scholar 

  3. Liu T, Liu J, Ma Y, He J, Han J, Ding X, Chen C (2021) Spatial feature fusion convolutional network for liver and liver tumor segmentation from ct images. Med Phys 48(1):264–272

    Article  PubMed  Google Scholar 

  4. Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention, Springer, pp 234–241

  5. Huang H, Lin L, Tong R, Hu H, Zhang Q, Iwamoto Y et al (2020) Unet 3\(+\): A full-scale connected unet for medical image segmentation. In: ICASSP IEEE International conference on acoustics, speech and signal processing (ICASSP), pp 1055–1059. IEEE

  6. Milletari F, Navab N, Ahmadi S-A (2016) V-net: Fully convolutional neural networks for volumetric medical image segmentation. In: Fourth international conference on 3D vision (3DV), pp 565–571. IEEE

  7. Khanna A, Londhe ND, Gupta S, Semwal A (2020) A deep residual u-net convolutional neural network for automated lung segmentation in computed tomography images. Biocybernetics Biomed Eng 40(3):1314–1327

    Article  Google Scholar 

  8. Li X, Chen H, Qi X, Dou Q, Fu C, Heng P-A (2018) Hdenseunet: hybrid densely connected unet for liver and tumor segmentation from ct volumes. IEEE Trans Med Imaging 37(12):2663–2674

    Article  PubMed  Google Scholar 

  9. Gu Z, Cheng J, Fu H, Zhou K, Hao H, Zhao Y, Zhang T et al (2019) Ce-net: Context encoder network for 2d medical image segmentation. IEEE Trans Med Imaging 38(10):2281–2292

  10. Mesejo P, Valsecchi A, Marrakchi-Kacem L, Cagnoni S, Damas S (2015) Biomedical image segmentation using geometric deformable models and metaheuristics. Comput Med Imaging Graph 43:167–178

    Article  PubMed  Google Scholar 

  11. Seo H, Badiei Khuzani M, Vasudevan V, Huang C, Ren H, Xiao R, Jia X et al (2020) Machine learning techniques for biomedical image segmentation: an overview of technical aspects and introduction to state-of-art applications. Med Phys 47(5):e148–e167

  12. Kushnure DT, Talbar SN (2021) Ms-unet: A multi-scale unet with feature recalibration approach for automatic liver and tumor segmentation in ct images. Comput Med Imaging Graph 89:101885

    Article  PubMed  Google Scholar 

  13. Jin Q, Meng Z, Sun C, Cui H, Su R (2020) Ra-unet: A hybrid deep attention-aware network to extract liver and tumor in ct scans. Front Bioeng Biotechnol 1471

  14. Li Y, Song S, Sun Y, Bao N, Yang B, Xu L (2022) Segmentation and volume quantification of epicardial adipose tissue in computed tomography images. Med Phys 49(10):6477–6490

    Article  PubMed  Google Scholar 

  15. Zhou Z, Rahman Siddiquee MM, Tajbakhsh N, Liang J (2018) Unet++: A nested u-net architecture for medical image segmentation. In: Deep learning in medical image analysis and multimodal learning for clinical decision support, Springer, pp 3–11

  16. Cheng F, Chen C, Wang Y, Shi H, Cao Y, Tu D et al (2020) Learning directional feature maps for cardiac mri segmentation. In: Medical image computing and computer assisted intervention–MICCAI 2020, Springer, pp 108–117

  17. Wang K, Zhang X, Zhang X, Lu Y, Huang S, Yang D (2022) Eanet: Iterative edge attention network for medical image segmentation. Pattern Recogn 127:108636

    Article  Google Scholar 

  18. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7132–7141

  19. Woo S, Park J, Lee JY, Kweon IS (2018) Cbam: Convolutional block attention module. In: Proceedings of the european conference on computer vision (ECCV), pp 3–19

  20. Schlemper J, Oktay O, Schaap M, Heinrich M, Kainz B et al (2019) Attention gated networks: Learning to leverage salient regions in medical images. Med Image Anal 53:197–207

    Article  PubMed  PubMed Central  Google Scholar 

  21. Hou Q, Zhou D, Feng J (2021) Coordinate attention for efficient mobile network design. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 13713–13722

  22. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  23. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4700–4708

  24. Szegedy C, Ioffe S, Vanhoucke V, Alemi AA (2017) Inceptionv4, inception-resnet and the impact of residual connections on learning. In: Thirty-first AAAI conference on artificial intelligence

  25. Jadon S (2020) A survey of loss functions for semantic segmentation. In: IEEE Conference on computational intelligence in bioinformatics and computational biology (CIBCB), pp 1–7. IEEE

  26. Umer J, Irtaza A, Nida N (2020) Maccai lits17 liver tumor segmentation using retinanet. In: 2020 IEEE 23rd International multitopic conference (INMIC), pp 1–5. IEEE

  27. Soler L, Hostettler A, Agnus V, Charnoz A, Fasquel J, Moreau J, Osswald A et al (2010) 3d image reconstruction for comparison of algorithm database: A patient specific anatomical and medical image database. IRCAD, Strasbourg, France, Tech. Rep, 1(1)

  28. Zhang C, Hua Q, Chu Y, Wang P (2021) Liver tumor segmentation using 2.5d uv-net with multi-scale convolution. Comput Biol Med 133:104424

  29. Wilson AC, Roelofs R, Stern M, Srebro N, Recht B (2017) The marginal value of adaptive gradient methods in machine learning. Adv Neural Inf Process Syst 30

  30. Pang S, Du A, Orgun MA, Wang Y, Yu Z (2021) Tumor attention networks: Better feature selection, better tumor segmentation. Neural Netw 140:203–222

    Article  PubMed  Google Scholar 

  31. Zhang Y, Liu H, Hu Q (2021) Transfuse: Fusing transformers and cnns for medical image segmentation. In: International conference on medical image computing and computer-assisted intervention, Springer, pp 14–24

  32. Sinha A, Dolz J (2020) Multi-scale self-guided attention for medical image segmentation. IEEE J Biomed Health Inform 25(1):121–130

    Article  Google Scholar 

  33. Yang Z, He X, Gao J, Deng L, Smola A (2016) Stacked attention networks for image question answering. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 21–29

  34. Silversmith W (2021) cc3d: Connected components on multilabel 3d images. Nat Methods

Download references

Acknowledgements

The first author would like to express his sincere gratitude to Professor Shuwei Mao for his guidance and encouragement over the years.

Author information

Author notes

  1. Zhendong Wang, Jiehua Zhu, and Shujun Fu contributed equally to this work and share first authorship.

Authors and Affiliations

  1. School of Mathematics, Shandong University, Jinan, Shandong, 250100, China

    Zhendong Wang & Shujun Fu

  2. Department of Mathematical Sciences, Georgia Southern University, Statesboro, GA, 30460, USA

    Jiehua Zhu

  3. Department of Mathematics, The University of Iowa, Iowa City, IA, 52242, USA

    Yangbo Ye

Authors
  1. Zhendong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiehua Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shujun Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yangbo Ye
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yangbo Ye.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Z., Zhu, J., Fu, S. et al. Context fusion network with multi-scale-aware skip connection and twin-split attention for liver tumor segmentation. Med Biol Eng Comput 61, 3167–3180 (2023). https://doi.org/10.1007/s11517-023-02876-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11517-023-02876-1

Keywords

Search

Navigation