Skip to main content

Advertisement

SpringerLink
Log in

mResU-Net: multi-scale residual U-Net-based brain tumor segmentation from multimodal MRI

  • Review Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

Brain tumor segmentation is an important direction in medical image processing, and its main goal is to accurately mark the tumor part in brain MRI. This study proposes a brand new end-to-end model for brain tumor segmentation, which is a multi-scale deep residual convolutional neural network called mResU-Net. The semantic gap between the encoder and decoder is bridged by using skip connections in the U-Net structure. The residual structure is used to alleviate the vanishing gradient problem during training and ensure sufficient information in deep networks. On this basis, multi-scale convolution kernels are used to improve the segmentation accuracy of targets of different sizes. At the same time, we also integrate channel attention modules into the network to improve its accuracy. The proposed model has an average dice score of 0.9289, 0.9277, and 0.8965 for tumor core (TC), whole tumor (WT), and enhanced tumor (ET) on the BraTS 2021 dataset, respectively. Comparing the segmentation results of this method with existing techniques shows that mResU-Net can significantly improve the segmentation performance of brain tumor subregions.

Graphical abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

The datasets are provided by BraTS 2021 Challenge and are allowed for personal academic research. The specific link to the dataset is https://ipp.cbica.upenn.edu/.

References

  1. Bakas S, Reyes M, Jakab A, Bauer S, Rempfler M, Crimi A et al (2018) Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the BRATS challenge. arXiv:1811.02629. https://doi.org/10.48550/arXiv.1811.02629

  2. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M, Berg AC, Fei-Fei L (2015) ImageNet LargeScale Visual Recognition Challenge. International Journal of Computer Vision 115. https://doi.org/10.1007/s11263-015-0816-y

  3. Zeng T, Wu B, Ji S (2017) DeepEM3D, approaching human-level performance on 3D anisotropic EM image segmentation. Bioinformatics 33(16):2555–2562. https://doi.org/10.1093/bioinformatics/btx188

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  4. Baid U et al (2021) The RSNA-ASNR-MICCAI BraTS 2021 benchmark on brain tumor segmentation and radio genomic classification. arXiv:2107.02314. https://doi.org/10.48550/arXiv.2107.02314

  5. Menze BH, Jakab A, Bauer S, Kalpathy-Cramer J, Farahani K, Kirby J et al (2015) The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Trans Med Imaging 34(10):1993–2024. https://doi.org/10.1109/TMI.2014.2377694

    Article  PubMed  Google Scholar 

  6. Bakas S, Akbari H, Sotiras A, Bilello M, Rozycki M, Kirby J et al (2017) Advancing The Cancer Genome Atlas glioma MRI collections with ex-pert segmentation labels and radiomic features. Sci Data 4. https://doi.org/10.1038/sdata.2017.117

  7. Bakas S, Akbari H, Sotiras A, Bilello M, Rozycki M, Kirby J et al (2017) Segmentation labels and radiomic features for the pre-operative scans of the TCGA-GBM collection. https://doi.org/10.7937/K9/TCIA.2017.KLXWJJ1Q

  8. Bakas S, Akbari H, Sotiras A, Bilello M, Rozycki M, Kirby J et al (2017) segmentation labels and radiomic features for the pre-operative scans of the TCGA-GBM collection. https://doi.org/10.7937/K9/TCIA.2017.GJQ7R0EF

  9. Ronneberger O, Fischer P, Brox T (2015) U-Net, Convolutional networks for biomedical image segmentation. In: Navab N, Hornegger J, Wells WM, Frangi AF (eds) Medical Image Computing and Computer-Assisted Intervention – MICCAI2015. Springer International Publishing, Cham, pp 234–241

    Google Scholar 

  10. Myronenko A (2019) 3D MRI brain tumor segmentation using autoencoder regularization. In: Crimi A, Bakas S, Kuijf H, Keyvan F, Reyes M, van Walsum T (eds) Brain lesion, glioma, multiple sclerosis, stroke and traumatic brain injuries. Springer International Publishing, Cham, pp 311–320

    Chapter  Google Scholar 

  11. Isensee F, Jaeger PF, Kohl SA, Petersen J, Maier-Hein K (2021) nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nature Methods 18(2):203–211. https://doi.org/10.1038/s41592-020-01008-z

    Article  CAS  PubMed  Google Scholar 

  12. Isensee F, Jäger PF, Full PM, Vollmuth P, Maier-Hein KH (2021) nnU-Net for brain tumor segmentation. In: Crimi A, Bakas S (eds) Brain lesion, glioma, multiple sclerosis, stroke and traumatic brain injuries. Springer International Publishing, Cham, pp 118–132

    Chapter  Google Scholar 

  13. Razzak MI, Imran M, Xu G (2019) Efficient brain tumor segmentation with multiscale two-pathway-group conventional neural networks. IEEE J Biomed Heal Informatics 23(5):1911–1919. https://doi.org/10.1109/JBHI.2018.2874033

    Article  Google Scholar 

  14. Kamnitsas K, Ledig C, Newcombe VF, Simpson JP, Kane AD, Menon DK, Rueckert D, Glocker B (2017) Efficient multiscale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med Image Anal 36(2):61–78

    Article  PubMed  Google Scholar 

  15. Oktay O, Schlemper J, Folgoc LL, Lee M, Heinrich M, Misawa K, Mori K, McDonagh S, Hammerla NY, Kainz B et al (2018) Attention u-net: learning where to look for the pancreas. arXiv, arXiv:1804.03999

  16. Wang W, Chen C, Ding M, Yu H, Zha S, Li J (2021) TransBTS: multimodal brain tumor segmentation using transformer. In: de Bruijne M et al (eds) Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science, vol 12901. Springer, Cham. https://doi.org/10.1007/978-3-030-87193-2_11

  17. Chen J, Lu Y, Yu Q, Luo X, Adeli E, Wang Y, Lu L, Yuille AL, Zhou Y (2021) TransUNet: transformers make strong encoders for medical image segmentation. arXiv:2102.04306. https://doi.org/10.48550/arXiv.2102.04306

  18. Zhang Z, Liu Q, Wang Y (2018) Road extraction by deep residual u-net. IEEE Geosci Remote Sens Lett 15:749–753

    Article  ADS  CAS  Google Scholar 

  19. Baid U, Talbar S, Rane S, Gupta S, Thakur MH, Moiyadi A, Sable N, Akolkar M, Mahajan A (2020) A novel approach for fully automatic intra-tumor segmentation with 3D U-Net architecture for gliomas. Front Comput Neurosci 14:1–11

    Article  Google Scholar 

  20. Milletari F, Navab N, Ahmadi SA (2016) V-Net: fully convolutional neural networks for volumetric medical image segmentation. In: 2016 Fourth International Conference on 3D Vision (3DV) pp 565-571. IEEE. https://doi.org/10.1109/3DV.2016.79

  21. Zhang J, Xie Y, Wang Y, Xia Y (2020) Inter-slice context residual learning for 3D medical image segmentation. IEEE Trans Med Imaging 40:661–672

    Article  Google Scholar 

  22. Çiçek Ö, Abdulkadir A, Lienkamp SS, Brox T, Ronneberger O (2016) 3D U-Net: learning dense volumetric segmentation from sparse annotation. In: Ourselin S, Joskowicz L, Sabuncu M, Unal G, Wells W (eds) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2016. MICCAI 2016. Lecture Notes in Computer Science, vol 9901. Springer, Cham. https://doi.org/10.1007/978-3-319-46723-8_49

  23. Rehman MU, Cho S, Kim JH, Chong KT (2020) Bu-Net: brain tumor segmentation using modified U-Net architecture. Electronics (Switzerland) 9(12):1–12

    CAS  Google Scholar 

  24. Rehman MU, Cho S, Kim J, Chong KT (2021) BrainSegNet: brain tumor MR image segmentation via enhanced encoder–decoder network. Diagnostics 11(2):169

    Article  PubMed  PubMed Central  Google Scholar 

  25. Zhang W, Yang G, Huang H, Yang W, Xu X, Liu Y, Lai X (2021) ME-Net: multi-encoder net framework for brain tumor segmentation. Int J Imaging Syst Technol 31(4):1834–1848

    Article  Google Scholar 

  26. Guan X, Yang G, Ye J, Yang W, Xu X, Jiang W, Lai X (2022) 3D AGSE-VNet: An automatic brain tumor MRI data segmentation framework. BMC Med Imaging 22(1):6

    Article  PubMed  PubMed Central  Google Scholar 

  27. Peiris H, Hayat M, Chen Z, Egan G, Harandi M (2022) A Robust Volumetric Transformer for Accurate 3D Tumor Segmentation. In: Wang L, Dou Q, Fletcher PT, Speidel S, Li S (eds) Medical Image Computing and Computer Assisted Intervention – MICCAI 2022. MICCAI 2022. Lecture Notes in Computer Science, vol 13435. Springer, Cham. https://doi.org/10.1007/978-3-031-16443-9_16

  28. Hatamizadeh A et al (2021) UNETR: transformers for 3D medical image segmentation. In: 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). IEEE, pp 1748–1758. https://doi.org/10.48550/arXiv.2103.10504

  29. Pham Q, Nguyen-Truong H, Phuong NN, Nguyen KN, Nguyen CD, Bui T, Truong SQ (2022) Segtransvae: hybrid CNN - Transformer with regularization for medical image segmentation. In: 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI). IEEE, Kolkata, India, pp 1-5

  30. Hatamizadeh A, Nath V, Tang Y, Yang D, Roth HR, Xu D (2022) Swin UNETR: swin transformers for semantic segmentation of brain tumors in MRI images. arXiv:2201.01266. https://doi.org/10.48550/arXiv.2201.01266

  31. Hatamizadeh A, Xu Z, Yang D, Li W, Roth HR, Xu D (2022) UNetFormer: a unified vision transformer model and pre-training framework for 3D medical image segmentation. arXiv:2204.00631. https://doi.org/10.48550/arXiv.2204.00631

  32. Lin Y, Goyal P, Girshick R, He K, Dollar P (2017) Focal loss for dense object detection. Proc IEEE Int Conf Comput Vis 2017:2999–3007. https://doi.org/10.1109/ICCV.2017.324

    Article  Google Scholar 

  33. Ibtehaz Nabil, Sohel Rahman M (2020) MultiResUNet: rethinking the U-Net architecture for multimodal biomedical image segmentation. Neural Netw 121:74–87

    Article  PubMed  Google Scholar 

Download references

Funding

(1) FC (No.61972117); (2) the Natural Science Foundation of Heilongjiang Province of China (ZD2019E007).

Author information

Authors and Affiliations

  1. School of Mechanical and Power Engineering, Harbin University of Science and Technology, Harbin, Heilongjiang, 150000, China

    Pengcheng Li, Zhihao Li, Zijian Wang, Chaoxiang Li & Monan Wang

Authors
  1. Pengcheng Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhihao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zijian Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chaoxiang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Monan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pengcheng Li.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, P., Li, Z., Wang, Z. et al. mResU-Net: multi-scale residual U-Net-based brain tumor segmentation from multimodal MRI. Med Biol Eng Comput 62, 641–651 (2024). https://doi.org/10.1007/s11517-023-02965-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11517-023-02965-1

Keywords

Search

Navigation