Skip to main content
SpringerLink
Log in

Lossless segmentation of cardiac medical images by a resolution consistent network with nondamage data preprocessing

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Convolutional neural networks originate from image classification tasks. The pooling operation can expand the receptive field and reduce the amount of calculation, but a large amount of pixel information can be lost, which is obviously harmful to pixel-level segmentation accuracy. Dilation convolution expands the receptive field and keeps the resolution unchanged, but it increases the amount of data storage and calculation. Therefore, dilation convolution can only be applied to a limited number of deep layers in a network. It is common that different samples have different sizes in medical image dataset. The resize operation is widely used in the field to obtain uniform sizes. However, the resize operation adds, deletes and modifies a large number of pixels based on the interpolation method. In this way, an image after resizing is damaged to a certain extent at the pixel level, which also significantly affects the performance of the segmentation network. We propose a resolution-consistent network (RCN), which removes all pooling layers and keeps all resolutions consistent to solve the data loss problem caused by downsampling operations. To solve the problem of the increased amount of data storage and calculation caused by dilated convolution and the data damage caused by the resize operation, we propose a nondamage data preprocessing method that includes a coarse segmentation network, cardiac center point positioning algorithm and nondamage cropping to avoid any resize operations. We achieve state-of-the-art performance and reach first place with respect to some indicators on the widely-used automated cardiac diagnosis challenge (ACDC) dataset. Our average dice scores are 0.951 (left ventricle), 0.915 (right ventricle) and 0.910 (myocardium) on the test set.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Algorithm 1
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Code Availability

Our code link is https://github.com/542250984/ACDC-cardiac-segmentation.

Notes

  1. https://github.com/MIC-DKFZ/ACDC2017

References

  1. Ang Y, Hong Y, Ha S et al (2017) Automatic segmentation of LV and RV in cardiac MRI. In: International workshop on statistical atlases and computational models of the heart. Springer, pp 161–169

  2. Badrinarayanan V, Kendall A, Cipolla R (2017) Segnet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39:2481–2495

    Article  Google Scholar 

  3. Bernard O, Lalande A, Zotti C et al (2018) Deep learning techniques for automatic MRI cardiac multi-structures segmentation and diagnosis: is the problem solved? IEEE Trans Med Imaging 37:2514–2525

    Article  Google Scholar 

  4. Bustamante M, Gupta V, Forsberg D et al (2018) Automated multi-atlas segmentation of cardiac 4D flow MRI. Med Image Anal 49:128–140

    Article  Google Scholar 

  5. Calisto MB, Lai-Yuen SK (2020) Adaen-net: An ensemble of adaptive 2D–3D Fully Convolutional Networks for medical image segmentation. Neural Netw 126:76–94

    Article  Google Scholar 

  6. Chaurasia A, Culurciello E (2017) Linknet: Exploiting encoder representations for efficient semantic segmentation. In: 2017 IEEE visual communications and image processing (VCIP). IEEE, pp 1–4

  7. Chen K, Franko K, Sang R (2021) Structured model pruning of convolutional networks on tensor processing units. https://doi.org/10.48550/arXiv.2107.04191. arXiv:2107.04191

  8. Chen LC, Papandreou G, Kokkinos I et al (2014) Semantic image segmentation with deep convolutional nets and fully connected crfs. arXiv:1412.7062

  9. Chen LC, Papandreou G, Kokkinos I et al (2017) Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans Pattern Anal Mach Intell 40:834–848

    Article  Google Scholar 

  10. Chen LC, Papandreou G, Schroff F, Adam H (2017) Rethinking atrous convolution for semantic image segmentation. arXiv:1706.05587

  11. Chen C, Wang G, Peng C, Zhang D, Fang Y, Qin H (2020) Exploring rich and efficient spatial temporal interactions for real time video salient object detection. arXiv:2008.02973

  12. Chen C, Wang G, Peng C, Zhang X, Qin H (2019) Improved robust video saliency detection based on long-term spatial-temporal information. IEEE Trans Image Process 29:1090–1100

    Article  MathSciNet  MATH  Google Scholar 

  13. Chen C, Wei J, Peng C, Zhang W, Qin H (2020) Improved saliency detection in RGB-d images using two-phase depth estimation and selective deep fusion. IEEE Trans Image Process 29:4296–4307

    Article  MATH  Google Scholar 

  14. Dou Q, Yu L, Chen H et al (2017) 3D deeply supervised network for automated segmentation of volumetric medical images. Med Image Anal 41:40–54

    Article  Google Scholar 

  15. Duan J, Bello G, Schlemper J et al (2019) Automatic 3D bi-ventricular segmentation of cardiac images by a shape-refined multi-task deep learning approach. IEEE Trans Med Imaging 38:2151–2164

    Article  Google Scholar 

  16. Fan DP, Wang W, Cheng MM, Shen J (2019) Shifting more attention to video salient object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 8554–8564

  17. Ganaye PA, Sdika M, Triggs B et al (2019) Removing segmentation inconsistencies with semi-supervised non-adjacency constraint. Med Image Anal 58:101551. https://doi.org/10.1016/j.media.2019.101551

    Article  Google Scholar 

  18. Geng L, Zhang S, Tong J, Xiao Z (2019) Lung segmentation method with dilated convolution based on VGG-16 network. Computer Assisted Surgery 24:27–33

    Article  Google Scholar 

  19. Guan D, Huang J, Lu S, Xiao A (2021) Scale variance minimization for unsupervised domain adaptation in image segmentation. Pattern Recogn 112:107764. https://doi.org/10.1016/j.patcog.2020.107764

    Article  Google Scholar 

  20. Ieracitano C, Paviglianiti A, Campolo M, Hussain A, Pasero E, Morabito FC (2021) A novel automatic classification system based on hybrid unsupervised and supervised machine learning for electrospun nanofibers. IEEE/CAA Journal of Automatica Sinica 8(1):64–76. https://doi.org/10.1109/JAS.2020.1003387

    Article  Google Scholar 

  21. Iglesias JE, Sabuncu MR (2015) Multi-atlas segmentation of biomedical images: a survey. Med Image Anal 24:205–219

    Article  Google Scholar 

  22. Isensee F, Jaeger PF, Full PM et al (2017) Automatic cardiac disease assessment on cine-MRI via time-series segmentation and domain specific features. In: International workshop on statistical atlases and computational models of the heart. Springer, pp 120–129

  23. Islam MA, Rochan M, Bruce NDB, Wang Y (2017) Gated feedback refinement network for dense image labeling. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3751–3759

  24. Jégou S, Drozdzal M, Vazquez D, Romero A, Bengio Y (2017) The one hundred layers tiramisu: Fully convolutional densenets for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 11–19

  25. Khened M, Alex V, Krishnamurthi G (2017) Densely connected fully convolutional network for short-axis cardiac cine MR image segmentation and heart diagnosis using random forest. In: International workshop on statistical atlases and computational models of the heart. Springer, pp 140–151

  26. Kirisli HA, Schaap M, Klein S et al (2010) Fully automatic cardiac segmentation from 3D CTA data: a multi-atlas based approach. In: Medical Imaging 2010: Image Processing. International Society for Optics and Photonics. 7623:762305. https://doi.org/10.1117/12.838370

  27. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25:1097–1105

    Google Scholar 

  28. Kudo Y, Aoki Y (2017) Dilated convolutions for image classification and object localization. In: 2017 Fifteenth IAPR international conference on machine vision applications (MVA). IEEE, pp 452–455

  29. Lecun Y, Bottou L (1998) Gradient-based learning applied to document recognition. Proc IEEE 86:2278–2324

    Article  Google Scholar 

  30. Lieman-Sifry J, Le M, Lau F, Sall S, Golden D (2017) FastVentricle: cardiac segmentation with ENet. In: International conference on functional imaging and modeling of the heart. Springer, pp 127–138

  31. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431–3440

  32. Painchaud N, Skandarani Y, Judge T et al (2020) Cardiac segmentation with strong anatomical guarantees. IEEE Trans Med Imaging 39:3703–3713

    Article  Google Scholar 

  33. Poudel RPK, Lamata P, Montana G (2016) Recurrent fully convolutional neural networks for multi-slice MRI cardiac segmentation. In: Reconstruction, segmentation, and analysis of medical images. Springer, pp 83–94

  34. Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 234–241

  35. Santos CFGD, Moreira TP, Colombo D et al (2020) Does removing pooling layers from convolutional neural networks improve results? SN Computer Science 1:1–10

    Article  Google Scholar 

  36. Skandarani Y, Painchaud N, Jodoin PM, Lalande A (2020) On the effectiveness of GAN generated cardiac MRIs for segmentation. arXiv:2005.09026

  37. Tran PV (2016) A fully convolutional neural network for cardiac segmentation in short-axis MRI. https://doi.org/10.48550/arXiv.1604.00494. arXiv:1604.00494

  38. Ullah I, Jian M, Hussain S et al (2020) DSFMA: Deeply supervised fully convolutional neural networks based on multi-level aggregation for saliency detection. Multimed Tools Appl 80:7145–7165

    Article  Google Scholar 

  39. Wang P, Chen P, Yuan Y, Liu D, Huang Z, Hou X, Cottrell G (2018) Understanding convolution for semantic segmentation. In: 2018 IEEE winter conference on applications of computer vision (WACV). IEEE, pp 1451–1460, DOI https://doi.org/10.1109/WACV.2018.00163, (to appear in print)

  40. Wei Z, Shi F, Song H, Ji W, Han G (2020) Attentive boundary aware network for multi-scale skin lesion segmentation with adversarial training. Multimed Tools Appl 79:27115–27136

    Article  Google Scholar 

  41. Wu H, Zhang J, Huang K, Liang K, Yu Y (2019) Fastfcn: Rethinking dilated convolution in the backbone for semantic segmentation. arXiv:1903.11816

  42. Xia P, He J, Yin J (2020) Boosting image caption generation with feature fusion module. Multimed Tools Appl 79(33):24225–24239

    Article  Google Scholar 

  43. Xiao Y, Tian Z, Yu J et al (2020) A review of object detection based on deep learning. Multimed Tools Appl 79(33):23729–23791

    Article  Google Scholar 

  44. Yamashita T, Furukawa H, Fujiyoshi H (2018) Multiple skip connections of dilated convolution network for semantic segmentation. In: 2018 25th IEEE International Conference on Image Processing (ICIP). IEEE, pp 1593–1597

  45. Yu F, Koltun V (2015) Multi-scale context aggregation by dilated convolutions. arXiv:1511.07122

  46. Zhang Q, Cui Z, Niu X, Geng S, Qiao Y (2017) Image segmentation with pyramid dilated convolution based on ResNet and U-Net. In: International Conference on Neural Information Processing. Springer, pp 364–372

  47. Zhang J, Lu C, Wang J et al (2019) Concrete cracks detection based on FCN with dilated convolution. Appl Sci 9(13):2686. https://doi.org/10.3390/app9132686

    Article  Google Scholar 

  48. Zhou R, Liao Z, Pan T et al (2017) Cardiac atlas development and validation for automatic segmentation of cardiac substructures. Radiother Oncol 122:66–71

    Article  Google Scholar 

  49. Zhou L, Zhang C, Wu M (2018) D-linknet: Linknet with pretrained encoder and dilated convolution for high resolution satellite imagery road extraction. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 182–186

  50. Zhuang X, Bai W, Song J, Zhan S, Qian X, Shi W, Lian Y, Rueckert D (2015) Multiatlas whole heart segmentation of CT data using conditional entropy for atlas ranking and selection. Med Phys 42(7):3822–3833

    Article  Google Scholar 

  51. Zhuang X, Li L, Payer C et al (2019) Evaluation of algorithms for multi-modality whole heart segmentation: an open-access grand challenge. Med Image Anal 58:101537. https://doi.org/10.1016/j.media.2019.101537

    Article  Google Scholar 

  52. Zotti C, Luo Z, Lalande A et al (2018) Convolutional neural network with shape prior applied to cardiac MRI segmentation. IEEE J Biomed Health Inform 23:1119–1128

    Article  Google Scholar 

  53. Zuluaga MA, Cardoso MJ, Modat M, Ourselin S (2013) Multi-atlas propagation whole heart segmentation from MRI and CTA using a local normalised correlation coefficient criterion. In: International conference on functional imaging and modeling of the heart. Springer, pp 174–181

Download references

Funding

This research is supported in part by Beijing Natural Science Foundation (5182018), the National Natural Science Foundation of China (No. 62172246), the Open Project Program of State Key Laboratory of Virtual Reality Technology and Systems (VRLAB2021A05), and the Youth Innovation and Technology Support Plan of Colleges and Universities in Shandong Province (2021KJ062).

Author information

Authors and Affiliations

  1. Beijing University of Chemical Technology, Beijing, China

    Yifan Yan & Jingyang Gao

  2. China University of Petroleum (East China), Qingdao, China

    Chenglizhao Chen

Authors
  1. Yifan Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chenglizhao Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jingyang Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Conceived and designed the experiments: YY CC JG. Performed the experiments: YY. Analyzed the data: YY CC. Wrote the paper: YY CC JG.

Corresponding author

Correspondence to Jingyang Gao.

Ethics declarations

Conflict of Interests

The authors declare that they have no competing interests.

Additional information

Availability of data and material

We use the public automated cardiac diagnosis challenge (ACDC) dataset. The data are downloaded from: https://acdc.creatis.insa-lyon.fr/

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yan, Y., Chen, C. & Gao, J. Lossless segmentation of cardiac medical images by a resolution consistent network with nondamage data preprocessing. Multimed Tools Appl 82, 20951–20973 (2023). https://doi.org/10.1007/s11042-022-14202-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-14202-2

Keywords

Search

Navigation