Skip to main content
SpringerLink
Log in

ECTransNet: An Automatic Polyp Segmentation Network Based on Multi-scale Edge Complementary

  • Published:
Journal of Digital Imaging Aims and scope Submit manuscript

Abstract

Colonoscopy is acknowledged as the foremost technique for detecting polyps and facilitating early screening and prevention of colorectal cancer. In clinical settings, the segmentation of polyps from colonoscopy images holds paramount importance as it furnishes critical diagnostic and surgical information. Nevertheless, the precise segmentation of colon polyp images is still a challenging task owing to the varied sizes and morphological features of colon polyps and the indistinct boundary between polyps and mucosa. In this study, we present a novel network architecture named ECTransNet to address the challenges in polyp segmentation. Specifically, we propose an edge complementary module that effectively fuses the differences between features with multiple resolutions. This enables the network to exchange features across different levels and results in a substantial improvement in the edge fineness of the polyp segmentation. Additionally, we utilize a feature aggregation decoder that leverages residual blocks to adaptively fuse high-order to low-order features. This strategy restores local edges in low-order features while preserving the spatial information of targets in high-order features, ultimately enhancing the segmentation accuracy. According to extensive experiments conducted on ECTransNet, the results demonstrate that this method outperforms most state-of-the-art approaches on five publicly available datasets. Specifically, our method achieved mDice scores of 0.901 and 0.923 on the Kvasir-SEG and CVC-ClinicDB datasets, respectively. On the Endoscene, CVC-ColonDB, and ETIS datasets, we obtained mDice scores of 0.907, 0.766, and 0.728, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data Availability

The codes are available online at https://github.com/liuweikang1112/polyp_seg.

References

  1. Wang J, Zhang X, Lv P et al. Automatic Liver Segmentation Using EfficientNet and Attention-Based Residual U-Net in CT, Journal of Digital Imaging; 35, pp. 1479–1493, 2022.https://doi.org/10.1007/s10278-022-00668-x.

  2. Sun Y, Li Y, Wang P et al. Lesion Segmentation in Gastroscopic Images Using Generative Adversarial Networks, Journal of Digital Imaging; 35, pp. 459–468, 2022.https://doi.org/10.1007/s10278-022-00591-1.

  3. Li M, Lian F, Guo S. Multi-scale Selection and Multi-channel Fusion Model for Pancreas Segmentation Using Adversarial Deep Convolutional Nets, Journal of Digital Imaging; 35, pp. 47–55, 2022.https://doi.org/10.1007/s10278-021-00563-x.

  4. Sung H, Ferlay J, Siegel R L et al. Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries, CA: A Cancer Journal for Clinicians; 71, pp. 209–249, 2021.https://doi.org/10.3322/caac.21660.

  5. Zhao S, Wang S, Pan P et al. Magnitude, Risk Factors, and Factors Associated With Adenoma Miss Rate of Tandem Colonoscopy: A Systematic Review and Meta-analysis, Gastroenterology; 156, pp. 1661–1674.e1611, 2019.https://doi.org/10.1053/j.gastro.2019.01.260.

  6. Favoriti P, Carbone G, Greco M et al. Worldwide burden of colorectal cancer: a review, Updates in Surgery; 68, pp. 7–11, 2016.https://doi.org/10.1007/s13304-016-0359-y.

  7. FIORI M, MUSÉ P, SAPIRO G. A COMPLETE SYSTEM FOR CANDIDATE POLYPS DETECTION IN VIRTUAL COLONOSCOPY; 28, pp. 1460014, 2014.https://doi.org/10.1142/s0218001414600143.

  8. Mamonov A V, Figueiredo I N, Figueiredo P N et al. Automated Polyp Detection in Colon Capsule Endoscopy, IEEE Transactions on Medical Imaging; 33, pp. 1488–1502, 2014.https://doi.org/10.1109/TMI.2014.2314959.

  9. Maghsoudi O H. Superpixel based segmentation and classification of polyps in wireless capsule endoscopy. In: 2017 IEEE Signal Processing in Medicine and Biology Symposium (SPMB). pp. 1–4, 2017.https://doi.org/10.1109/SPMB.2017.8257027.

  10. Ronneberger O, Fischer P, Brox T. U-Net: Convolutional Networks for Biomedical Image Segmentation. In: Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. Cham, pp. 234–241, 2015.https://doi.org/10.1007/978-3-319-24574-4_28.

  11. Zhou Z, Siddiquee M M R, Tajbakhsh N et al. UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation, IEEE Transactions on Medical Imaging; 39, pp. 1856–1867, 2018.https://doi.org/10.1109/TMI.2019.2959609.

  12. Jha D, Smedsrud P H, Riegler M A et al. ResUNet++: An Advanced Architecture for Medical Image Segmentation. In: 2019 IEEE International Symposium on Multimedia (ISM). pp. 225–2255, 2019.https://doi.org/10.1109/ISM46123.2019.00049.

  13. Drozdzal M, Vorontsov E, Chartrand G et al. The Importance of Skip Connections in Biomedical Image Segmentation. In: Deep Learning and Data Labeling for Medical Applications. Cham, pp. 179–187, 2016.https://doi.org/10.1007/978-3-319-46976-8_19.

  14. Jha D, Riegler M A, Johansen D et al. DoubleU-Net: A Deep Convolutional Neural Network for Medical Image Segmentation. In: 2020 IEEE 33rd International Symposium on Computer-Based Medical Systems (CBMS). pp. 558–564, 2020.https://doi.org/10.1109/CBMS49503.2020.00111.

  15. Fan D-P, Ji G-P, Zhou T et al. PraNet: Parallel Reverse Attention Network for Polyp Segmentation. In: Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. Cham, pp. 263–273, 2020.https://doi.org/10.1007/978-3-030-59725-2_26.

  16. Huang C-H, Wu H-Y, Lin Y-L J a p a. Hardnet-mseg: A simple encoder-decoder polyp segmentation neural network that achieves over 0.9 mean dice and 86 fps, 2021.https://doi.org/10.48550/arXiv.2101.07172.

  17. Chao P, Kao C Y, Ruan Y et al. HarDNet: A Low Memory Traffic Network. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV). pp. 3551–3560, 2019.https://doi.org/10.1109/ICCV.2019.00365.

  18. Shen Y, Jia X, Meng M Q H. HRENet: A Hard Region Enhancement Network for Polyp Segmentation. In: Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. Cham, pp. 559–568, 2021.https://doi.org/10.1007/978-3-030-87193-2_53.

  19. Zhong J, Wang W, Wu H et al. PolypSeg: An Efficient Context-Aware Network for Polyp Segmentation from Colonoscopy Videos. In: Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. Cham, pp. 285–294, 2020.https://doi.org/10.1007/978-3-030-59725-2_28.

  20. Wei J, Hu Y, Zhang R et al. Shallow Attention Network for Polyp Segmentation. In: Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. Cham, pp. 699–708, 2021.https://doi.org/10.1007/978-3-030-87193-2_66.

  21. Zhao X, Zhang L, Lu H. Automatic Polyp Segmentation via Multi-scale Subtraction Network. In: Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. Cham, pp. 120–130, 2021.https://doi.org/10.1007/978-3-030-87193-2_12.

  22. Srivastava A, Jha D, Chanda S et al. MSRF-Net: A Multi-Scale Residual Fusion Network for Biomedical Image Segmentation, IEEE Journal of Biomedical and Health Informatics; 26, pp. 2252–2263, 2022.https://doi.org/10.1109/JBHI.2021.3138024.

  23. Tomar N K, Jha D, Bagci U et al. TGANet: Text-Guided Attention for Improved Polyp Segmentation. In: Medical Image Computing and Computer Assisted Intervention – MICCAI 2022. Cham, pp. 151–160, 2022.https://doi.org/10.1007/978-3-031-16437-8_15.

  24. Hosseini H, Xiao B, Jaiswal M et al. On the Limitation of Convolutional Neural Networks in Recognizing Negative Images. In: 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA). pp. 352–358, 2017.https://doi.org/10.1109/ICMLA.2017.0-136.

  25. Vaswani A, Shazeer N, Parmar N et al. Attention is All you Need. 2017.https://doi.org/10.48550/arXiv.1706.03762

  26. Jha D, Smedsrud P H, Riegler M A et al. Kvasir-SEG: A Segmented Polyp Dataset. In: MultiMedia Modeling. Cham, pp. 451–462, 2020.https://doi.org/10.1007/978-3-030-37734-2_37.

  27. Bernal J, Sánchez F J, Fernández-Esparrach G et al. WM-DOVA maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physicians, Computerized Medical Imaging and Graphics; 43, pp. 99–111, 2015.https://doi.org/10.1016/j.compmedimag.2015.02.007.

  28. Vázquez D, Bernal J, Sánchez F J et al. A Benchmark for Endoluminal Scene Segmentation of Colonoscopy Images, Journal of Healthcare Engineering; 2017, pp. 4037190, 2017.https://doi.org/10.1155/2017/4037190.

  29. Bernal J, Sánchez J, Vilariño F. Towards automatic polyp detection with a polyp appearance model, Pattern Recognition; 45, pp. 3166–3182, 2012.https://doi.org/10.1016/j.patcog.2012.03.002.

  30. Silva J, Histace A, Romain O et al. Toward embedded detection of polyps in WCE images for early diagnosis of colorectal cancer, International Journal of Computer Assisted Radiology and Surgery; 9, pp. 283–293, 2014.https://doi.org/10.1007/s11548-013-0926-3.

  31. Gao S H, Cheng M M, Zhao K et al. Res2Net: A New Multi-Scale Backbone Architecture, IEEE Transactions on Pattern Analysis and Machine Intelligence; 43, pp. 652–662, 2021.https://doi.org/10.1109/TPAMI.2019.2938758.

  32. Woo S, Park J, Lee J-Y et al. Cbam: Convolutional block attention module. In: Proceedings of the European conference on computer vision (ECCV). pp. 3–19, 2018.https://doi.org/10.48550/arXiv.1807.06521.

  33. He K, Zhang X, Ren S et al. Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770–778, 2016.https://doi.org/10.1109/CVPR.2016.90

  34. Yu F, Koltun V. Multi-Scale Context Aggregation by Dilated Convolutions. In: ICLR. 2016.https://doi.org/10.48550/arXiv.1511.07122.

  35. Milletari F, Navab N, Ahmadi S A. V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation. In: 2016 Fourth International Conference on 3D Vision (3DV). pp. 565–571, 2016.https://doi.org/10.1109/3DV.2016.79.

  36. Chen L-C, Zhu Y, Papandreou G et al. Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the European conference on computer vision (ECCV). pp. 801–818, 2018.https://doi.org/10.1007/978-3-030-01234-2_49.

Download references

Funding

This work was supported in part by the Liaoning Provincial Education Department’s Service Local Project, grant/award number 2020FWDF04, and the Scientific Research Fund of Liaoning Provincial Education Department of China, grant/award number: 2020LNZD05.

Author information

Authors and Affiliations

  1. School of Electronic and Information Engineering, University of Science and Technology Liaoning, Anshan, 114051, China

    Weikang Liu, Zhigang Li, Chunyang Li & Hongyan Gao

Authors
  1. Weikang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhigang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chunyang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hongyan Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Weikang Liu (first author): conceptualization, methodology, software, data curation, and writing, original draft; Zhigang Li (corresponding author): conceptualization, supervision, funding acquisition, resources, and writing, review and editing; Chunyang Li: data curation, investigation, validation, and formal analysis; Hongyan Gao: visualization and writing, review and editing.

Corresponding author

Correspondence to Zhigang Li.

Ethics declarations

Ethics Approval

This research study was conducted retrospectively from data obtained for clinical purposes. We used only data coming from publicly available datasets.

Consent to Participate

Not applicable.

Consent for Publication

Not applicable.

Competing Interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, W., Li, Z., Li, C. et al. ECTransNet: An Automatic Polyp Segmentation Network Based on Multi-scale Edge Complementary. J Digit Imaging 36, 2427–2440 (2023). https://doi.org/10.1007/s10278-023-00885-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10278-023-00885-y

Keywords

Search

Navigation