Skip to main content
SpringerLink
Log in

Meta semi-supervised medical image segmentation with label hierarchy

  • Methodology
  • Published:
Health Information Science and Systems Aims and scope Submit manuscript

Abstract

Semi-supervised learning (SSL) has attracted increasing attention in medical image segmentation, where the mainstream usually explores perturbation-based consistency as a regularization to leverage unlabelled data. However, unlike directly optimizing segmentation task objectives, consistency regularization is a compromise by incorporating invariance towards perturbations, and inevitably suffers from noise in self-predicted targets. The above issues result in a knowledge gap between supervised guidance and unsupervised regularization. To bridge the knowledge gap, this work proposes a meta-based semi-supervised segmentation framework with the exploitation of label hierarchy. Two main prominent components named Divide and Generalize, and Label Hierarchy, are built in this work. Concretely, rather than merging all knowledge indiscriminately, we dynamically divide consistency regularization from supervised guidance as different domains. Then, a domain generalization technique is introduced with a meta-based optimization objective which ensures the update on supervised guidance should generalize to the consistency regularization, thereby bridging the knowledge gap. Furthermore, to alleviate the negative impact of noise in self-predicted targets, we propose to distill the noisy pixel-level consistency by exploiting label hierarchy and extracting hierarchical consistencies. Comprehensive experiments on two public medical segmentation benchmarks demonstrate the superiority of our framework to other semi-supervised segmentation methods, with new state-of-the-art results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Zhou Y, Huang W, Dong P, Xia Y, Wang S. D-UNet: a dimension-fusion U shape network for chronic stroke lesion segmentation. IEEE/ACM Trans Comput Biol Bioinform. 2019;18(3):940–50.

    Article  Google Scholar 

  2. Chen Q-Q, Sun Z-H, Wei C-F, Wu EQ, Ming D. Semi-supervised 3D medical image segmentation based on dual-task consistent joint learning and task-level regularization. IEEE/ACM Trans Comput Biol Bioinform. 2022. https://doi.org/10.1109/TCBB.2022.3144428

  3. Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3431–3440; 2015.

  4. Hesamian MH, Jia W, He X, Kennedy P. Deep learning techniques for medical image segmentation: achievements and challenges. J Digit Imaging. 2019;32(4):582–96.

    Article  Google Scholar 

  5. He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778; 2016.

  6. Isensee F, Jaeger PF, Kohl SA, Petersen J, Maier-Hein KH. nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nat Methods. 2021;218(2):203–211.

  7. Liu C, Xie H, Zhang S, Mao Z, Sun J, Zhang Y. Misshapen pelvis landmark detection with local-global feature learning for diagnosing developmental dysplasia of the hip. IEEE Trans Med Imaging. 2020;39(12):3944–54.

    Article  Google Scholar 

  8. Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention, pp. 234–241. Cham: Springer; 2015.

  9. Feyjie AR, Azad R, Pedersoli M, Kauffman C, Ayed IB, Dolz J. Semi-supervised few-shot learning for medical image segmentation. arXiv preprint; 2020. arXiv:2003.08462.

  10. Yao H, Hu X, Li X. Enhancing pseudo label quality for semi-supervised domain-generalized medical image segmentation. arXiv preprint; 2022. arXiv:2201.08657.

  11. Tajbakhsh N, Jeyaseelan L, Li Q, Chiang JN, Wu Z, Ding X. Embracing imperfect datasets: a review of deep learning solutions for medical image segmentation. Med Image Anal. 2020;63: 101693.

    Article  Google Scholar 

  12. Yan K, Wang X, Lu L, Summers RM. Deeplesion: automated mining of large-scale lesion annotations and universal lesion detection with deep learning. J Med Imaging 2018;5(3):036501.

  13. Cheplygina V, de Bruijne M, Pluim JP. Not-so-supervised: a survey of semi-supervised, multi-instance, and transfer learning in medical image analysis. Med Image Anal. 2019;54:280–96.

  14. Yu L, Wang S, Li X, Fu C.-W, Heng P-A. Uncertainty-aware self-ensembling model for semi-supervised 3D left atrium segmentation. In: International conference on medical image computing and computer-assisted intervention. Berlin: Springer; 2019. p. 605–613.

  15. Seo H, Yu L, Ren H, Li X, Shen L, Xing L. Deep neural network with consistency regularization of multi-output channels for improved tumor detection and delineation. IEEE Trans Med Imaging. 2021;40(12):3369–78.

    Article  Google Scholar 

  16. Van Engelen JE, Hoos HH. A survey on semi-supervised learning. Mach Learn. 2020;109(2):373–440.

    Article  MathSciNet  Google Scholar 

  17. Li X, Yu L, Chen H, Fu C.-W, Xing L, Heng P-A. Transformation-consistent self-ensembling model for semisupervised medical image segmentation. IEEE Trans Neural Netw Learn Syst. 2020;32(2): 523–34

  18. Zheng Z, Wang X, Zhang X, Zhong Y, Yao X, Zhang Y, Wang Y. Semi-supervised segmentation with self-training based on quality estimation and refinement. In: International workshop on machine learning in medical imaging. Berlin: Springer; 2020. p. 30–39.

  19. Min S, Chen X, Xie H, Zha Z-J, Zhang Y. A mutually attentive co-training framework for semi-supervised recognition. IEEE Trans Multimedia. 2020;23:899–910.

    Article  Google Scholar 

  20. Malach E, Shalev-Shwartz S. Decoupling “when to update” from “how to update”. In: 31st Conference on advances in neural information processing (NIPS 2017), Long Beach. vol. 30; 2017.

  21. Zhang Y, Yang L, Chen J, Fredericksen M, Hughes DP, Chen DZ. Deep adversarial networks for biomedical image segmentation utilizing unannotated images. In: International conference on medical image computing and computer-assisted intervention. Berlin: Springer; 2017. p. 408–416

  22. Li S, Zhang C, He X. Shape-aware semi-supervised 3d semantic segmentation for medical images. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 552–561 (2020). Springer

  23. Ben-David S, Lu T, Pál D, Sotáková M. Learning low density separators. In: Artificial intelligence and statistics, PMLR; 2009. p. 25–32.

  24. Wu Y, Xu M, Ge Z, Cai J, Zhang L. Semi-supervised left atrium segmentation with mutual consistency training. In: International conference on medical image computing and computer-assisted intervention. Berlin: Springer; 2021; p. 297–306.

  25. Tarvainen A, Valpola H. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. In: Advances in neural information processing systems, vol. 30; 2017.

  26. Luo X, Chen J, Song T, Wang G. Semi-supervised medical image segmentation through dual-task consistency. In: Proceedings of the AAAI conference on artificial intelligence, vol. 35; 2021. p. 8801–8809.

  27. Wang K, Zhan B, Zu C, Wu X, Zhou J, Zhou L, Wang Y. Tripled-uncertainty guided mean teacher model for semi-supervised medical image segmentation. In: International conference on medical image computing and computer-assisted intervention. Berlin: Springer; 2021. p. 450–460.

  28. Kendall A, Gal Y. What uncertainties do we need in bayesian deep learning for computer vision? In: Advances in neural information processing systems, vol. 30; 2017.

  29. Shi Y, Zhang J, Ling T, Lu J, Zheng Y, Yu Q, Qi L, Gao Y. Inconsistency-aware uncertainty estimation for semi-supervised medical image segmentation. IEEE Trans Med Imaging. 2021;41(3):608–20.

    Article  Google Scholar 

  30. Liu S.-A, Xie H, Xu H, Zhang Y, Tian Q. Partial class activation attention for semantic segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition; 2022. p. 16836–16845.

  31. Xiong Z, Xia Q, Hu Z, Huang N, Bian C, Zheng Y, Vesal S, Ravikumar N, Maier A, Yang X, et al. A global benchmark of algorithms for segmenting the left atrium from late gadolinium-enhanced cardiac magnetic resonance imaging. Med Image Anal. 2021;67:101832.

    Article  Google Scholar 

  32. Menze BH, Jakab A, Bauer S, Kalpathy-Cramer J, Farahani K, Kirby J, Burren Y, Porz N, Slotboom J, Wiest R, et al. The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Trans Med Imaging. 2014;34(10):1993–2024.

    Article  Google Scholar 

  33. Laine S, Aila T. Temporal ensembling for semi-supervised learning. In: 5th International conference on learning representations, ICLR 2017, Toulon, France, 24–26 April 2017, conference track proceedings. OpenReview.net; 2017. https://openreview.net/forum?id=BJ6oOfqge

  34. Perone C.S, Cohen-Adad J. Deep semi-supervised segmentation with weight-averaged consistency targets. In: Deep learning in medical image analysis and multimodal learning for clinical decision support.Berlin: Springer; 2018. p. 12–19.

  35. Vu T.-H, Jain H, Bucher M, Cord M, Pérez P. Advent: adversarial entropy minimization for domain adaptation in semantic segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition; 2019. p. 2517–2526.

  36. Han B, Yao Q, Yu X, Niu G, Xu M, Hu W, Tsang I, Sugiyama M. Co-teaching: robust training of deep neural networks with extremely noisy labels. In: Advances in neural information processing systems, vol. 31; (2018.

  37. Zhang Y, Liao Q, Jiao R, Zhang J. Uncertainty-guided mutual consistency learning for semi-supervised medical image segmentation. arXiv preprint; 2021. arXiv:2112.02508.

  38. Hospedales T, Antoniou A, Micaelli P, Storkey A. Meta-learning in neural networks: a survey. IEEE Trans Pattern Anal Mach Intell. 2021;44(9):5149–69.

    Google Scholar 

  39. Vilalta R, Drissi Y. A perspective view and survey of meta-learning. Artif Intell Rev. 2002;18:77–95.

    Article  Google Scholar 

  40. Sun Q, Liu Y, Chua T-S, Schiele B. Meta-transfer learning for few-shot learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition; 2019, p. 403–412.

  41. Ravi S, Larochelle H. Optimization as a model for few-shot learning. In: International conference on learning representations; 2017.

  42. Finn C, Abbeel P, Levine S. Model-agnostic meta-learning for fast adaptation of deep networks. In: International conference on machine learning, PMLR; 2017; p. 1126–1135.

  43. Ren M, Zeng W, Yang B, Urtasun R. Learning to reweight examples for robust deep learning. In: International Conference on Machine Learning, pp. 4334–4343 (2018). PMLR

  44. Li J, Wong Y, Zhao Q, Kankanhalli MS. Learning to learn from noisy labeled data. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition; 2019. p. 5051–5059.

  45. Wang J, Lan C, Liu C, Ouyang Y, Qin T, Lu W, Chen Y, Zeng W, Yu P. Generalizing to unseen domains: a survey on domain generalization. IEEE Trans Knowl Data Eng. 2022. https://doi.org/10.24963/ijcai.2021/628

  46. Muandet K, Balduzzi D, Schölkopf B. Domain generalization via invariant feature representation. In: International conference on machine learning, PMLR; 2013. p. 10–18.

  47. Ghifary M, Kleijn WB, Zhang M, Balduzzi D. Domain generalization for object recognition with multi-task autoencoders. In: Proceedings of the IEEE international conference on computer vision; 2015. p. 2551–2559.

  48. Li H, Pan S.J, Wang S, Kot A.C. Domain generalization with adversarial feature learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2018. p. 5400–5409.

  49. Li D, Yang Y, Song Y.-Z, Hospedales T. Learning to generalize: Meta-learning for domain generalization. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 (2018)

  50. Balaji Y, Sankaranarayanan S, Chellappa R. Metareg: Towards domain generalization using meta-regularization. Advances in neural information processing systems 31 (2018)

  51. Lehner PE, Adelman L, Cheikes BA, Brown MJ. Confirmation bias in complex analyses. IEEE Trans Syst Man Cybern Part A Syst Humans 2008;38(3):584–592.

  52. Li G, Hari SKS, Sullivan M, Tsai T, Pattabiraman K, Emer J, Keckler SW. Understanding error propagation in deep learning neural network (dnn) accelerators and applications. In: Proceedings of the international conference for high performance computing, networking, storage and analysis; 2017. p. 1–12.

  53. Milletari F, Navab N, Ahmadi S-A. V-Net: fully convolutional neural networks for volumetric medical image segmentation. In: 2016 4th international conference on 3D vision (3DV). Piscataway: IEEE; 2016; p. 565–571.

  54. Buslaev A, Iglovikov V.I, Khvedchenya E, Parinov A, Druzhinin M, Kalinin AA. Albumentations: fast and flexible image augmentations. Information. 2020;11(2):125.

  55. Wang Y, Zhang Y, Tian J, Zhong C, Shi Z, Zhang Y, He Z. Double-uncertainty weighted method for semi-supervised learning. In: International conference on medical image computing and computer-assisted intervention. Berlin: Springer; 2020. p. 542–551.

  56. Zhang C, Bengio S, Hardt M, Recht B, Vinyals O. Understanding deep learning (still) requires rethinking generalization. Commun ACM 2021;64(3):107–15.

    Article  Google Scholar 

Download references

Funding

This work is supported by the CAS Project for Young Scientists in Basic Research (Grant No. YSBR-067)，the National Nature Science Foundation of China (62022076, 61972105, 61976008, U19A2057), Joint Research Fund of Guangzhou and University (202201020181).

Author information

Authors and Affiliations

  1. School of Information Science and Technology, University of Science and Technology of China, Hefei, 230026, Anhui, China

    Hai Xu, Hongtao Xie & Yongdong Zhang

  2. Cyberspace Institution of Advanced Technology, Guangzhou University, Guangzhou, 511442, Guangdong, China

    Qingfeng Tan

Authors
  1. Hai Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hongtao Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qingfeng Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yongdong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hongtao Xie.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, H., Xie, H., Tan, Q. et al. Meta semi-supervised medical image segmentation with label hierarchy. Health Inf Sci Syst 11, 26 (2023). https://doi.org/10.1007/s13755-023-00222-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s13755-023-00222-1

Keywords

Search

Navigation