Skip to main content

Advertisement

SpringerLink
Log in

Deep anatomy learning for lung airway and artery-vein modeling with contrast-enhanced CT synthesis

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

Endobronchial intervention requires detailed modeling of pulmonary anatomical substructure, such as lung airway and artery-vein maps, which are commonly extracted from non-contrast computed tomography (NCCT) independently using automatic segmentation approaches. We aim to make the first attempt to jointly train a CNN-based model for airway and artery-vein segmentation along with synthetic contrast-enhanced CT (CECT) generation.

Methods

A multi-task framework is proposed to simultaneously generate three segmentation maps and synthesize CECTs. We first design a collaborative learning model with tissue knowledge interaction for lung airway and artery-vein segmentation. Meanwhile, a conditional adversarial training strategy is applied to generate CECTs from NCCTs guided by artery maps. Additionally, CECT identity and reconstruction help to regularize the model for plausible NCCT to CECT translation.

Results

Extensive experiments were conducted to evaluate the performance of the proposed framework based on three datasets (90 NCCTs for the airway task, 55 NCCTs for the artery-vein task and 100 CECTs for the artery task). The results demonstrate the effective improvement of our proposed method compared to other methods and configurations that can achieve more accurate segmentation maps (Dice score coefficients for these three tasks are: 93.6%, 80.7% and 82.4%, respectively) and realistic CECTs simultaneously. The ablation study further verifies the effectiveness of the components of the designed model.

Conclusion

This study demonstrates the model potential in multi-task learning that integrates anatomically relevant segmentation and performs NCCT to CECT translation. Such an interaction approach promotes mutually for both promising segmentation results and plausible synthesis.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Gu Y, Gu C, Yang J, Sun J, Yang G-Z (2022) Vision-kinematics interaction for robotic-assisted bronchoscopy navigation. IEEE Trans Med Imaging 41(12):3600–3610

    Article  PubMed  Google Scholar 

  2. Shen M, Gu Y, Liu N, Yang G-Z (2019) Context-aware depth and pose estimation for bronchoscopic navigation. IEEE Robot Autom Lett 4(2):732–739

    Article  Google Scholar 

  3. Fabijanska A (2009) Results of applying two-pass region growing algorithm for airway tree segmentation to MDCT chest scans from EXACT database. In: The 2nd international workshop on pulmonary image analysis, CreateSpace, USA, pp 251–260

  4. Zhang M, Zhang H, Yang G-Z, Gu Y (2022) Cfda: Collaborative feature disentanglement and augmentation for pulmonary airway tree modeling of COVID-19 CTs. In: International conference on medical image computing and computer-assisted intervention, pp 506–516. Springer

  5. Meng Q, Roth HR, Kitasaka T, Oda M, Ueno J, Mori K (2017) Tracking and segmentation of the airways in chest CT using a fully convolutional network. In: International conference on medical image computing and computer-assisted intervention, pp 198–207. Springer

  6. Qin Y, Chen M, Zheng H, Gu Y, Shen M, Yang J, Huang X, Zhu Y-M, Yang G-Z (2019) Airwaynet: a voxel-connectivity aware approach for accurate airway segmentation using convolutional neural networks. In: International conference on medical image computing and computer-assisted intervention, pp 212–220. Springer

  7. Garcia-Uceda Juarez A, Selvan R, Saghir, Z, Bruijne Md (2019) A joint 3d UNet-graph neural network-based method for airway segmentation from chest CTs. In: International workshop on machine learning in medical imaging, pp. 583–591. Springer

  8. Charbonnier J-P, Brink M, Ciompi F, Scholten ET, Schaefer-Prokop CM, Van Rikxoort EM (2015) Automatic pulmonary artery-vein separation and classification in computed tomography using tree partitioning and peripheral vessel matching. IEEE Trans Med Imaging 35(3):882–892

    Article  PubMed  Google Scholar 

  9. Nardelli P, Jimenez-Carretero D, Bermejo-Pelaez D, Washko GR, Rahaghi FN, Ledesma-Carbayo MJ, Estépar RSJ (2018) Pulmonary artery-vein classification in CT images using deep learning. IEEE Trans Med Imaging 37(11):2428–2440

    Article  PubMed  PubMed Central  Google Scholar 

  10. Pu J, Leader JK, Sechrist J, Beeche CA, Singh JP, Ocak IK, Risbano MG (2022) Automated identification of pulmonary arteries and veins depicted in non-contrast chest CT scans. Med Image Anal 77:102367

    Article  PubMed  PubMed Central  Google Scholar 

  11. Qin Y, Zheng H, Gu Y, Huang X, Yang J, Wang L, Yao F, Zhu Y-M, Yang G-Z (2021) Learning tubule-sensitive CNNs for pulmonary airway and artery-vein segmentation in CT. IEEE Trans Med Imaging 40(6):1603–1617

    Article  PubMed  Google Scholar 

  12. Payer C, Pienn M, Bálint Z, Shekhovtsov A, Talakic E, Nagy E, Olschewski A, Olschewski H, Urschler M (2016) Automated integer programming based separation of arteries and veins from thoracic CT images. Med. Image Anal 34:109–122

    Article  PubMed  Google Scholar 

  13. Namasivayam S, Kalra MK, Torres WE, Small WC (2006) Adverse reactions to intravenous iodinated contrast media: a primer for radiologists. Emerg Radiol 12:210–215

    Article  PubMed  Google Scholar 

  14. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2020) Generative adversarial networks. Commun ACM 63(11):139–144

  15. Armato SG III, McLennan G, Bidaut L, McNitt-Gray MF, Meyer CR, Reeves AP, Zhao B, Aberle DR, Henschke CI, Hoffman EA (2011) The lung image database consortium (LIDC) and image database resource initiative (IDRI): a completed reference database of lung nodules on CT scans. Med. Phys 38(2):915–931

    Article  PubMed  PubMed Central  Google Scholar 

  16. Lo P, Van Ginneken B, Reinhardt JM, Yavarna T, De Jong PA, Irving B, Fetita C, Ortner M, Pinho R, Sijbers J (2012) Extraction of airways from CT (EXACT’09). IEEE Trans Med Imaging 31(11):2093–2107

    Article  PubMed  Google Scholar 

  17. Yushkevich PA, Piven J, Hazlett HC, Smith RG, Ho S, Gee JC, Gerig G (2006) User-guided 3d active contour segmentation of anatomical structures: significantly improved efficiency and reliability. Neuroimage 31(3):1116–1128

    Article  PubMed  Google Scholar 

  18. Wang K, Qiu Z, Wang W, Song T, Cao S, Zhao Y, Liu J, He Y, Gan S, Liang X, Xu M, Guo Z (2022) Pulmonary artery segmentation challenge 2022. Zenodo. https://doi.org/10.5281/zenodo.6361906

    Article  Google Scholar 

  19. Zhang H, Li J, Shen M, Wang Y, Yang G-Z (2019) Ddu-nets: distributed dense model for 3d MRI brain tumor segmentation. In: International MICCAI Brainlesion workshop, pp 208–217. Springer

  20. Zhang H, Shen M, Shah PL, Yang G-Z (2020) Pathological airway segmentation with cascaded neural networks for bronchoscopic navigation. In: 2020 IEEE international conference on robotics and automation (ICRA), pp 9974–9980. IEEE

  21. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention, pp 234–241. Springer

  22. Isensee F, Jaeger PF, Kohl SA, Petersen J, Maier-Hein KH (2021) nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nature Methods 18(2):203–211

    Article  CAS  PubMed  Google Scholar 

  23. Wu Y, He K (2018) Group normalization. In: Proceedings of the European conference on computer vision (ECCV), pp 3–19

  24. Ioffe S, Szegedy C (2015) Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: International conference on machine learning, pp. 448–456. PMLR

  25. He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE international conference on computer vision, pp 1026–1034

Download references

Acknowledgements

This work was partly supported by Shanghai Sailing Program (20YF1420800) and National Nature Science Foundation of China (No. 62003208).

Author information

Authors and Affiliations

  1. Institute of Medical Robotics, Shanghai Jiao Tong University, Shanghai, China

    Hanxiao Zhang, Minghui Zhang, Yun Gu & Guang-Zhong Yang

  2. Shanghai Center for Brain Science and Brain-Inspired Technology, Shanghai, China

    Yun Gu

Authors
  1. Hanxiao Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Minghui Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yun Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Guang-Zhong Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Yun Gu or Guang-Zhong Yang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed consent

The study conducted experiments based on two publicly available datasets. The access of our airway dataset should be applied for research purposes upon reasonable request.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, H., Zhang, M., Gu, Y. et al. Deep anatomy learning for lung airway and artery-vein modeling with contrast-enhanced CT synthesis. Int J CARS 18, 1287–1294 (2023). https://doi.org/10.1007/s11548-023-02946-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-023-02946-7

Keywords

Search

Navigation