Skip to main content
SpringerLink
Log in

Generative adversarial network for newborn 3D skeleton part segmentation

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Childbirth simulations have been studied in order to predict and prevent difficult delivery issues. The reconstruction of the maternal pelvic model, which consists of a comprehensive fetal model with articulated joints, is important for therapeutic purposes. However, it is difficult and time-consuming to segment the various bones using classical image processing approaches. The aim of this study is to develop and evaluate a generative adversarial network to automatically segment the bony structures of the complete neonatal skeleton. A database of 124 newborn CT images was collected and segmented. Each 3D reconstructed skeleton was divided into 23 distinct bony segments. We proposed the generative adversarial network based on PointNet to perform the automated segmentation directly on the 3D point clouds. Our method was compared to the pointwise convolutional neural network to demonstrate its accuracy and efficiency. The GAN model produced highly accurate results with an IoU of 93.68% ± 7.37%, a Dice of 96.56% ± 4.41% and an accuracy score of 96.72% ± 3.56%, compared to 72.30% ± 5.10% for IoU, 83.82% ± 3.44% for Dice and 84.81% ± 3.25% for accuracy respectively by the pointwise convolutional neural network. In addition, our model behaved better on skeletons in anatomical postures than ones in fetal positions. This study opens new avenues for fast and accurate 3D part segmentation of the newborn 3D skeleton. In the future, further study should focus on segmenting fused bones like vertebrae and integrating the whole articulated skeleton into the maternal pelvic model to simulate complex vaginal delivery and perform associated preventive actions.

Graphical abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Data availability

All data and models used in this study will be provided upon request.

References

  1. Shah V, Coroneos CJ, Ng E (2021) The evaluation and management of neonatal brachial plexus palsy. Paediatr Child Health 26:493–497. https://doi.org/10.1093/pch/pxab083

    Article  Google Scholar 

  2. Macedonia CR, Gherman RB, Satin AJ (2003) Simulation laboratories for training in obstetrics and gynecology. Obstet Gynecol 102:388–392. https://doi.org/10.1016/S0029-7844(03)00483-6

    Article  Google Scholar 

  3. Dupuis O, Silveira R, Zentner A, Dittmar A, Gaucherand P, Cucherat M, Redarce T, Rudigoz R-C (2005) Birth simulator: reliability of transvaginal assessment of fetal head station as defined by the American College of Obstetricians and Gynecologists classification. Am J Obstet Gynecol 192:868–874. https://doi.org/10.1016/j.ajog.2004.09.028

    Article  Google Scholar 

  4. Parente MPL, Jorge RMN, Mascarenhas T, Fernandes AA, Martins JAC (2009) The influence of an occipito-posterior malposition on the biomechanical behavior of the pelvic floor. Eur J Obstet Gynecol Reprod Biol 144:S166–S169. https://doi.org/10.1016/j.ejogrb.2009.02.033

    Article  Google Scholar 

  5. Chen S, Grimm MJ (2021) Childbirth computational models: characteristics and applications. J Biomech Eng 143:50801. https://doi.org/10.1115/1.4049226

    Article  Google Scholar 

  6. Lapeer R, Gerikhanov Z, Sadulaev S-M, Audinis V, Rowland R, Crozier K, Morris E (2019) A computer-based simulation of childbirth using the partial Dirichlet-Neumann contact method with total Lagrangian explicit dynamics on the GPU. Biomech Model Mechanobiol 18:681–700. https://doi.org/10.1007/s10237-018-01109-x

    Article  Google Scholar 

  7. Ami O, Maran JC, Gabor P, Whitacre EB, Musset D, Dubray C, Mage G, Boyer L (2019) Three-dimensional magnetic resonance imaging of fetal head molding and brain shape changes during the second stage of labor. PLoS ONE 14:e0215721. https://doi.org/10.1371/journal.pone.0215721

    Article  Google Scholar 

  8. Dao TT (2019) From deep learning to transfer learning for the prediction of skeletal muscle forces. Med Biol Eng Comput 57:1049–1058. https://doi.org/10.1007/s11517-018-1940-y

    Article  Google Scholar 

  9. Ballit A, Dao T-T (2022) Recurrent neural network to predict hyperelastic constitutive behaviors of the skeletal muscle. Med Biol Eng Comput 60:1177. https://doi.org/10.1007/s11517-022-02541-z

    Article  Google Scholar 

  10. Nguyen-Le DH, Ballit A, Dao T-T (2023) A novel deep learning-driven approach for predicting the pelvis soft-tissue deformations toward a real-time interactive childbirth simulation. Eng Appl Artif Intell 126:107150. https://doi.org/10.1016/j.engappai.2023.107150

    Article  Google Scholar 

  11. O’Mahony N, Campbell S, Carvalho A , Harapanahalli S, Hernandez GV, Krpalkova L, Riordan D, Walsh J (2020) Deep learning vs. traditional computer vision. In: Arai K, Kapoor S (ed) Advances in Computer Vision. Springer International Publishing, Cham, pp 128–144

  12. Bai Z, Zhang X-L (2021) Speaker recognition based on deep learning: an overview. Neural Netw 140:65–99. https://doi.org/10.1016/j.neunet.2021.03.004

    Article  Google Scholar 

  13. Esteva A, Robicquet A, Ramsundar B, Kuleshov V, DePristo M, Chou K, Cui C, Corrado G, Thrun S, Dean J (2019) A guide to deep learning in healthcare. Nat Med 25:24–29. https://doi.org/10.1038/s41591-018-0316-z

    Article  Google Scholar 

  14. Norgeot B, Glicksberg BS, Butte AJ (2019) A call for deep-learning healthcare. Nat Med 25:14–15. https://doi.org/10.1038/s41591-018-0320-3

    Article  Google Scholar 

  15. Gürünlü B, Öztürk S (2022) A novel method for forgery detection on lung cancer images. Int J Inform Secur Sci 11:13–20

    Google Scholar 

  16. Liu P, Han H, Du Y, Zhu H, Li Y, Gu F, Xiao H, Li J, Zhao C, Xiao L, Wu X, Zhou SK (2021) Deep learning to segment pelvic bones: large-scale CT datasets and baseline models. Int J Comput Assist Radiol Surg 16:749–756. https://doi.org/10.1007/s11548-021-02363-8

    Article  Google Scholar 

  17. Liu X, Han C, Wang H, Wu J, Cui Y, Zhang X, Wang X (2021) Fully automated pelvic bone segmentation in multiparameteric MRI using a 3D convolutional neural network. Insights Imaging 12:93. https://doi.org/10.1186/s13244-021-01044-z

    Article  Google Scholar 

  18. Zhang L, Wang H (2020) A novel segmentation method for cervical vertebrae based on PointNet + + and converge segmentation. Comput Methods Programs Biomed 200:105798. https://doi.org/10.1016/j.cmpb.2020.105798

    Article  Google Scholar 

  19. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2020) Generative adversarial networks. Commun ACM 63:139–144. https://doi.org/10.1145/3422622

    Article  Google Scholar 

  20. Odena A, Olah C, Shlens J (2017) Conditional Image Synthesis with Auxiliary Classifier GANs. In: Precup D, Teh YW (eds) Proceedings of the 34th International Conference on Machine Learning. PMLR, pp 2642–2651

  21. Zhu J-Y, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp 2242–2251

  22. Ma C, Yang Y, Guo J, Pan F, Wang C, Guo Y (2022) Unsupervised point cloud completion and segmentation by Generative Adversarial Autoencoding Network. In: Koyejo S, Mohamed S, Agarwal A, Belgrave D, Cho K, Oh A (eds) Advances in Neural Information Processing systems. Curran Associates, Inc, pp 3556–3568

  23. Qi CR, Su H, Mo K, Guibas LJ (2017) PointNet: Deep learning on point sets for 3D classification and segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 652–660. https://doi.org/10.1109/CVPR.2017.16

  24. Hua B, Tran M, Yeung S (2018) Pointwise Convolutional Neural Networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 984–993

  25. Edgar HJH, Daneshvari Berry S, Moes E, Adolphi NL, Bridges P, Nolte KB (2020) New Mexico Decedent Image Database (NMDID). https://doi.org/10.25827/5S8C-N515

  26. Garcia-Garcia A, Orts-Escolano S, Oprea S, Villena-Martinez V, Martinez-Gonzalez P, Garcia-Rodriguez J (2018) A survey on deep learning techniques for image and video semantic segmentation. Appl Soft Comput 70:41–65. https://doi.org/10.1016/j.asoc.2018.05.018

    Article  Google Scholar 

  27. Nguyen T-N-T, Ballit A, Lecomte-Grosbras P, Colliat J-B, Dao T-T (n.d.) The uncertainty quantification of hyperelastic properties using precise and imprecise probabilities toward reliable in silico simulation of the second-stage labor. J Mech Med Biol 0:2350083. https://doi.org/10.1142/S0219519423500835

  28. Ballit A, Hivert M, Rubod C, Dao T-T (2023) Fast soft-tissue deformations coupled with mixed reality toward the next-generation childbirth training simulator. Med Biol Eng Comput 61:2207–2226. https://doi.org/10.1007/s11517-023-02864-5

    Article  Google Scholar 

  29. Achlioptas P, Diamanti O, Mitliagkas I, Guibas L (2018) Learning representations and generative models for 3D point clouds. In: Dy J, Krause A (eds) Proceedings of the 35th International Conference on Machine Learning, PMLR, pp 40–49

  30. Yu Y, Huang Z, Li F, Zhang H, Le X (2020) Point Encoder GAN: a deep learning model for 3D point cloud inpainting. Neurocomputing 384:192–199. https://doi.org/10.1016/j.neucom.2019.12.032

    Article  Google Scholar 

  31. Qin H, Zhang S, Liu Q, Chen L, Chen B (2020) PointSkelCNN: deep learning-based 3D human skeleton extraction from point clouds. Comput Graphics Forum 39:363–374. https://doi.org/10.1111/cgf.14151

    Article  Google Scholar 

  32. Takmaz A, Schult J, Kaftan I, Akçay M, Leibe B, Sumner R, Engelmann F, Tang S (2023) 3D Segmentation of Humans in Point Clouds with Synthetic Data. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)

  33. Qi CR, Yi L, Su H, Guibas LJ (2017) PointNet++: deep hierarchical feature learning on point sets in a metric space. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. Curran Associates Inc., Red Hook, NY, USA, pp 5105–5114

  34. Wang Z, Lu F (2018) VoxSegNet: volumetric CNNs for semantic part segmentation of 3D shapes. IEEE Trans Vis Comput Graph 26:2919–2930

    Article  Google Scholar 

  35. Yu F, Liu K, Zhang Y, Zhu C, Xu K (2019) PartNet: A Recursive Part Decomposition Network for Fine-Grained and Hierarchical Shape Segmentation. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 9483–9492. IEEE Computer Society, Los Alamitos

Download references

Funding

The authors would like to thank the Métropole Européenne de Lille (MEL) and ISITE ULNE (R-TALENT-20-009-DAO) for providing financial support to this project.

Author information

Authors and Affiliations

  1. Univ. Lille, CNRS, Centrale Lille, UMR 9013 - LaMcube - Laboratoire de Mécanique, Multiphysique, Multiéchelle, F-59000, Lille, France

    Hien-Duyen Nguyen-Le, Morgane Ferrandini & Tien-Tuan Dao

  2. Université de Technologie de Compiègne, CNRS, Biomechanics and Bioengineering, Centre de Recherche Royallieu, CS 60 319 – 60 203, Compiègne Cedex, France

    Duc-Phong Nguyen

  3. Ho Chi Minh City University of Technology and Education, Ho Chi Minh City, Vietnam

    Vi-Do Tran

  4. University of Science, Vietnam National University, Ho Chi Minh City, Vietnam

    Hoai-Danh Vo

  5. School of Biomedical Engineering, International University, Ho Chi Minh City, Vietnam

    Tan-Nhu Nguyen

  6. Vietnam National University, Ho Chi Minh City, Vietnam

    Tan-Nhu Nguyen

Authors
  1. Hien-Duyen Nguyen-Le
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Morgane Ferrandini
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Duc-Phong Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Vi-Do Tran
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hoai-Danh Vo
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Tan-Nhu Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Tien-Tuan Dao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tien-Tuan Dao.

Ethics declarations

Competing Interests

The authors declare that there is no conflict of interest related to this present study.

Ethics approval and consent

There is no ethical issue for the present study.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nguyen-Le, HD., Ferrandini, M., Nguyen, DP. et al. Generative adversarial network for newborn 3D skeleton part segmentation. Appl Intell (2024). https://doi.org/10.1007/s10489-024-05406-0

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10489-024-05406-0

Keywords

Search

Navigation