Abstract
Face-skull transformation, i.e., shape transformation between facial surface and skull structure, has a wide range of applications in various fields such as forensic facial reconstruction and craniomaxillofacial (CMF) surgery planning. However, this transformation is a challenging task due to the significant differences between the geometric topologies of the face and skull shapes. In this paper, we propose a novel coarse-to-fine face-skull transformation network(i.e., FAST-Net) that has a pyramid architecture to gradually improve the transformation level by level. Specifically, using face-to-skull transformation for instance, in the first pyramid level, we use a point displacement sub-network to predict a coarse skull shape of point cloud from a given facial shape of point cloud with a skull template of point cloud as prior information. In the following pyramid levels, we further refine the predicted skull shape by first dividing the skull shape together with the given facial shape into different sub-regions, individually feeding the regions to a new sub-network, and merging the outputs as a refined skull shape. Finally, we generate a surface mesh model for the final predicted skull point cloud by non-rigidly registration with a skull template. Experimental results show that our method achieves the state-of-the-art performance on the task of face-skull transformation.
L. Zhao and L. Ma—Equal Contributions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Aspert, N., Santa-Cruz, D., Ebrahimi, T.: MESH: measuring errors between surfaces using the hausdorff distance. In: Proceedings of the IEEE International Conference on Multimedia and Expo, vol. 1, pp. 705–708. IEEE (2002)
Guleria, A., Krishan, K., Sharma, V., Kanchan, T.: Methods of forensic facial reconstruction and human identification: historical background, significance, and limitations. Sci. Nat. 110(2), 8 (2023)
Ichim, A.E., Kadleček, P., Kavan, L., Pauly, M.: Phace: physics-based face modeling and animation. ACM Trans. Graph. (TOG) 36(4), 1–14 (2017)
Li, Y., Harada, T.: Non-rigid point cloud registration with neural deformation pyramid. arXiv preprint arXiv:2205.12796 (2022)
Lorensen, W., Cline, H.: Marching cubes: a high resolution 3D surface construction algorithm. ACM SIGGRAPH Comput. Graph. 21, 163 (1987)
Ma, L., et al.: Deep simulation of facial appearance changes following craniomaxillofacial bony movements in orthognathic surgical planning. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12904, pp. 459–468. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87202-1_44
Ma, L., et al.: Bidirectional prediction of facial and bony shapes for orthognathic surgical planning. Med. Image Anal., 102644 (2022)
Ma, L., et al.: Simulation of postoperative facial appearances via geometric deep learning for efficient orthognathic surgical planning. IEEE Trans. Med. Imaging (2022)
Ma, X., Qin, C., You, H., Ran, H., Fu, Y.: Rethinking network design and local geometry in point cloud: a simple residual MLP framework. arXiv preprint arXiv:2202.07123 (2022)
Madsen, D., Lüthi, M., Schneider, A., Vetter, T.: Probabilistic joint face-skull modelling for facial reconstruction. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5295–5303 (2018)
Valsecchi, A., Damas, S., Cordón, O.: A robust and efficient method for skull-face overlay in computerized craniofacial superimposition. IEEE Trans. Inf. Forensics Secur. 13(8), 1960–1974 (2018)
Wang, Y., Cao, M., Fan, Z., Peng, S.: Learning to detect 3D facial landmarks via heatmap regression with graph convolutional network (2022)
Wu, T., Hung, A., Mithraratne, K.: Generating facial expressions using an anatomically accurate biomechanical model. IEEE Trans. Vis. Comput. Graph. 20(11), 1519–1529 (2014)
Xiao, D., et al.: Estimating reference shape model for personalized surgical reconstruction of craniomaxillofacial defects. IEEE Trans. Biomed. Eng. 68(2), 362–373 (2020)
Yin, K., Huang, H., Cohen-Or, D., Zhang, H.: P2P-NET: bidirectional point displacement net for shape transform. ACM Trans. Graph. (TOG) 37(4), 1–13 (2018)
Acknowledgement
This work was supported in part by The Key R &D Program of Guangdong Province, China (grant number 2021B0101420006), National Natural Science Foundation of China (grant number 62131015), and Science and Technology Commission of Shanghai Municipality (STCSM) (grant number 21010502600).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zhao, L. et al. (2024). FAST-Net: A Coarse-to-fine Pyramid Network for Face-Skull Transformation. In: Cao, X., Xu, X., Rekik, I., Cui, Z., Ouyang, X. (eds) Machine Learning in Medical Imaging. MLMI 2023. Lecture Notes in Computer Science, vol 14349. Springer, Cham. https://doi.org/10.1007/978-3-031-45676-3_11
Download citation
DOI: https://doi.org/10.1007/978-3-031-45676-3_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-45675-6
Online ISBN: 978-3-031-45676-3
eBook Packages: Computer ScienceComputer Science (R0)
