Abstract
As a result of the development of Artificial Intelligence (AI) techniques, in recent years, machine learning (ML) and deep learning (DL) approaches have been widely used to semantically enrich 3D architectural cultural heritage (ACH) point clouds. While existing approaches for analyzing and interpreting point clouds continue to improve, the generalizability of pre-trained ML and DL methods to various types of historic buildings remains uncertain. In this context, a comprehensive understanding of both methodologies can enable us to make more effective use of AI techniques in the ACH domain (e.g., data exploitation, model definition, analysis, and preservation). This work presents and compares two very different approaches for the 3D ACH semantic segmentation task. Specifically, we train and test a ML method based on the Random Forest (RF) classifier on the point cloud of three chapels part of the “Sacromonte Calvario di Domodossola” and on the two test scenes of the ArCH dataset. Then, we employ dynamic graph convolutional neural network (DGCNN) as our DL method, training on the ArCH dataset and testing on both the two unseen test scenes of the ArCH dataset and on the “Sacrimonti” chapel point clouds. We provide empirical experiments to illustrate the efficiency of applying ML and DL methodologies to ACH point clouds. Following that, the advantages and limitations of these two approaches are evaluated through a systematic study of the classification results.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Sánchez-Aparicio, L.J., Del Pozo, S., Ramos, L.F., Arce, A., Fernandes, F.: Heritage site preservation with combined radiometric and geometric analysis of TLS data. Autom. Constr. 85, 24–39 (2018). https://doi.org/10.1016/j.autcon.2017.09.023
Bosché, F.: Automated recognition of 3D CAD model objects in laser scans and calculation of as-built dimensions for dimensional compliance control in construction. Adv. Eng. Inform. 24(1), 107–118 (2010). https://doi.org/10.1016/j.aei.2009.08.006
Czerniawski, T., Leite, F.: Automated digital modeling of existing buildings: a review of visual object recognition methods. Autom. Constr. 113, 103131 (2020). https://doi.org/10.1016/j.autcon.2020.103131
Ham, Y., Golparvar-Fard, M.: Three-dimensional thermography-based method for cost-benefit analysis of energy efficiency building envelope retrofits. J. Comput. Civ. Eng. 29, B4014009 (2015). https://doi.org/10.1061/(ASCE)CP.1943-5487.0000406
Teruggi, S., Grilli, E., Russo, M., Fassi, F., Remondino, F.: A hierarchical machine learning approach for multi-level and multi-resolution 3D point cloud classification. Remote Sens. 12(16), 2598 (2020). https://doi.org/10.3390/rs12162598
Weinmann, M., Jutzi, B., Mallet, C., Weinmann, M.: Geometric features and their relevance for 3D point cloud classification. ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci., IV-1/W1, 157–164 (2017).https://doi.org/10.5194/isprs-annals-IV-1-W1-157-2017
Grilli, E., Farella, E. M., Torresani, A., Remondino, F.: Geometric features analysis for the classification of cultural heritage point clouds. Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. XLII-2/W15, 541–548 (2019). https://doi.org/10.5194/isprs-archives-XLII-2-W15-541-2019
Grilli, E., Remondino, F.: Machine learning generalization across different 3D architectural heritage. ISPRS Int. J. Geo-Inf. 9, 379 (2020). https://doi.org/10.3390/ijgi9060379
Wang, Y., Sun, Y., Liu, Z., Sarma, S.E., Bronstein, M.M., Solomon, J.M.: Dynamic graph CNN for learning on point clouds. ACM Trans. Graph. Tog. 38, 1–12 (2019). https://doi.org/10.1145/3326362
Matrone, F., et al.: A benchmark for large-scale heritage point cloud semantic segmentation. Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. XLIII-B2–2020, 1419–1426 (2020). https://doi.org/10.5194/isprs-archives-XLIII-B2-2020-1419-2020
Tommasi, C., Fiorillo, F., Jiménez Fernández-Palacios, B., Achille, C.: Access and web-sharing of 3D digital documentation of environmental and architectural heritage. Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. XLII-2/W9, 707–714 (2019). https://doi.org/10.5194/isprs-archives-XLII-2-W9-707-2019
Mathias, M., Martinovic, A., Weissenberg, J., Haegler, S., Van Gool, L.: Automatic architectural style recognition. In: Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XXXVIII-5-W16, 171–176 (2011). https://doi.org/10.5194/isprsarchivesXXXVIII-5-W16-171-2011
Ho, T.K.: Random decision forests. In: 3rd International Conference on Document Analysis and Recognition, vol. 1, pp. 278–282. IEEE, Montreal, QC, Canada (1995). https://doi.org/10.1109/ICDAR.1995.598994
Breiman, L.: Random forests. Mach. Learn. 45, 5–32 (2001). https://doi.org/10.1023/A:1010933404324
Griffiths, D., Boehm, J.: A review on deep learning techniques for 3D sensed data classification. Remote Sens. 11, 1499 (2019). https://doi.org/10.3390/rs11121499
Grilli, E., Dininno, D., Petrucci, G., Remondino, F.: From 2D to 3D supervised segmentation and classification for cultural heritage applications. Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. XLII-2, 399–406 (2018). https://doi.org/10.5194/isprs-archives-XLII-2-399-2018
Qi, C.R., Su, H., Mo, K., Guibas, L.J.: PointNet: Deep learning on point sets for 3D classification and segmentation. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 652–660. IEEE, Honolulu, HI, USA (2017). https://doi.org/10.48550/arXiv.1612.00593
Thomas, H., Qi, C.R., Deschaud, J.E., Marcotegui, B., Goulette, F., Guibas, L.J.: KPConv: Flexible and deformable convolution for point clouds. In: IEEE/CVF International Conference on Computer Vision, pp. 6411–6420. IEEE, Seoul, South Korea (2019). https://doi.org/10.48550/arXiv.1904.08889
Pierdicca, R., et al.: Point cloud semantic segmentation using a deep learning framework for cultural heritage. Remote Sens. 12, 1005 (2020). https://doi.org/10.3390/rs12061005
Matrone, F., Grilli, E., Martini, M., Paolanti, M., Pierdicca, R., Remondino, F.: Comparing machine and deep learning methods for large 3D heritage semantic segmentation. ISPRS Int. J. Geo-Inf. 9, 535 (2020). https://doi.org/10.3390/ijgi9090535
Cao, Y., Scaioni, M.: 3DLEB-Net: Label-efficient deep learning-based semantic segmentation of building point clouds at LoD3 level. Appl. Sci. 11, 8996 (2021). https://doi.org/10.3390/app11198996
Achille, C., Fassi, F., Mandelli, A., Fiorillo, F.: Surveying cultural heritage: summer school for conservation activities. Appl. Geomatics 10(4), 579–592 (2018). https://doi.org/10.1007/s12518-018-0225-3
Goutte, C., Gaussier, E.: A probabilistic interpretation of precision, recall and F-score, with implication for evaluation. In: Losada, D.E., Fernández-Luna, J.M. (eds.) Advances in Information Retrieval. Lecture Notes in Computer Science, vol. 3408, pp. 345–359. Springer, Heidelberg (2005). https://doi.org/10.1007/978-3-540-31865-1_25
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Cao, Y., Teruggi, S., Fassi, F., Scaioni, M. (2022). A Comprehensive Understanding of Machine Learning and Deep Learning Methods for 3D Architectural Cultural Heritage Point Cloud Semantic Segmentation. In: Borgogno-Mondino, E., Zamperlin, P. (eds) Geomatics for Green and Digital Transition. ASITA 2022. Communications in Computer and Information Science, vol 1651. Springer, Cham. https://doi.org/10.1007/978-3-031-17439-1_24
Download citation
DOI: https://doi.org/10.1007/978-3-031-17439-1_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-17438-4
Online ISBN: 978-3-031-17439-1
eBook Packages: Computer ScienceComputer Science (R0)