Local quality assessment of point clouds for indoor mobile mapping
Introduction
Indoor 3D models are essential sources in acquiring information for many applications such as earthquake rescue tasks, cultural heritage protection, and intelligent building design. The quality and accuracy of creating an indoor 3D model are influenced by the quality of data collected from the real world. Point cloud data, a way to describe the 3D indoor environment, are widely exploited in building indoor 3D models [1], [2], [3], [4], [5]. With the rapid development of indoor mobile mapping systems (IMMSs), many IMMSs have been used to collect indoor point cloud data [6], [7], [8], [9], [10]. Typical IMMSs include wheeled mobile systems, backpacked mobile systems, and hand-held devices, etc. The wheeled mobile system smoothly integrates multi-sensors, including cameras, laser scanners, and inertial measurement units on a mobile platform, e.g., pushcart or robot. In the backpacked mobile system, the user backpacks a multi-sensor integrated system to collect data in motion. In the hand-held system, a data acquiring device, e.g., Kinect, is held by users when acquiring data. This way, it is convenient for these hand-held devices to collect data under certain conditions, especially in areas that are difficult for other IMMSs to access. In our proposed framework, the dataset is collected by a Kinect camera mounted on a mobile robot [7].
Some quality problems, or data degradation, such as missing data, occluded data, sparse data, blurred data, and very dark or very bright data, are inevitable for IMMS point clouds. Causes of data degradation include the characteristics of the sensing device, large rotation angle of the mobile platform in motion, and uneven illumination distribution in an indoor environment. Compared with image degradation, the reasons for the degradation of point clouds differ in different local areas, leading to an uneven distribution of point cloud quality. Moreover, different reasons of degradation lead to a diverse degradation of point clouds. Therefore, the qualities of point clouds have the characteristics of diversity and locality for indoor mobile mapping.
The local quality assessment of point clouds are to handle the data quality assessment by considering the diverse and local degradation of the IMMS point clouds. In general, good indoor point cloud data should not only have complete structure information but also maintain consistency between the appearance and structural information. Poor quality data need to be discriminated because they will not provide effective and sufficient information. However, in our dataset, there is an imbalanced problem, i.e., the amount of low-quality data is much greater than high-quality data. The local quality assessment of point clouds can classify these data into different degradation types and prepare for the further repair of these data based on different strategies. One main challenge for automated quality assessment of IMMS point clouds is the establishment of a training dataset in terms of time and cost. On one hand, it is difficult to classify different quality types in one point cloud by manually labeling because of the diverse and local degradation of point clouds. On the other hand, it is difficult, or impossible, to obtain the ground-truth (or reference data) without degradation, which indicates that our quality assessment problem needs to be considered in the absence of reference situations. Thus, it is essential to know how to use limited labeled data to predict the labels of a large number of unlabeled data. A semi-supervised learning method, i.e., a method requiring only a small amount of labeled training data, provides an efficient way to address this challenge.
In this paper, we propose a new framework to assess the local quality of indoor mobile mapping point clouds. First, we describe the IMMS point cloud degradation by geometric feature descriptors. To effectively analyze the essential components of these geometric feature descriptors, a feature selection method is integrated into the proposed framework to reduce the redundancy of these used features. To avoid the intensive labor costs of manual labels, a semi-supervised method, named Safe Semi-supervised Support Vector Machines (S4VMs) [11], is integrated into our proposed framework to conduct quality assessment tasks by manually labeling a small portion of the training dataset. Additionally, we establish a point cloud dataset (benchmark) with multiple kinds of quality problems to evaluate the proposed framework.
The rest of this paper is organized as follows: first, Section 2 reviews the related work; next, Section 3 details our proposed framework in three parts, including the establishment of a dataset with multiple kinds of quality problems, the feature description of the degraded data, and the local quality assessment of indoor point clouds; then, Section 4 reports the experimental results and presents the comparative experiments; finally, Section 5 concludes the entire paper.
Section snippets
Related works
Most recent works on data quality assessment focused on 2D images [12], [13], [14], [15], [16]. Xue et al. [12] established a codebook to assess the quality of images by computing quality-aware centroids of each patch in the training images. Ref. [13] presented a sparse feature representation method to learn a dictionary on the spatial correlations between training images. Two deep neural network methods in Refs. [15] and [16] were introduced to address non-referenced image quality assessment
Point clouds acquisition
An IMMS integrating a 2D laser scanner and a RGB-D camera is adopted for collecting 3D indoor point clouds in this paper [7]. The 2D laser scanner and RGB-D camera are used to build 2D map and to obtain point clouds, respectively. Moreover, this IMMS can achieve a 2D trajectory of the mobile platform while building the maps. However, the acquired data have some quality problems, such as missing data, occluded data, too sparse data, blurriness, and too much darkness or brightness. To deeply
Experiments and results
The proposed framework was coded with C++, and implemented on a personal computer with a single core 3.2 GHz and a RAM of 16 GB. To assess the data quality of local regions of a point cloud, the local search radius of 3D patch is set to 6 cm. According to [11], the regularization parameters , , and , are set to 100, 0.1 and 0.1, respectively. Besides, the sampling size , cluster number , and risk parameter , are set to 100, 10, 3, respectively. The kernel type of S4VM is KBF.
Conclusion
In this paper, we proposed a semi-supervised learning framework to solve the problem of local quality assessment of IMMS point clouds with limited labeled data and a large amount of unlabeled data. First, we collected IMMS point cloud data by a depth camera mounted on a mobile robot, and created a point cloud dataset with multiple kinds of quality problems. Next, we used feature extraction and feature selection methods to obtain optimal feature sets that are exploited to describe local
Acknowledgments
This work is supported by the National Science Foundation of China (Grant no. 61401382).
Fangfang Huang received the B.Sc degree in computer science from Fuzhou University, China in 2013. She is currently a master student with the Fujian Key Laboratory of Sensing and Computing for Smart City in the School of Information Science and Engineering, Xiamen University, China. Her current research interests include computer vision, machine learning, and mobile LiDAR point clouds data processing.
References (44)
- et al.
Handling occlusions in augmented reality based on 3D reconstruction method
Neurocomputing
(2015) - et al.
High-level attributes modeling for indoor scenes classification
Neurocomputing
(2013) - et al.
No reference image quality assessment using sparse feature representation in two dimensions spatial correlation
Neurocomputing
(2016) - et al.
Image quality assessment: a sparse learning way
Neurocomputing
(2015) - et al.
No-reference image quality assessment with shearlet transform and deep neural networks
Neurocomputing
(2015) - et al.
Deviation analysis method for the assessment of the quality of the as-is building information models generated from point cloud data
Autom. Constr.
(2013) - et al.
Combinative hypergraph learning for semi-supervised image classification
Neurocomputing
(2015) - et al.
Empirical characterization of random forest variable importance measures
Comput. Stat. Data Anal
(2008) - et al.
Mining data with random forests: A survey and results of new tests
Pattern Recognit.
(2011) - S. Izadi, D. Kim, O. Hilliges, D. Molyneaux, R. Newcombe, P. Kohli, J. Shotton, S. Hodges, D. Freeman, A. Davison,...
Indoor localization algorithms for an ambulatory human operated 3D mobile mapping system
Remote Sens.
Three-dimensional indoor mobile mapping with fusion of two-dimensional laser scanner and RGB-D camera data
IEEE Geosci. Remote Sens. Lett.
Zebedee: Design of a spring-mounted 3-d range sensor with application to mobile mapping
IEEE Trans. Robot.
RGB-D mapping: using Kinect-style depth cameras for dense 3D modeling of indoor environments
Int. J. Robot. Res.
Towards making unlabeled data never hurt
IEEE Trans. Pattern Anal. Mach. Intell.
Quality analysis on 3D building models reconstructed from airborne laser scanning data
ISPRS J. Photogramm. Remote Sens.
Cited by (15)
Indoor mapping using low-cost MLS point clouds and architectural skeleton constraints
2023, Automation in ConstructionABC-PSO for vertical handover in heterogeneous wireless networks
2017, NeurocomputingCitation Excerpt :Nowadays, the mobile cloud computing has been advanced with much attention worldwide. Generally, the mobile cloud computing is one of the most rapidly emerging future technologies [1–4]. Although mobile devices has limited capacity to store and execute processes, information properties query process is still considered as one of the main issues for implementing mobile cloud computing environment.
Visual Saliency and Quality Evaluation for 3D Point Clouds and Meshes: An Overview
2022, APSIPA Transactions on Signal and Information ProcessingPoint cloud visualization methods: A study on subjective preferences
2021, European Signal Processing ConferenceMapping for Indoor Walking Environment from Point Clouds by Using Mobile Mapping Systems
2021, International Journal of Geospatial and Environmental Research
Fangfang Huang received the B.Sc degree in computer science from Fuzhou University, China in 2013. She is currently a master student with the Fujian Key Laboratory of Sensing and Computing for Smart City in the School of Information Science and Engineering, Xiamen University, China. Her current research interests include computer vision, machine learning, and mobile LiDAR point clouds data processing.
Chenglu Wen received the Ph.D. degree in mechanical engineering from China Agricultural University, Beijing, China in 2009. She is currently an Associate Professor with Fujian Key Laboratory of Sensing and Computing for Smart City, School of Information Science and Engineering, Xiamen University, Xiamen, China. She has coauthored more than 30 research papers published in refereed journals and proceedings. Her current research interests include machine vision, machine learning, and point cloud processing. She is the secretary of ISPRS WG I/3 on Multi-Platform Multi-Sensor System Calibration (2012–2016).
Huan Luo received the B.Sc degree in software engineering from Nanchang University, China in 2009. He is currently a Ph.D. student with the Fujian Key Laboratory of Sensing and Computing for Smart City in the School of Information Science and Engineering, Xiamen University, China. His current research interests include computer vision, machine learning, and mobile LiDAR point clouds data processing.
Ming Cheng received the Ph.D. degree in Biomedical Engineering from Tsinghua University, China in 2004. He is currently an Associate Professor with the Fujian Key Laboratory of Sensing and Computing for Smart Cities, School of Information Science and Engineering, Xiamen University, China. His research interests include remote sensing image processing, computer vision, and machine learning.
Cheng Wang received the Ph.D. degree in information and communication engineering from the National University of Defense Technology, Changsha, China in 2002. He is currently a Professor with and the Associate Dean of the School of Information Science and Technology, Xiamen University, Xiamen, China. He has authored more than 80 publications. His research interests include remote sensing image processing, mobile LiDAR data analysis, and multisensor fusion. He is a council member of China Society of Image and Graphics, IEEE Senior Member, and Co-Chair of ISPRS Working Group I/3 on Multi-Platform Multi-Sensor System Calibration (2012-2016).
Jonathan Li received the Ph.D. degree in geomatics engineering from the University of Cape Town, Cape Town, South Africa. He is currently a Professor with the Fujian Key Laboratory of Sensing and Computing for Smart Cities, School of Information Science and Engineering, Xiamen University, China. He is also Professor and Head of the Mobile Sensing and Geodata Science Lab at the University of Waterloo, Canada. He has co-authored more than 300 publications, over 130 of which were published in refereed journals, including IEEE-TGRS, IEEE-JSTARS, IEEE-TITS, IEEE-GRSL, ISPRS-JPRS, IJRS, PE&RS, and RSE. His current research interests include information extraction from mobile LiDAR point clouds and from Earth observation images. Prof. Li is the Chair of the ISPRS Working Group I/Va on Mobile Scanning and Imaging Systems (2012-2016), Chair of the ICA Commission on Sensor-driven Mapping (2015-2019), and Associate Editor of IEEE-TITS, IEEE-JSARRS and Geomatica.