Elsevier

Neurocomputing

Volume 196, 5 July 2016, Pages 59-69
Neurocomputing

Local quality assessment of point clouds for indoor mobile mapping

https://doi.org/10.1016/j.neucom.2016.02.033Get rights and content

Highlights

  • A point cloud dataset with multiple kinds of quality problems is created.

  • The main causes of point cloud data degradation in indoor mobile mapping is analyzed.

  • A novel semi-supervised framework is proposed for quality assessment of indoor mobile mapping point cloud data.

Abstract

The quality of point clouds obtained by RGB-D camera-based indoor mobile mapping can be limited by local degradation because of complex scenarios such as sensor characteristics, partial occlusions, cluttered backgrounds, and complex illumination conditions. This paper presents a machine learning framework to assess the local quality of indoor mobile mapping point cloud data. In our proposed framework, a point cloud dataset with multiple kinds of quality problems is first created by manual annotation and degradation simulation. Then, feature extraction methods based on 3D patches are treated as operating units to conduct quality assessment in local regions. Also, a feature selection algorithm is deployed to obtain the essential components of feature sets that are used to effectively represent local degradation. Finally, a semi-supervised method is introduced to classify quality types of point clouds. Comparative experiments demonstrate that the proposed framework obtained promising quality assessment results with limited labeled data and a large amount of unlabeled data.

Introduction

Indoor 3D models are essential sources in acquiring information for many applications such as earthquake rescue tasks, cultural heritage protection, and intelligent building design. The quality and accuracy of creating an indoor 3D model are influenced by the quality of data collected from the real world. Point cloud data, a way to describe the 3D indoor environment, are widely exploited in building indoor 3D models [1], [2], [3], [4], [5]. With the rapid development of indoor mobile mapping systems (IMMSs), many IMMSs have been used to collect indoor point cloud data [6], [7], [8], [9], [10]. Typical IMMSs include wheeled mobile systems, backpacked mobile systems, and hand-held devices, etc. The wheeled mobile system smoothly integrates multi-sensors, including cameras, laser scanners, and inertial measurement units on a mobile platform, e.g., pushcart or robot. In the backpacked mobile system, the user backpacks a multi-sensor integrated system to collect data in motion. In the hand-held system, a data acquiring device, e.g., Kinect, is held by users when acquiring data. This way, it is convenient for these hand-held devices to collect data under certain conditions, especially in areas that are difficult for other IMMSs to access. In our proposed framework, the dataset is collected by a Kinect camera mounted on a mobile robot [7].

Some quality problems, or data degradation, such as missing data, occluded data, sparse data, blurred data, and very dark or very bright data, are inevitable for IMMS point clouds. Causes of data degradation include the characteristics of the sensing device, large rotation angle of the mobile platform in motion, and uneven illumination distribution in an indoor environment. Compared with image degradation, the reasons for the degradation of point clouds differ in different local areas, leading to an uneven distribution of point cloud quality. Moreover, different reasons of degradation lead to a diverse degradation of point clouds. Therefore, the qualities of point clouds have the characteristics of diversity and locality for indoor mobile mapping.

The local quality assessment of point clouds are to handle the data quality assessment by considering the diverse and local degradation of the IMMS point clouds. In general, good indoor point cloud data should not only have complete structure information but also maintain consistency between the appearance and structural information. Poor quality data need to be discriminated because they will not provide effective and sufficient information. However, in our dataset, there is an imbalanced problem, i.e., the amount of low-quality data is much greater than high-quality data. The local quality assessment of point clouds can classify these data into different degradation types and prepare for the further repair of these data based on different strategies. One main challenge for automated quality assessment of IMMS point clouds is the establishment of a training dataset in terms of time and cost. On one hand, it is difficult to classify different quality types in one point cloud by manually labeling because of the diverse and local degradation of point clouds. On the other hand, it is difficult, or impossible, to obtain the ground-truth (or reference data) without degradation, which indicates that our quality assessment problem needs to be considered in the absence of reference situations. Thus, it is essential to know how to use limited labeled data to predict the labels of a large number of unlabeled data. A semi-supervised learning method, i.e., a method requiring only a small amount of labeled training data, provides an efficient way to address this challenge.

In this paper, we propose a new framework to assess the local quality of indoor mobile mapping point clouds. First, we describe the IMMS point cloud degradation by geometric feature descriptors. To effectively analyze the essential components of these geometric feature descriptors, a feature selection method is integrated into the proposed framework to reduce the redundancy of these used features. To avoid the intensive labor costs of manual labels, a semi-supervised method, named Safe Semi-supervised Support Vector Machines (S4VMs) [11], is integrated into our proposed framework to conduct quality assessment tasks by manually labeling a small portion of the training dataset. Additionally, we establish a point cloud dataset (benchmark) with multiple kinds of quality problems to evaluate the proposed framework.

The rest of this paper is organized as follows: first, Section 2 reviews the related work; next, Section 3 details our proposed framework in three parts, including the establishment of a dataset with multiple kinds of quality problems, the feature description of the degraded data, and the local quality assessment of indoor point clouds; then, Section 4 reports the experimental results and presents the comparative experiments; finally, Section 5 concludes the entire paper.

Section snippets

Related works

Most recent works on data quality assessment focused on 2D images [12], [13], [14], [15], [16]. Xue et al. [12] established a codebook to assess the quality of images by computing quality-aware centroids of each patch in the training images. Ref. [13] presented a sparse feature representation method to learn a dictionary on the spatial correlations between training images. Two deep neural network methods in Refs. [15] and [16] were introduced to address non-referenced image quality assessment

Point clouds acquisition

An IMMS integrating a 2D laser scanner and a RGB-D camera is adopted for collecting 3D indoor point clouds in this paper [7]. The 2D laser scanner and RGB-D camera are used to build 2D map and to obtain point clouds, respectively. Moreover, this IMMS can achieve a 2D trajectory of the mobile platform while building the maps. However, the acquired data have some quality problems, such as missing data, occluded data, too sparse data, blurriness, and too much darkness or brightness. To deeply

Experiments and results

The proposed framework was coded with C++, and implemented on a personal computer with a single core 3.2 GHz and a RAM of 16 GB. To assess the data quality of local regions of a point cloud, the local search radius of 3D patch is set to 6 cm. According to [11], the regularization parameters C1, C2, and β, are set to 100, 0.1 and 0.1, respectively. Besides, the sampling size N, cluster number T, and risk parameter λ, are set to 100, 10, 3, respectively. The kernel type of S4VM is KBF.

Conclusion

In this paper, we proposed a semi-supervised learning framework to solve the problem of local quality assessment of IMMS point clouds with limited labeled data and a large amount of unlabeled data. First, we collected IMMS point cloud data by a depth camera mounted on a mobile robot, and created a point cloud dataset with multiple kinds of quality problems. Next, we used feature extraction and feature selection methods to obtain optimal feature sets that are exploited to describe local

Acknowledgments

This work is supported by the National Science Foundation of China (Grant no. 61401382).

Fangfang Huang received the B.Sc degree in computer science from Fuzhou University, China in 2013. She is currently a master student with the Fujian Key Laboratory of Sensing and Computing for Smart City in the School of Information Science and Engineering, Xiamen University, China. Her current research interests include computer vision, machine learning, and mobile LiDAR point clouds data processing.

References (44)

  • R.B. Rusu, N. Blodow, Z.C. Marton, M. Beetz, Close-range scene segmentation and reconstruction of 3D point cloud maps...
  • M. Ye, Y. Zhang, R. Yang, D. Manocha, 3D reconstruction in the presence of glasses by acoustic and stereo fusion, In:...
  • N. Corso et al.

    Indoor localization algorithms for an ambulatory human operated 3D mobile mapping system

    Remote Sens.

    (2013)
  • C. Wen et al.

    Three-dimensional indoor mobile mapping with fusion of two-dimensional laser scanner and RGB-D camera data

    IEEE Geosci. Remote Sens. Lett.

    (2014)
  • M. Bosse et al.

    Zebedee: Design of a spring-mounted 3-d range sensor with application to mobile mapping

    IEEE Trans. Robot.

    (2012)
  • P. Henry et al.

    RGB-D mapping: using Kinect-style depth cameras for dense 3D modeling of indoor environments

    Int. J. Robot. Res.

    (2012)
  • T. Liu M. Carlberg G. Chen J. Chen J. Kua A. Zakhor Indoor localization and visualization using a human-operated...
  • Y.-F. Li et al.

    Towards making unlabeled data never hurt

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2015)
  • W. Xue, L. Zhang, X. Mou, Learning without human scores for blind image quality assessment, In: Proceedings of the 2013...
  • H. Tang, N. Joshi, A. Kapoor, Blind image quality assessment using semi-supervised rectifier networks, In: Proceedings...
  • S.O. Elberink et al.

    Quality analysis on 3D building models reconstructed from airborne laser scanning data

    ISPRS J. Photogramm. Remote Sens.

    (2011)
  • J. Feng, R. Zhong, Y. Yang, W. Zhao, Quality evaluation of spatial point-cloud data collected by vehicle-borne laser...
  • Cited by (15)

    • ABC-PSO for vertical handover in heterogeneous wireless networks

      2017, Neurocomputing
      Citation Excerpt :

      Nowadays, the mobile cloud computing has been advanced with much attention worldwide. Generally, the mobile cloud computing is one of the most rapidly emerging future technologies [1–4]. Although mobile devices has limited capacity to store and execute processes, information properties query process is still considered as one of the main issues for implementing mobile cloud computing environment.

    • Visual Saliency and Quality Evaluation for 3D Point Clouds and Meshes: An Overview

      2022, APSIPA Transactions on Signal and Information Processing
    • Mapping for Indoor Walking Environment from Point Clouds by Using Mobile Mapping Systems

      2021, International Journal of Geospatial and Environmental Research
    View all citing articles on Scopus

    Fangfang Huang received the B.Sc degree in computer science from Fuzhou University, China in 2013. She is currently a master student with the Fujian Key Laboratory of Sensing and Computing for Smart City in the School of Information Science and Engineering, Xiamen University, China. Her current research interests include computer vision, machine learning, and mobile LiDAR point clouds data processing.

    Chenglu Wen received the Ph.D. degree in mechanical engineering from China Agricultural University, Beijing, China in 2009. She is currently an Associate Professor with Fujian Key Laboratory of Sensing and Computing for Smart City, School of Information Science and Engineering, Xiamen University, Xiamen, China. She has coauthored more than 30 research papers published in refereed journals and proceedings. Her current research interests include machine vision, machine learning, and point cloud processing. She is the secretary of ISPRS WG I/3 on Multi-Platform Multi-Sensor System Calibration (2012–2016).

    Huan Luo received the B.Sc degree in software engineering from Nanchang University, China in 2009. He is currently a Ph.D. student with the Fujian Key Laboratory of Sensing and Computing for Smart City in the School of Information Science and Engineering, Xiamen University, China. His current research interests include computer vision, machine learning, and mobile LiDAR point clouds data processing.

    Ming Cheng received the Ph.D. degree in Biomedical Engineering from Tsinghua University, China in 2004. He is currently an Associate Professor with the Fujian Key Laboratory of Sensing and Computing for Smart Cities, School of Information Science and Engineering, Xiamen University, China. His research interests include remote sensing image processing, computer vision, and machine learning.

    Cheng Wang received the Ph.D. degree in information and communication engineering from the National University of Defense Technology, Changsha, China in 2002. He is currently a Professor with and the Associate Dean of the School of Information Science and Technology, Xiamen University, Xiamen, China. He has authored more than 80 publications. His research interests include remote sensing image processing, mobile LiDAR data analysis, and multisensor fusion. He is a council member of China Society of Image and Graphics, IEEE Senior Member, and Co-Chair of ISPRS Working Group I/3 on Multi-Platform Multi-Sensor System Calibration (2012-2016).

    Jonathan Li received the Ph.D. degree in geomatics engineering from the University of Cape Town, Cape Town, South Africa. He is currently a Professor with the Fujian Key Laboratory of Sensing and Computing for Smart Cities, School of Information Science and Engineering, Xiamen University, China. He is also Professor and Head of the Mobile Sensing and Geodata Science Lab at the University of Waterloo, Canada. He has co-authored more than 300 publications, over 130 of which were published in refereed journals, including IEEE-TGRS, IEEE-JSTARS, IEEE-TITS, IEEE-GRSL, ISPRS-JPRS, IJRS, PE&RS, and RSE. His current research interests include information extraction from mobile LiDAR point clouds and from Earth observation images. Prof. Li is the Chair of the ISPRS Working Group I/Va on Mobile Scanning and Imaging Systems (2012-2016), Chair of the ICA Commission on Sensor-driven Mapping (2015-2019), and Associate Editor of IEEE-TITS, IEEE-JSARRS and Geomatica.

    View full text