LidarCSNet: A Deep Convolutional Compressive Sensing Reconstruction Framework for 3D Airborne Lidar Point Cloud

https://doi.org/10.1016/j.isprsjprs.2021.08.019Get rights and content

Abstract

Lidar scanning is a widely used surveying and mapping technique ranging across remote-sensing applications involving topological, and topographical information. Typically, lidar point clouds, unlike images, lack inherent consistent structure and store redundant information thus requiring huge processing time. The Compressive Sensing (CS) framework leverages this property to generate sparse representations and accurately reconstructs the signals from very few linear, non-adaptive measurements. The reconstruction is based on valid assumptions on the following parameters- (1) sampling function governed by sampling ratio for generating samples, and (2) measurement function for sparsely representing the data in a low-dimensional subspace. In our work, we address the following motivating scientific questions- Is it possible to reconstruct dense point cloud data from a few sparse measurements? And, what could be the optimal limit for CS sampling ratio with respect to overall classification metrics? Our work proposes a novel Convolutional Neural Network based deep Compressive Sensing Network (named LidarCSNet) for generating sparse representations using publicly available 3D lidar point clouds of the Philippines. We have performed extensive evaluations for analysing the reconstruction for different sampling ratios {4%, 10%, 25%, 50% and 75%} and we observed that our proposed LidarCSNet reconstructed the 3D lidar point cloud with a maximum PSNR of 54.47 dB for a sampling ratio of 75%. We investigate the efficacy of our novel LidarCSNet framework with 3D airborne lidar point clouds for two domains - forests and urban environment on the basis of Peak Signal to Noise Ratio, Haussdorf distance, Pearson Correlation Coefficient and Kolmogorov-Smirnov Test Statistic as evaluation metrics for 3D reconstruction. The results relevant to forests such as Canopy Height Model and 2D vertical profile are compared with the ground truth to investigate the robustness of the LidarCSNet framework. In the urban environment, we extend our work to propose two novel 3D lidar point cloud classification frameworks, LidarNet and LidarNet++, achieving maximum classification accuracy of 90.6% as compared to other prominent lidar classification frameworks. The improved classification accuracy is attributed to ensemble-based learning on the proposed novel 3D feature stack and justifies the robustness of using our proposed LidarCSNet for near-perfect reconstruction followed by classification. We document our classification results for the original dataset along with the point clouds reconstructed by using LidarCSNet for five different measurement ratios - based on overall accuracy and mean Intersection over Union as evaluation metrics for 3D classification. It is envisaged that our proposed deep network based convolutional sparse coding approach for rapid lidar point cloud processing finds huge potential across vast applications, either as a plug-and-play (reconstruction) framework or as an end-to-end (reconstruction followed by classification) system for scalability.

Section snippets

Introduction and related work

Lidar (Light Detection and Ranging) is an optical remote sensing data acquisition technique that produces 3D data with highly accurate x,y,z measurements. The x , y, and z measurements are derived from the Time of Travel (ToT) after applying mathematical transformations. Lidar scanner works similarly to the principle of Electronic Distance Measuring Instrument (EDMI). In EDMI, a laser pulse is fired from the transmitter of the transceiver and the reflected energy of the pulse is captured by the

Methods and materials

We describe dataset details and acquired methodology for our proposed LidarCSNet, LidarNet, and LidarNet++ in the following sections.

Experiments and results

We present comprehensive results obtained for extensive experiments based on the two case studies for forest and urban environment in the coming sections.

Conclusion and future directions

Lidar point cloud data has been widely used in research as well as industrial applications. Geospatial applications involving analysis of forest dynamics, urban environment mapping and surveying uses lidar point cloud data. In our work, we are proposing a deep Convolutional Neural Network based Compressive Sensing framework, named LidarCSNet, to generate sparse approximation of the lidar point cloud data for selected {4%, 10%, 25%, 50% and 75%} CS measurement ratios. We perform multiple

CRediT authorship contribution statement

Rajat C. Shinde: Conceptualization, Methodology, Software, Validation, Formal analysis, Investigation, Resources, Data curation, Writing - original draft, Visualization. Surya S. Durbha: Conceptualization, Methodology, Resources, Data curation, Writing - review & editing, Supervision, Project administration. Abhishek V. Potnis: Methodology, Resources.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgements

The authors express their gratitude towards the Training Center for Applied Geodesy and Photogrammetry (UP TCAGP) and the PHIL-Lidar Program of Philippines for publishing the open lidar data. The authors also express their gratitude to the Google Cloud Team for providing the Google Cloud Platform with GPU enabled computing facility for implementing the architectures under the Google Cloud Platform Research Credits Program.

Funding

This research is partially funded under the Prime Minister’s Research Fellowship issued by the Ministry of Education, Government of India. The high-performance computation for lidar data processing is performed using the Google Cloud Platform Credits received under the Google Cloud Platform Research Credits Program.

References (77)

  • Antonarakis, A.S., Richards, K.S., Brasington, J., 2008. Object-based land cover classification using airborne LiDAR....
  • Bater, C.W., Wulder, M.A., Coops, N.C., Nelson, R.F., Hilker, T., Næsset, E., 2011. Stability of sample-based...
  • Bhangale, U., Durbha, S.S., King, R.L., Younan, N.H., Vatsavai, R., 2017. High performance GPU computing based...
  • Buades, A., Coll, B., Morel, J.M., 2005. A review of image denoising algorithms, with a new one. Multiscale Model....
  • Candès, E.J., Romberg, J., Tao, T., 2006. Robust uncertainty principles: Exact signal reconstruction from highly...
  • Candès, E.J., Wakin, M.B., Boyd, S.P., 2008. Enhancing Sparsity by Reweighted. J. Fourier Anal....
  • Q. Chen et al.

    Isolating individual trees in a savanna woodland using small footprint lidar data

    Photogramm. Eng. Remote Sensing

    (2006)
  • X. Chen et al.

    A fast reconstruction method of the dense point-cloud model for cultural heritage artifacts based on compressed sensing and sparse auto-encoder

    Opt. Quantum Electron.

    (2019)
  • Chen, Y., Huang, T.Z., He, W., Yokoya, N., Zhao, X. Le, 2020. Hyperspectral Image Compressive Sensing Reconstruction...
  • Daubechies, I., Defrise, M., De Mol, C., 2004. An iterative thresholding algorithm for linear inverse problems with a...
  • Donoho, D.L., 2006. Compressed sensing. IEEE Trans. Inf. Theory....
  • Duncanson, L., Neuenschwander, A., Hancock, S., Thomas, N., Fatoyinbo, T., Simard, M., Silva, C.A., Armston, J.,...
  • Durbha, S. S., King, R. L., Younan, N. H., 2007. Support vector machines regression for retrieval of leaf area index...
  • Fowler, J.E., Mun, S., Tramel, E.W., 2010. Block-based compressed sensing of images and video. Found. Trends Signal...
  • P. Gamba et al.

    Digital surface models and building extraction: a comparison of IFSAR and LIDAR data

    IEEE Trans. Geosci. Remote Sens. doi

    (2000)
  • Goodwin, N.R., Coops, N.C., Culvenor, D.S., 2006. Assessment of forest structure with airborne LiDAR and the effects of...
  • L. Graham

    The LAS 1.4 Specification

    Photogramm. Eng. Remote Sensing.

    (2012)
  • Gregor, K., LeCun, Y., 2010. Learning fast approximations of sparse coding, in: ICML 2010 - Proceedings, 27th...
  • H. Guan et al.

    Deep learning-based tree classification using mobile LiDAR data

    Remote Sens. Lett.

    (2015)
  • Y. Guo et al.

    Deep learning for 3D point clouds: A survey

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2019)
  • Holmgren, J., Persson, Å., 2004. Identifying species of individual trees using airborne laser scanner. Remote Sens....
  • J. Hyyppä et al.

    A segmentation-based method to retrieve stem volume estimates from 3-D tree height models produced by laser scanners

    IEEE Trans. Geosci. Remote Sens. doi

    (2001)
  • M. Iliadis et al.

    Deep fully-connected networks for video compressive sensing

    Digit. Signal Process. A Rev. J.

    (2018)
  • Jiang, M., Wu, Y., Zhao, T., Zhao, Z., Lu, C., 2018. PointSIFT: A SIFT-like Network Module for 3D point cloud semantic...
  • Jin, K.H., McCann, M.T., Froustey, E., Unser, M., 2017. Deep Convolutional Neural Network for Inverse Problems in...
  • K. Klasing et al.

    Comparison of surface normal estimation methods for range sensing applications

    IEEE International Conference on Robotics and Automation

    (2009)
  • K. Kulkarni et al.

    ReconNet: Non-iterative reconstruction of images from compressively sensed measurements

  • G. Kutyniok

    Theory and applications of compressed sensing

    GAMM Mitteilungen

    (2013)
  • Cited by (15)

    • Ultrasmall fully-convolution GVA-net for point cloud processing[Formula presented]

      2023, Applied Soft Computing
      Citation Excerpt :

      As the availability and accuracy of 3D scanners, LiDAR tools, and photogrammetry software s, the use of point clouds is becoming a dominant approach in a variety of industrial branches. Deformation detection in civil engineering [1], 3D roof reconstruction [2], forest dynamic observations [3], or disaster damage detection [4] are just cases in point. Among tasks related to point clouds, we may differentiate two groups: semantic tasks and auxiliary tasks.

    • Fore-Net: Efficient inlier estimation network for large-scale indoor scenario

      2022, ISPRS Journal of Photogrammetry and Remote Sensing
      Citation Excerpt :

      With the increasing maturity of three-dimensional (3D) scanning technology, 3D sensors such as Lidar or depth cameras have become the primary option for sensing the environment. Point cloud registration, which is the process of finding rigid transformation between different scans of a target scene, has become a critical problem in 3D reconstruction (Shinde et al., 2021; Jung et al., 2014), autonomous driving (Li and Ibanez-Guzman, 2020; Li et al., 2021), robot positioning (Röwekämper et al., 2012; Lee et al., 2019), and many other applications. However, when used in the real-world environment, especially in cluttered indoor spaces, point clouds scanned by sensors are often corrupted by noise because of sensor limitations, occlusions, or artifacts.

    View all citing articles on Scopus
    View full text