Estimation of Accuracy and Reliability of Terrestrial Laser Scanner in the Detection of Object Shape

The influence of the angle of incidence on the quality of the point cloud has always been a contentious issue in the field of terrestrial laser scanning. The influence is appreciable when determining the shape (2D) and form (3D) of objects. The present study investigates two issues, first, the influence of the angle of incidence on the detection of the correct size of a plane object, and secondly, its influence on the accuracy of laser scanning point clouds. Experiments were carried out using six different modern-day terrestrial laser scanners: Leica C10, Leica P40, Leica HDS6200, Trimble TX5, Faro X330, and Faro S70. The results of this study indicate that a large angle of incidence in scanning, in general, can produce relatively large deviations in a point cloud. However, the maximum deviations are still only on a millimetre scale. The effects of a large angle of incidence on the detection of an object's correct size were found to be relatively marginal.


Introduction
Since terrestrial lasers scanners (TLS) were first introduced in the field of surveying, the empirical testing of the accuracy of instruments has always been a subject of interest. This testing involves a comparison of empirically obtained results with those provided by the manufacturer (e.g., Calders et al., 2017). The lack of standard testing procedures for TLS instruments made such testing the ideal breeding ground for different studies. The first standard for testing the accuracy of TLS systems was developed only quite recently. The standard was published in December 2018 as ISO 17123-9. One of the first significant comprehensive studies on TLS accuracy is that by Boehler and Marbs (2003), who outlined specific steps for testing TLS metrological capabilities. Other studies, like those by Schulz and Ingensand (2004), Reshetyuk (2006Reshetyuk ( , 2010, Kersten et al. (2009) and Zhengchun et al. (2016), have all made significant contributions to research on the accuracy behavior of TLS systems.
As in the case of other geodetic surveying results (e.g., total station survey), laser scanning results are also influenced by random and systematic errors. Studies, for example, by Holst and Kuhlmann (2014) and Medić et al. (2019), have investigated the influence of these errors on surveying results. It was concluded that systematic errors produced by TLS instruments are generally due to internal misalignments in the instrument itself. Detailed descriptions of the relation between the working principles of various TLS instruments and error sources are, for example, provided by Reshetyuk (2009).
TLS is a reflectorless measurement technology that uses laser signals to gather 3D information from the surrounding environment. When using lasers, the following factors affect the quality of measurement results: atmospheric attenuation, surface reflectability and roughness, surveying distance, beam divergence angle, and the angle of incidence (Calders et al., 2017;Soudarissanane et al., 2011). The angle of incidence ( Fig. 1), in particular, plays an important role in the determination of the shape (considered as a 2D closed contour) and the form (considered as a 3D figure) of objects (Boehler and Marbs 2003;Nurunnabi et al., 2017;Forsman et al., 2018). The angle of incidence ranges over the interval [0° < α < 90°] Many scholars, e.g., Lichti (2007), Kersten et al. (2009), Soudarissanane et al. (2011), Soudarissanane (2016, Mill and Ellmann (2017), have studied the influence of the angle of incidence on laser scanning accuracy. According to these studies, scanning under a large angle of incidence (beyond the 60° critical limit) will result in an increased level of noise in the point cloud. In surveying projects, only the surveying distance and the angle of incidence can be controlled. Removing or mitigating the influences of the other factors mentioned is in fact generally impossible. Thus, only the metrological capabilities of TLS devices can be relied upon. Most of the earlier studies investigating the influence of the angle of incidence focused on specific types of TLS (either time-offlight or phase shift). Due to differences in TLS working principles, the results of these studies do not apply to the accuracy behaviour of TLS systems in general. As these studies were carried out nearly a decade ago, it is also maintained here that the conclusions reached do not apply to the accuracy of modern-day instruments. The aim of the present study is to investigate the influence of the angle of incidence on the detection of the correct size of a plane object and on the accuracy of laser scanning point clouds.

Data processing methodology
The data in this study consisted of the actual survey data, i.e., the empirical data and the accuracy values for the scanner provided by the manufacturer in the datasheet, i.e., the theoretical data.
The processing of the empirical data involved several steps. Firstly, the raw scanning data was manipulated using the software provided by the manufacturer. This work included the removal of all unnecessary survey points from the point clouds and the export of the data in ASCII format. The data was then processed in the Matworks Matlab environment.
To assess empirical deviations, point clouds for a plate were iteratively fitted to a plane surface (Eq. 1) using the least squares method (Eq. 2 and 3), Where: a, b, and ccomponents of the normal vector (N), x, y, and zcoordinates of a point on the surface. Aobservables, La column vector, eresiduals, 0 2is a posteriori variance factor that is estimated using the least squares residuals ̂, nthe number of measured points, and k is the number of plane parameters (k = 3). Taking the square root of the posteriori variance factor yields the standard deviation for the fitting: The estimated residuals (̂) based on the least squares fitting method describe deviations from the modelled surface. Residuals also indicate the deviations of the point cloud with respect to the plate, and they can also be used to describe the range noise (Maar and Zogg, 2014). As the least squares method uses an iterative method to find the best-fit location of the plane surface in the point cloud, the average of the residuals is close to zero. To find the maximum thickness of the point cloud layer, the sum of maximum ( ) and minimum ( ) residuals is calculated for each plane orientation case (0°, 10°, 20,…, 80°) using: Figure 2 .The triangle formed. The red dashed line indicates the true path of the laser beam, the blue line the erroneous path due to and . The surface S 1 denotes the true surface that is horizontally rotated, for example, by 40°, S 2 denotes the shifted surface due to the influence of , and S 3 denotes the shifted surface due to the combined uncertainty .

Mill
In addition to the empirically based calculations, theoretical calculations of uncertainties were also carried out, where standard deviation serves as one measure of uncertainty. The theoretical values of uncertainties were calculated using a method suggested by Mill and Ellmann (2017). This method, which uses the geometric relations of a triangle (Fig. 2), applies to the accuracy specification in the datasheets provided by the manufacturers of the scannersthe distance accuracy and angular accuracy (see Table 1). Using the Law of Sines, the formula for the combined uncertainty . is derived from the error propagation law: Where: ρthe measured distance from the scanner to the plane surface (20 m), taken as one side of a triangle, τthe angle opposite the side ρ, γthe angle opposite the side , .
combines the associated uncertainties of angle measurement ( − ) and distance measurement ( ). Lastly, the apparent size of the plate is determined by fitting a plane primitive to the point cloud. For this, Cloud Compare (Lague et al., 2013), open source software, is used. The extents of the fitted plane are generated automatically (Fig. 3). the plate is rotated to 60°

Description of the experiment
The experiment was set up under laboratory conditions to ensure homogeneous lighting, temperature, and humidity. The influence of the angle of incidence was studied using a specially designed 80.1 × 59.6 cm fiberboard plate mounted on a tripod using a modified tribrach fitted with a 360° (arc degrees) protractor (Fig. 4). The surface of the fibreboard had a smooth matt white coating, which provided good surface reflectability and isotropic light reflection. With the aid of the protractor, the board was rotated horizontally from 0° to 80° by 10-degree increments. For each TLS test, the zero-angle orientation of the plate was determined using a total station Leica TS12 R1000 (with an angular and reflectorless distance measurement accuracy of 3" and 2 mm + 2 ppm, respectively), which ensured the plate was perpendicular to the line of sight of the scanner. The scanners used were placed at a fixed location 20 meters from the plate. A similar measurement setup is described in Soudarissanane et al., (2011).
Six different terrestrial laser scanners from different vendors were used: Leica C10, Leica P40, Leica HDS6200, Trimble TX5, Faro X330, and Faro S70. These scanners differ in their distance measuring principles, accuracy specifications, and year of production. Table 1 summarizes the specifications provided by the manufacturers for the TLS instruments used.

Results and discussion
The number of measured points from the plate depends on the type of scanner used. The maximum difference is up to ten times. Considering the fact that Leica C10 and P40 are long-range scanners, the differences in the number of survey points on the plate is reasonable when the plates are perpendicular. As the angle of incidence increases, the number of points on the surface decreases (Fig. 5). The most dramatic decrease in measured points occurs in the case of Faro X330, with a decrease of up to 85%. The smallest decrease is shown by Trimble TX5, with a decrease of 78%. The average decrease is 81%. This last figure indicates that regardless of scanner type, the decrease in measured points due to the change of the angle of incidence is roughly the same, with only marginal differences. The reasons for the decrease in measured points were the following: a) as both the scanner and vertical surface rotated, the spacing between points became larger due to the orientation of the plate; b) at larger angle of incidence values the energy of the backscatter laser signal was too low for the detection unit to detect it; thus signals were not being recorded.
The results of the empirical study indicate that deviations undergo a sharp increase after the angle of incidence exceeded approximately 70° (Fig. 6).
In all cases, standard deviation values do not change significantly until the angle of incidence exceeds 60°, the limit reported, e.g., in Soudarissanane et al., (2011). Although changes in standard deviation values are clearly noticeable, the largest standard deviation is ±1.11 mm, which still indicates relatively good accuracy and higher than that reported, e.g., in Soudarissanane et al., (2011). A similar trend (Fig. 6) was reported in previous studies, e.g., Lichti (2007), Kersten et al. (2009), Soudarissanane et al. (2011) and Soudarissanane (2016. The theoretical uncertainties ( . ) show a pattern similar to those of the empirical results (Fig. 7); however, the magnitudes of the uncertainties differ by more than a factor of ten. The reasons for the significant difference between the theoretical and empirical results are arguably as follows: c) the calculation of standard deviation based on Eq. 4 will result in a small value due to a very large number of measurements (n); d) the input values used in theoretical calculations taken from manufacturer specifications are arguably rather conservative. The maximum thicknesses of the point clouds (Eq. 6) obtained indicate that the maximum and minimum values for the deviations (Fig. 8) are significantly larger than the standard deviation at a 99.73% (i.e., three sigma) level of confidence. Such a result indicates the existence of gross errors in the point clouds.

Figure 8. Maximum layer thickness
The scanner using the time-of-flight working principle (Leica C10) provided the best empirical results. This supports the results reported by Kersten et al., (2009), which concluded that the time-of-flight principle is not as sensitive as the phase shift principle to a large angle of incidence. Better results at a large angle of incidence, were, however, expected but not obtained from the newest scanner, the Faro S70 (produced in 2017).
Regarding the influence of the angle of incidence on the magnitude of the detectable surface area of the plate, the results indicated some systematic trends (Fig. 9). Scanners C10, P40 and HDS6200 all produced systematically larger surface areas. The total average deviation from the true surface area, however, was only 4%. The maximum deviation from the true area was up to 11%. Other scanners, S70, TX5 and X330, produced, in general, systematically smaller surface areas. The total average in this case was -2%. The maximum deviation from the true area was up to -9%. In general, the maximum deviations were obtained at an 80° angle of incidence.

Conclusions
In conclusion, the results indicate that despite the dramatic (more than 80%) decrease in the number of measured points on the surface with respect to the orientation of the plate, there was no significant impact on the accuracy of the point clouds. The largest standard deviation obtained was ±1.11 mm. The accuracy of the point cloud was, in fact, more realistically described by the maximum point cloud layer thicknesses obtained. The maximum value for the point cloud layer thickness was as high as 6.99 mm (Leica HDS 6200, = 80°), and the minimum value was as low as 0.29 mm (Faro S70, = 0°). Deviations in the layer thicknesses obtained were also in greater agreement with the theoretically estimated standard deviation than with the empirically obtained deviations.
The effect of a large angle of incidence on the detection of the correct size of an object was found to be relatively marginal. The maximum deviation from the true surface area of the object was up to 11% (Leica HDS 6200, = 80°), and the minimum deviation was as low as -9% (Faro X330, = 80°). The extreme values were generally obtained using point clouds measured at a large angle of incidence. The largest deviations were obtained using older generation PS scanners. This can be explained by the fact that older generation PS scanners are more likely to be influenced by laser signal attenuation and beam divergence angle, as well as the angle of incidence.
It was concluded that modern-day laser scanners provide data with a level of accuracy sufficient for projects demanding high accuracy, even at an angle of incidence exceeding 70° with approximately 80% data loss.