A Comparative Study of Multiple 2D Laser Scanners for Outdoor Measurements ^†

Abstract

This research work examines the performance of several compact and lightweight 2D laser scanners, including the Hokuyo URG-04LX, RP Lidar and Hokuyo UTM-30LX-EW, for their potential use in scanning and mapping applications. A detailed study was conducted to evaluate the performance of each scanner specifically for outdoor mapping operations. Multiple experiments were performed to characterize each scanner through statistical analysis. The paper discusses the results of testing for the drift effect, the impact of direct sun exposure to different materials and the effect of range measurement in outdoor environments on range and measurement accuracy.

1. Introduction

Laser scanners have become a ubiquitous technology in a wide range of fields, including surveying, robotics, autonomous vehicles, construction, agriculture and road-network mapping. In particular, indoor and outdoor surveying tasks have become the most popular to construct detailed models of structures and their surroundings. Researchers have presented the revolution of mapping system technologies for land, sea and aerial applications in which laser scanners plays a vital role in mapping systems [1]. The information collected during inspections and modeling of buildings under construction or fully furnished is extremely valuable for a variety of applications [2].

There are different types of laser scanners on the market that have varying capabilities, such as single- and multi-layer scanning, and it is important to choose the right one for a specific application. The two portable laser scanners hand-held ZEB-REVO (GeoSLAM, Alexandia, VA, USA) and the Leica Pegasus backpack (Leica Geosystems AG, Heerbrugg, Switzerland) were compared in both indoor and outdoor environments to verify the measuring capability for different environments and applications [3]. Some of the most commonly used 2D laser scanners are Hokuyo (Hokuyo Automatic Co. Ltd., Tomakomai, Japan), SICK (SICK, Minneapolis, MN, USA) and RP Lidar (SLAMTEC, Shanghai, China).

OEMs often provide basic information about a device’s type and accuracy, but do not typically provide detailed information about how factors such as light, materials or colors on the measurement surface may affect the device or the extent to which the device may drift over time or produce accurate measurements at different distances. These factors can significantly impact how the device is used and operated in different environments. As designers often need to use sensors in a variety of environments, it is important to consider these characteristics in order to account for potential inaccuracies in measurement.

2. Related Work

Many researchers have conducted performance evaluations and characterizations of various 2D laser scanners for indoor and outdoor applications. These studies have examined different technical parameters that can impact the performance and highlight the limitation of these scanners. Prior knowledge of the outcome of various scanners enables the designer to select and develop its mapping system according to the specific conditions and scene recognition.

A team of scientists carried out an in-depth evaluation of the Hokuyo URG-04LX scanner (Hokuyo Automatic Co. Ltd., Tomakomai, Japan). They examined the scanning quality by altering the characteristics of the targeted surfaces such as texture and color and different lighting conditions on various surfaces [4]. In this paper, the SICK LMS 511 laser scanner (SICK, Minneapolis, MN, USA) was thoroughly analyzed to determine its metrological capabilities along with the effects of direct sunlight on the sensor’s performance in outdoor environments [5]. The authors evaluated the performance of the 2D Sick LMS-200 scanner (SICK, Minneapolis, MN, USA) in comparison to the Hokuyo URG-04LX scanner for complex shapes at various surfaces [6].

The researchers characterized the canopy of a paired plantation system in tomato crop greenhouses using a 2D LiDAR scanner [7]. Similarly, researchers performed UTM-30LX (Hokuyo Automatic Co. Ltd., Tomakomai, Japan) indoor characterization for its use in the detection of obstacles with overhead power lines [8]. The same research group further extended the research to outdoor environmental conditions by considering the variation of lighting, magnetic fields for high-power line disturbances [9]. A group of researchers have performed an evaluation of UTM-30LX with multi-layer IBEO LD-ML scanner (Ibeo Automotive Systems GmbH, Hamburg, Germany) for outdoor environmental conditions [10].

3. Characteristics of Multiple 2D Laser Scanners

The Hokuyo UTM-30LX provides a 270-degree scanning range with a maximum measurement distance of 60 m [11]. The Hokuyo URG-04LX offers a low-cost and low-range compact 2D laser scanner that can scan up to 4 m with 240-degree rotation [12]. The RP Lidar can scan a full 360-degree rotation and can detect objects up to 6 m away [13]. Figure 1 shows the pictorial model of 2D laser scanners.

4. Test Methodology

First, a mounting platform to hold all scanners has been developed. This platform consists of multiple parts for assembling the scanners on a single rigid platform. A wooden base enclosure is chosen for the foundation of the test mounting platform. An acrylic hollow cubicle structure is developed to place each scanner at each floor to test and evaluate the same environment. This hollow cubicle box is mounted on a base wooden enclosure with the help of screws. The three scanners are mounted on three floors of cubicle structure in the orientation of URG-04LX in the bottom, RP Lidar in middle and UTM-30LX on the top surface. All scanners are powered up with the USB port and data is also transmitted with the same port. The hardware setup of the test platform is shown in Figure 2.

A laptop with an ROS (Robot Operating System)-based program has been connected to each scanner through a USB interface. ROS is an open-source software package used for interfacing various hardware for creating complex and customized models for research and development of real-world robotic applications [14]. After the data have been recorded on the laptop, it is then post-processed by using Matlab (MathWorks, Natick, MA, USA) for calculation and comparison of statistical values for mean, standard deviation and absolute error of multiple target objects. Figure 3 shows the block diagram and workflow for both online and offline procedures and a 2D layout of this experimental test setup.

5. Performance Evaluation of Laser Scanners in Outdoor Environment

To evaluate the performance of the distinct characteristics of multiple laser scanners, several tests and experiments in an outdoor environment were carried out. The open hall with no roof was selected to perform measurements on multiple objects during daylight. The objects that were typically available in an open hall were concrete walls, floor and glass doors. Moreover, to perform surface reflection and the effects of sunlight, wooden table was also placed according to sensor’s maximum detection capabilities. The sensor test platform was placed at a defined distance from these objects to conduct multiple tests. The laser sensors were tested for drift effect, the influence of target distance and the dependency of range measurement in day and sunlight conditions. The data for the tests were recorded at specified time intervals according to the test’s requirement and sensors’ evaluation.

Mean value ($\overline{x}$) and standard deviation (σ) were computed for each measurement followed by a relative error, which represents the deviation from the nominal values. The observed range mean value ${\overline{x}}_m$ for the specific object distance can be determined using Equation (1).

$${\overline{x}}_m=\frac{1}{N}{\sum }_{i=1}^N{x}_i$$

(1)

Similarly, the sample for standard deviation ${\sigma }_m$ at mth time interval is computed by Equation (2).

$${\sigma }_m=\sqrt{\frac{1}{N}{\sum }_{i=1}^N(x_i-{\overline{x}}_m{)}^2}$$

(2)

The absolute error $e_m$ is calculated by $e_m=x_T-{\overline{x}}_m$ where $x_T$ is the true value. Similarly, the percent relative error ${\eta }_m$ is determined by ${\eta }_m=\frac{e_m}{x_T}\times 100$.

5.1. Drift Effect

Laser scanners can produce inaccurate range values for stationary objects due to internal warming caused by the embedded circuit and motor confined in a small, packed area. This is generally known as the drift phenomenon, and it is an important aspect to recognize it effects on measurement with respect to time. The test was conducted for half an hour to analyze this effect. The results for each laser scanner are shown in Figure 4. The range plots $x_i$ of measured scans for each scanner have been expressed in blue color and the sampled mean values ${\overline{x}}_m$ in green color in the same figure. It is clearly observed that after almost 10 min, the values are stabilized and repeatable.

It can be depicted that variations were higher in RP Lidar as compared with URG-04LX and UTM-30LX.

Table 1. Results of drift effects at 3 m distance.

Scanner	Mean Value ${\overline{\mathit{x}}}_{\mathit{M}}$ (mm)	Standard Deviation ${\overline{\mathit{\sigma }}}_{\mathit{M}}$ (mm)	Relative Error ${\overline{\mathit{\eta }}}_{\mathit{M}}$ (%)
URG-04LX	2985	15.4	0.5
RP Lidar	3023	22.7	0.76
UTM-30LX	3004	9.8	0.13

shows the summarized test results of mean, standard deviation and relative error.

5.2. Exposure to Direct Sun Light

The quality of scanning for multiple objects may be affected by the outdoor environment during sunlight. To study this effect, an experiment was conducted for sunlight and non-sunlight conditions at 3 m distance. The test was performed twice at 9:00 AM and 12:00 PM on the same day for recording the effects of sunlight and non-sunlight conditions. The results were analyzed to observe how the range measurements changed depending on the lighting conditions of the objects being exposed to the sun. Figure 5 shows the graph of the average variations of three scanners during no and direct exposure to sunlight. URG-04LX and UTM-30LX have shown less dependency on direct exposure to sunlight due to better build quality and reliability and RP Lidar shows significant variations.

5.3. Multiple Range Test

The ability of a scanner to accurately measure the distance of objects in its environment is crucial for its use in mapping applications. To study this, a wooden table was located at 3 m distance from the scanner platform. The experiment was carried out for 30 min, the range measurements were analyzed for the last 5 min and the results are shown in Figure 6. The graph shows the mean range values and standard deviations of each scanner. Both measurements show variations at the beginning, but after 3–4 min of operation, they become stable with minimal deviation as per OEM specification. UTM-30LX has again proven for better immunity to these conditions.

6. Conclusions

In summary, a study was conducted to evaluate the performance of three compact and cost-effective 2D laser scanners, URG-04LX, RP Lidar and UTM-30LX, for use in mapping applications for outdoor environments. The results showed that all scanners performed well within their specified limits, with some minor exceptions. The Hokuyo UTM-30LX was found to be the best performer in terms of range stability, standard deviations, and error values. It also had a higher angular resolution which resulted in more points being returned for scanned objects. However, its higher cost could be a downside. The Hokuyo URG-04LX was found to be a good choice for low-range operations and provided stable and nearly accurate results. The RP Lidar was the least accurate of the three scanners, but its low cost could make it suitable for economic system development. Overall, this study gives valuable information for those who are planning to use these laser scanners in outdoor mapping and scanning applications.