AUTOMATIC ROAD STRUCTURE DETECTION AND VECTORIZATION USING MLS POINT CLOUDS

: Accurate three-dimensional road structures and models are of great significance to intelligent transportation applications, such as vehicle navigation, inventory evaluation, construction quality control, self-driving vehicles and so on. This paper proposes an efficient and robust method to automatically extract structured road curbs from mobile laser scanning (MLS) data. The proposed method mainly consists of three steps: efficient supervoxel generation, road curbs detection and driving free space estimation. First, supervoxels are generated by assigning ground points with similar geometrical characteristics into the same group. Second, supervoxels with higher local projection density and height difference are identified and clustered as initial road curbs, which are continuous vertical curb facets. The continuous facades consisting of lots of scanned points on the road shoulder can be modeled as multi-dimensional boundary models depending on the requirements of the application, such as vector lines with or without height, micro-facades, etc. Finally, driving free space is obtained due to the road limits can be defined by road boundary in most scenarios. The proposed method is tested on two complex datasets acquired by an Alpha3D mobile laser scanning system from the urban area of Shanghai, China. Experimental results show that the road boundaries and driving free space can be accurately and efficiently extracted, which also demonstrates the superiority of the proposed method.


INTRODUCTION
Roads are the main components of the transportation infrastructure, and residents' daily travel basically depends on it.To build a more efficient and convenient urban transportation and environment, national finances are heavily invested in road construction and safety inspection every year (Antonio Martí n-Jimé nez et al.).Intelligent transportation system emerges at the strong demand for a better transportation system, aiming to make safer, more coordinated and smarter use of transport networks.With the growing demand of fine road structure models in intelligent transportation system, researchers have proposed kinds of road extraction algorithms from various surveyed data, including satellite and aerial imagery, aerial laser scanning (ALS), panoramic imagery, mobile laser scanning (MLS) data and so on (Elberink and Vosselman, 2009;Fang et al., 2015;Soilá n et al., 2018;Wang et al., 2016).However, accurate road boundary extraction from satellite, aerial imagery, and ALS data is quite challenging due to the limited resolution and missing data caused by the occlusion of tall buildings and trees.Although panoramic imagery is a low-cost data source to detect road structures and build road models, the accuracy of the detected road boundaries is very much dependent on the quality of disparity images.For instance, the presence of shadows on the road affects the performance of the road boundary extraction algorithms.The result may not meet the accuracy demand of several applications, such as HD (High-Definition) Map.By contrast, MLS system is not sensitive to the shadows on the road surface and can acquire three-dimensional point clouds quickly and accurately by recording the geometry and intensity information along the road, which is considered as the costeffective alternative for road inspection.

RELATED WORKS
The methods for road extraction using mobile laser scanning data can be summarized into two categories: projection-based methods and point-based methods.The projection-based methods first generate the raster images by projecting points on the reference plane, in most instances, which is a plane.Then road boundaries are extracted by applying image processing algorithms like RANSAC and Hough transform based on intensity and height attributes.Although the projection-based methods are computationally efficient compared to the intensive characteristic calculation point by point, the precision is decreased during the rasterizing process (Kumar et al., 2013).To overcome accuracy loss in projectionbased methods, the methodology of point-based methods using local three-dimensional road features and patterns are applied to extract road boundaries.For instance, some point-based methods detect road surfaces emphatically for considering road curbs as the split facets among separate roads.Zai et al. (2018) proposed a method for extracting road boundary including supervoxel generation and graph-cut smoothing.Supervoxels near the road are generated by selecting smooth points as seeds and assigning points with several attributes into facets centered on these seeds.Adjacent co-planar facets are merged as road surfaces, the boundaries of which are extracted using the alphashape algorithm.Graph-cuts algorithm and curve fitting are applied finally to make the extracted boundary continuous and smooth (Zai et al., 2018).This algorithm performs well on Road boundaries in straight shape parallel with the vehicle trajectory, but cannot do well with the situations where boundary and trajectory are not really parallel.Qiu et al. (2016) extracted planes using RANSAC from coarse to fine, and road boundaries are considered as edges of the fined-tuned planes.Road width and slope continuity of adjacent points are employed to refine extracted edge points (Qiu et al., 2016).The width and roughness of roads vary a lot in real-world situations, therefore using this methodology to extract boundary automatically is a tough task.While others employ local patterns, such as local tiny curb planes, height difference, and intensity, to detect road boundaries.Sheng et al. (2016) extracted the candidate curb points based on the energy function and refined them using least-cost path model.(Miraliakbari et al., 2015;Sheng et al., 2016;Zai et al., 2018).
In summary, the challenge of structured road boundary extraction in urban scenarios are as follows.First, straight, curved and intersection shaped roads come along with various boundary morphology.Second, point cloud gathered by MLS has the uneven, unorganized, incomplete and massive nature.Above all, this paper aims to provide a novel road boundaries extraction method using mobile laser scanning data that the integrity, accuracy and computational efficiency are taken into account.This goal is achieved by using efficient supervoxel generation method without time-consuming normal estimation point by point.Then super voxels are classified into two categories, including candidate vertical facades and horizontal surfaces.Since the detected vertical facades are isolated microfacades, Euclidean distance clustering is employed to get the continuous boundary entities.Furthermore, driving free space is estimated combining the boundary limits and road surface geometrical consistency.

METHODOLOGY
In summary, a new three-dimensional road boundary extraction and driving free space estimation method using mobile laser scanning data is proposed in this paper, which makes full use of the MLS data and trajectory gathered from Global Navigation Satellite System (GNSS).Our method mainly consists of three stages, including supervoxel generation, road boundary extraction and driving free space estimation.The workflow is shown in Figure 1.
Data preprocessing, isolating the on-ground and off-ground points, exists before the boundary extraction.In the ground filtering stage, point clouds are partitioned into several blocks along the trajectory.For each block, off-ground points are filtered by removing the points whose height difference compared to the local lowest point is larger than a given threshold (Hernandez and Marcotegui, 2013).This threshold should preserve the curb points but filter the off-ground points as much as possible.Figure 3(a) shows the source point clouds, and Figure 3(b) shows the ground data after ground filtering.In Figure 3(b) ground is in black, and off-ground points below the trajectory are in gray.

Supervoxel Generation
Two-dimensional regular grids are generated as initial voxels uniformly distributed on the horizon, as shown in Figure 2( ( where , and are the point in the voxel, the number of points in the voxel and projection area, respectively.is the mean points distance of ground points, and is the points number in the neighborhood of the radius .

Boundary Extraction
Supervoxels with height difference larger than a threshold are clustered as the initial road boundaries, as shown in equation ( 8). , the minimum height difference threshold, is used to select the super voxels belonging to the boundary facades.Since the foot of building walls, light-poles, and trees are preserved on the ground after ground filtering, the local height distribution histogram combining on-ground and off-ground data is used to describe the clustered boundaries (Yang et al., 2013).The foot of the pole-like object is short while most scanned points on the wall are assigned to the off-ground.Based on the two above mentioned attributes, the foot of walls and pole-like objects in initial clustered boundaries are rejected as false positive curbs.
(8) where and are the height of the point whose height is the maximum in the super voxel and the height of the point whose height is the minimum in the same super voxel respectively, and is the minimum height difference threshold.

Driving Free Space Estimation
The continuous facades consisting of lots of scanned points on the road shoulder can be modeled as multi-dimension boundary models depending on the requirements of the applications as vector lines with or without height, micro-facades, etc.The boundary vectorization follows close to the boundary extraction.For instance, as for large-scale urban topographic maps, threedimensional vector lines with height are used to represent the road boundary.
As for driving free space, it is limited by the left and right road boundaries most scenarios (Fernandez et al., 2014).Once road boundaries are detected, driving free space can be obtained using region growing algorithm with the boundary limitation.Supervoxels never selected as part of the initial boundary are defined as left horizontal distributed ones.Those horizontally distributed voxels where sampled trajectory points are projected in are selected as the seeds of driving free space.Region growing is applied for merging those eight-connected left supervoxels around the seeds when they meet the growing criteria that height difference between the seed and the connected one is less than a given threshold.The threshold is empirically set as 3cm, due to the road surface is flat in the urban area.

Dataset
The feasibility of the proposed algorithm is validated through two datasets by an Alpha3D mobile laser scanning system from the urban area of Shanghai, China, which includes a long-range and precise RIEGL VUX-1 laser scanner, an inertial measurement unit (IMU), a global navigation satellite system (GNSS), and a panoramic camera.The scanner records the ranges from the object and the scanner.The GNSS locates the position of the system and records the timestamps in data collection.Two datasets covering two complex urban areas are Xinchang residential district and Zhangjiang industrial district in Shanghai, China respectively.The Xinchang dataset, as shown in Figure 6(a), is a residential area, including about 500 million points (20.59GB), and the length of the road is about 10.9 km.The Zhangjiang dataset is the industrial area, as shown in Figure 6(c), including about 650 million points(21.74GB),and the length of the road is about 13.04 km.Besides, both urban datasets include grass stripes, fences and a large number of vehicles, which brings a huge challenge to the algorithm.

Boundary Extraction Results
The values of the parameters used in the two datasets are listed in Table 1. is the threshold used to select candidate voxels.To extract tiny height step curbs, this value is given as 3cm.

Parameters
Description Values A distance threshold to determine if the voxel containing boundary.The weight of the height difference 0.25 Table 1.Parameters setting for the experiment The completeness, correctness, computational efficiency and lightweight storage also prove the flexibility and practicability of the algorithm.In Figure 3 and 4, where original data, ground points, and extracted road boundary are showed respectively.In Figure 3

CONCLUSION
An algorithm of automatic structured road curbs extraction and driving free space is proposed in this paper.After super-voxel generation and classification, Euclidean distance clustering is applied to obtain the continuous road boundary entity rather than discrete points.Therefore, although the method is valid for the dataset with even and dense points, it is not the usual case.A more advanced method taking uneven density and sparse data into account will be developed as part of our work.In practice, points density is affected by many factors, including laser scanner quality, range from object to the scanner, driving speed, etc.On the other hand, there are many missing boundaries because of the occlusion of the obstacles on road, like vehicles and pedestrian.A more advanced method with the capacity of completing the missing boundary automatically will be analyzed in our future work.Road horizontal markers, such as road marking, will be extracted.Furthermore, the lightedweighted descriptions of road makers should be developed for storage and further applications.After obtaining road boundaries and markings, road parameters containing lane numbers, road width, road slope, and curvatures will be estimated for assisting road management and safety evaluation in the future.Transactions on Intelligent Transportation Systems PP, 1-12.doi.org/ 10.1109/TITS.2017.2701403.

Figure 1 .
Figure 1.The workflow of the proposed method.
a), and the grids with the height difference, , larger than a threshold are labeled as candidate boundary voxels , as shown in equation (1).Points in candidate voxels are reallocated to generate super-voxels based on the local density, height, and Euclidean distance, as shown in equation (2).Points in the candidate boundary voxel stay in place or are reassigned to one of the eight connected voxels based on the maximum character similarity, and that is minimum feature space difference calculated in equation (2).As for the height of the points on the profile of boundary is not a fixed value, between the point in the candidate voxel and the located voxel is calculated by equation (3).The location of the voxel is represented by the geometrical center of all points in the voxel in equation (5).Projection density of the voxel is calculated in equation (6).Projection density of the point is calculated in equation (7).The result of the points reassignment is shown in Figure2(b) and (c).These supervoxels where points are reassigned are located nearby the road shoulder.The morphology is no longer regular in horizon after reassigning the points for better geometrical consistency and border preservation in a supervoxel.Other regular voxels remain unchanged meanwhile.boundary voxel, the height difference of the voxel and height difference threshold used to select candidate boundary voxels, respectively.distance, projection density distance and height distance from the point in the candidate boundary voxel to its eight connected voxels around it or the located voxel respectively; , and are the weight of , and respectively.
Super voxel generation: (a) regular grid cells; (b) irregular super voxels in top view; (c) irregular super voxels in side view.
(d), the blue points are extracted road boundary points and the red line is the simplified representation.Besides, extracted road boundary and estimated driving free space are superimposed on the original point clouds, as shown in Figure 5(b) and (d).In Figure 6, two big urban datasets, including Xinchang residential district (a) and Zhangjiang industrial district (c) in Shanghai, China, are tested.The results of the boundary extraction are shown in Figure (b) and (d) respectively.