DEVELOPMENT OF A MULTI-SENSOR SYSTEM FOR ROAD CONDITION MAPPING

Abstract. We present a concept for a vehicle based road condition mapping system using infrared spectrometers, high resolution RGB cameras and a laser scanner. Infrared spectrometry is employed to monitor the deterioration of the surface material and pavement condition, in particular by aging. High resolution RGB imaging enables automatic asphalt crack detection and provides base images for spectrometry spots. Laser scanning aims at the detection of geometrical road irregularities and pavement failures such as potholes and ruts. These three major recordings contribute to the analysis of the pavements condition. All mapping sensors are synchronised with a navigation sensor to collect geo-referenced data. The concept of road condition mapping relies on a separate analysis of the different sensor data which are related to road sections. Processing results like the percentage of the road section area related to cracks, pot holes, ruts etc. are merged to achieve an assessment for the road section. The processes for assessing deterioration from the spectrometer data, the detection of ruts from the laser data and cracks from the images are discussed in detail and outlined with some experiments.


INTRODUCTION
Precise information about the pavement condition is a key issue for the overall management of transportation infrastructure. The evaluation of the pavement quality is crucial for the maintenance management of the roads. In our contribution, we consider a vehicle-supported non-contact sensor combination including infrared spectrometers, high resolution RGB cameras and a laser scanner. Infrared spectrometry is employed to monitor the deterioration of the surface material and pavement condition, in particular by aging. High resolution RGB imaging enables automatic asphalt crack detection and provides base images for spectrometry spots. Vehicle-based mobile laser scanning aims at the detection of geometrical road irregularities and pavement failures such as potholes, ruts etc. These three major recordings contribute to the analysis of the pavements condition.

SENSOR SPECIFICATION AND INTEGRATION
At the University of Applied Sciences in Stuttgart, we have started to equip vehicles with sensors for mobile data collection some years ago. The overall goal of this development is to support research and teaching with a highly configurable system. One of these configurations is visualized in Error! Reference source not found..  Figure 2 shows the laser scanner, RGB camera and the spectrometer head. Aim and purpose of this sensor setup is to collect suitable data for investigating the condition of road pavement. All recordings are to be geo-referenced and synchronized using the GPS/INS position and orientation system. The synchronization is based on a TTL signal sent from the sensors and received by the GPS/INS system. All three sensors are connected to the Applanix POS LV 420 and are triggered depending on settings. For the laser scanner these are the measurement rate in points per second or the line scan speed specified by the number of scan rotations per second. Images or spectrometer data are recorded by selecting a suitable time interval. In addition, there is a possibility to trigger the RGB imaging based on the desired path increment. Thereby recording superfluous images is avoided, which otherwise happens when the vehicle is standing still e.g. due to traffic jam.

DATA ACQUISITION
Depending on the speed of the vehicle (typically between 20 to 30 km/h on small streets and alleys) the footprint of the spectrometer spot on the ground has a length of 16 to 24 cm and a width of 5 cm. The Polytec PPS 2221 captures 256 pixels with a resolution of 7.9 nm in the spectral range of 1253 to 2259 nm. Digital RGB images are taken together with the spectrometry data. By mounting the camera in nadir view the back side of the vehicle would be visible in the images. Therefore the images are recorded in an oblique view ( Figure 2) and are ortho-rectified in a post-processing step. The image capturing is performed either with a constant time increment, e.g. every second or with a constant path increment e.g. every 2 m. The RGB images support the investigations carried out with the spectrometer data. Further they will be used for automatic detection of asphalt failures, in particular for cracks. The laser scanner is pointing nadir in order to achieve a maximum point density on the road. For mobile laser scanning, the FARO Focus 3D X330 instrument is equipped with a socalled helical adapter. 8533 points have been recorded per scan rotation (limited to 300°) at a line scan speed of 97 lines per second. The laser line spacing in the direction of travel amounts to 8.6 cm when driving at the speed of 30 km/h. Within the line, a sampling distance of a few mm is achieved on the road surface. Figure 3 shows the location of the spectrometer spots overlaid onto the RGB image. The vehicle was driving at a speed of ca.17 km/h. One of the spectrometer spots covers a crack (Figure 3) which shows up in the spectral signatures with a lower reflectance than those spectra recorded at neighbouring locations, see Figure 4. This indicates that the lower reflectivity appears not only in the visible ( Figure 3 shows the colour image) but also in the infrared domain between 1200 nm and 2300 nm.   Integrating the processing results of the spectrometer data, the 3D surface failures from the laser data and the cracks detected in the RGB image will be further investigated in this paper. Figure 6. Crack detection (below) using RGB image (above)

INVESTIGATIONS RELATED TO DATA ACQUISITION
The approach followed in this paper is first to consider each sensor separately and analyse its mutual benefit for the analysis of road pavement condition. In a second step, a road condition mapping procedure integrates the results obtained from the three sensors.
The investigations shown in this contribution focus on the first step. Spectrometer data have been recorded in areas which are characterized by good, intermediate and bad road pavement conditions. The spectral signatures reveal these conditions with some restriction. Concerning the spectrometer recording, it is necessary to understand how the integration time influences the noise of the spectral signature. Further, the movement during the exposure time influences the area covered by a recorded spectrometer spot. Whether this "averaging" effect impacts the classification of the spectral signature with respect the mentioned road condition classes will be analysed as well. Segmentation and line filtering algorithms are applied to automatically detect cracks in the RGB images. The analysis of images recorded with different ground resolution aims at identifying a most suitable ground resolution for crack detection.
A homogenous point cloud distribution on the road surface might be beneficial for spatial analysis of pavement failures. An almost rasterized recording can be achieved by adjusting the measurement rate to the laser line spacing in the direction of travel. The limiting factor in this case is the line scan rate of the sensor. Alternatively, a common practice of road condition mapping is to record transversal profiles with high point density and accept a significantly larger spacing in the longitudinal direction. Finding proper settings in this regard is another issue of the detection of geometrical road irregularities.