PAVEMENT DISTRESS DETECTION WITH PICUCHA METHODOLOGY FOR AREA-SCAN CAMERAS AND DARK IMAGES

The PICture Unsupervised Classification with Human Analysis (PICUCHA) refers to a hybrid human-artificial intelligence methodology for pavement distresses assessment. It combines the human flexibility to recognize patterns and features in images with the neural network ability to expand such recognition to large volumes of images. In this study, the PICUCHA performance was tested with images taken with area-scan cameras and flash light illumination over a pavement with dark textures. These images are particularly challenging for the analysis because of the lens distortion and non-homogeneous illumination, generating artificial joints that happened at random positions inside the image cells. The chosen images were previously analyzed by other software without success because of the dark coluor. The PICUCHA algorithms could analyze the images with no noticeable problem and without any image pre-processing, such as contrast or brightness adjustments. Because of the special procedure used by the pavement engineer for the key patterns description, the distresses detection accuracy of the PICUCHA for the particular image set could reach 100%.


INTRODUCTION
Road pavements are among the most expensive infrastructures for any country.The budgets to maintain the highways are always limited while a continuous demand for its expansion is present.The available funds are often insufficient for any country to keep all the road pavements in an excellent condition all the time.On the top of this, a large portion of the road infrastructure, constructed in the 1960/70, is approaching the end of its service life, which squeezes the budgets even more in the coming years.To help manage such situations and maximize the value of the existing budgets for roads, the effective pavement management is inevitable.
Together with the structural condition, the pavement surface condition assessment is the most important information for a proper pavement management as well as for pavement rehabilitation design.The handmade assessments, common decades ago, are no longer viable for many reasons, including the high costs, high labour demand and risks for staff on the current highly-trafficked highways.
Over the years different imaging technologies were developed or adapted for automated pavement surveys, including traditional area-scan cameras that take an image covering a rectangular area on every shot, line-scan cameras that take one single line of pixels on every shot, and 3D cameras that read one transversal lane profile on every shot.Despite the progress on the hardware used in the field survey, the distresses detection and analysis software for generating field data remains relatively underdeveloped.
Puan and others developed an automated pavement imaging program (APIP) for pavement distresses assessment, capable to work with longitudinal, transverse and alligator cracks, and able to analyze also the crack severity by using a number of different algorithms [1].Ting and others used an approach based on the k-means and classification algorithms to identify the pavement distresses on pictures.The images were processed and filtered in order to keep only black-andwhite pixels that were assumed as related to cracks.Then the images were grouped in clusters as distresses being detected by a decision-tree algorithm capable of recognizing horizontal, vertical, alligator and man-hole-like cracks [2].
To detect the pavement distresses in pictures Ouyang et al., tried an approach based on filtering the images to remove the background, or pavement texture, and image enhancements, segmentation and Canny edge detection [3].Nguyen and others did the cracks detection using a kind of conditional texture anisotropy to characterize and classify the pixels as "crack" or "crackfree" pixels.The idea is to detect variations of features, including noise, continuity, homogeneity and others.The authors claimed the method can detect also other patterns, like joints [4].
Tsai and others worked on image segmentation as a kind of preprocessing for distresses detection and classification.Six different algorithms were used and evaluated with images taken near the city of Atlanta, USA, with varying lighting conditions, shadows and cracks [5].Lin and Liu used Support Vector Machine (SVM), a topology of artificial intelligence similar to neural networks, for assessment of potholes on pavement pictures.The pavement texture was detected by using the histogram and a non-linear SVM was built to identify if the target region was a pothole.The experimental results show that the approach can achieve satisfactory results [6].
An approach combining analytic hierarchy process (AHP) and fuzzy logic theory for pavement condition assessment was developed by Sun and Gu.AHP is used to determine weight from a paired comparison matrix and an evaluation with fuzzy relations, combining the evaluation of five different indicators, including roughness, deflection, surface deterioration, rutting and skid resistance.A maximum grade principle and a defuzzified weighted cumulative index are proposed to assess the condition of a road [7].
Salman and others proposed a novel approach to automatically distinguish cracks in digital pavement images with the "Gabor" filter that is related to the mammalian visual perception.In their study the pavement images with a prominent surface macrotexture that are frequently harder to get analyzed [8].
Potholes were also the subject in Koch and Brilakis' study.The images were first segmented into defect and non-defect regions using the histogram-based thresholding and then, the potential pothole shape was approximated by utilizing morphological thinning and elliptic regression.After that, the texture inside the potential pothole was compared with the texture of the surrounding area to decide if it represented a pothole or not.The routine was implemented in MATLAB [9].After the procedure was improved by using a vision tracker to reduce the computational effort and improve the detection and pothole counting [10].
To detect distresses in images Avila and others used a "minimum path" technique, defined as when the sum of pixels intensity is the smallest.It is a single step procedure that works with geometry and intensity property of pixels, made viable for practical use by dynamic programing [11].For the distresses assessment Díaz-Vilariño and others used data from a mobile Light Detection And Range (LiDAR) system, a surveying technology that measures distances by illuminating the target with a laser light.Their method is based on the evaluation of roughness descriptors from LiDAR data to segment and classify the asphalt pavement [12].
The Kinect device is a motion sensor originally developed to provide a natural interface for gamers.Because of its low cost, interesting detection capabilities and easy connection to computers, it has been adapted for different uses in many industries.Xie worked with Kinect generated data to detect pavement distresses by considering the measurements of depth, width, length, and area, and various cracking types, such as alligator cracking, traverse cracking and longitudinal cracking [13].

PICUCHA FOR PAVEMENT DISTRESSES ASSESSMENT
The PICture Unsupervised Classification with Human Analysis (PICUCHA) method is a new approach for pavement distresses assessment that combines the human flexibility to recognize patterns on images with the neural network ability to match patterns by similarity, expanding the (human) pavement engineer decisions over large image sets.
PICUCHA has been designed to circumvent the limitations commonly found on other methodologies such as the variations on the pavement colour and texture, handling all the patterns registered on images, distresses or not.PICUCHA can detect good pavement, raveling, complex or isolated cracks, block or alligator cracks, sealed cracks, patches, potholes, painted horizontal signals, like white or yellow strips, reflective signs attached to the pavement, drainage devices, embedded inductive loops, joints, asphalt bleeding, or any combination of two or more of such patterns, among others.It can analyze road sections with mixed pavement types, like asphalt and concrete, with any type of surface texture, colour or pattern, including anti-slippery strips or cuttings, and with the presence of complex or solid shadows.
PICUCHA is capable of analyzing orthogonal images ("downward facing") taken in the field with any device technology (line-scan or area-scan cameras, 3D cameras, …), different sources of illumination (laser beam, incandescent lamps, LED, …) and different image resolutions and dimensions such as 512 x 2048 or 2048 x 2048 pixels.The PICUCHA approach is structured in a few steps including field survey, key patterns extraction, key patterns analysis by a pavement engineer, and the engineer decisions expansion to all the images in a given set, as shown in Fig. 1.A study evaluating the PICUCHA method performance with images at different resolutions is also available [14].
The PICUCHA method is an extension of our previous developments on new methodologies and artificial intelligence applied for pavement engineering, including pavement management with genetic algorithms [15], pavement modelling with neural networks [16], the aside failure criteria that opens a new frontier of possibilities [17], and a deflection basin geometry analysis to calculate strains [18], among others.

CAMERA TECHNOLOGIES FOR PAVEMENT SURVEY
Over the years the companies developed different cameras technologies and other devices for pavement surface condition assessment with growing image resolutions.The most common are the area-scan cameras, line-scan cameras and 3D cameras.Those devices are available with different image resolutions and from different suppliers and integrators ("brands").The camera type and resolution have a direct impact on the image quality and the equipment's cost.
Area-scan cameras were the first generation of equipment for pavement survey.They are adaptations of general-purpose cameras (Fig. 2.a) and are notorious for its low cost.Some integrators assemble two cameras together for a higher resolution.This type of camera takes a rectangular image on every shot (e.g.2048x512 pixels) resulting in a number of problems, especially image distortions and non-homogeneous illumination.Those problems are especially visible when two or more images are assembled together and generate an extra challenge for the distresses detection by software.Area-scan cameras are not compatible with laser illumination, requiring many lamps that are usually strobe flashlights.

Field survey
The field survey is done with any equipment capable to take downward facing pictures.The PICUCHA algorithms can process images taken with any device brand or technology including:  Line-scan or area-scan camera, laser crack measurement systems (LCMS) or other;  With laser, incandescent, LED or other types of lamps, or just natural illumination; and,  Images with any size and resolution.

Key patterns identification
The images are sliced into cells (e.g.128 x 128 pixels) and a special algorithm will analyze and group the cells to identify the key patterns.There is no predefinition of distresses or limitations, the selflearning algorithm deals with any kind of pattern existing in a given image set.The key patterns represent the characteristics of every group.

The human pavement expert analysis
The key patterns are analyzed by a human pavement expert that will describe the distresses and other desired characteristics with base on any standard or manual for distresses assessment.At least one key pattern cell per group must be analyzed.This procedure avoids the problem to rely just on software tools and keeps the human expert on top of the process.

The human pavement expert analysis expanded to all images
The human pavement expert description is used to refeed the algorithms that will expand such decisions to all the images in the respective image set and groups, generating the final report.

Fig. 1 -Flow chart of PICUCHA Method's main steps
Line-scan cameras represent an advance in technology and image quality (Fig. 2.b).As its name suggests, it takes one single line of pixels on every shot (e.g.2048 x 1 pixels).Image distortions and illumination heterogeneity are minimal.The line-scanned pixels can be assembled together to generate a consistent and flat rectangular image.It is compatible with laser (beam) illumination in addition to incandescent, LED and other types of lamps.
The 3D cameras, also known as laser crack measurement systems (LCMS), collect two different information at the same time: a 2D top-view image and a transverse profile (Fig. 2.c).The image is usually a by-product for this equipment and its quality tends to be low.The analysis is performed by using the profile data to locate the cracks over the image.Some integrators install the 3D cameras together with line-scan cameras for a better image quality.This technology is relatively new and well-known for its very high costs.

PICUCHA PERFORMANCE WITH AREA-SCAN CAMERA AND DARK IMAGES
The objective of this study is to verify the PICUCHA algorithms ability to process images taken with area-scan cameras and showing a dark asphalt pavement.
Images taken with area-scan cameras and flash illumination represent an extra challenge for processing and distresses detection.Some pavement areas are closer to the camera than others, creating lens distortions and non-perfectly flat images, and the illumination is not homogeneous over the image because some areas are closer to the lamps than others, generating an artificial joint when two or more images are assembled together for the analysis, as shown in Fig. 3.
Dark images are an important challenge for the automatic distresses detection.Almost all software are capable to analyse only images with distresses patterns, colours and visual texture similar to those taken in consideration on the algorithms development, failing in all other cases.The images used in this study were previously analysed by other commercial software without success because of the pavement dark colour and texture.

Fig. 3 -Distortions and illumination joints are evident with image are assembled together
The images analysed in this study were taken by the Road and Transport Research Institute of Lithuania with a vehicle survey system equipped with two area-scan cameras, providing a final resolution of 1.1157 mm/pixel, and flash lights illumination (strobe light bar) (Fig. 4).The area-scan cameras took a rectangular image on every shot.Every image is 2921x444 pixels in size, resulting of the data fusion of both cameras, and covering an area of 3.25x0.50meters (width x length), which is slightly narrow than a typical road lane.

Fig. 4 -Vehicle for pavement surveys with dual area-scan cameras and flashlights
For this study 40 sequential images with 2921x444 pixels each one are used, assembled together as shown in Fig. 5, with a final size of 2921x17760 pixels.No image pre-processing has been used, like contrast or brightness adjustments, or others; they are raw as generated by the cameras.
The analysis followed PICUCHA methodology and the 2921x17760 pixels image (Fig. 5) was broken down into cells.For the processing the cell's size was set to 133x133 pixels and the algorithms were set to generate 25 groups.This procedure, the 40 images assembling and successive slicing into cells, is required to have the illumination joints and lens distortion happening in random positions inside the cells (Fig. 6), preventing a bias and making the analysis more challenging.
PICUCHA algorithms had no problems to process the image (Fig. 5) and perform the cells' grouping.The distribution of the 2814 cells among the 25 groups are shown on Fig. 7.The group 5 is the largest, with 349 cells while the group 19 has just 17 cells.The cells groups is a key step to reduce, simplify and rationalize the subsequent pavement engineering work describing the patterns.
Following the PICUCHA method, the pavement engineer did the key patterns description according to the Lithuanian standard for distresses assessment [19].Five patterns were identified in the images: (1) good pavement, (2) crack medium severity, (3) longitudinal crack medium severity, (4) alligator cracking, and (5) pavement patch.The characteristics (patterns and distresses) of every group are shown in Fig. 8 and the groups of cells over the original image are shown in Fig. 9.The final results, containing the distresses assessment for this study, are shown in Fig. 10.Because of the special procedure used by the pavement engineer analysing the key patterns, the PICUCHA accuracy for this particular study is 100%.Different image sets may lead to different accuracies, which depend on the images quality, used camera, illumination type, pavement characteristics, distresses patterns complexity, pavement engineer work analysing the key patterns and other variables.
PICUCHA was able to perform the analysis despite of the lens distortion, illumination joints and dark aspect of the pavement.No image pre-processing was used.The algorithms made 25 groups, which is a relatively low number, and the distresses are well characterized in every group.The pavement engineer followed the Lithuania pavement distresses assessment standard to identify five key patterns: good pavement, crack medium severity, longitudinal crack medium severity, alligator cracking and patch.Because of the special procedure used by the pavement engineer, the distresses detection accuracy for this study could reach 100%.Different image sets may lead to different accuracies.