Lane detection and tracking using B-Snake

https://doi.org/10.1016/j.imavis.2003.10.003Get rights and content

Abstract

In this paper, we proposed a B-Snake based lane detection and tracking algorithm without any cameras' parameters. Compared with other lane models, the B-Snake based lane model is able to describe a wider range of lane structures since B-Spline can form any arbitrary shape by a set of control points. The problems of detecting both sides of lane markings (or boundaries) have been merged here as the problem of detecting the mid-line of the lane, by using the knowledge of the perspective parallel lines. Furthermore, a robust algorithm, called CHEVP, is presented for providing a good initial position for the B-Snake. Also, a minimum error method by Minimum Mean Square Error (MMSE) is proposed to determine the control points of the B-Snake model by the overall image forces on two sides of lane. Experimental results show that the proposed method is robust against noise, shadows, and illumination variations in the captured road images. It is also applicable to the marked and the unmarked roads, as well as the dash and the solid paint line roads.

Introduction

Autonomous Guided Vehicles (AGV) have found many applications in the industries. Their applications had been explored in areas, such as patient transportation in hospitals, automated warehouses and other hazardous related areas. In most applications, these AGVs have to navigate in the unstructured environments. Path findings and navigational control under these situations are usually accomplished from the images captured by camera mounted on the vehicles. These images are also interpreted to extract meaningful information such as positions, road markings, road boundaries, and direction of vehicle's heading. Among many extraction methods, the lane marking (or road boundary) detection from the road images had received great interest. As the captured images are usually corrupted by noises, lots of boundary-detection algorithms have been developed to achieve robustness against these noises.

The main properties that the lane marking (or boundary) detection techniques should possess are:

  • The quality of lane detection should not be affected by shadows, which can be cast by trees, buildings, etc.

  • It should be capable of processing the painted and the unpainted roads.

  • It should handle the curved roads rather than assuming that the roads are straight.

  • It should use the parallel constraint as a guidance to improve the detection of both sides of lane markings (or boundaries) in the face of noises in the images.

  • It should produce an explicit measurement of the reliability of the results obtained.

Up to present, various vision-based lane detection algorithms have been developed. They usually utilized different lane patterns (solid or dash white painted line, etc.) or different road models (2D or 3D, straight or curve), and different techniques (Hough, template matching, neural networks, etc.). Basically, there are two classes of approaches used in lane detection: the feature-based technique and the model-based technique. The feature-based technique localizes the lanes in the road images by combining the low-level features, such as painted lines [5], [6], [7], [8], [9], [10] or lane edges [1], [2], etc. lane segments that are detected by traditional image segmentation. Accordingly, this technique requires the studied road having well-painted lines or strong lane edges, otherwise it will fail. Moreover, as it has the disadvantage of not imposing any global constraints on the lane edge shapes, this technique may suffer from occlusion or noise.

On the other hand, the model-based technique just uses a few parameters to represent the lanes. Assuming the shapes of lane can be presented by either straight line [11], [12], [13], [16] or parabolic curve [3], [4], [14], [15], the processing of detecting lanes is approached as the processing of calculating those model parameters. This way, the model-based technique is much more robust against noise and missing data, compared with the feature-based technique. To estimate the parameters of lane model, the likelihood function [3], [4], [11], [12], [16], Hough transform [13], and the chi-square fitting [14], [15], etc. are applied into the lane detection. However, as the most lane models are only focused on certain shapes of road, thus they lack the flexibility to modeling the arbitrary shape of road.

Motivated by the above problems, we here present a new B-Snake based lane detection and tracking algorithm for the outdoor application of AGV. The main characters of our method are the following:

  • 1.

    A novel B-Snake based lane model which describes the perspective effect of parallel lines is constructed with dual external forces for generic lane boundary or marking, it is able to describe a wider range of lane structures than other lane models such as straight and parabolic models. In addition, it is robust against shadows, noises, etc. due to the use of the parallel knowledge of roads on the ground plane. The lane detection problem is formulated by determining the set of lane model control points.

  • 2.

    A robust algorithm called Canny/Hough Estimation of Vanishing Points (CHEVP) is presented for providing a good initial position for the B-Snake lane model. This algorithm is robust to noises, shadows, and illumination variations in the captured road images, and is also applicable to both the marked and the unmarked, dash paint line and solid paint line roads.

  • 3.

    Using Gradient Vector Flow (GVF) to construct the B-Snake external force field for lane detection, a minimum error method called Minimum Mean Square Error (MMSE) that finds the correspondence between B-Snake and the real edge image is presented to determine the parameters of road model iteratively. Road tracking is carried on after successful lane detection, by a simple external force field and MMSE method, tracking is efficient and speed is fast.

Besides B-Spline, other kind splines also can be used in our lane model. Our early version of lane model used Catmull-Rom spline [24], [25], [26]. The different between the B-Spline and the other kind splines is the locations of the control points.

The remained structure of this paper is arranged as follows. Section 2 introduces a novel B-Spline lane model with dual external forces. In Section 3, the CHEVP is described for B-Snake lane model initialization. Section 4 presents a minimum error method, MMSE, to determine the parameters for lane detection and lane tracking. This section also shows some representative results of applying the proposed algorithm to various types of roads under different environments. This paper concludes in Section 5.

Section snippets

The modeling of lane boundaries

Lane model plays an important role in lane detection. The lane modeling has to make some assumptions about the road's structure in the real world in order to fully recover 3D information from the 2D static image. In this paper, we focus on constructing the 2D lane model, by assuming that the two sides of the road boundaries are parallel on the ground plane as shown in Fig. 1(a).

In addition, let us assume that the right side of road is the shifted version of the left side of road at a distance, D

Initialization of B-Snake lane model: CHEVP algorithm

Some lane detection algorithms required the operator to provide the initial estimate of the road location, while others required the specific road structure scene (such as straight road) as the first road image. These requirements on the road initializations are clumsy for the automatic road detection task. Therefore, automatic initialization technique, able to extract the location of any type of the lane shapes, is important and necessary.

B-Snake parameters updated from image data

Based on the initial location of the control points that are determined either by CHEVP algorithm or lane detection result of previous frame, the B-Snake would further approach to road edge accurately in the current frame. This Section deals with this problem.

Conclusion

In this paper, a novel B-Snake based lane model, that describes the perspective effect of parallel lines, has been established for generic lane boundaries (or markings). It is able to describe a wider range of lane structures than other lane models, such as straight and parabolic models. The problems of detecting both sides of lane markings (or boundaries) are merged here as the problem of detecting the mid-line of the lane. A robust algorithm, called CHEVP, is presented for providing a good

References (28)

  • S.P Liou et al.

    Road following using vanishing points

    Computer Vision, Graphics, and Processing

    (1987)
  • T McInerney et al.

    A dynamic finite element surface model for segmentation and tracking in multidimensional medical with application to cardiate 4D image analysis

    Computerized Medical Imaging and Graphics

    (1995)
  • Y Wang et al.

    Lane detection using spline model

    Pattern Recognition Letters

    (2000)
  • B. Serge, B. Michel, Road segmentation and obstacle detection by a fast watershed transform, Proceedings of the...
  • Y. Xuan, B. Serge, B. Michel, Road tracking lane segmentation and obstacle recognition by mathematical morphology,...
  • K Kluge et al.

    A deformable template approach to lane detection

  • S Lakshmanan et al.

    Lane detection for automotive sensor

    ICASSP

    (1995)
  • A Broggi

    Robust real-time lane and road detection in critical shadow conditions

    Proceedings IEEE International Symposium on Computer Vision, Coral Gables, Florida, November 19–21

    (1995)
  • A Broggi et al.

    Vision-based road detection in automotive systems: a real-time expectation-driven approach

    Journal of Artificial Intelligence Research

    (1995)
  • A Broggi

    A massively parallel approach to real-time vision-based road markings detection

  • M Bertozzi et al.

    GOLD: a parallel real-time stereo vision system for generic obstacle and lane detection

    IEEE Transactions of Image Processing

    (1998)
  • H Andrew et al.

    Lane detection by orientation and length discrimination

    IEEE Transactions On Systems, Man and Cybernetics, Part B

    (2000)
  • S.G Jeong et al.

    Real-time lane detection for autonomous navigation

    IEEE Proceedings Intelligent Transportation Systems 2001

    (2001)
  • D Grimmer et al.

    A deformable template approach to detecting straight edges in radar images

    IEEE Transactions on Pattern Analysis and Machine Intelligence

    (1996)
  • Cited by (0)

    View full text