Real Time Lane Detection in Autonomous Vehicles Using Image Processing

The aim of this research is to detect and analyze the drivable path ahead of the autonomous vehicles by implementing path detection algorithms in MATLAB. This study presents a technique for real time detection of lanes marked on the road that can be used in autonomous vehicles. Since, for safe navigation of autonomous vehicles, detection of drivable path is a key requirement. Painted road markings on rural, urban roads and highways help to detect precisely the road area for the vehicle to move. Lane detection algorithms aim at finding the edges or boundaries of road within which vehicle can move. This target is accomplished by the use of camera set up on the vehicle that tracks road ahead of the vehicle. Lanes are detected through processing of each frame of video by edge detection and Hough transform techniques. This system can be further make compatible with the hardware of the vehicle to take certain actions depending on the detections made. This study presents potential approaches and requirements that are needed for detection of lanes, its algorithm and result.


INTRODUCTION
Autonomous vehicles are a kind of robots provided with intelligence to move autonomously without human assistance.These systems are liable to grow fast and become more challenging in the coming years.The main task in the development of these vehicles is the perception, having two elements.
Road and lane detection and the other is obstacle (vehicles as well as other objects present on the road) detection (Amit, 2007).In this study, we consider the detection of first element only.Color and texture of road, road boundaries and markings are main cues for human to drive a vehicle (Aharon Bar et al., 2014).The techniques used for road navigation apply image analysis methods, in conjunction with camera models, to obtain three-dimensional data from images of the visual scene (Sowers and Mehrotra, 1989).Though there have been many image segmentation methods for this purpose but the results are not always satisfactory (Guowei et al., 2002).Since the autonomous vehicles are to navigate on the same road on which the other vehicles with human drivers do, the system to be designed should rely on these cues upon which humans do.However, the similar detection can be made through sensors also such as LIDAR and updating through Global Position System (GPS) (Yuan and Zheng, 2004) but, we use division system (that is one video camera) because it is more prominent area of research in autonomous ground vehicles since, markings on the road are made for human vision only.

METHODOLOGY
Lane detection: Marking of lanes has become more important because while driving in foggy season or heavy rain, where road is not clearly visible, then the lanes marked on the road can help in driving.Hence, we choose detection of lanes for the purpose of finding drivable path for the moving vehicle.In the absence of lane markings, color and texture of the road are mainly used for detection of path (Qingquan et al., 2014).Despite the simplicity of finding white markings on a dark road, difficulties raised by shadows, other vehicles occlusion, road surfaces and different types of markings (Tolga and Aytül, 2007).The model used for detecting lanes assumes that the lanes marked are straight and flat.Earlier, most of the approaches used Kalman and particle filter for lane detection (Stefan et al., 2007).Lanes can also be detected in image sequences of a camera mounted behind the windshield of vehicle (Samadzadegan et al., 2007).But here we use edge detection and Hough transform.
The task of identification of road marking includes processing of available data from vision system and combining it with information we already have in order to determine if a lane marking is present in the surrounding of vehicle.The block diagram for the process of lane detection is shown in Fig. 1.It mainly consist of acquiring the video through vision system and applying image processing unit (applying edge detection and hough transform) on each frame of video and displaying the output detections.
The algorithm which is working in the above mentioned processing units is discussed in next part Algorithm.

ALGORITHM
The captured video is processed through MATLAB.After video to frame conversion, each frame is read and pre processed (such as gray scale conversion) and then the nearly same color boundaries are extracted from those frames then all those boundaries are filtered through RGB channels as given in Fig. 2. Extra boundaries that are other than the road are removed by color masking.For examples all the shades of green color (trees or grass present on the sides of road) are removed and only the road boundaries are kept are processed.The remaining colors are processed to hue, saturation values to separate the shades of gray color to filter out the boundaries and finally get the desired boundaries only.

Pre processing:
The input video is then divided into frames and each frame is converted to gray scale.This conversion of RGB to gray scale in each frame is done so that to extract the boundaries from different shades of gray only.
Since the video covers the road as well as other parts also such as trees on both sides of road.Processing the complete view would be difficult.So we first confine the view that is, we focus only on the road part.For this, only the base section of each frame is taken and processed out.
Edge detection: Convert RGB frame to intensity values and detect all the edges.Since each RGB channel has three color masks.We are performing color masking basically to remove all the green color components present in the frame and thus extracting boundaries of road only.After color making we saturate the values of edge detected image between 0 and 1 such as: Again edges are calculated from this bounded image.
Sometimes, while driving when the sun rays affect the intensity and colors of the video being captured then, we have to remove the other color components also from one color.This is done by thresholding.Here, we removed the red component from the green color.
Rewriting this equation: Though, this whole process can be done by morphological, linear structuring element or correlation but then we will have to manage rotation, zoom, distortions etc. Edge detection is used as pre processing before applying Hough transform.The input to Hough transform is a threshold edge image: • Quantize or divide the parameter space into cells (This Quantized space is also called as accumulator cells).• Then for each edge point, increment the value of array (made from parameter space) as in Eq. ( 4) for all possible values of m: Then array with maximum value is taken as solution.
• Find local maxima in the parameter space.
• Polar representation of lines: where, θ is calculated from magnitude of gradient vectors (˟ , ˟ { as: Each Hough line (line segments) can be identified by its two end points (Stefan et al., 2007).Hough transform can be modified to predict the curves which represent lane markers in particularly during turning (Yuan and Zheng, 2004).
Lane extraction: After calculating Hough transform, define range of viewing angle to cover the road portion only.Here we wipe out the area <-78° and >78°.Select value of ρ and θ corresponding to the Hough peaks (ρ is the distance from origin to line along a vector which is perpendicular to that line).

RESULTS
The captured video is processed in MATLAB as shown in Fig. 3.
The view of the road marked with white painted lanes is shown in Fig. 4 before applying any detection algorithm.The specifications of the vision system and video taken as input for lane detection algorithm are as follows.

Specifications:
• Camera used: Nokia Lumia 1520  And the lanes are detected in each frame of video and are marked with white thin line as shown in Fig. 5  and 6.Each processed frame is saved with the detected lanes and thus saving the whole video.This detection confines the vehicle to move within the boundaries detected.

CONCLUSION AND RECOMMENDATIONS
The requirement is to minimize the number of incorrect classifications and minimizing their risks which is of great importance in autonomous navigation.If accurately calibrated, the system can provide robust response to false detections even in unstructured environments having slopes, shadows and different textures of road surface.To reach the required reliability, the path detection algorithm is achieved by detection of lanes marked on the road which ensures the drivable road for the moving vehicle.Since nowadays almost all the roads urban, rural and highways all have lanes marked on it for driving.
The future work can be to extend the road analysis and finding the obstacles present on road that are needed to be avoided for complete autonomous navigation (Tamer and Bayoumy, 2012).Along with analysis of arrows marked on the road and classification of these markings depending on the database and actions depending on the detections made.

Fig. 1 :
Fig. 1: Block diagram for lane detection Vision system: Vision system (one video camera) is mounted on the vehicle to capture the front view of the road in the forward direction in which the vehicle is moving.Then image processing algorithms are carried out to detect the lanes in the video being captured.Lanes are detected on the basis of road markings.The block diagram for the process of lane detection is shown in Fig.1.It mainly consist of acquiring the video through vision system and applying image processing unit (applying edge detection and hough transform) on each frame of video and displaying the output detections.The algorithm which is working in the above mentioned processing units is discussed in next part Algorithm.

Fig. 2 :
Fig. 2: Algorithmic steps for detection and marking of lanes Hough transform: Hough transform can be used to detect straight lines, circles, ellipse and other arbitrary shapes.It finds location of lines in an image as in Eq. (1): Tracking and generation of lanes: Track set of lane marking lines.Convert those lines from polar to Cartesian space.Draw the detected lane marker lines.While driving, if a change of lane is done to avoid the traffic in current lane, detect the right lane or the third lane present on other side of road.Otherwise find the lanes on left side.

Fig. 6 :
Fig. 6: Lanes detected and marked on right side of road when vehicle make a change of lane