Avoiding Forward Car Collision using Stereo Vision System

This paper presents a model of stereo vision system for intelligent vehicles to avoid forward car collision. Approached vehicle extraction and distance estimation algorithm based on dense edge map calculation and ROI filter. Primary experiments have been executed modeling vision system in a virtual computer reality. Special developed software has been used to simulate virtual environment and vision system. Results shows that approached algorithm successfully extracted not only vehicles but and other potential obstacles, such as, pedestrians.


I. INTRODUCTION
Car accidents are the key problem of today's traffic world. Particularly this strongly felt in urban areas where cars density is largest. According to Lithuanian 2011 year car accidents statistic most common type of car accidents are collisions. It represents 44% of total car accidents. Accidents happened with pedestrians and obstacles also could be assigning to similar car accidents type. By summing of everything, we are getting 82% of total accidents [1]. Most of car accidents happened due to human factor: somnolence, inattention, absence of mind and slow reaction. All these factors are critical for a driving safety. Using stereo vision system is a way to reduce number of car accidence. Stereo vision based traffic observation systems are in active development period. Scientist and many famous car manufacturers are developing durable, accurate and real time car driver assistance systems. Many of works done by researching and developing vision system for lane tracking [2], pedestrian or bicyclists detection [3], car detection [4] or driver monitoring [5]. The main purpose of using stereo vision system is capability to transform 2D view into 3D information usually called depth map. Typical visual system consists of two digital video cameras which are displaced horizontally from one another to obtain different views of a scene, in a manner similar are used to human binocular vision (Fig. 1). 3D information derived by finding the corresponding points across multiple cameras images. Correspondences lie to the same epipolar line. To simplify the matching process, camera images are often rectified. The result is epipolar line corresponds on image scan line. Before image rectification system has to meet these requirements: cameras image planes are parallel, focal points height and lengths are the same. To get a depth map a disparity value must be calculated for every image pixel. A disparity value for a target point p calculated using following expression Disparity value estimation directly related to main stereo vision problemcorrespondences. Local and global methods are used to find correspondences. Local methods usually use less computation resources and are much faster while global opposite.
A depth map estimation based on a triangulation. Given camera geometry with a focal length f and the distance B between two cameras are main system parameters. Depth map can be computed using Depth map is a main 3D data source for car extraction and distance estimation.

II. STEREO VISION SYSTEM ARCHITECTURE
Stereo vision system architecture consists of three data processing levels ( Fig. 2):  Low level procedures covers image capturing from digital cameras, data buffering and image rectification according to system calibration data.  Medium level is the most critical for all vision system accuracy. Selected stereo matching algorithms and estimated 3D information strongly influence higher level results. Today one of the most advanced stereo matching algorithms is based on segmentation, adaptive belief or reliable propagation [6]- [8].  High level is the last processing stage where depth and image data are used for object extraction and recognition. Further object list is used for front vehicle tracking, distance or velocity estimation. Calculated data are used for car indication or car control.

III. CAR EXTRACTION APPROACH
Suggested approach based on dense edge map calculation and ROI filter. Fig. 3 illustrates a block diagram of vision system data processing. Totally eight steps sequence to extract and mark car from 2D image set. Car extraction process starts from capturing of view images. Next, captured 2D images processed in the middle data processing level. Here 2D data transformed into 3D information, well known as depth map. ROI filter eliminates redundant 3D information. The rest of data is processed by dense edge map estimation algorithm. The result of algorithm is top view of depth map with highlighted object edges. Edges detected using binary large object method. After this process we have all useful information about object position, height, width and distance to it. These results directly can be use for vehicle control or representation. To represent detected object on 2D source image, three more steps sequence. Source view background eliminated using depth map and dense edge map data. Blob bounds detected using same binary large object search method. These bounds further used for object marking on 2D view image.

IV. ROI FILTER
Region of Interest filter is used to eliminate redundant 3D information. ROI can be represented as a frustum in a 3D environment (Fig. 4). According to fact that system designed to observe front vehicles a frustum is projected on a roadway and covers one or more drive lines. All data outside this space are eliminated leaving only significant information. The main benefit of applying ROI filter is saved extra computation resources for further object extraction processes.

ROI filter
where ( , ) min( ) 1, , ( , , ) 0, , where A is 3D data array; B is dense edge data array, i∈{0,1..h-1}, h is image height, j∈{0,1..w-1}, w is image width, c is a factor calculated This transformation is modified following the idea to analyze not whole object but only their edges. The result of this transformation is an edge's map in a perspective of top view. Fig. 5 depicts the transformation principle in a 3D environment. The most density of points is located on object's edges. Edges are highlighted by summing points in a vertical direction. The more object's edge is higher and sharper the more points will be counted in a vertical direction. Flat and low altitude objects have a small amount of points; therefore, objects like a road can be easily eliminated using simple threshold.

VI. SYSTEM TESTING PLATFORM
A virtual 3D computer reality was chosen as very first vision system and approached car extraction algorithm test environment. Specially designed software allows simulating stereo vision system behavior in a virtual 3D reality. Software based on OpenGL and OpenCV programming libraries and is written in a C++ language. Software runs on Core 2 Duo 2.53GHz CPU and 4 GB memory of RAM. Similar kind of test method has been used by other authors [9] and [10] to test stereo vision systems for relative applications. In order that tested situations would be more realistic, a segment of street with all following infrastructure: cars, road signs, buildings, pedestrians were designed and rendered in a virtual 3D reality. In this test stage a depth map was generated using OpenGL graphic API methods. This allows getting the best depth map quality. Depth map can be estimated and using any kind of stereo matching algorithm. Simulation software allows imitating stereo camera view and capture left and right camera images. Estimated depth map can be represented as a point cloud converting 3D data to real word coordinates. Different traffic situations, vision systems configurations can be emulated and repeated many times. This is the main advantage of described simulation software.

VII. EXPERIMENT RESULTS
As was mention before, approached car extraction algorithm was tested simulating different traffic situations in a virtual 3D reality. Some situations examples presented in Fig. 8. Left image depicts a camera view and marked obstacle on the road. Right image is a representation of a dense edge map. Situation no. 1 represents a traffic circumstance then forward car is far away from driving car. Vision system detected potential obstacle and marked with a green rectangle. Green color indicates that vehicle is in safe distance. Objects away from a camera represented as small blobs in a dense edge map. Blob position in a vertical direction determined by distance from camera to object. Greater object distance leads greater blob distance from map bottom. The opposite situation presented in the situation no. 2. Detected car marked with a red colored rectangle. Red color indicates that car is in the unsafe distance and driver must pay attention. Close objects represent as big blobs located near to dense edge map bottom. The rest two situations (no. 3, 4) shows system ability detect not only forward cars but and different kind of obstacles, for instance pedestrians crossing the street. Another aspect of vision system is ability to detect multi numbers of obstacles.

VIII. CONCLUSIONS
After range of generated different traffic situations, testing results shows a good approached car extraction algorithm ability to detect objects and measure distance to it on the road. Vision system was able to detect not only massive object such as vehicle, but and less massive, such as pedestrians.
To reduce computational recourses ROI filter was used to eliminate redundant 3D data. This filter actually determines system analysis space. To simplify object extraction and safe calculation time, edge dense map is used. Here object examined only as dense edge curve. The average time of object extraction and marking took ~125 ms. This was achieved using Core 2 Duo 2.54 GHz processor and 4GB memory of RAM. To apply vision system for cars need to reach at least 30 ms for all data processing. This can be done using optimized code, multi threading, and new generation or GPU processors.
To use approached vision system model for avoidance of