An integrated framework for knowledge based obstacle information system with image processing techniques

One of the most important reasons of car accidents are collisions with vehicles that are not visible to a driver. This is why driver warning systems are developed. The main threat for a driver on the highway comes from the surrounding vehicles especially when the driver is not aware of the close presence generally known as driver’s blind spot. An area in and around the vehicle that cannot be directly observed by the driver are known as blind spots. Image processing plays a vital role in this scenario to safe guard drivers from sudden obstacles and blind spots. The pre-processed sequences of images acquired using front and rear camera of a vehicle are considered to train the case base reasoning (CBR) model, which detects the presence of dangerous objects in the blind spot area. To distinguish near and far obstacles, the same CBR model is used with a specified threshold in the vehicle blind spot area. The proposed knowledge based obstacle information system obtains promising results with standard blind spot camera which can improve safety of the driver and has the potential to be applied in vehicular applications for the detection of obstacles and blind spot area.


Introduction
In India, especially on the highways, vehicle accidents are the major cause of mortality and can cause severe even life threatening injuries. The main threat for a drive on the highway comes from the surrounding vehicle especially when the driver is not aware of their close presence generally known as driver's blind spot. An area in and around the vehicle that cannot be directly observed by the driver are known as blind spots, under existing circumstances. Motivated by this factor, An integrated framework for knowledge based obstacle information system is proposed with image processing techniques for Intelligent Vehicle Applications. Vision is the main sensor for providing detection of blind and hidden obstacle on the roads. Each year, passenger vehicles are produced with new innovative mechanical and electronic features installed to enhance drivability and safety. Blind spot monitoring systems are an example of this type of feature. The system is capable to detect a moving vehicle in blind spot area under three main condition of which static, dynamic speed operation at 60 and 100 km/h and also overtake position. Computer-vision-based BSM use digital camera imaging technology to sense the presence of vehicles in blind zones. Cameras mounted on or near the outside rear-view mirror housings on both sides of the vehicle provide views of the blind zones to the system. Images from the cameras are processed by the systems using algorithms that identify image characteristics associated with vehicles. Systems may identify vehicle shapes in daylight, vehicle headlights at night, and/or some combination of shapes and lighting. The systems can be disrupted by snowfall and rainfall, and by snow, ice, and dirt present on the camera lenses. Shadows, light reflected from wet roadways, and sunlight at sunrise and sunset can cause the systems to produce false positive alerts.

Related works
Research on active safety systems is a hot issue in recent car technology. Such systems are usually based on radar, ultrasound or optical sensors that help them to perceive the environment around the car and if it is possible to predict and prevent potentially dangerous situations. The active systems can warn the driver of the risk of accident, prepare the passive safety systems (like airbags) or even change the car trajectory to avoid collisions. Existing solutions use in general video cameras that provide the richest information about nearby objects. Kwon [3] proposed a blind spot detection and warning system named BSDW for driver assistance under daytime and night time conditions. Yu, F., Kamiska (2007) and Park, S (2010) [7] proposed approaches based on ultrasound sensors and microwave radar. Chen et al. (2007) [6] proposed an algorithm for vehicle detection based on two CCD cameras mounted on side mirrors. Vehicles were detected by comparing the grey intensity with the road surface and a method for the distance estimation. Mahapatra et al (2009) [4] A panoramic sensor based automotive vehicle monitoring system was described. Yu, F., Kamiska et al (2007) [3] implemented a simple ultrasonic sensors system reporting the close-by objects to the driver. Krips (2004) et al [1] concluded that there is also a tracking method for vehicles approaching from the rear, where they are classified as potential targets by means of a shadow based classification algorithm. Mahapatra et al (2009) [5] developed a system using the microwave radar to detect objects and a FNN to classify targets. Miguel Angel et al (2007) [4] focuses on the development of SVM-based vehicle recognition, as proposed in Blanc and Steux, 2017 for increasing the detection rate and decreasing the false alarm rate. It has also been demonstrated in sotelo et al., [4] 2015, where SVM was used for vehicle detection in an ACC (adaptive cruise control) application.

Proposed Framework
In general, research has been conducted on lane change assist systems, blind spot warning systems, road sign recognition systems, or adaptive cruise control systems. There are also other ideas such as pedestrian protection system, or systems detecting driver fatigue, etc. The active safety systems are typically rather expensive because they make use of advanced sensors and sophisticated algorithms. The proposed approach is to design and create an image based method using machine learning algorithm that helps in detecting vehicles in the blind spot area. Figure 1 is presented with an integrated framework for knowledge based obstacle information system.   The installation of the camera was carried out using a supplementary element attached to the left-hand side lateral mirror of the car. Thus, the driver maintains full visibility of the scene by means of the lateral mirror, while incorporating a new element that provides additional safety by using the visionbased processing. The car was manually driven for several hours in real highways and roads.

Image Pre-processing
The proposed method is based on the observation that a moving vehicle has usually much more details and contrasts than its environment. This means that the density of edges in the area where a vehicle is present is much higher than in the surrounding areas. Histogram equalization increases the contrast of the analysed image. This operation makes edges more distinct, so they can be easier to be found. The idea is applying noise removal then to detect edges on the image frame, divide the frame into several segments and estimate the edge density for each of them.

Segmentation
The segmentation is proposed to use very simple edge detector based on the Sobel mask. Sobel operator calculates spatial gradient measure of the image. In this method we perform such operation in both directions and then sign all the points with the gradient value exceeding a certain threshold as belonging to edges. To do it we define a pair of Sobel convolution masks of the size 3x3 to approximate the derivative in the horizontal and vertical direction. Such masks are shifted over the whole image calculating gradient of each pixel. The resulting binary image, after otsu thresholding contains only detected edges (white pixels) and the black background.

Estimation of Moving Direction
Once we detect the object in Region of Interest of window, we need to estimate moving direction of the vehicle. There are two possible directions for the detected vehicle to move. One is approaching toward the ego vehicle from behind and the other is moving backward as the ego vehicle overtakes. It is natural for the purpose of BSM to concern only the approaching one. So, we must estimate the motion vector (MV) of the detected vehicle to determine if it moves forward or backward. We can easily determine if the detected vehicle is approaching from the direction of motion vector. When a vehicle is detected in the right detection window and the MV is towards the ego vehicle that means the vehicle moves forward and approaches the ego vehicle in the blind-spot area.

Generation of alarm/signal to alert driver
The final step is to determine an indication signal must be generated or not, based on event value from the vehicle detection in blind spot area and motion vector of detected vehicle.  Figure 13. Flow chart for obstacle driver alert 3.8 Evaluation of Algorithm Performance and Computational Complexity 1) Accuracy performance of the proposed algorithm, need to be measured using recall and precision approach 2) Complexity of algorithm is measured using computing time on embedded board. Additional components that be added: These will be added based on the progress shown above.

iConclusion iand iFuture iwork
In this paper, an integrated framework for knowledge based obstacle information system is proposed to monitor the blind-spot area using only a rear-view camera and implemented it within inexpensive lightweight embedded device. The proposed method revealed perfect performance on the highway regardless of the use of basic or high-speed algorithm. The embedded device is developed for vehicle navigator and demonstrated that it is suitable for real-time processing.