A Novel Method to Handle the Partial Occlusion and Shadow Detection of Highway Vehicles

This study proposes a novel method for resolving the partial occlusion and shadow of moving vehicles seen in a sequence of highway traffic images captured from a single roadside fixed camera position. The proposed method detects the shadow regions of foreground/moving vehicles in any direction with no human intervention. Also, it handles the partially occluded vehicles in three different states on the highway traffic scene. The moving vehicles are detected using the background subtraction method followed by using the dilation and erosion operations. In this step, every segmented moving vehicle has an extracted Feature Vector (FV) which is used to initialize the Shadow Searching Window (SSW). Then, a chromatic based analysis technique uses the semi mean value of RGB colour space of moving vehicle region pixels as a threshold value to discriminate the shadow of moving vehicles despite the direction of movement. Finally, the partially occluded vehicles are handled using a new training procedure based on previous estimations to calculate the new moving vehicle sizes. These sizes are employed to separate and highlight the partially occluded vehicles by drawing the bounding boxes for each moving vehicle.


INTRODUCTION
Highway surveillance is one of the significant applications of video-based traffic supervision systems.Highway vehicle detection and tracking from video frames is an active research area which is believed by many researchers worthy of investigation due to the significant impact of providing an efficient traffic analysis data (vehicle flow, trajectory, speed, etc.).
However, some effects may deteriorate the detection accuracy of vehicles such as changing weather conditions, the wave motion of tree leaves and light fluctuations, vehicle shadows, obstruction or occlusion by visual obstacles such as road signs, trees and other vehicles.These previously mentioned problems are gradually deteriorating the performance of the detection and tracking of moving vehicles which is dependent on an efficient and accurate traffic image analysis (Hadi et al., 2014a;Jalal and Singh, 2012).
The handling of partial occlusion and shadow detection of moving vehicles is an important and challenging task.In most cases, partial occlusion means the way in which some parts of the vehicle are hidden by other objects (vehicle, road sign, tree, etc.) and it occurs under different projection views due to camera position and the high congestion on the highway because of dense vehicle movements and trajectories (Kanhere and Birchfield, 2008).Usually, shadows occur as a result of object movement under different conditions of illumination and the appearance of regions simultaneously changes with the direction of vehicle motion.The shadow is made from eight directions on the frame due to the projection of sunlight blocked by objects (Tian et al., 2012).

LITERATURE REVIEW
Detection of occluded vehicles and moving shadows is a critical process in the generation of efficient, accurate and robust detection and tracking of moving vehicles.Therefore, many studies have attempted to detect and handle vehicle occlusions and shadow.
Mao-Chi and Shwu-Huey (2004) presented a system based on analyzing the color of an image using the YCbCr color space for vehicle detection, shade elimination and tracking.In this study, the scoring function method known as Shadow Confidence Score (SCS) is adopted for discrimination of the pixels of vehicle shadow.A new algorithm based on edge detection method to identify and remove the shadow of moving blobs (vehicles) is proposed by Zhang et al. (2007).This algorithm utilized the canny edge detector to produce an edge image of each moving vehicle.The produced edges are represented as features to identify and remove the shade region of each moving vehicle.Also, another proposed work by Huang et al. (2012) used the HSV (hue, saturation and value) color space to detect and remove the shadow of moving objects.They suggest a comparsion of the different of saturation and hue intensity values of any dedicated segmented region pixels with specified thresholds.The final value is called the ratio of light intensity which determines the shadow region.Also, using the brightness and chromatic values of pixels to detect shadow region is proposed by Mandellos et al. (2011).The authors employed the LUV color space by defining the UV component as the chromaticity and the L component as the brightness of a pixel, respectively, in their work.They distinguished the pixels as a shadow of moving objects when the brightness of any pixel is less than the background scene pixel value.The researchers Joshi and Papanikolopoulos (2008) suggested a learning method which is combination of Support Vector Machine (SVM) with co-training algorithm to detect moving shadow in dynamic environments.The objective of this proposed method is to classify the shadow and foreground regions based on analysis of pixel-level with minimum training samples using extracted features (color, illumination and edges).
The probability of partial occlusion occurring in a vehicle has already been demonstrated for varying projection views (front, rear, or side views) and environments (outdoor or aerial).For occlusion handling, a statistical approach using Principal Component Analysis (PCA) and independent component analysis (ICA) is offered by Chi-Chen Raxle and Lien (2008) for automatic vehicle detection in the presence of partial occlusion matter.The approach detects the none and partially occluded vehicles using the local features which are located in significant sub regions of the image.Another study was introduced by Kanhere and Birchfield (2008) to detect and track the partially occluded vehicle using feature points such as vehicle corners.This study employed the 3D world box model which is estimated from 2D image coordinates with a relative height constraint to group the feature points of each individual vehicle for tracking process.Another useful method was proposed by Chunyu and Shihui (2012) for occlusion handling using skeleton features.This method used the morphological skeleton operation for analysis and to detect the occlusion effect between the blobs by extracting the skeleton nodes.These nodes are represented as features which are used to segment the obstructed vehicles in one blob with related concave spots.A multilevel framework consisting of three levels (intraframe, interface and tracking level) to detect the occluded vehicles is proposed by Wei et al. (2008).The vehicle occlusion is handled by evaluating the compactness ratio and interior distance ratio of vehicles in the intraframe level.In the second (interframe) level, the occlusion is detected and separated based on binary classification of motion vectors of vehicles which implements the modified subtractive clustering method.Finally, the tracking level performs the bidirectional occlusion reasoning algorithm to track the occluded vehicle throughout the video sequence.Also, the authors Chung-Lin and Wen-Chieh (2004) suggested improvements in three processes: occlusion detection, motion vector calibration and motion field clustering to separate the occluded vehicles.The occlusion is handled by removing the "cutting region" between the occluded vehicles.

PROPOSED METHOD
Figure 1 shows a block diagram of the proposed method based on the background subtraction method to detect shadow and handle the partial occlusion of moving vehicles.Background Subtraction (BS) is the most versatile and widely used computational method for the detection and tracking of moving objects in video surveillance.First of all, a post-processing mechanism is used to generate an empty background or so called reference image, which is further exploited for background construction or extraction (Hadi et al., 2014b;Zheng et al., 2006) following four phases.First, a set of RGB frame images are loaded and analyzed to three components namely, R, G and B. Second, these components are stored in vectors.Third, the most frequent values that construct an RGB image are located (the mode of pixel values for three components of RGB frame images).Finally, the background image is achieved by converting the resultant image to grayscale.
The proposed method begins with gray-scale conversion (reading frame by frame) of the RGB video image frames to gray-scale color space.In the second step, by using the foreground detection procedure, the background image is subtracted from the newly converted gray-scale frame to acquire the resultant image referred to as difference image.Then, the thresholding procedure is applied to the difference image to display the state of motion of the objects and to create the Foreground (FG) binary mask images.Finally, morphological binary operations such as dilation and erosion are employed to refine the FG images.After that, vehicle counting and labeling are carried out using a new model in two Dimensions (2D) to count the moving vehicles.The region growing method is utilized to segment the foreground pixels (vehicle) from the background pixels.Meanwhile, the

MOVING VEHICLES DETECTION
We adopt the proposed approach (Hadi et al., 2014b) for detecting and counting the moving vehicles.First, foreground detection is used to detect\extract the moving vehicles from the image sequence by subtracting the new frame pixels F img (x, y) from background scene pixels R img (x, y) (reference image).Then, the difference image D img (x, y) is the resultant image which will be compared with a pre-defined threshold (Th mv ).The pixels of difference image will be classified as a foreground object (moving vehicle) when its values are larger than (Th mv ); otherwise, it will represent background.After the thresholding process, the resultant image is called Foreground binary mask (FG) image and this image consists of foreground objects (white pixels) and background (black pixels).Meanwhile, the dilation and erosion morphological binary operations are applied on FG to highlight the foreground regions and to clear the background from spots or undesirable objects which influence the vehicle detection and tracking process for traffic surveillance (Fig. 2).In Fig. 3, 2D model is employed to count the vehicles and the region growing algorithm will be used to segment every numbered vehicle with extraction of the Feature Vector (FV) for each one (Hadi et al., 2014b).The FV represents the width and height coordinate features for each moving vehicle and it will be used for following shadow detection and partial occlusion handling processes.

SHADOW DETECTION
The shadow is detected via a two-steps process including Shadow Searching Window (SSW) and Fig. 4: Schematic of shadow detection module Shadow Region Segregation (SRS).Figure 4 represents the steps of this module.It determines the SSW for every segmented vehicle area which is used to detect and separate the shadow from the body region of the vehicle.
Shadow searching window (SSW): Accordingly, the searching window is determined by the segmented regions of the vehicle's dimension.The in Fig. 5 is constructed from eight directions in the frame because the projection of sunlight is blocked by the object.
Once the segmentation process is completed, the acquired feature vector consisting of width and height (W×H) dimensions of the vehicle region as shown in Fig. 6 is used as SSW.The searching procedure is further employed to detect shadow pixels from body pixels of vehicle's region regardless of the sunlight projection direction in video dataset.Here, the width difference is defined as δW = X max -X min X min implying the maximum and the minimum of pixel value along x-axis.Similarly, the height difference is denoted as δH = Y max -Y min Fig. 5: Construction of shadow Shadow Region Segregation (SRS).Figure 4 represents the steps of this module.It determines the SSW for every segmented vehicle area which is used to detect body region of the Accordingly, the searching window is determined by the segmented regions of the vehicle's dimension.The shade as shown directions in the projection of sunlight is blocked by Once the segmentation process is completed, the acquired feature vector consisting of width and height (W×H) dimensions of the vehicle region as shown in Fig. 6 is used as SSW.The searching procedure is further employed to detect shadow pixels from body pixels of vehicle's region regardless of the sunlight projection direction in video dataset.Here, the width min with X max and implying the maximum and the minimum position axis.Similarly, the height min with Y max and Y min indicating the maximum and the minimum position of pixel value along y-axis.

Shadow region segregation:
In the segmented (foreground/vehicle) region, the shade which is included within the entire object area requires isolation.Therefore, the following steps in the developed algorithm (Tian et al., 2012) are used to detect and segregate out the shade pixels from vehicle body pixels: • where, Ps (x, y) indicates the pixel of the segmented foreground region.These procedures are capable of producing two regions such as a big homogenous region and a shadow region depending on the number of Fs pixels greater or smaller than a specified threshold Th sh .Once the shadows are distinguished, it replaces the pixels with zero (background) values to remove it from the binary mask.Finally, the detected shadow regions are then isolated from foreground parts leaving only the moving objects (vehicles) areas as furnished in Algorithm1.Moreover, this algorithm can detect the shadow in any of the eight directions despite the direction of movement.

OCCLUSION HANDLING
Once the shadow detection for each vehicle is completed the next step is to handle the occlusion.The problem of partial occlusion or overlapping in highway traffic surveillance primarily originates from vehicle obstruction, the shadow cast by other vehicles and detection of two vehicles in close proximity that appear as one large object.The regions of vehicles may be merged or split from frame to frame.Hence, scenarios such as a single region in the current frame image might split into several regions in the next frame or various regions in the current frame image may merge into a single region.These two cases indicate that vehicles are separated and occluded.The merge can be found in three different states: • Vertical merges: For a vehicle which nearly merges with another vehicle located along the rear or front of it, • Horizontal merge: For a vehicle which merges with another vehicle located on the right or left side of it.• Diagonal merges: For a vehicle which merges diagonally with another vehicle located in the rear or front of it (Rad and Jamzad, 2005).
In this study, the problem of partial occlusion is handled and resolved using a new method.As the vehicles are homogenous objects with geometrical shapes, during the movement their sizes seem to gradually grow and appear larger due to angle and view capturing of the video camera.In addition, camera positioning and quality of the recording play a critical role in improving the vehicle detection and tracking processes.However, the new method used in this study relies on vehicle sizes only.The sizes of blobs those counted using the 2D vehicle counting model remain unknown.Meanwhile, the segmentation process provides the required and complete information or features regarding vehicle width, height and center.
It is essential to train the system about the sizes of vehicles.Following previous estimations based on the behavior of vehicles (sizes and trajectories in the dataset) the sizes are analyzed and calculated using the expressions: The coefficients of regular increase of the bounding box coordinates around the vehicle during counting and tracking processes are represented by the parameters a and b for the width and c and d for the height.Size of vehicles are calculated by taking the first frame in the counting process and the last frame in the tracking process as displayed in Fig. 7.These coefficients represent the types of vehicles such as Saloon, SUV, Van, to cite a few.In addition, the bounding box coordinates surrounding the vehicle are indicated by (nxu, nx2u, nyu and ny2u) and (nxt, nx2t, nyt and ny2t), where u and t signify counting and tracking, respectively.
Using this new method and calculated sizes, the partial occlusion arises in three different situations as stated before can be detected and isolated by surrounding the vehicles through the bounding boxes.This allows one to distinguish the vehicles individually via counting and tracking processes.
Algorithm 2 presents the various steps for occlusion handling.The estimated vehicle size can be compared with an obtained threshold from any specific kind of vehicle size computation.The value of the parameter occ state (occlusion state) is acquired from one of the determined values on Horizontal, Vertical and Diagonal directions.It is further used to indicate and distinguish the nature of partial occlusion which occurs between vehicles.Finally, the values of S width and S height are calculated.Furthermore, the parameters for occluded (ox, oy, ox2 and oy2) and non-occluded (nox, noy, nox2 and noy2) vehicle size coordinates are achieved for the tracking process.

EXPERMINATAL RESULTS
The proposed method has been tested on the baseline category of changedetection.net(CDNET) dataset acquiring highway video sequence (Goyette et al., 2012) and sample of frames of traffic video sequence (Kanhere and Birchfield, 2008;Kanhere et al., 2006).We have implemented the proposed method using Microsoft visual C# on a Pentium IV Core i3-2.13GHz Laptop with 4 GB RAMS.for some samples of shadow detection in multi directions.

No. of consecutive frames
The result of shadow detection is presented in Fig. 9 highlighting the shadow with a cyan color for better explanation.
The partial occlusion states may occur as a horizontal, vertical or diagonal merging of vehicles bodies.The key idea of this new module is to solve the partial occlusion by using the vehicles' sizes as a new contribution in the traffic monitoring systems for vehicles overlapping problems.
The vehicles' sizes are calculated manually and are used to train the system.For instance, the following equations describe the calculation of parameters: a, b, c and d which are previously mentioned in section (Occlusion Handling) and these parameters are used for calculating the vehicle sizes S width and S height (Saloon type) in lane one of road of highway video sequence (CDNET data set), as follows: Table 1 tabulates the kind of vehicles' sizes (Saloon, SUV and Van) in the highway video sequence (CDNET data set) with its bounding box coordinates (nxu, nx2u, nyu and ny2u) and (nxt, nx2t, nyt and ny2t) at counting and tracking frames that are represented as a feature vector.
When the vehicle parameters (a, b, c and d) (which are called the regular increasing coefficients of bounding box coordinates) are calculated for each type of vehicle, the vehicle sizes S width and S height will be known and it will distinguish each occluded vehicle from another by using the aforementioned coefficients.Thus, any segmented vehicle (width w and height h) features will be compared with the predefined vehicle's width and height thresholds for each vehicle type.For Next, the result of the occurrence of partial occlusion as a horizontal and diagonal view of the traffic road of the second data set is shown in Fig. 11 and 12, respectively.

CONCLUSION
This study proposed a new method for handling partial occlusion and to detect the shadow of highway vehicles as seen in the image sequence.First, by adopting the background subtraction method the moving vehicles are detected and the dilation and erosion morphological binary operations are used to refine the resultant FG.Second, the region growing algorithm is used to segment the foreground objects (vehicles) from the background and to extract the geometrical shape features (width and height dimensions) of each segmented vehicle.These features will be used as a Feature Vector (FV) to determine the SSW.Third, the chromatic based analysis technique uses the semi mean value of RGB color space of moving vehicle region pixels as a threshold value to detect the shadow region in any direction of sunlight projection with no human intervention.Finally, a new training procedure utilizes the vehicle sizes which are employed to handle the effect of partial occlusion (three states between the overlapped vehicles).Therefore, previous estimations are performed based on the behavior of moving vehicles (sizes and trajectories in the dataset) to yield the increasing regular coefficients, which are used to compute the size of moving vehicles and to separate the occluded vehicles by drawing the bounding boxes for each one.

Fig. 1 :
Fig. 1: Diagram of proposed method shadow of each vehicle is detected using a new chromatic analysis technique and partial occlusion is enabled to separate out the occluded vehicles by using a new vehicle size procedure during the detection of moving vehicles.

Fig. 6 :
Fig. 6: Features of shadow searching window • Obtain the threshold value (Th sh ) by dividing the mean values of 2, which represent the lower bound for the detection of the shadow region • Denote Sh (Th sh ) as the shadow region detected via Th sh to yield ˟ℎ{ˠℎ GB { = {P_{˲ ˳{ < ˠℎ GB } (2)

Algorithm 1 :
Shadow detection algorithm Input: Segmented vehicle region Output: Detecting shadow region of vehicle Begin: Set sum ← 0 \\ Summing of RGB intensities Set Th sh ← 0 \\ Threshold of shadow detecting Set co ← 0 Set ∆W ← Xmax-Xmin Set ∆H ← Ymax-Ymin for q = 1 → difference between Ymin and ∆H do for p = 1 → difference between Xmin and ∆W do if IFG (p, q) = 255 // White pixel co ← co + 1 set sum ← sum + RGB color space pixel values end if end for end for m ← sum / co Th sh ← m / 2 for b = 1 → difference between Ymin and ∆H do for a = 1 difference between Xmin and ∆W do if IFG (p, q) = 255 // white pixel if RGB color space pixel values < Th sh set X position ← a // Collecting the pixels` set Y position ← b // positions of vehicle end and store the RGB pixel values of vehicle region Th sh : Threshold used to detect the shadow of vehicle co : Store the white pixel numbers ∆W : Width of vehicle size ∆H : Height of vehicle size p, q, a and b : Counter variables Xmin, Ymin, Xmax and Ymax : The coordinate of vehicle size X position and Y position : Store the X and Y of detecting the shadow of vehicle region The width of the vehicle S height = The height of the vehicle

Algorithm 2 :
Fig. 7: Features of shadow searching window else if S vehicle = THV width and THV height Set OCC state ← "D" //diagonal partial occlusion end if Assign the parameters (a, b, c and d) new values of specified detected vehicles S width ← (a*ny2u) + b S height ← (c*ny2u) + d Set ox ← nxu // Gathering the vehicle size Set oy ← nyu // coordinate Set ox2 ← nx2u Set oy2 ← ny2u // Determine the non-occluded vehicle: If S vehicle != one of three partial occlusion states Assign the parameter vehicle parameters (a, b, c and d) new values of Specified detected vehicles S width ← (a*ny2u) + b S height ← (c*ny2u) + d Set nox ← nxu // Gathering the vehicle size Set noy ← nyu // coordinate Set nox2 ← nx2u Fig. 8: A step by step illustration of the shadow detection process; (a): original Image; (b): bounding box area (SSW); (c): shadow identification; (d): shadow removed The shadow occurred due to sunlight projection and changed by movement of object under different lighting conditions.This module can detect the shadow in any of eight directions without any heuristic training stage.The results in Fig.8demonstrated the steps of detection and segregation of a moving vehicle's shadow

Fig. 12 :
Fig. 12: Diagonal view of partial occlusion occurrence example, the vehicle of the saloon type has w = 37 and h = 30 pixels and it will be compared with the predefined thresholds (45 and 40); if w and h are larger than these thresholds, partial occlusion occurs or there is no partial occlusion state: If ((w>=45) or (h>=40)) then The partial occlusion is occurring Else This vehicle is not occluded by other Fig. 10 presents the result of the occurrence of partial occlusion as a vertical view of the traffic road of CDNET data set.Next, the result of the occurrence of partial occlusion as a horizontal and diagonal view of the traffic road of the second data set is shown in Fig.11and 12, respectively.

Table 1 :
Summary of parameters of vehicle's sizes coordinates