Efficient Drowsiness Detection by Facial Features Monitoring

: With increase in technology fatigue detection systems with more accuracy are overcoming their previous versions. The main focus of these systems is on robustness, accuracy and cost. Based on these factors this study presents a driver fatigue detection system design. This design uses facial features (eyes and mouth) to determine driver’s vigilance. A hybrid of two commonly known techniques Viola Jones and skin color detection is used as detection technique. Lastly some experimental results are given showing the accuracy and robustness of the proposed system.


INTRODUCTION
Driving with drowsiness is one of the major reasons causing traffic accidents.Driver's fatigue not only affects the alertness and response time of the driver but it also increases the chance of being involved in car accidents.Statistics show that between 10 and 20% of the traffic accidents are due to driver's fatigue or diminished vigilance level (Bergasa et al., 2004).The National Highway Traffic Safety Administration (NHTSA) estimates that 100000 police-reported crashes are directly caused by driver fatigue each year, which result in an estimated 1550 deaths, 71000 injuries (NHTSA).This problem has increased the need of developing active safety systems inside vehicle that monitor driver's level of vigilance and alert drivers after detecting their drowsiness.Researchers have been developing many techniques for drowsiness detection which can be broadly divided into three major categories.

Driver's physiological parameters based fatigue detection:
In this approach detection of drowsiness is based on driver's physiological parameters like recording brain activates in alpha and gamma band EEG (Electroencephalography) (Ming et al., 2010;Garces and Laciar, 2010), heart rate variability (Vicente et al., 2011), pulse rate and respiration.However sensitivity of physiological approach is still low compared with visual approach because external devices like electrodes attachment to the driver are required.Application of electrodes on driver's body usually proves to be an annoyance and is in most situations impractical.

Fatigue detection based on driving performance:
Second approach is driving behavior, in which driver's fatigue can be detected by analyzing the transportation hardware systems, such as changes of vehicle lateral position, longitudinal and lateral speeds, accelerations, steering wheel angle (Takei and Furukawa, 2005;Yulan et al., 2007), braking and turning angle, etc. Different sensors are used to acquire such data from vehicles and roads.However, long time driving can result in perspiration on the sensors, decreasing their ability to monitor accurately.

Fatigue detection from facial expression:
Visual behaviors that reflect driver's level of drowsiness include slow eyelid movement, smaller degree of eye openness, sluggish in facial expression, yawning, sagging posture and frequent nodding.Using a video camera to monitor driver's fatigue by analyzing yawning and eye closure (Abtahi et al., 2011;Hariri et al., 2011;Omidyeganeh et al., 2011b) become more practical and popular with the rapid development of camera and computer vision technologies.
Numerous results demonstrates that facial expressions provides significant information about driver's fatigue (Ingre et al., 2006) and that if such visual behavior can be detected then it will be possible to predict driver's level of vigilance.Our proposed system integrates yawning feature along with the eye state analysis to detect the fatigue level of driver, using hybrid method of Viola Jones and skin color detection techniques for face detection, which is the first step towards yawning and eyes detection.

LITERATURE REVIEW
The work in Omidyeganeh et al. (2011a) proposed the detection of face region using skin color detection technique.SSIM (Structural Similarity Measure) technique is then used to detect eyes and mouth map based on YCbCr is then generated to locate mouth and yawning in detected face.In Yufeng and Zengcai (2007) face area is detected using difference image between two images.The yawning of driver is then detected by measuring the distance between the midpoint of nostrils and chin.The work in Weiwei et al. (2010) detects the face using skin color properties and grey image linearization method and use PERCLOS principle to detect rate of eyes blinks.Mouth is detected based on geometrical feature of mouth by using a threshold value.Wu et al. (2010) Detects the face using Harr-like features and adaboost algorithm and then use Radial symmetry and distance transform along with Support Vector Machine (SVM) classifier to detect status.Azim et al. (2009) Presents a design where face is located through Viola Jones face detection method in a video frame.A mouth window is then extracted from the detected face area for lips detection through spatial Fuzzy c Means (s-FCM) clustering.

Genetic algorithm:
The fatigue detection system consists of 4 levels which can be classified as: • Face detection • Eyes and mouth detection • Yawning detection • Drowsiness detection using eye state analysis Face detection: Face detection is the first step of fatigue detection.So far, many algorithms has been presented for face detection, in which Viola Jones and Skin color pixel detection are most commonly used.In this design, we used hybrid design of Skin color detection and Viola johns (Viola and Jones, 2001) techniques for better accuracy and more robustness.Skin color pixel detection technique is used to extract the entire skin color pixels from the image.The system then, applies Viola Johns method on extracted skin color pixels to detect face area.This increases the efficiency of Viola Jones technique and decreases the processing time.

Eyes and mouth detection:
In second step the detected face area is extracted from input image for further processing.The system divides the extracted face image into three sub portions upper left half, upper right half and lower half.
These divisions are made on the basis of physical approximation of eyes and mouth locations.Figure 1 shows the process done in second phase.This helps in phase three in which Viola Jones is separately applied on image sub portions to detect eyes and mouth.
Yawning detection: Yawning state can be detected by measuring the height of mouth.When a person is in yawning state, height of his mouth increases by a specific amount.By obtaining normal height of mouth and a threshold value, they can be used to compare and determine wither mouth is in yawning state or not.Every human has his own facial features.To obtain reference of driver a data base of fifty frames is obtained.This data base is refreshed by new data after every one thousand frames.Hence in first fifty frames the detected information of mouth i.e., height of mouth is saved and average of these heights of mouth is obtained.The average will give normal state height of mouth.To check yawning state a threshold value is used which can be calculated by using Eq. ( 1) and the condition of yawning state can be mathematically expressed in Eq. ( 2): Threshold IHB = Mouth height -JA @ '" @F ?G + Mouth height -JA @ '" @F ?G 3 9 (1) Not Yawning other wise (2) Fig. 1: Face image division based on physical approximated location of eyes and mouth Yawning state is detected in different experimentation using threshold, one is shown with experimental details in Table 1.

Drowsiness detection using eye state analysis:
In case of eye, after detection comes the part to determine wither eyes are open or closed.Since every person has his own facial features so it becomes difficult to determine eyes status by using general data.To do so the system detected eyes information from first fifty frames of every coming one thousand frames.The detected eye region is extracted and converted in binary image.Then the number of black and white pixels in that region are calculated and stored in data.In case of open eyes the ratio of black pixels in the region is greater than in case of closed eyes.Since blinking is negligible, the data of first fifty frames can provide the approximate ratio of black and white pixel in case of open eyes.This approximate ratio is used to calculate a threshold value which can be used in coming frames to determine state of eyes.
The mathematical expression to calculate threshold value is: Threshold ?M?G = black white JA @ '" @F ?G − black white JA @ '" @F ?G 3 : (3) The condition to determine eye status can also be mathematically represented as: Closedif black white ?M?F?AC < Threshold ?M?G Open eyes other wise (4) Experiments were done on images of different persons proving this condition.One image with experimental details is given Table 2. Once closed eyes are detected, their blinking frequency and time in closed state can be monitored.This will help in alerting driver just before micro sleep.A threshold value of closed time and blinking frequency of eyes is set and if the system detects blinking frequency or time of closed state greater than threshold values, it will consider driver to be drowsy.

NUMERICAL RESULTS
The system parameters used for experimentation are given in Table 3. Outcomes of three different test videos with 985, 890 and 1232 frames are shown in Fig. 2. The frame rate of each video is 3 msec.Figure 2 also shows average time consumed in processing of each frame and drowsiness detection accuracy of system for the input video.

CONCLUSION
This study describes an accurate and robust fatigue detection system which detects the fatigue level by analyzing yawning and eyes status.A hybrid approach of Viola Johns and skin color pixels detection method is used to detect the face accurately.The detected face area is then divided into three parts based on physical approximated location and Viola Johns method is applied on each part of the image to detect eyes and mouth efficiently, in detected face area.To analyze the status of eyes and yawning, a threshold value is calculated from first fifty frames of the video, individually and then the current status of eyes and mouth is compared to corresponding threshold values, that describe the vigilance level of the driver.The system uses VGA camera and works accurately during daylight.The design can easily be implemented on FPGA as well.The system can be used to increase road side safety at day time while with using night vision camera and some further improvements, fatigue level of driver can be observed at night as well.

Fig. 2 :
Fig. 2: (a) 985 frames in video the average time consumed by each frame is 0.0349 sec with system accuracy of 95.6,(b) input video frames are 890; average time consumed per frame is 0.0298 sec and accuracy of system is 96.2, (c) number of frames are 1232 and each frame is processed in average of 0.0314 sec with accuracy of 95.8

Table 1 :
Yawning state detection using mouth threshold

Table 2 :
Closed eye detection using eye threshold

Table 3 :
System parameters