You Take Care of the Drive, I Take Care of the Rule: A Traffic-Rule Awareness System Using Vehicular Sensors and Mobile Phones

Traffic rules are used to regulate drivers' behaviours in modern traffic systems. In fact, all driving behaviours are presented by vehicles' behaviours. If vehicles have awareness of their behaviours, it is possible that traffic rules are able to regulate vehicles instead of drivers. There are three advantages of vehicle regulation: (1) without worrying about violations of traffic rules and searching for traffic signs, drivers can pay more attention on emergency situations, such as jaywalking. (2) Many traffic violations are due to attention distraction; machines do not have the attention issues; therefore they can provide good traffic-rule obeying. (3) New traffic rules can be spread and applied more quickly and effectively through the Internet or Vehicular Ad hoc NETworks (VANETs). In this paper, we propose a novel traffic-rule awareness system using vehicular sensors and mobile phones. It translates traffic rules into combinations of vehicular sensors, GPS device, and Geography Information System (GIS); the system can tell whether a driver violates the traffic rules and help him to amend his driving behaviour immediately. Experiments in real driving environments show that our system can be aware of the traffic rules accurately and immediately.


Introduction
During the year 2011, there were 11856 traffic accidents that happened in Sichuan province of China and 95% of the accidents are caused by traffic violations [1].If trafficviolation rate can be significantly decreased, a lot of lives can be saved from traffic accidents.In fact, most traffic violations are not on purpose; the reason of high trafficviolation rate is that the traffic rules are designed to regulate drivers' behaviours.As long as drivers are human beings, they will suffer from memory issues and attention distraction.Their memory and concentration will be severely affected by mood, alcohol, drugs, and environment.Even a short conversation during driving will distract drivers and cause unnecessary traffic violations.
Google self-driving car [2] is a good attempt to decrease the traffic-violation rate, because machines do not have the memory and concentration issues like humans.However, google self-driving car only focuses on self-driving; it has not yet taken traffic rules into consideration except for traffic lights.Some other works like [3] monitor dangerous driving behaviours like aggressive turns, acceleration, and braking and help drivers to correct these unsafe behaviours.However, they did not take traffic rules into consideration.Actually, even if driving behaviours are not aggressive, as long as the driver violates the traffic rule, there is still a big possibility that traffic accidents may happen.
If vehicles have awareness of whether they violate traffic rules, they can warn the drivers about the violations and help them to amend their driving behaviours.In fact, although it is a human who drives the vehicle, the driving behaviours are presented by vehicles' behaviours.No matter who drives a vehicle or how he drives a vehicle, the driving behaviours can be described by two factors of vehicle's behaviours: speed and direction.Traffic rules are location based; different location has different traffic rule; therefore location is the third factor to describe driving behaviours for traffic rules.For example, if we want to describe a traffic rule of which speed limit is 40 kilometers per hour and one way from south to north at road A, we can set the three factors as Table 1 shows.
Modern mobile phones are equipped with various sensors like accelerometer, gyroscope, and orientation.They also equipped with GPS localization devices.Accelerometer can be used to monitor the speed of a vehicle; gyroscope and orientation can be used to monitor the direction changes.GPS device can be used to monitor the location.It seems that mobile phones alone are enough to build the traffic-rule awareness system.In fact, traffic rules not only regulate the driving vehicles, but also regulate the parking vehicles.We need to know whether a vehicle is parking or not.No speed does not mean the vehicle is parking; it can be waiting for traffic lights or something else.We need to know whether the engine has been flamed out for a while to judge if the vehicle is parking, therefore we have to take advantage of vehicular sensors to monitor the engine states.
In this paper, we propose a novel traffic-rule awareness system.It translates traffic rules into combinations of the three aforementioned factors.It will judge whether there is the tendency of violating the traffic rules while driving and warn drivers of this tendency.Therefore, it will free drivers from searching for traffic signs and remembering all the traffic rules; drivers can put more energy on driving.Mobile phones can provide 2G/3G/4G data connection for downloading translated traffic-rule from remote servers or construct Mobile Ad-hoc NETwork (MANET) to get the traffic rules from neighbors.
The rest of paper is organized as follows.Section 2 describes the details of system design.Section 3 shows the experiment results.Related works and conclusions will be given in Sections 4 and 5.

System Design
As mentioned before, mobile phones alone can not meet all the requirements of the system; vehicular sensors are good complements.On-Board Diagnostics (OBD) is widely equipped in modern automobiles.The OBD implementations use a standardized digital communications port to provide real-time data in addition to a standardized series of diagnostic trouble codes [4], or DTCs, which allow one to rapidly identify and remedy malfunctions within the vehicle [5].With the help of OBD scanner, the values of vehicular sensors can be easily obtained by our system.

System Overview.
As Figure 1 shows, there are three modules in the system: communication module, process module, and alert module.Communication module is used to download traffic rules from remote server via internet or neighbors via MANET; also it provides download service for its neighbors.Process module contains two parts: information collecting part and information processing part.Information collecting part collects three kinds of information which are related to the vehicle: speed, direction, and location.Speed can be obtained from accelerometer of mobile phone; however, as mentioned before, we also need to know whether the vehicle is parking when the speed is zero.
Engine state is a way to judge the parking state, and it can be read from OBD scanner.ELM 327 Diagnostic Scanner [6] is a common off-the-shelf product which provides an interface to access the OBD system.The scanner uses a female 16-pin SAE J1962 connector to connect to the OBD system, and it uses Bluetooth to connect to a control device such as mobile phone.Gyroscope and orientation which are embedded in mobile phones are used to generate the direction information.GPS device which is embedded in mobile phone is used to obtain the location information.The information process part will compare the collected information with the traffic rules stored in DB to judge whether the driver violates the traffic rules.Once there is a trend of traffic violation, the violation alert module will warn the driver.

Traffic Rules Translation.
As mentioned before, vehicles' behaviours can be described by three factors: speed, direction, and location.In this research, we use a collection of triples TR (speed, direction, and location) to represent the traffic rules.Speed is a 2-tuple Speed (lower, upper); the lower and upper elements are the lower and upper bounds of an interval of speed limit.For example, for the highway, the speed limit is not less than 80 km/h and not more than 120 km/h, so the 2-tuple is Speed (80, 120).Direction is a 2-tuple Direction (road direction, turns); road direction is the direction allowed on the road and it is a set of five values: NORTH, EAST, WEST, SOUTH and TWOWAY.If the road is one way street, road direction gives the heading direction.If the road is two way street, road direction gives the TWOWAY value.Turns is a quadruples Turn (Left, Right, U, Lane Change), each element indicates whether this kind of turn is allowed.Location is a 2-tuple Location (road, area).
Element road is the road name on the map; area is a 2-tuple area (from, to) which indicate the area of road applying this traffic rule.From and to are GPS coordinates.For example, as Figure 2 shows, Chengdu road is an one-way street and the speed limit is not more than 40 km/h.Between Locations 2 and 3, vehicles can choose to turn right or go straight.The traffic rules at Chengdu road can be translated to three TR (speed, direction, location) triples as Table 2 shows.We can see that the lower bound of Speed (lower, upper) is 1; the reason is that parking on the road is not allowed.If a location allows parking, the lower bound of speed will be set to zero.

Information Collection.
There are three kinds of information need to be collected: speed, direction and location.
The process module uses OBD parameter IDs (PIDs) codes to request data from a vehicle.SAE standard J/1979 defines many PIDs, but manufacturers also define many more PIDs specific to their vehicles [4].In this research, we use standard PIDs for good compatibility.
2.3.1.Speed.Speed can be calculated through GPS coordinates or accelerometer, but a more easier way is that sending a PID code VEHICLE SPEED to OBD scanner, the OBD system will return the vehicle speed.However, speed is not enough for traffic-rule translation; we also need to know the engine states.To detect whether the vehicle is parking, we need to know whether the engine is off for a period of time.There is an OBD PID code ENGINE RPM which indicate the rotation speed of engine, if the engine is off, the rotation speed is zero.Therefore, a vehicle is parking when VEHICLE SPEED is zero and ENGINE RPM is zero for a period of time.As (1) shows, only when the vehicle is parking, we set the speed of the vehicle to zero, otherwise the speed is larger than zero:

Direction.
To detect the direction, we need three kinds of sensors: accelerometer, orientation, and gyroscope.Before discussing about how to collect direction information, we will first discuss the coordinate system of mobile phone.As Figure 3 shows, the x-axis is horizontal and points to the right, the y-axis is vertical and points up, and the z-axis points towards the outside of the front face of the screen [7].Data from all the sensors embedded in mobile phone is threedimensional and obeying this coordinate system.
To obtain the value of road direction, we need to use orientation sensor.The data from orientation sensor follows three dimensions: Azimuth (degrees of rotation around the z-axis); Pitch (degrees of rotation around the x-axis); Roll (degrees of rotation around the y-axis) [8].In this research, we fix the mobile phone head forward and flatwise; therefore, Azimuth is the degree we needed.Table 3 shows the mapping relation between Azimuth and road direction.
We can use accelerometer to detect the turn event of driving behaviour, as Figure 4 shows; three kinds of turns, lane change, left turn and U turn are easily distinguished from each other.There are many works discussed about the driving patterns and gestures recognition [9][10][11].However, we cannot tell the driver: "You cannot turn left" when he already turned left.We have to make prediction of drivers' turning intentions.Gyroscope is a very sensitive device to detect the rotation action; we can recognize the turning intentions at the very beginning of drivers' turning International Journal of Distributed Sensor Networks  behaviours by taking full advantage of gyroscope.Detailed discussion is in the next section.

Location.
Modern mobile phones are equipped with GPS devices; therefore, the location of vehicles can be easily obtained.With the help of Geography Information System (GIS), we can locate the vehicle on the map by inputting GPS coordinates.

Challenges.
The main role that the traffic-rule awareness system plays is to warn drivers about the possible future traffic violations.Therefore, different from other works of recognizing driving patterns which are already happened, the system must predict drivers' intentions.Existing researches in prediction of driving intentions are mostly using cameras which would bring inconvenience and privacy issue to the drivers.In this research, we only take advantage of sensors from mobile phones to achieve the prediction.

How to Predict Drivers' Intention?
According to velocity formula v = v 0 + at, to predict the velocity of a vehicle, we need to know current velocity v 0 and the acceleration a. v 0 can be obtained through OBD system a can be obtained through accelerometer; therefore, v can be predicted.In the coordinate system of mobile phone, data from accelerometer are three dimensions: A x (acceleration at xaxis), A y (acceleration at y-axis), and A z (acceleration at zaxis).Acceleration can be calculated as (2) shows.To predict the velocity v, we need to average a, as (3) shows, to reduce measurement error.t 0 is the current time and t is the time interval of prediction.Equation ( 4) gives the final prediction.Once the prediction v does not belong to Speed (lower, upper), a warning will be alerted: Predicting the turning event is a great challenge.Firstly, turning event is a short-time process; the prediction must be made at the very beginning to give enough response time for drivers.Secondly, normal driving behaviours contain a lot of turning actions, such as dodging and parking.To distinguish these turning actions from real turning event, we have to take advantage of turn signal.Turning on turn signal is a standard process in turning event; every time drivers want to make turns, they will firstly turn on the turn signal.Therefore, if we can detect the turn signal, we can recognize the real turning event.

How to Detect the Right/Left
Turn Signal?OBD system is no doubt the first choice of detecting the left/right turn signal; it can accurately tell whether the left or right turn signal is on.For example, Chevrolet Corvette OBD-II Codes contain the codes to return the information of left and right turn signals [12].However, the OBD codes are not the standard OBD codes; they are exclusive to a certain vehicle manufacturer.To find a more compatible way to recognize the turn signal, we use a method called Sound Cross-Correlation (SCC).
The main idea of SCC is that the sound features of turn signal are precaptured; every time a sound signal received, the system will cross-correlate the received sound signal with the sound of turn signal.When a spike occurs in the correlation, it means THAT turn signal is in the received sound signal.Figure 5 shows the sound wave feature of turn signal.Cross-correlation is a widely used method in recent research works [13].Suppose the known symbol pattern of sound wave of turn signal is ts of length L, X(n) is the complex number representing the nth received symbol, then the cross-correlation at a shift position p is ts * is the complex conjugate of ts and C(ts, X, p) is the correlation coefficient.Once the X[n] is aligned with ts, there will be a sudden spike in the correlation as Figure 6 shows.SCC cannot distinguish right turn and left turn; however, it will not affect the recognition of left turn and right turn in out system, because turn signal is only used for detecting drivers' turning intention.

How to Distinguish Lane Change, Left Turn, and U
Turn?Only if we can hack in drivers' brains, otherwise we cannot know whether they want to turn left or change lane or make U turn in advance.Now the turn signal is detected, the driver's turning intention is known, and the problem is there are three kinds of turning; all of them are related to the traffic rules; therefore the three kinds of turning must be distinguished.Only when the driver makes turns, we can know what kind of turns he makes.If we recognize the kind of turning after the turning is finished like the pattern recognition or gesture recognition does, it is meaningless to issue a warning to the driver after the traffic violation.The challenge is that the recognition must be finished at the very beginning of the turning behaviour.
The movement of turning can be divided into two kinds of movements: translation movement and spin movement.Spin movement is the key factor to distinguish turning style.Accelerometer measures the acceleration of vehicles; it can be used to detect the translation movement, but it can not detect the spin movement; therefore it is not suitable for turning detection.Gyroscope is a sensitive device which can detect angular speed in three dimensions according to the coordinate system of mobile phone.If the mobile phone is fixed flatwise, the rotation around z-axis can be used to reflect the spin movement; therefore gyroscope can be used to distinguish the turning style.
Figure 7 shows the gyroscope readings of lane change (from right to left), left turn, and U turn (from right to left); we choose the Azimuth to reflect the spin movement.We can see that the first positive peak of three turning style  all appears at the beginning (there is a process of approaching to right side before U turn, so the first positive peak appears a little late).The peak represents the start of spin movement of the three turning styles, and the three turning style have different peak intervals.The peak interval of lane change is (0.5, 0.7); the peak interval of left turn is (0.7, 0.9); the peak interval of U turn is (1.2, 1.6).Therefore, we can distinguish the three turning styles at the beginning of turning by which intervals of the first positive peak belong to.

Implementation and Evaluation
The experiment evaluates three aspects of the system: turn signal detection, turning event prediction, and traffic rules awareness.All the experiments are running in real driving environment.The automobile we use is Buick Regal which provides a standard OBD system.The mobile phone we use is Lenovo S2 which has 1 GB ram and 1.4 GHz CPU core International Journal of Distributed Sensor Networks frequency which can provide enough processing ability, also the phone has all the sensors we need including orientation sensor, acceleration sensor, and gyroscope sensor.Loud and noisy driving environment is bad for driving safety; therefore, in most cases, driving environment is quiet and suitable for turn signal recognition.

Turning Event Prediction.
We test the turning event prediction in the campus.Once the system recognizes any of three turning patterns, it will give a warning and the driver stops the car.Figure 9 shows the results of experiments; we can see that the car stops at the very beginning of turning movement and gives the driver chances to amend their driving behaviours.

Traffic Rules Awareness.
We choose an area in the campus which is sparsely populated and convenient for us to do the tests.As Figure 10 shows, we choose four road segments and label them with four colors: red, green, blue, and yellow.Red road is for testing lane change and left turn restriction; green road is for testing U turn and one-way restriction; blue road is for testing speed limit; yellow road is for testing parking restriction.The traffic rules are listed as Table 4 shows.
The results show that our system has good awareness of traffic rules; however, there are a few misjudgments during the tests; when we do the same tests again at another time, misjudgments disappear.The reason probably is that the accuracy of GPS localization is sometimes not good; it will cause a vehicle at road B violates a rule at a neighbor road A.

Intentions Prediction.
Most researches use vision-based method to recognize body gestures to predict driving intentions.[14] uses camera to capture body pose to predict turning intention; [15] uses camera to capture head pose judge whether the driver focuses on driving; [16] generates 3D visions to assist driving.[17] establishes a probabilistic model to analyse and predict driving intentions to assist brake controls.

Behaviours Recognition.
Driving behaviours recognition can be used to detect some dangerous behaviours, like drunk drive [18].Most recognition works are using accelerometer [19,20]; it is enough for driving pattern recognition, but not enough to describe more complex driving behaviours.
[3] takes advantage of more sensors: gyroscope, magnetometer, to detect more complex and dangerous driving behaviours.

Traffic Lights.
Traffic lights detection is related to traffic rules; [21] uses camera from mobile phone to detect and predict the traffic light schedule.It is a complement to our research.

Conclusion
Traffic rules regulate drivers' behaviours so that there could be fewer traffic accidents and fewer lost lives.However, a human is not a machine; he has memory and attention troubles, he is easily affected by many things; therefore traffic violation rates never come down too much.Modern vehicles already have powerful processing ability and various sensors for sensing the world; they can be aware of the traffic rules for human so that drivers can focus on dealing with emergency situations such as jaywalking; it will no doubt significantly decrease traffic-accident rates and save thousands of lives.However, how to predict drivers' intentions as soon as possible and improve localization accuracy will be the future works.

Figure 3 :
Figure 3: Coordinate system of mobile phone.

Figure 5 :
Figure 5: Sound wave of turn signal.

Figure 7 :
Figure 7: Gyroscope Azimuth readings of three turning patterns.

3. 1 .
Turn Signal Detection.Cross-correlation can well recognize a signal in a mixed signal; however, it has difficulties in recognizing when SINR is low; here we treat the sound wave of turn signal as Signal, the others as Interference and Noise.We test the cross correlation in three different environments: quiet, talking, and playing music.

Figure 8
shows the results of cross correlation in the three environments.We can see from Figure8(a) that cross correlation can well recognize turn signal in a quiet environment; four separate spikes indicate four turn signals; in a talking environment as Figure8(b) shows, human voices sometimes submerge the turn signals and make the recognition impossible.However, talking is a discrete process, it has nontalking gap for us to recognize the turn signal, and there is a spike at the beginning which indicates the existence of turn signal.Other spikes are caused by interference which must be excluded; the spikes are usually mixed together; therefore we choose the separated spikes as the indication of turn signal; Figure8(c)shows the cross correlation in a music playing environment; we can see that there is no separated spike; therefore recognition of turn signal is almost impossible in a loud and continuous interfering environment.

Figure 8 :
Figure 8: Cross-correlation of turn signal in different environments.

Figure 9 :
Figure 9: Tests of turning event prediction.

Table 1 :
Three factors of traffic rule.

Table 3 :
Mapping relation between Azimuth and road direction.

Table 4 :
Traffic rules for tests.