Development of a Smartphone Application for Safe Car Driving Using Google API and Built-in Sensor

In the world where streets brimmed with vehicles are no longer an foreign sight, vehicle accidents come along as one of unwanted consequences. Accidents are due to many reasons, though however the reason is, most of the times it leads to damaged vehicle(s), injuries or even fatality. In this research, we present Car Safe, a smartphone application which can make phone call reception decision, text readout to prevent distraction, driving time alert and send the location of the driver upon the occurrence of a detected crash. While the Google Activity Recognition API only detects driving state approximately 89% of the time, our proposed method yields more than 93% detection. Keywords—Smartphone application; Car Safe; sensor; detection


Introduction
Traffic accidents have always been a thorny problem in recent decades and will still be in the foreseeable future. According to WHO's report in 2018 [1], nearly 1.35 million people die in road crashes each year, meaning approximately 3700 deaths per day on average along with 20-50 million injured, disabled. With an average of 27.5 deaths per 100,000 populations, low-income countries pose a threefold higher risk than that of high-income countries (8.3 deaths per 100,000 population). Traffic accidents clearly do not just affect the live of people, but also threaten the economy and human resources, given that the injuries from traffic accidents has become the leading cause of death for children and young adult aged from 5 to 29 (more than even HIV/AIDS, tuberculosis or diarrheal diseases). Though, observation [2] shows that political commitment and financial investment for this issue is no where near what have been made to combat these afore-mentioned diseases (919-985 million USD of research and development funding for HIV compare to 24-33 million USD for traffic crashes [3]). Moreover, traffic injuries are predicted to be #3 DALYs (Disabilityadjusted life year) for the 10 leading causes of the global burden of disease in 2020, climbing from its former #9 position in 1990 [4].
In Figure 1, the percentage of death by different road user types [1] is presented:

Fig. 1. Percentage of different road user type worldwide
As seen in Figure 1, although car is usually considered a safer mean of road participation than walking or 2-and 3-wheel vehicles, the percentage of car occupants' death is still slightly higher than the rest. Having acknowledged the issue, we chose car driver's well-being as the target of this research and development.
There are a number of significant reasons that often lead to traffic crash: bad weather, speeding, drunk driving, fatigue, mobile phone distraction, etc. Since some of these causes are not systematically reported in crash reports in most countries and witness statement is also sometimes unreliable, it can be troublesome to draw a thorough conclusion of each cause's proportion in the accident chart. Given the previously mentioned causes, it is both essential and possible to prevent and diminish the consequences of an accident through numerous ways: following the safety instruction, raising awareness and education or through explicit law enforcement, etc. The approach in this paper is focused on allowing application to take control of some smartphone features, given that distraction by phone is one of the major reasons that lead to car crash but it also can also provide services to aid safe driving and notify when accident occurs. The application offers utilities such as driving state recognition, making phone call reception decision, read out text message, drive time alert, accident detection and accident location notification to a preset emergency list.

Related Work
A number of smartphone applications have been developed for the purpose of safe driving. Apple Carplay [5] and Google Android Auto [6] are great applications that utilize voice control through AI assistants to help control the phone utilities, therefore make driving easier and safer. However, there are problems concerning these two applications: they are mostly compatible with high-end cars and latest models of phone. While some middle-level cars are capable of using them, it is only 1-year free trial and later on, if the driver wants to keep using, a fee is charged annually.
Not every application offers automatic Bluetooth switching like DashDroid [7] through driving state recognition, but still it lacks the ability to give warning about the accident that happens to the driver or provide vocal reading with each received message. With a different orientation from DashDroid, DriveSafe [8] focuses on auto messages and call receiving as its main features. There are a number of other applications similar to the two mentioned above which block incoming calls. In general, that feature reduces the distraction, however under some circumstances there are exceptions that the driver should answer the call, for examples to receive urgent news or the call is from an exceptionally important person.
There also have been studies related to safe driving in recent year, which mostly concentrate on the use of smartphone services, IMU including car built-in sensors or phone built-in sensors. RK Megalingam et al. utilized car built-in impact sensor composed of accelerometers ADXL 335 in their work [9], but due to the fact that they focused on building a communication network through radio frequency (RF), the car accident recognition was only performed by the feature of exploding airbag, which set some flag bits of microcontroller to notify. Simply put, it proves to depend wholly on the vehicle's crash detection. Moreover, the work requires a dedicated hardware setup, which is difficult for combination with other features and is often inflexible.
Unlike RK Megalignam et al., J Zaldivar et al. [10] approached by developing an Android smartphone application that Bluetooth-communicating with car system with On-board Diagnostic II (ODB-II) Standard for monitoring and accident detection. However, they could only handle at most 4 sensors a time to maintain per-sensor sampling rate above one per second. Their proposed accident detection harnessed airbag explosion recognition; acceleration derived from both smartphone sensor, vehicle sensor, GPS information and set a threshold. P. Parmar and M. Sapkal [11] used the combination of IMU and smart phone for the same purpose. By using gyroscopes and an accelerometer processed by MPU6050, they can detect through threshold-based method not just crash, but also rollover, whose information of location is then sent to the nearest hospital for approval. Though it shows a promising result of accident detection, the whole system setup requires both IMU and smartphone, which already has its own built-in sensors. It is also impractical in terms of waiting approval from the hospital before sending the alert to family members, which should be simultaneous in case of non-immediate response from any medical service. I.A. Zualkernan et al. published their research [12] using artificial-intelligencebased classifier for 2 types of accident, which are car to non-deformable, rigid object or car to human(s). This research applied Discrete Wavelet Transform (DWT) to features extracted and for noise suppression, therefore resulted in improved classification due to better defined features. The DWT-coded data is then processed with many classifiers: Naïve Bayes, Bayesian Networks, Logistic Regression, Random Forest, REPTree and J48. The conclusion was drawn as Naïve Bayes, Random Forest and Logistic Regression showed the best performance. These fore-mentioned algorithms were applied in an application named Crash Detech, which can send data of location, type of accident to local authority, Emergency Management System (EMS) and user's emergency contacts via SMS. Their work excels in dealing with false positives such as phone drop, sudden change of direction or sudden brake, however, it requires a huge amount of training, computation and numerous second, third-world countries do not offer sufficient EMS or infra-structures.
J. White et al. presented WreckWatch, an application available on Android platform that acts like an event recorder, which too can detect accident and alert to emergency contacts in their work [13]. WreckWatch consists of 2 elements: server and client. The clientinstalled on Android Smartphoneutilizes built-in GPS receiver, built-in accelerometer and sound recorder for accident detection, showing acceleration, speed, route and location of user. Their algorithm is superior in terms of harnessing multiple parameters (11-tuple model of phone state) for detection, which is triggered if one of two conditions is met:  A high acceleration event and a high sound event are recorded while the velocity is larger than the threshold speed  The distance moved since the last time the speed threshold was exceeded is less than distance threshold at the same time as an acceleration, sound event happen WreckWatch can aggregate imagery, voice over internet protocol (VOIP) or by people nearby to send to server then the emergency responder(s). Despite its amazing features, WreckWatch does have drawbacks due to large battery consumption, intensive computation, possible deactivation while driving slowly or smoothly, requirement of WreckWatch installation on nearby people phones and of cellular data for emergency responder(s) while using some utilities. Most of researches and applications until now do not recognize user's current activity to automatically toggle the utilities, therefore battery is depleted quicker. Furthermore, the majority of proposed applications are not available on Google's Play Store or App Store by Apple, thus making it harder for users/customer to approach.
Developed by Dynamus Technologies, CrashDetech [14] is an excellent application for post-crash solution. Their work is portrayed in 3 words: monitor (drive detection technology), detect (detect crash) and respond (respond to the accident). Their strong side is that they are able to collaborate with not just medical service to dispatch them in need, but also with insurance service, legal assistance and road protection. However, the application services are only available in South Africa, and monthly fee is charged if the user wish to use more than just crash alert in the free version, ranging from about 3 USD classic subscription to more than 7 USD elite subscription.
Having acknowledged the insufficiencies of other applications, we developed Car Safe, which is compatible with every genre of Android Smartphone and uses GAR_API (Google Activity Recognition Application Programming Interfaces) along with built-in sensors. A number of utilities are activated after the user is detected to be driving car.

3
Proposed Application

Proposed algorithm
In Car Safe, we utilize Google API, including Google Text-to-Speech [15], Google Speech-to-Text [16], Google Location Service [17] and Google Activity Recognition [18] along with the built-in in order to combine many features for one ultimate purpose: safety of the driver and passengers. The features presented in our application are reading out loud messages (SMS, email, Facebook Message), making decision on receiving the call or not, warning when the driver has driven too long without rest or is weaving, detecting when accident happens and inform people from VIP list of the driver's location. Google Text-to-Speech and Speech-to-Text are used for message vocal reading and vocal control. Google Location Service helps detecting the location of user, but is used only when accident happens and the information is needed to be sent to those it may concerns. The reason behind Google Location Service selectedtime use is to conserve the battery power. The last and most important API is Google Activity Recognition, which participates in detecting car-driving state that triggers other utilities. Built-in accelerometer is used to recognize an accident happens. In this development, we design Car Safe to work mainly with phone being held on phone holder.
Google Activity Recognition API is developed to detect activities based on data collected from built-in sensors of mobile phone. With provided data, GAR_API returns a list of possible actions that may be happening along with their confidence, since until now, activity recognition has only been able to give a potential result in real life or a precise one given a limited set of test activities. The confidence of each returned activity ranges from 0 to 100: the higher it gets, the more likely that it is the action being taken. At the moment, GAR_API can detect up to 8 different activities, which are in-vehicle, on-bicycle, on-foot, running, staying still, titling, walking and the rest are marked unknown. In Car Safe, the sampling rate is set at 50Hz, which indicates there are 50 samples recorded each second. Each sample is distinguished into 2 types which indicate either the user is driving or not based on the confidence. If the confidence of vehicle driving state is larger than 70, it is then considered driving state, else it indicates the user is not driving. Below is flowchart of our enhanced driving recognition based on GAR_API shown in Figure 2: The user is driving The user is not driving N Y

Fig. 2. Flowchart of Car Safe's enhanced driving recognition based on GAR_API
In order to improve the performance, we process activity recognition by assenting to the majority of every 3 consecutive samples as each refers to the most likely current state of the user (driving car or not). We also consider circumstances which user stops for red light or is driving too smoothly, which may cause the utilities to cease working. In that case, Car Safe is set to continue until it confirms again 2 minutes later if the user is still driving or not. If the user is still driving, the state remains unchanged, utilities keep functioning and vice versa. Figure 3 summaries our application's utilities in a flowchart as below:  Fig. 3. Flowchart of Car Safe's utilities iJIM -Vol. 14, No. 2, 2020 The explanation of each feature is presented in the following sub-sections.

Message readout
By registering for Broadcast Receiver, which is an Android component that allow the application acknowledge Intents, Car Safe can "listen" and process incoming SMS. A SMS consists of the content and phone number of sender, which is compared to phone contacts through Content Provider in case the message is from a known contact. Facebook message and email are trickier to handle. Facebook Graph API once gave the permission to read mailbox, however, it is no longer supported since 2015 [18] due to new privacy policy. Owing the fact, automatical read-out-loud-Facebook-message rarely features among applications. In our study, we propose to extract the content through on-screen notifications. Unlike Facebook message, Google does provide support for email reading, however it requires username and password input, which may prove to be inconvenient and affect user's privacy. Therefore, email is read by the same method applied to Facebook message. To achieve content through on-screen notifications, Notification Listener Service is utilized. Notification Listener Service can return packet name in order to know whence the message is achieved ('com.facebook.orca' for Facebook or 'com.google.android.gmail' for email), title (name of sender defined during email registration or Facebook conversation) and text (content of the message) along with other information. Upon receiving the message, Car Safe automatically processes it and then reads out loud the user without a need for manual instruction.

Drive time warning
In EU, a break period of 45 minutes should be taken after 4.5 hours driving [20]. As in other regions, the same term may not be explicitly mentioned in any regulation, therefore regularly, it depends solely on driver's awareness. Statistics [21] show that driver fatigue is a problem responsible for up to 20% of traffic accidents. This type of accident often cause more serious injuries or death than others since a driver who passes out or lose concentration due to tiredness cannot brake or avoid the direct impact of the crash. In Car Safe, user is allowed to set the driving time as in Figure 5 below: The default driving time threshold is set to be 3 hours, which can be manually adjusted in the application as the user wishes. While participating in traffic, relentless driving time excludes and does not reset at red light stops, extremely short stops or false positive detection while still driving. To detect these fore-mentioned unqualified stops, we consider every stop less than 5 minutes, which is especially short to driving time threshold and most of the time larger than any red light stop, to be ineligible and do not reset the driving time. If the driving time exceeds the threshold, an on-screen warning is shown along with strong vibration to remind the user to take rest.

VIP list and call reception decision
Car Safe allows user to set up a VIP list in the settings that composes of contact name and phone number as below:

Fig. 5. VIP list setup
Inevitably, contact read must by allowed for the setup completion. By tapping Setup, user is taken to the VIP list screen which is empty at default, which then can be added directly from his/her contact list. When a call comes, it is firstly checked whether the phone number is from VIP list for further actions. If the contact is VIP listed, the call is then received. Otherwise, the call is neglected and an email saying "I'm driving. I will contact you later. Sent from Car Safe" is sent unless the same number has been dialing 3 times in a predefined time, which we consider to be urgent and be a temporary VIP.

Crash detection and post-crash aid
Car Safe does not only help and give advices to user to avoid car accident; it is also developed to aid the user when one happens. Since the phone is hung on the holder, which can be considered stationary to the car, they share the same coordinate while using accelerometer. Figure 6 below shows the flowchart of accident detection performed by our algorithm, given the user is detected to be driving car: In order to aid the user, Car Safe first has to be able to detect accident. Accident is recognized by processing data from built-in accelerometer, which can indicate the abrupt change of acceleration during the crash.
From analysis of experiments, we propose a threshold-based method to detect accelerometer data. Since the axes can be random as the user may put the phone down in various ways or hang it by the holder in either direction, Car Safe processes with Root Mean Square (RMS) of the acceleration: with Ax, Ay, Az are acceleration recorded from 3 axes. During a period of 2 seconds, we calculate the difference between the maximum and minimum acceleration of RSM, and if it exceeds the threshold of 2G, we consider it to be a potential accident. To ensure precision and energy preservation, the recognition only occurs when driving state is detected. After Car Safe detects a potential accident, it shows a warning dialog, which can be shut down by tapping "I'm OK" if that is a false alarm, or only a small crash happens and the user is still ok. The warning dialog lasts 60 seconds and in case there is no response from the user, Car Safe concludes that the accident happens and the user needs help. Google Location Service is then turned on to get the position, which is then attached to email (formerly SMS, but since it has been prohibited by Google for application to take control of sending SMS, email is selected) and sent to VIP list along with warning to notify them of the user's state. The reason why we combine VIP list and emergency list into one is to remind the user of the importance of selective setup.

Result and Analysis
After several practical test, we finally present Car Safe to Google Play Store as a non-charged and available-to-everyone application. Upon the first startup, the user is requested to grant Car Safe the permission to access location as shown in Figure 8: The Bluetooth pairing requires one-time manual connection between the phone and the car, which Car Safe then implements automatically every time it starts. The user is given the choice to toggle utilities in setting as seen above, even Bluetooth connection since there are genres of car which do not support hand-free call and it would be redundant, wasting battery keeping Bluetooth on.

Crash detection and notification
Continue from the previous sub-section 3.6, crash data form can be seen in Figure  9 below: Fig. 9. Crash data from 3-axes accelerometer In Figure 9, the abrupt change in data resulted from a crash is so large that it diverges from normalcy. If the user does not confirm his/her safety, the application will send the message as follow:

Fig. 10. Alert message form and map pinpoint
Upon the arrival of the message as in Figure 10, contact from the other end can pinpoint the location of the accident and send help. Since sending SMS through application is now forbidden, the feature is forfeited from the application and email sending is selected as the replacement in the next update. The form of message, however, remains the same.

4.2
Feature comparison It is difficult to compare the types of utilities since they depend on application's purpose. However, as seen in Table 2, Car Safe offers a wide range of utilities to cover as much smartphone's distracting sources as possible. Google Android Auto and Apple Carplay mainly focus on multimedia-related services, which improve user's entertainment, navigation, etc. experience but may also be a cause of distraction. As in CrashDetech, the orientation is solely crash-related, covered from crash detection, medical services to road secure, insurance and legal issues. The problem is, as exceptional as it is, most of the services are available in South Africa only. Even if CrashDetech is internationally collaborated, few countries are capable of providing such services. Furthermore, to experience above applications in full, monthly or annually charge is applied. In terms of giving a free experience of car safety aid, Car Safe outstands as one worthy choice.

Driving state recognition test
Of all the features presented in Car Safe, driving state recognition is considered the fundamental "switch" on account of toggling other utilities. Since most of Car Safe's applications are used only when driving is detected, it is essential that the state is recognized accurately. In this section, its tests and results are discussed.
We performed the tests on a range of Android smartphones, from low-middle to high end: OPPO F1s, Samsung A50, Samsung Galaxy S6, Samsung Galaxy S9 plus, etc. The testing scenario is daily life driving for maximum examination and evaluation. We consider the driving time recognized per total driving time (including short stop from red line, traffic jam, etc.) as the test metric. While the Google Activity Recognition API only detects driving state approximately 89% of the time, our proposed method yields more than 93% detection. Due to the paramount importance of this step in Car Safe, we aim to further improve our approach to improve user's experience.

Conclusion
In this work, we introduce a new Android application called Car Safe. By using Google API and processing data from built-in sensors, we offer various utilities, which are ultimately for car driver's safety and convenience. Car Safe requires access to read and send SMS, read user's contact list, read notifications and use location service. It now features: reading out loud messages (SMS, email, Facebook Message), making decision on receiving the call or not, warning when the driver has driven too long without rest, detecting when accident happens and inform people from VIP list of the driver's location. Furthermore, user can freely decide which services he/she wants to use. With a simple user interface, Car Safe can be effortlessly perceived and used as a trusted partner for car driver. IOS version is being in development.
In our current and future work, we aim to enhance our algorithm along with using sensor fusion to add more features so that users can have the best experience using Car Safe. In the upcoming update, more features we are also working on other supported languages and replacement database for Vietnamese, English. Carsafe is a good example of mobile technologies which can be applied to real applications [22][23][24][25][26]. over 150 research papers. His main research interests include the representation, processing, analysis, and communication of information embedded in signals and datasets. He serves as a TPC co-chair, a technical committee program member, track chair, session chair and reviewer of many international conferences and journals.