Influence of automated driving on driver’s own localization: a driving simulator study

Purpose – Level 3 automated driving, which has been defined by the Society of Automotive Engineers, may cause driver drowsiness or lack of situation awareness, which can make it difficult for the driver to recognize where he/she is. Therefore, the purpose of this study was to conduct an experimental study with a driving simulator to investigate whether automated driving affects the driver’s own localization compared to manual driving. Design/methodology/approach – Seventeen drivers were divided into the automated operation group and manual operation group. Drivers in each group were instructed to travel along the expressway and proceed to the specified destinations. The automated operation group was forced to select a course after receiving a Request to Intervene (RtI) from an automated driving system. Findings – A driver who used the automated operation system tended to not take over the driving operation correctly when a lane change is immediately required after the RtI. Originality/value – This is a fundamental research that examined how the automated driving operation affects the driver's own localization. The experimental results suggest that it is not enough to simply issue an RtI, and it is necessary to tell the driver what kind of circumstances he/she is in and what they should do next through the HMI. This conclusion can be taken into consideration for engineers who design automatic driving vehicles.


Introduction
Automated vehicles are now in the spotlight. Automated vehicle technology has raised expectations for the reduction of human error while driving (Brookhuis et al., 2001) and may release people from the boring and distressing task of driving (Stanton and Young, 2005), enhancing traffic safety and fuel economy (Fagnant and Kockelman, 2015). The five levels of autonomous driving as defined by the Society of Automotive Engineers are widely accepted in the automotive industry (SAE, 2016). Nowadays, research on automated driving has mainly focused on the practical use of partial (Level 2; L2) and conditional (Level 3; L3) automated driving systems. In the partial automated driving system (L2) that executes both lateral and longitudinal controls on behalf of the human driver, the driver is required to continuously monitor the road environment and status of the automated system. The conditional automated driving system (L3) executes not only vehicle dynamic controls but also monitors surroundings. This means that the human driver does not need to monitor the driving environment during L3 driving. However, some circumstances (e.g. sensor failures, misunderstanding marked lanes) can make L2 and L3 systems reach their limits; then, the system issues a Request to Intervene (RtI). At this time, the driver should take manual control of the vehicle immediately and appropriately.
Although driving automation may promise such benefits, several human driver-related issues have been raised with regard to automated driving systems. Some researchers reported that automated driving can cause drivers to fall asleep compared to manual driving. Also, Naujoks et al. (2016) pointed out that drivers are more likely to pay attention to nondriving tasks while driving with adaptive cruise control (ACC) and lane keeping assist (LKA). These matters may cause such as out-of-the-loop (OoTL) performance problem (Endsley and Kiris, 1995) where drivers away from control loops cannot adequately respond to system errors. This is partly due to lack of an operator's situation awareness. Merat and Jamson (2009) investigated the awareness and comprehension of driver's peripheral traffic during manual driving and automated driving, and as a result, under automated driving, the driver's response to the dangerous events will be delayed. In addition, there are several studies that show automated driving causes driver drowsiness (Jamson et al., 2013;De Winter et al., 2014), which will lead to a lack of situation awareness of driver.
Many studies have also been conducted on driver response to an RtI under different circumstances. Naujoks et al. (2014) focused on the modality of RtI and reported that a driver responded to an RtI sooner if visual-auditory RtI was used instead of just visual RtI. Zeeb et al. (2016) pointed out that the quality of the takeover can be attributed to driver behavior before the RtI. Louw et al. (2017) proposed that humanmachine interface (HMI) design should focus on helping the driver take evasive action quickly rather than emphasizing the time to takeover. Most studies have focused on the time span and driver behavior required for an RtI. However, there are other issues to consider with regard to RtI. Current vehicles are equipped with assorted features. For instance, car navigation systems are installed in most cars and will tell the driver the current location and destination. However, the driver may be deeply involved in non-driving task or be absentminded, a result based on past studies. Such a driver may not consciously look at the car navigation system and confirm the current location. This situation is not a huge problem as long as an automated driving system is performed without any abnormality. What can be a problem is when a tight RtI is issued. If the time margin given to the driver is small, or the tasks required for the drive is large, such as changing the driving lane after an RtI, the driver may not be able to immediately grasp the situation. Gold et al. (2013) revealed that it takes more than 5 s for the driver to respond to an Rtl owing to (the lack of or decline in) situation awareness. Thus, it is important to investigate the vehicle design such that the driver can grasp the current situation correctly in a limited time. Also, HMI will play a major role when and RtI is issued. Louw et al. (2017) proposed that HMI design should focus on helping the driver take evasive action quickly rather than emphasizing the time to takeover. In L3 automated driving, drivers do not need to do drive tasks and have a supervisory control of the system and environment. As a result, it is possible that the driver will not be able to identify his/her location and does not know which direction to go in a limited time. In fact, in our past research, we confirmed a branch failed case despite presenting the correct branch lane after an RtI to the driver and the driver was able to grasp the steering wheel (NEDO, 2018). It is necessary to further investigate whether this result was caused by automated driving. Thus, the aim of this study is to investigate the influence of automated driving on the driver's localization. In this study, driver's localization means the ability to grasp what position on the route a car is currently travelling to the destination. As HMI design is as an important concern for elderly driving performance (Freund et al., 2005;Körber et al., 2016), this driving simulator study is targeted to elderly drivers.

Participants
Seventeen elderly participants (men, 12; women, 5), of age 66 to 82 years (M = 72.0, SD = 4.0), participated in the experiment. The participants were recruited via a local human resource agency. All of them hold a valid driver's license and drove daily. The experiment was conducted after obtaining approval from the University of Tsukuba Research Ethics Committee.

Experimental design
In the experiment, one factor in the "existence or nonexistence of automated driving" was taken into consideration. It was a betweensubject design with two levels of autonomous operation (AO) and manual operation (MO). The participants were randomly assigned to each group. The average age and composition of each group were as follows (AO: M = 73.6, men = 6, women = 3; MO: M = 71.3, men = 6, women = 2).

Driving simulator
As presented in Figure 1, the experiment was conducted with a motion-based driving simulator (Honda Motor Co., Ltd.). The simulator has three LCD monitors that display the left and right mirror image and rearview mirror image. The front field-of-view was approximately 120°horizontally and 45°vertically. Two LCD monitors for displaying the HMI (automated status) and Internet TV (AbemaTV: https://abema.tv/) were mounted on the dashboard (Figure 2). We used the TV programs of AbemaTV with their permission. The simulator provided vehicle traveling sounds. The sounds were not too high to enable the  View from the driver's seat of the driving simulator drivers to hear the TV and an RtI request. To prevent motion sickness, the motion function was not activated.

Automated driving system
The automated driving system can be activated by pressing a button. The system maintained the vehicle's speed at 70 km/h and kept the vehicle at the center of the driving lane. Note that an automatic lane-changing function was not installed. Drivers did not have to hold the steering wheel during autonomous operation. Drivers could disengage the autonomous operation by steadily pushing the brake pedal more than 30 per cent the amount of depression or moving the steering wheel more than the absolute value of 30°.

Human-machine interface
The HMI was used to display automation status using an 8-inch LCD monitor ( Figure 2). When automation was activated, a green image appeared; when deactivated, an orange image with an "off" sign appeared, as shown in Figure 3(a) and 3(b), respectively. Figure 3(c) shows that the automated driving system required the driver to resume control of the vehicle. The RtI was issued with two consecutive beeping sounds. Even if the driver did not take any action following the RtI, the system was deactivated 10 s later.

Driving task
All participants were instructed to proceed to a junction or a branch on an expressway specified in advance by an experimenter. They were also asked to drive safely. In addition, the MO participants were asked to keep the vehicle's speed at approximately 70 km/h and stay in the left lane (this is the driving lane in Japan). The AO participants were asked to watch Internet TV during automated driving and to resume control of the vehicle as needed.
The experimental driving course was based on a section of the Kita-Kanto expressway in Japan ( Figure 4). It has two lanes on each side and no other car existed. The starting point was the same in all scenarios, which was the Kaminokawa IC. Since the automated driving system of this experiment operated only a single lane, and it was impossible to change lanes automatically. Thus, in the AO, the RtI request was issued approximately 200 m before the Tsuga IC exit, 250 m before Tochigi-Tsuga JCT. This means that because the car was travelling at 70 km/h, it gave the driver approximately 10 to 13 s of grace time before branching. Because the Tochigi-Tsuga JCT had a large left curve before branching, an RtI was issued in a straight section 250 m before branch considering the smooth handover. The scenarios were as follows (Figures 4 and 5): get off the expressway at Tsuga IC; proceed to the right of Tochigi-Tsuga JCT; and proceed to the left of Tochigi-Tsuga JCT.

Procedure
Each participant signed an informed consent form after receiving an explanation of the experiment's purpose and overview. A practice drive was given to each participant to get used to the simulator (approximately 5 min). Subsequently, the AO participants received an explanation about the automated system and HMI, and practiced automated driving for approximately 5 min. They also performed exercises on how to engage and disengage the automated system by pressing the brake pedal or moving the steering wheel.
Afterwards, each participant drove under three scenarios. Before each driving scenario, they received instructions from the experimenter about the route using a map. There were no navigation devices to indicate the vehicle's location and route to participants while driving. Thus, they had to remember the destination. Figure 5 presents a detailed outline of each scenario with an actual simulator photo and illustrations. It took approximately 15 to 20 min to complete each driving scenario, and the order of drives was randomized for each participant. All participants took a 10-min break in between each drive. After all the drives, the participants were interviewed and took the Trail Making Test A/B (TMT A/B) (Tombaugh, 2004). TMT is a kind of cognitive function test used to confirm whether participants were assigned uniformly to each group. The total time of this experiment per participant was within 2 h.

Dependent measures
To investigate our hypothesis, i.e. drivers may have difficulty in appropriately driving to the instructed destination during automation compared to manual operation, the data collected from the simulator were analyzed with following dependent measures.
Results of the TMT A/B. This was used as supplemental data to confirm whether participants were equally allocated to each group. In this study, a significant decline in cognitive function  Figure 4 Outline map of the driving course may cause differences in results because participants were tasked with memorizing their destination. Thus, it is necessary to ensure that the experimental results are reliable.
Number of participants who drove appropriately to the instructed destination. For each driving condition, all participants experienced three scenarios. Whether they drove to the instructed direction was counted to investigate the driver's ability to understand the location of the host vehicle.
Time elapsed to hold the steering wheel. This is the elapsed time from when the system asked the driver to take control of the vehicle until the driver takes hold of the steering wheel in the AO condition. To investigate why the driver failed to reach the destination, the elapsed time was calculated.
Point where lane change began. This indicates how far before the lane change was made from the branch point in Scenario B, in which drivers needed to change lanes to the right. This was used as an indicator to show how much the driver's maneuvering attitude against branching differs between the AO and the MO conditions.
In this experiment, we focused on the driver's ability to grasp driver's own location, and thus we will analyze the precise analysis of vehicle behavior as necessary in the future.

Results of TMT-A/B
The results of TMT-A and TMT-B are shown in Figures 6 and 7  participants were homogeneous between the two groups and the experimental results are worthy of analysis. Figure 8 shows "the proportion of drivers who proceeded to the appropriate course. Between groups, a Friedman test of success rate provided a chi-square value of 0.33, which was not significant (p > 0.1). In the MO, the ratio is almost equal in each scenario. On the other hand, approximately half of the AO participants failed only in Scenario B (to proceed to the right of the JCT), but had no errors in the other scenarios. A chi-square test of the success rate between Scenarios A-B and B-C in the AO provided a chi-square value of 2.89, which was marginally significant (p < 0.1). Although branch failures were certainly confirmed even in MO, an approximately half of AO drivers failed in Scenario B where a driver should proceed to the right lane. The driver may be able to properly deal with an RtI in the case of going straight to the left or leaving to the left lane like Scenarios A and C. On the other hand, in Scenario B where lane change was required to the right lane after an RtI, the driver may have proceeded to the left lane as they were trying to grasp the situation. Despite the relatively long grace period after an RtI, it is noteworthy that some drivers did not proceed to correct lane. This matter may have been more significant in more critical situations. Failure to do the appropriate lane change may cause traffic accidents. This finding suggests that automated driving hinders the driver's situation awareness and as a result may affect the ability to grasp self-location. Table I shows the classification of the number of participants who held the steering wheel in the AO condition before and after the RtI. Note that data set of one participant in Scenarios A and B is missing. Figure 9 shows the time from issuing the RtI to taking over the wheel in the AO (0 s = RtI issue point). Error bars indicate the standard deviation. As a result of a one-factor ANOVA with three levels of Scenarios A, B and C, the main effect was not significant [F(2, 21) = 0.99, p > 0.05]. However, as seen in Figure 9 and Table I, half of the participants gripped the steering wheel before the RtI and prepared for the coming situation. The average time was À18.6, À6.8 and À5.0 s, from the left of the graph. Some studies have revealed that a driver took preliminary action for an RtI if the circumstances in which the RtI was issued is predictable (Merat et al., 2014); the results of this study support this conclusion. Especially in Scenario A, which involves leaving the expressway, the tendency to grasp the steering wheel earlier than in other scenarios was confirmed. On the other hand, even if the driver held the steering wheel before the RtI, a case of making a mistake in branching was confirmed. This also occurred in our previous experiments. Therefore, when suddenly assuming control from automatic operation, the driver may not distinguish the course that he/she should go to; as a result, there is a risk of creating a dangerous situation.

Point where lane change began
In this experiment, although participants were not explicitly instructed to do so, almost all drivers used the turn signal when they began to change lanes. Thus, the point where the turn signal was made was analyzed as the starting point of lane changing. Six out of eight participants in the MO and five out of nine participants in the AO condition succeeded in branching in Scenario B. Even if the driver did not take control, the automated driving system was disengaged at the beginning of the zebra zone. The zebra zone was approximately 300 m. It was also possible to change lanes within the area: however, it is highly recommended to change lanes before entering the zebra zone. Figure 10 shows the point where the driver who successfully changed lanes began to change lanes in Scenario B (AO: six participants, MO: five participants). The result of F-test showed that the two populations are not equally distributed [F(4,5) = 84.325, p < 0.001)]. The result of Welch's test showed no significant difference between the two groups [t(4.07) = À1.269, p > 0.05]. However, most participants in MO seemed to start

Discussions
The aim of this study was to investigate the influence of automated driving on the driver's own localization. The driving simulator experiment involved two types of operationmanual and autonomous operationand all participants experienced three different road environments that simulated a Japanese highway. By forcing participants to choose an instructed lane before junctions or interchanges in all driving scenarios, we investigated the impact on the ability of the driver to grasp own position.
The statistical results suggest that the driver is more likely to rely on the automated driving system. Although there is no significant difference between the two groups, it seems that automated driving influenced the advancement/movement toward the correct course and the position at which the lane change was made. For instance, in Scenario B, lane changing was required after an RtI, but drivers were unable to quickly resume control owing to the expectation, which caused approximately half of failures. Also, it was suggested that it may take longer time to execute lane change under such a situation. On the other hand, in the other two scenarios where lane change was not required, there is no driver who failed to take over. These results suggest that if the driver does not know why the RtI was issued and how to take over, a failed takeover may occur more frequently. In addition, despite holding the steering wheel after an RtI, cases where the lane change was made incorrectly or where the lane change was made in the zebra zone were confirmed. This result suggests that even if the driver responds to an RtI, the driver may not always understand the situation. Further investigation in a more complex road environment will be necessary. It would be useful to explain a driver via HMI why an RtI was issued and what to do next.
There are also several limitations in this experiment. The number of participants in this experiment is not sufficient to discuss the impact/effects of the results. Thus, it is important to consider which information can be beneficial or them in terms of designing an HMI for such circumstances. Moreover, even though the drivers were asked to watch TV during automated driving, most drivers were not always looking at the TV; they sometimes monitored their surroundings. This may be because one trial for each condition lasted at most approximately 20 min. If drivers are driving for a longer time, the driver may fall into the OoTL more. However, several failures at branch occurred even in the MO group. We need to keep in mind that there could be a participant who just cannot read the letters of the sign because of a problem with the resolution of the simulator. In this experiment, to alleviate this problem, the speed of the host vehicle was limited to 70 km/h, and the participants received supplementary explanation during instructions about the route (e.g. get off at the third exit to the left/go to the fourth branch to the right).
Some drivers who failed to branch remarked that they missed the branch point unintentionally. This could be due to their driving experience or highway experience. Thus, it is necessary to analyze in detail whether the experience of manual driving will affect branch failures.
To design an HMI that allows drivers to take over safely and to recognize which direction they should go, it will be important to investigate whether the driver's maneuvers and gaze will change depending on the manner of presenting the course. Larsson et al. (2015) suggested that a simpler HMI would be preferable for smooth RtI.
From the viewpoint of safety, if measures based on HMI alone are insufficient, then methods using tactile sense such as directional and vibrotactile RtI (Fricke et al., 2015;Petermeijer et al., 2016;Petermeijer et al., 2017) or haptic shared control (Abbink et al., 2012) are also conceivable. These approaches can convey information to the driver through tactile. It is necessary to further investigate how to design an HMI for automated driving and the limitations of RtI.
It is also important to design a system that does not issue the RtI at the scene where the driver is forced to change lanes suddenly. However, there are various situations when a system must suddenly issue an RtI (e.g. accident ahead, road construction, system failure). Even if driving authority is suddenly transferred, an HMI design that allows safe handover that/allows safe handover requires further investigation.

Conclusions
Automated driving may lower a driver's situation awareness; therefore, the main objective of this study was to investigate the influence of automation on understanding of driver's own localization. The driving simulator experiment involved two types of operation: manual and automated operations, and all participants experienced three different scenarios. All drivers were instructed how to go to the course in advance. The car was not equipped with a navigation system; thus, the driver had to remember the destination.
Furthermore, the automated operation group was instructed to watch an Internet TV program during automated driving, and to take over the operation if an RtI was issued. The experimental results suggested that the driver may not drive to the expected destination when lane changing is forced right after resuming manual control from automated driving. In the scene of the junction where a lane change to the right was required after an RtI, nearly half of the drivers could not change course correctly. This result suggests that it is insufficient to just issue an RtI, and shows the necessity of presenting what the driver should do afterwards. It also seems to be important that the HMI should provide information on where a driver is and where to go upon regaining control of the driving task.
It would be important to investigate how much information is appropriate and how much instruction should be provided, while changing these levels step by step. Furthermore, it will be necessary to investigate what should be presented to the driver not only during the RtI but also during normal automated driving via the HMI.