Assessing the Effects of Modalities of Takeover Request, Lead Time of Takeover Request, and Traffic Conditions on Takeover Performance in Conditionally Automated Driving

Abstract

When a conditionally automated vehicle controlled by the machine faces situations beyond the capability of the machine, the human driver is requested to take over the vehicle. This study aims to assess the short-term effects of three factors on the takeover performance: (1) traffic conditions (complex and simple); (2) modality of takeover request (auditory and auditory + visual); (3) lead time of takeover request (TORlt, 5 s and 7 s). The scenario is the obstacle ahead. Indicators include: (1) Take Over Reaction Time (TOrt); (2) approximate entropy (ApEn), operating order of steering wheel Angle and pedal torque; (3) the choice of target lane and speed of lane-changing; (4) mean and standard deviation of acceleration and velocity; (5) quantifiable lateral cross-border risk and longitudinal collision risk. A driving simulation experiment is conducted to collect data for analysis. The effects of the three factors on takeover performance are analyzed by analysis of variance (ANOVA) and non-parametric tests. The results show that when the traffic conditions are complex, drivers have a larger ApEn of the steering wheel angle and brake pedal torque, and a smaller ApEn of acceleration pedal torque. In the 5 s TORlt case, drivers have a smaller ApEn of brake pedal torque the interaction between TORlt, traffic conditions, and modality of TOR affects ApEn of accelerator pedal torque. 5 s TORlt/complex traffic condition makes the scene more urgent, which is easy to cause driver to make sudden and simultaneous turning and sudden braking dangerous behavior meanwhile. Compared with other combinations of modality and TORlt, the combination of 7 s and auditory + visual significantly reduces the lateral cross-border risk and longitudinal collision risk.

1. Introduction

The J3016 standard of the Society of Automotive Engineers (SAE) categorizes autonomous driving from level 0 to level 5 [1]. Level 3 driving automation includes conditionally automated driving, in which the driver and the vehicle operate it together. The autonomous driving function will be disabled when the system encounters a scenario outside the Operational Design Domain (ODD). Human drivers of conditionally automated vehicles would inevitably take control of the vehicle from the system. When an L3 vehicle needs to transfer control authority from the system to the human drivers, the system sends the human drivers a takeover request (TOR), and the human drivers take over control authority within a limited period. The effectiveness of autonomous driving and the standard of takeover have both significantly improved as a result of recent research advancements. The majority of intelligent vehicles on the road today use driving assistance technologies, but there are still security flaws that hinder a secure takeover. The National Highway Traffic Safety Agency (NHTSA) published studies on autonomous driving in June 2022. Automobile manufacturers reported a total of 392 accidents in the L2 Advanced Driver Assistance system report [2], and a total of 130 accidents were reported for the L3-L5 Autonomous Driving System [3]. The majority of accidents are caused by human error or the limitations of automatic driving in addition to the causes associated with conventional vehicles. The driver could still not be able to take control in time even if the car properly detects an accident and delivers a warning. To assist with optimizing TOR safety design, therefore the characteristics of this type of driver takeover behavior must be examined.

The effectiveness of human drivers during takeover is influenced by numerous factors: traffic conditions, modalities of the takeover request (TOR), the lead time of takeover request (TORlt), the weather, the state of the roads, etc. are included. The TOR can be transmitted to human drivers through a variety of communication modalities by the human machine interface (HMI). How different TOR modalities affect takeover performance is currently a hot topic of research. The modality of the TOR could be a visual, haptic, auditory, or multi-modal warning. Lee et al. [4] discovered that drivers may react more slowly with static visual displays on screens than auditory warnings. Drivers’ average takeover times in scenes with static visual warnings were 3.44 s while they were 1.79 s in scenes with static visual and auditory interactions. Static visual warnings might divert attention away from crucial information. Some academics have researched new visual displays in recent years, for instance, steering wheel lights and background flashing LED [5] have the potential to transmit emergency information and enhance takeover quality. Auditory warning consists of semantics, intonation, and speed. If the auditory information is more warning than moderate persuasion [6] or speaks faster [7], or more frequently [8], the urgency of the message will be increased. The impact of the haptic modality on takeover has drawn a lot of attention in recent years. The haptic modality has a higher transmission rate than the other modalities [9], however, it is not appropriate for transmitting multiple warning information [10]. When two or more different modalities of warning are combined, it is known as a multi-modal warning. Diederich and Colonius [11] found that the multi-modal warning might achieve better takeover performance by reducing takeover response time. VanErp et al. [12] studied the impact of multi-modal warnings on the urgency perceived by human drivers. They found that higher signal rates resulted in higher perceived urgency. Bazilinskyy P [13] investigated the combination of auditory, vibratory haptic, and visual display warnings and found that drivers prefer multi-modal warnings when facing emergencies.

The effect of TORlt on takeover performance has also attracted much attention. TORlt is defined as the lead time from TOR sending to a critical event. Takeover performance varies with the TORlt. Most studies have chosen several discrete TORlts for comparison. Young and Stanton [14] found that the shorter the TORlt, the faster the driver reacts, but the more urgent the driver behaves. There are many studies with TORlt, including 5 s [15,16,17,18], and 7 s [18,19,20,21]. Gold et al. [18] compared 5 s and 7 s TORlt (HMI Modality is auditory or visual/auditory) and found that under the 5 s TORlt, drivers can grip the wheel faster and look around faster. Eriksson and Stanton [22] showed that takeover times were longer when non-driving related tasks (NDRT) were performed, and the exact differences varied with traffic conditions. Wan and Wu [23] found that takeover requests with a longer TORlt of more than 10 s had a lower collision rate, longer minimum time to collision (TTC), and smaller lateral acceleration.

Driving is a dynamic, interactive activity that necessitates making accurate predictions and judgments while also assessing the current driving circumstances of other road users. Therefore, traffic condition around a conditionally automated vehicle is also an important factor affecting the takeover performance. Several researchers [17,21,24] have looked into the influence of traffic density on drivers’ takeover operation behavior. In complex conditions, drivers will be more conservative, with fewer steering and acceleration operations and more braking operations. Du et al. [25] found that poor takeover quality would come from the driver’s inability to focus his or her eyes during the takeover due to the complex environment. Another study [26] by their team shows that drivers in high-density traffic situations are less attentive to their surroundings, more likely to dismiss certain pieces of information, and had higher heart rates (23% higher than in low-density traffic situations). Scharfe et al. [27] studied the influence of traffic density on takeover behavior and found that drivers would focus more on the left and right lanes, as well as their rearview mirrors when turning in high-density situations. Baldwin and Coyne [28] found that the higher the traffic density, the smaller the driver’s judgment ability when he takes over the control authority. Radlmayr et al. [29] investigated the effect of traffic density on takeover performance. It has been discovered that the Take Over Reaction Time (Tort) will be longer and the lateral acceleration will be greater when the surrounding traffic volume increases. Gold et al. [18] found that there are more conflicts, a longer TOrt, and worse takeover performance when other vehicles are present.

The interaction effect among multiple factors on takeover performance is also an interesting research topic, interaction is a phenomenon in which the difference in the amount of response between levels of a factor varies with different levels of other factors. The fact that it exists suggests that the impacts of multiple variables investigated concurrently are not independent. Korber et al. [29] studied the interaction effect between age and traffic density on takeover performance. Feldhutter et al. [30] and Wu et al. [31] studied the interaction effect between age and traffic conditions. The traffic condition was represented by traffic speed. Du and Kim [26] found that drivers’ takeover performance was poor under the conditions of high cognitive load, short TORlt, and high oncoming traffic density. The factors mentioned above have a single-factor effect on takeover behavior, and there may be interaction effects between factors. Multi-factor interaction research on the aforementioned scenario components is currently lacking. Few studies have examined the interaction between traffic conditions, the TORlt, and the modality of takeover request and taken into account the interaction between the three factors on takeover performance, despite the significant contributions made by earlier studies on the interaction effect between two of the factors.

This study aims to explore the short-term effects (including interaction effects) among traffic conditions, the TORlt, and the modality of TOR, on human driver’s takeover performance.

The rest of the paper is structured as follows. Section 2 introduces the methodology for analyzing the effects of the three factors on takeover performance. The results are presented and discussed in Section 3 and Section 4, respectively. Section 5 concludes this work and recommends further research directions.

2. Materials and Methods

To assess the effect of lead time, modalities of TOR, traffic conditions on takeover performance in conditionally automated driving, a takeover experiment with multiple factors was designed. Metrics related to driving operations, vehicle dynamics, and safety situations were used to evaluate driver behavior. Several participants performed individually to complete the experiment. A multivariate analysis was conducted in this study to assess the effect of multiple factors on the drivers’ behavioral performance.

2.1. Multiple Factor Analysis

To assess the effects of various elements on the experimental scenario and examine the characteristics of drivers’ takeover responses, a multiple-factor analysis between groups is required in this study. The analysis process of the data is shown in Figure 1. It is necessary to evaluate each group’s additivity, independence, randomness, normality, and homogeneity of variance first. The Levene test, a homogeneity of variance test technique appropriate for evaluating both normal and non-normal data, can be used to determine the homogeneity of variance. The Kolmogorov-Smirnov test (K-S test) can be used to check the normality of the data. The samples that pass the above hypothesis testing conditions can be used for the analysis of variance (ANOVA) or the multivariate form of analysis of variance (MANOVA). The majority of experimental data are, however, non-normal or non-homogeneous between groups. In this case, non-parametric tests such as the Mann-Whitney U rank-sum test (one factor), Kruskal-Wallis test (one factor), or Scheirer-Ray-Hare test (Multiple factors and their interaction terms) can be adopted can be considered.

2.2. Indicators

This study enriches evaluation indexes and analyzes driving behavior from the following metrics: TOrt, approximate entropy (ApEn), and operating order of steering wheel Angle and pedal torque, Mean and standard deviation of velocity and acceleration, lane-changing behavior and lateral cross-border risk, and longitudinal collision risk.

2.2.1. Take Over Reaction Time (TOrt)

The TOrt refers to the period in which an automated system sends TOR to reclaim control by the human driver. According to the TOrt calculated by a meta-analysis [32], when the driver turns the steering wheel, releases the brake, or presses the takeover button, the takeover begins. As the driver’s feet are largely free during NDRT and can be placed on the pedal, in this study, the takeover was initiated by handing the steering wheel.

2.2.2. ApEn and Operating Order of Steering Wheel Angle and Pedal Torque

ApEn can be used to assess the irregularity and complexity of a period series which can be used to assess whether the driver’s operation is not smooth.

The idea of ApEn analysis is to detect the probability of new sub-sequence generation in time series, and it has a certain anti-noise and anti-outfield ability. ApEn can be used to measure the irregularity and complexity of a period series, especially the sequence with the deterministic trend and random fluctuation. The driver’s operation is just such a sequence because when facing the obstacle scene in this study, most of the drivers will significantly slow down and turn the steering wheel, which is a deterministic trend and the subsequent driver’s operation is random. ApEn can be used to measure whether a driver is not operating smoothly. The time series data is {$t\left(i\right)$, $i$ = 0, 1, ..., N}, the mode dimension is $m$, and the similarity tolerance is $r$. The calculation steps are as follows:

• Sequence $\left\{t\left(i\right)\right\}$ to form m as vector $Y\left(i\right)$.

$$Y\left(i\right)=\left[t\left(i\right),t\left(i+1\right),\dots ,t\left(i+m-1\right)\right]$$

(1)

2.

For each $i$, calculate the distance between the vector and the remainder:

$$d\left[Y\left(i\right)\right],Y\left(j\right)]=\underset{0-m-1}{\mathit{max}}\left|t\left(i+k\right)-t\left(j+k\right)\right|$$

(2)

3.

Calculate the ratio of the number $n$ less than r in each $i$ calculated by (2) to the total number, i.e.,

$$C_r^m\left(i\right)=\frac{n}{N-m+1}$$

(3)

4.

Take the logarithm of $C_r^m\left(i\right)$ and average it.

$${\Psi }^m\left(r\right)=\frac{{{\displaystyle \sum }}_{i=1}^{N-m+1}ln{C}_r^m\left(i\right)}{N-m+1}$$

(4)

5.

Repeat calculation of (1)~(4) for $m+1$, get ${\Psi }^{m+1}$

6.

Calculate ApEn, where $N$ is not equal to ∞

$$ApEn\left(m,r\right)=\underset{N\to \infty }{\mathit{lim}}\left[{\Psi }^m\left(r\right)-{\Psi }^{m+1}\left(r\right)\right]$$

(5)

In this study, $m$ is 2 and $R$ is 0.2*Standard deviation of data.

The more varied frequencies there are in the transmission, the higher the entropy and the more complex and irregular the driver’s behavior is. There are three indexes to measure the regularity and complexity of driver operation: ApEn of steering wheel Angle, ApEn of accelerator pedal torque, and ApEn of brake pedal torque, which are all used to measure the regularity and complexity of driver takeover process.

In addition, a series of operating times can also be used to judge the order of driving operations, and to judge the driver’s operating habits and reaction speed under specific experimental situations, including the takeover reaction time, steering wheel trigger time, and pedal trigger time. Lane change time refers to the duration between the driver accepting TOR and the car crossing into an adjacent lane.

2.2.3. Mean and Standard Deviation of Velocity and Acceleration

Speed, acceleration, and position indications that show how the vehicle is moving are related to the driver’s numerous operating inputs. For instance, steering wheel, brake, and accelerator pedal inputs are related to vehicle lateral indicators like lateral speed and lateral acceleration. The mean and standard deviation of vehicle acceleration and velocity are examined in this study. In this study, the velocity and acceleration are decomposed according to the vehicle coordinate system. The longitudinal direction (positive direction) of the vehicle is defined as the forward direction of the vehicle’s central axis through the vehicle’s center of mass, the lateral direction is defined as the longitudinal vertical direction, and the positive direction is the left side of the vehicle when looking down. In this study, the mean and standard deviation of Lateral acceleration, longitudinal acceleration, lateral velocity, and longitudinal velocity was analyzed.

The trend chart of the mean and standard deviation of vehicle acceleration and velocity within 7 s of takeover was also examined in this study.

2.2.4. Lane-Changing Behavior

Human drivers must take control safely when changing lanes in specific situations, such as when there are obstructions in the front lane. To thoroughly assess lane-changing behavior, this study uses two metrics: lane-changing rate, lane-changing speed. When a vehicle entirely crosses the edge line of the original lane and enters the opposite lane, it has successfully changed lanes. The successful lane change ratio of human drivers in all driving experiments conducted after the TOR is issued but before the occurrence is what is meant by the term “lane-changing rate”. Under all successful lane changing tests, lane changing speed is defined as the ratio of the lateral displacement of the vehicle’s center of mass to the time from when the vehicle starts to press over the edge until the vehicle’s wheels are entirely in the other lane.

2.2.5. Lateral Cross-Border Risk and Longitudinal Collision Risk

The safety of takeover should also be taken into account in the examination of takeover quality because the analysis presented above is not exhaustive. To assess the safety performance of takeover, a lateral and longitudinal risk score was defined. The risk was described as the likelihood of a collision between the primary vehicle and the nearby structures and vehicles, which was then broken down into two categories: lateral cross-border risk and longitudinal collision risk.

Minderhoud and Bovy [33] developed time exposure to collisions (TET) as a measure of safety, which is the sum of TTC in a period lower than the safe time threshold. In this study, the longitudinal collision risk is adapted from this idea, and the lateral risk calculation method is proposed.

The following is a definition of lateral cross-border risk: Once the time of the vehicle touching the edge line is less than a specific safety threshold, if the state of time t is maintained, the synthesis of the difference value is calculated. The study employs the Time to Lane Crossing (TLC) model, which takes into account the lateral acceleration, speed, and yaw speed while assuming that the steering wheel angle remains constant (Figure 2). The time needed for the vehicle to touch the lane edge line is determined using the TLC model. The calculation steps are as follows: (6) to (8).

$$s=vTLC·tan\theta +\frac{1}{2}a{T}^2$$

(6)

$$a=\frac{v^2}{\frac{1}{C_r-C_v}}$$

(7)

$$TLC=\frac{-vtan\theta +\sqrt{v^{2}tan{\theta }^2+2v^{2}s{C}_r-\frac{\omega }{v}}}{v^2{C}_r-\frac{\omega }{v}}$$

(8)

Among them, $s$ is the distance from the lane edge line to the vehicle in front of the vehicle, $v$ is the longitudinal velocity, $\theta$ is the vehicle steering Angle, $a$ is the lateral acceleration, $v$ is the lateral speed, $TLC$ is the time of side collision lane line, $C_r$ is the road curvature, $C_v$ is the vehicle running track curvature, which can be expressed by $\frac{\omega }{v}$, where ω represents the horizontal pendulum Angle.

$TLC$ of the TLC model mentioned above can be calculated by the data of each time stamp given in takeover time. When $TLC$ is less than the critical time $TL{C}_{th}$, the vehicle is considered to have lateral risk, Z. Yan et al. [34] believed that when TLC is 5 s, drivers can safely change lanes, in this study the critical time threshold in this paper is 5 s and the potential overflow risk value at this time $r_1$ is defined as

$$r_{1}t=T_{th}-TLCt$$

(9)

Therefore, the total risk degree in the takeover process is the accumulation of the overflow risk value in the takeover time, i.e., $R_1$

$$R_1={\delta }_1\left(t\right)\left(T_{th}-{{\displaystyle \sum }}^{\hspace{0.17em}}\frac{-vtan\theta +\sqrt{v^{2}tan{\theta }^2+2v^{2}s{C}_r-\frac{\omega }{v}}}{v^2{C}_r-\frac{\omega }{v}} \right)$$

(10)

$${\delta }_1\left(t\right)=\{\begin{array}{c}1, TL{C}_{th}>TLCt \\ 0, TL{C}_{th}<TLCt \end{array}$$

(11)

The possibility of a collision between a vehicle and a vehicle or obstacle in the same lane is known as the longitudinal collision risk: under the presumption that the acceleration of the vehicle in front and the vehicle behind is identical, the collision time formula is employed. The Time to Collision (TTC) is the period before a collision occurs when the speed of the vehicle in front is greater than the speed of the vehicle in the back.

$$TTC=\frac{x_i-x_{i-1}}{v_i-v_{i-1}}=\frac{v_i{h}_s}{v_i-v_{i-1}}$$

(12)

$x_{i }$is the position of the vehicle behind, $x_{i-1}$ is the speed of the vehicle in front, the speed of the obstacle is assumed to be 0, and $h_s$ is the time headway of vehicles. When $TTC$ is less than the collision time threshold $TT{C}_{th}$, the longitudinal collision overflow risk $r_2$ is defined as

$$r_2=TT{C}_{th}-TTC$$

(13)

The collision time threshold is divided into different standards, previous studies have suggested several optimal thresholds: 1.5 s [35], 2.6 s [36], and 3 s [37,38,39]. In this paper, 3 s is selected as the risk threshold. The total risk of longitudinal collision $R_2$ during takeover time is

$$R_2={\delta }_2\left(t\right)\left(TT{C}_{th}- {{\displaystyle \sum }}^{\hspace{0.17em}}\frac{v_i{h}_s}{v_i-v_{i-1}}\right)$$

(14)

$${\delta }_2\left(t\right)=\{\begin{array}{c}1, TT{C}_{th}>Tt \\ 0, TT{C}_{th}<Tt \end{array}$$

(15)

2.3. Driving Simulation Experiment

2.3.1. Participants

46 participants were selected, including 23 males and 23 females, and drivers range in age from 22 to 53 with average 34.3 years. All participants have driver’s licenses, their visual acuity is above 0.8. They are in a good state without any serious medical histories, psychological or physical illnesses, or poor habits. Each participant was rewarded 150 yuan after the experiment.

2.3.2. Apparatus

The simulation control software is SCANeR Studio, a commercial software developed by OKTAL Company in France, which is used for setting up scenarios, simulation, and data output. Hardware simulator is a high simulation driving simulator from Tongji University can be seen in Figure 3a, and logitech G29 driving simulation set is adopted can be seen in Figure 3b, which is composed of steering wheel, pedal, and seat. The steering wheel is equipped with dual-motor force feedback technology. The rotation degree is 900 degrees, and the maximum rotation angle to the left or right is 450 degrees, which is the same as that of the real car steering wheel to realistically simulate the feeling of controlling the steering wheel in the real driving process. Three enormous HD monitors with dimensions of 906 mm × 660 mm × 225 mm, a 1920 × 1080 resolution, a 90-degree angle of view, and a field of vision that can accommodate drivers are used to present the driving scenario.

2.3.3. Experiment Design and Procedure

Road modeling was designed according to Chinese highway standards. The experimental route is a 10-km-long, straight stretch of a two-way, six 3.75$\mathrm{m}$ lanes motorway (see Figure 3c) with a maximum speed limit of 120 $\mathrm{km}·{\mathrm{h}}^{-1}$ and no minimum speed limit. The isolation belt is set in the center The speed of 100 $\mathrm{km}·{\mathrm{h}}^{-1}$ is maintained during automatic driving, and there are street lamps and street trees 50 m apart. A few buildings, land, and other facilities are set on both sides of the road. The main car runs in the central lane and the theoretical speed range is 90 $\mathrm{km}·{\mathrm{h}}^{-1}$−120 $\mathrm{km}·{\mathrm{h}}^{-1}$.

In this experiment, the auditory warning was a non-phonetic beep signal and the beep of simulated sound, with a duration/interval of 800/100 and 200/1600 ms, the warning doesn’t go off until the driver takes over. The visual warning was displayed in the form of a picture warning displaying red text: “Danger, please take over”, and the picture did not contain scenario information. The TORlt adopts 5 s and 7 s. There are two main types of traffic conditions as illustrated in Figure 3c, one is the complex traffic conditions: there are vehicles in the lanes on both sides of the main vehicle, and there is little space between them, but the safe spacing for vehicle insertion is met. The road traffic volume is 1800$\mathrm{p}\mathrm{c}\mathrm{u}·{\mathrm{h}}^{-1}$. Another one is simple traffic conditions with 800 $\mathrm{pcu}·{\mathrm{h}}^{-1}$traffic volume.

As is shown in

Table 1. Eight scenarios were designed for the driving simulation experiment.

Number	Modality of Takeover Request (TOR)	Lead Time of Takeover Request (TORlt)	Traffic Conditions	Number	Modality of TOR	TORlt	Traffic Conditions
1	A (auditory waring)	5 s	complex	5	A+V (auditory and visual waring)	5 s	complex
2	A	5 s	simple	6	A+V	5 s	simple
3	A	7 s	complex	7	A+V	7 s	complex
4	A	7 s	simple	8	A+V	7 s	simple

, a 2 × 2 × 2 factorial experiment was designed to assess the effects of modalities of TOR, TORlt, and traffic conditions on takeover performance in conditionally automated driving. 46 drivers are summoned to complete the driving simulation experiment. The driver was asked to hold a mobile phone and watch the video. Questions related to the video popped up during the watching, and the driver had to answer the questions manually. The purpose is to simulate non-driving related tasks.

Each driver in this experiment is required to complete every scenario, and the scenes’ order is randomly disorganized. The driver must complete 8 min manual driving and 7 min rest between the two experiments. The driver has encountered construction, overtaking, car following, pedestrian, and other scenes while practicing manual driving. The practice effect and fatigue effect are significantly diminished in this study. Communication between each participant was prohibited, and each experiment was conducted independently.

The output data fields of the experimental design include simulator time ($\mathrm{s}$), vehicle acceleration along the X direction of the body coordinate system ($\mathrm{km}·{\mathrm{h}}^{-1}$), vehicle acceleration along the Y direction of the body coordinate system $(\mathrm{km}·{\mathrm{h}}^{-1}$), vehicle velocity along the X direction of the body coordinate system ($\mathrm{m}·{\mathrm{s}}^{-2})$, and vehicle acceleration along the body coordinate system Y direction speed ($\mathrm{m}·{\mathrm{s}}^{-2}$), steering wheel Angle ($\mathrm{rad}$), accelerator pedal torque $(\mathrm{daN}·\mathrm{m}$), brake pedal torque ($\mathrm{daN}·\mathrm{m}$), distance from vehicle axis to lane sideline ($\mathrm{m}$), vehicle centroid position X ($\mathrm{m})$, vehicle centroid position Y ($\mathrm{m})$, trigger time of takeover event ($\mathrm{s}$), trigger time of takeover ($\mathrm{s}$). A total of 368 experiments with 46 participants. The time range of analysis in this study is 10 s after the driver responds to the takeover, the state data is output once every 0.05 s, and each data contains 200 timestamps with 0.05 s interval.

3. Results

Following the multiple factor analysis process in Figure 1, two tests were performed. Test1 is a test for normality (The null hypothesis: the normal distribution is satisfied) for all groups and homogeneity (The null hypothesis: the homogeneity of variance of each group is satisfied) between groups, $p_1$ is the significance level of the normality test, and $p_2$ is the significance level of homogeneity of variance test. Test 2 is a test for multiple factor analysis (The null hypothesis: all groups have the same mean value.), the $p$ value was bilateral test value, α = 0.05.

3.1. Take Over Reaction Time (TOrt)

Table 2. All factors affecting TOrt, this table only lists the factors that affect the experimental metric.

The Dependent Variable	Factors	Test 1	Test 2	Statistics
TOrt	TORlt	$p_1$($5s$) = 0.054, $p_1\left(7s\right)$ = 0.004, $p_2$ = 0.246,	Mann-Whitney U	U = 17,506, $p$ = 0.031
	traffic condition	$p_1$($5s$) = 0.078, $p_1\left(7s\right)$ = 0.005, $p_2$ = 0.246,	Mann-Whitney U	U = 17,475, $p$ = 0.029

shows all the factors affecting Tort, for example, the impact of TORlt on TOrt is studied, all TOrt samples are divided into two groups by 5 s and 7 s, and the results of the normality tests are as follows: $p_1$($7s$) = 0.054, $p_1\left(7s\right)$ = 0.004. The $p_1$ value of the latter group is less than α, rejecting the null hypothesis which does not meet the normality test and initial hypothesis of ANOVA. Therefore, a non-parametric test method namely the Mann-Whitney U test is adopted.

Figure 4 displays the TOrt distribution as a histogram and kernel density curve, with the vertical axis denoting the probability density calculated by Equation (16), $bin$ is the width of the histogram grouping.

$$probability density=\frac{Grouped sample·bin}{Total sample}$$

(16)

The mean TOrt is 1.678 s (See Figure 4). Considering the difference of people, the 85% quantile of TOrt is 2.3561 s, which is more valuable. The TOrt under 7 s TORlt (AVG. = 1.804 s) is greater than that under 5 s (AVG. = 1.552 s), which shows that under the 7 s lead time, the driver has more time to react (U = 12.36, $p$ = 0.031) (Table 2). When traffic conditions are complex, the TOrt of drivers will increase appropriately (complex traffic conditions: AVG. = 1.796 s; simple traffic conditions: AVG. = 1.56 s; U = 18.15, $p$ = 0.029) (Table 2). The modality of TOR has little influence on TOrt. There is no interaction among TORlt, traffic conditions, and the modality of TOR.

3.2. ApEn and Operating Order of Steering Wheel Angle and Pedal Torque

Table 2 shows all the factors affecting ApEn of steering wheel Angle and pedal torque, Figure 5, Figure 6 and Figure 7 show the ApEn of the steering wheel angle, accelerator pedal torque, and brake pedal torque, respectively, when traffic conditions are taken into consideration. For ease of comparison, the mean points of the upper and lower distribution maps are joined to form solid lines. Traffic conditions significantly affect the regularity and complexity and regularity of the steering wheel Angle, torque input of accelerator and brake pedal torque, when the traffic conditions are complex, the ApEn of the driver’s steering wheel Angle is smaller (U = 17,562, $p$ = 0.035) (

Table 3. All factors affecting ApEn of steering wheel angle and pedal torque, this table only lists the factors that affect the experimental metric.

The Dependent Variable	Factors	Test 1	Test 2	Statistics
ApEn of Steering wheel Angle	traffic condition	$p_1\left(complex\right)$ = 0.0002 $p_1\left(simple\right)$ = 0.2000 $p_2$ = 0.008	Mann-Whitney U	U = 17,562, $p$ = 0.035
ApEn of accelerator pedal torque	TORlt	$p_1$($5s$) = 0.089, $p_1\left(7s\right)$ = 0.102, $p_2$ = 0.124	ANOVA	F = 13.509, $p$ = 0.00027
	traffic condition	$p_1\left(complex\right)$ = 0.015 $p_1\left(simple\right)$ = 0.291 $p_2$ = 0.241	Mann-Whitney U	U = 14,722, $p$ = 0.000005
	TORlt * 1 modality of TOR * traffic condition	$p_1\left(5s/A+V/complex\right)$ = 0.081 $p_1\left(5s/A/complex\right)$ = 0.097 $p_1\left(7s/A+V/complex\right)$ = 0.241 $p_1\left(7s/A/complex\right)$ = 0.031 $p_1\left(5s/A+V/simple\right)$ = 0.081 $p_1\left(5s/A/simple\right)$ = 0.097 $p_1\left(7s/A+V/simple\right)$ = 0.241 $p_1\left(7s/A/simple\right)$ = 0.041 $p_2$ = 0.001	Scheirer-Ray-Hare test	H = 6.311, $p$ = 0.012
ApEn of brake pedal torque	TORlt	$p_1$($5s$) = 0.120, $p_1\left(7s\right)$ = 0.079, $p_2$ = 0.251	ANOVA	F = 4.20, $p$ = 0.041
	traffic condition	$p_1\left(complex\right)$ = 0.078 $p_1\left(simple\right)$ = 0.200 $p_2$ = 0.089	ANOVA	F = 6.70, $p$ = 0.010

¹ A * B means the interaction of factor A and factor B.

) and the ApEn of the accelerator pedal torque is smaller (U = 14,722, $p$ = 0.000005) (Table 3), while the ApEn of the brake pedal torque is larger (F = 6.700, $p$ = 0.010) (Table 3).

The study additionally discovered that, in difficult traffic conditions, the value range and variance of each driver’s ApEn index were higher, indicating that different drivers’ takeover performance varied significantly.

Figure 8 shows that the operation complexity, regularity of accelerator pedal is also affected by TORlt (F = 13.509, $p$ = 0.00027) (Table 3). On average, the accelerator pedal torque in 5 s TORlt is larger than the ApEn in 7 s TORlt.

Moreover, Figure 9 shows that TORlt has an impact on the ApEn of brake pedal torque (F = 4.200, $p$ = 0.041) (Table 3). When opposed to 5 s, the braking behavior is less smooth when 7 s is warning in advance due to the lower ApEn of brake pedal torque under the 7 s TORlt.

Exploring the interaction between multiple factors, the study found the interaction between TORlt, traffic conditions and modality of TOR affects ApEn of accelerator pedal torque (H = 6.311, $p$ = 0.012) (Table 3), and ApEn with TORlt 5 s is significantly higher than that with 7 s. There is no interaction between other factors and TORlt dual factors. Figure 8a–c show that the ApEn distributions of various groups are comparable. The ApEn of the accelerator pedal torque of the 5 s and 7 s TORlt showed the opposite tendency in all samples of A+V and complex traffic conditions (See Figure 8d) caused by the interaction of three factors. The operation sequence provides additional insight into the driver’s driving habits,

Table 4. Driving operation time (relative to the time issued by TOR).

Action	Turn Steering Wheel (s)	Brake (s)	Accelerate (s)	Change Lane (s)
The timing	1.672	0.430	4.005	8.034

describes the time order of takeover operations.

Most drivers believed that, in the experimental scenario, they would first depress the brake pedal to slow down and then turn the steering wheel to lessen risks. The brake and gas pedals would be pressed simultaneously by around half of the drivers. On average, the brake pedal operates 1.242 s in advance of the steering wheel whereas the accelerator pedal operates 2.333 s behind the latter.

3.3. Mean and Standard Deviation of Velocity and Acceleration

All the factors affecting the mean and standard deviation of velocity and acceleration are shown in

Table 5. All factors affecting velocity and acceleration, this table only lists the factors that affect the experimental metric.

The Dependent Variable	Factors	Test 1	Test 2	Statistics
Mean longitudinal velocity	traffic condition	$p_1\left(complex\right)$ = 0.029 $p_1\left(simple\right)$ = 0.251 $p_2$ = 0.058	Mann-Whitney U	U = 17,240, $p$ = 0.017
	TORlt * traffic condition	$p_1\left(5s/A+V\right)$ = 0.232 $p_1\left(5s/A\right)$ = 0.094 $p_1\left(7s/A+V\right)$ = 0.067 $p_1\left(7s/A\right)$ = 0.121 $p_2$ = 0.076	MANOVA	F = 4.380, $p$ = 0.037
Mean of longitudinal acceleration	TORlt	$p_1\left(5s\right)$ = 0.102, $p_1\left(7s\right)$ = 0.059, $p_2$ = 450	ANOVA	F = 6.890, $p$ = 0.009
	traffic condition	$p_1\left(complex\right)$ = 0.130 $p_1\left(simple\right)$ = 0.045 $p_2$ = 0.063	Mann-Whitney U	U = 15,907, $p$ = 0.0004
Standard deviation of longitudinal acceleration	TORlt * traffic condition	$p_1\left(5s/A+V\right)$ = 0.104 $p_1\left(5s/A\right)$ = 0.097 $p_1\left(7s/A+V\right)$ = 0.065 $p_1\left(7s/A\right)$ = 0.060 $p_2$ = 0.102	MANOVA	F = 5.060, $p$ = 0.025

* B means the interaction of factor A and factor B.

.

As is shown in Figure 10, the difference between A and A+V does not significantly affect velocity and acceleration, therefore, the changes in mean and variance of longitudinal acceleration and velocity were studied without considering the modality of TOR.

Traffic conditions affect the longitudinal acceleration (U = 17,240, $p$ = 0.017) (Table 5). When the traffic conditions are complex, the longitudinal deceleration of the vehicle is larger shown in Figure 10c.

Except for the 7 s TORlt and complex traffic conditions, the driver has slowed down before the takeover in all other scenarios and maintains a speed of 100 $\mathrm{km}·{\mathrm{h}}^{-1}$ when it starts. In every situation, the speed will drop to its lowest point between 4 and 7 s after the takeover condition is triggered. The lowest speed can be sustained. Smaller speeds are similar among vehicles in complex traffic conditions and 5 s TORlt (F = 4.380, $p$ = 0.037) (Table 5), reaching about 22 $\mathrm{km}·{\mathrm{h}}^{-1}$ at 5 s and the speed of other takeover scenarios is reduced to about 50–60 $\mathrm{km}·{\mathrm{h}}^{-1}$ (see Figure 10a).

Figure 10b,d reflect the group differences in drivers’ speed and acceleration behaviors in the takeover stage. 5 s TORlt/complex traffic condition has the smallest variance (F = 5.060, $p$ = 0.025) (Table 5), and drivers’ control behaviors of speed and acceleration are relatively consistent.

3.4. Lane-Changing Behavior

All the factors affecting lane-changing behavior are shown in

Table 6. All factors affecting lane changing speed, this table only lists the factors that affect the experimental metric.

The Dependent Variable	Factors	Test 1	Test 2	Statistics
Lane changing speed	TORlt	$p_1\left(5\mathrm{s}\right)$ = 0.047 $p_1\left(7\mathrm{s}\right)$ = 0.089 $p_2$ = 0.024	Mann-Whitney U	U = 16,934, $p$ = 0.008

. In this scenario, most drivers had lane departure behavior and crossed the boundary, and the lane-changing rate was 93%. The average lane-changing speed was 0.302 $\mathrm{m}·{\mathrm{s}}^{-1}$. There is little difference in the lane-changing speed between 5 s and 7 s TORlt (U = 16,934, $p$ = 0.008) (Table 6). Under 7 s TORlt, the driver has a slower lane change speed-. This reasonably explains that the driver has less situational judgment time in 5 s TORlt.

3.5. Lateral Cross-Border Risk and Longitudinal Collision Risk

All the factors affecting lateral cross-border risk and longitudinal collision risk are shown in

Table 7. All factors affecting lateral cross-border risk and longitudinal collision risk, this table only lists the factors that affect the experimental metric.

The Dependent Variable	Factors	Test 1	Test 2	Statistics
Lateral Cross-border risk	TORlt * modality of TOR	$p_1(5s/A+V)$ = 0.320 $p_1\left(5s/A\right)$ = 0.102 $p_1\left(7s/A+V\right)$ = 0.132 $p_1\left(7s/A\right)$ = 0.200 $p_2$ = 0.117	MANOVA	F = 4.686, $p$ = 0.031
Longitudinal collision risk	traffic condition	$p_1\left(complex\right)$ = 0.042 $p_1\left(simple\right)$ = 0.070 $p_2$ = 0.025	Mann-Whitney U	U = 17,096, $p$ = 0.012
	TORlt * modality of TOR	$p_1\left(5s/A+V\right)$ = 0.081 $p_1\left(5s/A\right)$ = 0.097 $p_1\left(7s/A+V\right)$ = 0.241 $p_1\left(7s/A\right)$ = 0.191 $p_2$ = 0.078	MANOVA	F = 6.530, $p$ = 0.011

* B means the interaction of factor A and factor B.

. Complex traffic conditions will increase the complexity, interference degree of traffic, sense of urgency of the driver, and the risk of crossing the boundary when takeover. TORlt and modality of TOR interactively affect the experiment scenario of lateral risk (F = 4.686, $p$ = 0.031) and longitudinal risk (F = 6.530, $p$ = 0.011) shown in Figure 11, under the scenario of 5 s TORlt, A and A+V have little effect on lateral and longitudinal risk, but A+V warning can reduce the risk of takeover the lateral cross-border and longitudinal collision risk under the scenario of 7 s TORlt.

4. Discussion

The characteristics of takeover behavior across all drivers and the effects of different factors are outlined in this section. Figure 12 summarizes the factors influencing takeover behavior.

4.1. Operation Order of Steering Wheel and Pedals

The operation order of the steering wheel and pedals provides additional insight into the driver’s driving habits. In the obstacle scenario, the general operation order of the driver is: press down the brake pedal → adjust the steering wheel → accelerate and change lanes. The source of this habit in drivers may be common sense. It is preferable to turn after reducing the speed as opposed to turning when driving at a high speed and hitting a barrier. Most drivers will drive cautiously and avoid obstacles because turning unexpectedly while moving at a high speed increases the probability of losing control of the car.

4.2. Effects of the Single Factor

4.2.1. TORlt

TORlt affects TOrt. If a warning is sent 7 s earlier than if it is sent 5 s earlier, the driver will have more time to react when an obstacle arises in front of them (5 s TORlt: AVG. = 1.552 s, 7 s TORlt: AVG. = 1.804 s). This conclusion is consistent with the study (5 s TORlt: AVG. = 1.45 s, 7 s TORlt: AVG. = 1.78 s) of Gold et al. [18], the experimental result of the latter one, however, was predicated on the requirement that only 28 drivers received A+V modality warnings. There are more driving scenarios and drivers in this experiment.

TORlt affects the smoothness of acceleration and braking operation after takeover. The driver will accelerate more cautiously but brake more smoothly if the warning information is sent 5 s in advance; If it is sent 7 s in advance, the driver will accelerate more smoothly but brake more cautiously.

TORlt affects lane-changeing speed. Under 7 s TORlt, the driver has a slower lane change speed. This reasonably explains that the driver has less situational judgment time in 5 s TORlt.

4.2.2. Modalities of TOR

In all of the scenarios examined, the difference in driver behavior between the two modalities A+V and A was not significant. The static vision was conveniently disregarded in the obstacle scenarios.

4.2.3. Traffic Conditions

Traffic conditions affect the smoothness of steering wheel and pedal operation when the driver takes over. When traffic is complex, the driver won’t make complicated detours when taking control, and the braking action is more complicated and irregular while the accelerating action is simpler. This suggests that in an emergency, the driver is more likely to focus on the two sides closest to them. The driver will undertake more cautious vehicle offset and acceleration operations, as well as more explicit braking when there are vehicles on both sides.

The experimental conclusions can be complementary to previous studies [17,21,24]. Drivers will lessen steering and acceleration actions when there are more vehicles around but will enhance braking operations. While braking behaviors seem more agitated, steering and acceleration are smoother.

4.3. Interaction Effects of TORlt, Modalities of TOR, and Traffic Conditions

The interaction between TORlt, Traffic Conditions and Modalities of TOR affects ApEn of accelerator pedal torque. When categorized as a single or two-factor, the ApEn of the 5 s TORlt group was higher than that of the 7 s TORlt group. When grouped by 3 factors, the groups with the lowest ApEn of accelerator pedal torque were those with 5 s TORlt, complex traffic conditions, and A+V waring. The A+V warning message, to some extent, causes the driver to accelerate more smoothly assuming there is just a 5 s warning period and several other vehicles are present.

The interaction between TORlt and Traffic Conditions affects the magnitude of speed reduction. As can be observed, the reverse acceleration increases with scene urgency, indicating an increase in braking amplitude. The conclusion is consistent with the study by Harbluk et al. [40]. where the more urgent the condition, the more frequent the deceleration. Du et al. [26] concluded that low TORlt and high traffic scenarios were associated with poorer driver performance and higher mental stress load, this could be the cause of drivers panicking and slowing down.

If the warning is too early, static visual signals do not serve the driver. If the warning is sent 5 s in advance, there is no difference between the two warning modalities (A+V, A) in terms of Lateral cross-border risk or Longitudinal collision risk. The combined A+V warning, however, can significantly lower Lateral cross-border risk if it is sent 7 s in advance. In 5 s TORlt scenarios, valuable information can not be provided by visual warnings, and 5 s TORlt is too urgent for drivers to receive visual information in a timely and thorough manner. The conclusion is in line with other studies that showed how simple it is for drivers to overlook static visual information [41]. The combination of static visual and auditory information is anticipated to make autonomous driving safer if emergencies are broadcasted 7 s earlier when perception levels of future autonomous cars are enhanced.

5. Conclusions

The short-term effects (including interaction effects) among traffic conditions, the TORlt, and the modality of TOR on human driver’s takeover performance are explored. The ApEn to measure the smoothness of operation, lateral cross-border risk and longitudinal collision risk, operation sequence, and velocity and acceleration are used to analyze takeover behavior and the following conclusions were reached:

(1)

The experimental scenario to take over the order of operation is similar: Step on the brake pedal, adjust the steering wheel, accelerate, and change lanes.

(2)

When the traffic conditions are complex, drivers have a larger ApEn of steering wheel Angle and brake pedal torque, and a smaller ApEn of acceleration pedal torque. In the 5 s TORlt case, drivers have a smaller ApEn of brake pedal torque.

(3)

The interaction between TORlt, traffic conditions and modality of TOR affects ApEn of accelerator pedal torque, generally speaking, the ApEn of accelerator pedal torque under 5 s TORltis higher affected by just one or just two factors including TORlt. but if the warning modality is A+V, the traffic conditions are more complex, the ApEn of accelerator pedal torque under 5 s TORlt is lower than that under 7 s TORlt

(4)

In 5 s TORlt/ difficult traffic conditions, the situation gets more urgent, which is likely to raise the level of risk compared to other groups. The driver slows down more, which makes dangerous turning and braking actions simple to perform. When there is a straightforward traffic situation or a considerable TORlt (7 s), the motorist slows down less.

(5)

Auditory and auditory + visual combinations did not singly affect the behavioral characteristics of experimental scenarios.

(6)

In this study, only under the interaction effect of 7 s TORlt and the “visual + auditory” modality, the lateral cross-border risk and longitudinal collision risk are significantly reduced.

Several points needs to be improved in this study:

(1)

Due to equipment limitations, this study only assesses the driver’s takeover behavior based on how smoothly the vehicle operates after the takeover, how smoothly the driver operates, and how likely it is that the driver will collide transversely or longitudinally while driving. The physical and psychological responses of the driver may also be considered as one of the evaluation criteria. The addition of an eye tracker and a heart rate monitor will be helpful for the upcoming study.

(2)

The experimental setting in this study attempts to mimic the actual driving environment as closely as possible. There are differences between the real car and the driving simulation equipment even if all drivers utilize the same equipment. Additional comparative testing is necessary to determine whether the experimental results are applicable to other road conditions or real-world driving situations.