Take-over performance in evasive manoeuvres

We investigated after e ﬀ ects of automation in take-over scenarios in a high-end moving-base driving simulator. Drivers performed evasive manoeuvres encountering a blocked lane in highway driving. We compared the performance of drivers 1) during manual driving, 2) after automated driving with eyes on the road while performing the cognitively demanding n-back task, and 3) after automated driving with eyes o ﬀ the road performing the visually demanding SuRT task. Both minimum time to collision (TTC) and minimum clearance towards the obstacle disclosed a substantial number of near miss events and are regarded as valuable surrogate safety metrics in evasive manoeuvres. TTC proved highly sensitive to the applied de ﬁ nition of colliding paths, and we prefer robust solutions using lane position while disregarding heading. The extended time to collision (ETTC) which takes into account acceleration was close to the more robust conventional TTC. In line with other publications, the initial steering or braking intervention was delayed after using automation compared to manual driving. This resulted in lower TTC values and stronger steering and braking actions. Using automation, e ﬀ ects of cognitive distraction were similar to visual distraction for the intervention time with e ﬀ ects on the surrogate safety metric TTC being larger with visual distraction. However the precision of the evasive manoeuvres was hardly a ﬀ ected with a similar clearance towards the obstacle, similar overshoots and similar excursions to the hard shoulder. Further research is needed to validate and complement the current simulator based results with human behaviour in real world driving conditions. Experiments with real vehicles can disclose possible systematic differences in behaviour, and naturalistic data can serve to validate surrogate safety measures like TTC and obstacle clearance in evasive manoeuvres.


Introduction
Vehicles with increasing levels of automation will allow drivers to delegate longitudinal and lateral control, to take their eyes off the road and engage in activities unrelated to driving (SAE J3016, 2016).Drivers will have to resume manual driving in conditions not yet supported by automation such as complex urban traffic and adverse weather.Transitions between manual driving and various levels of automation can be initiated by the driver while in other cases the automation will take the initiative and will request the driver to resume control (Lu et al., 2016).Knowledge of human performance in these so-called take-over requests (TOR) is essential in particular to design fallback procedures dealing with automation limitations and failure.
Extensive experimental research on TOR has shown that after using automation, drivers need a sufficient time budget to generate effective control actions (e.g.Damböck, 2013;Zeeb et al., 2015).Here we define the available time budget as the time between the TOR or an equivalent stimulus in manual driving and the moment when an accident would occur when the driver would take no action.Thus, the time budget captures the time available for perception and rebuilding of situation awareness, response selection and response execution.
The take-over process has been extensively analysed in terms of reaction times towards the TOR including the "gaze reaction time", indicating the first glance from non-driving related objects towards the road or driving related interfaces, the "intervention time" at which a first steering or brake/throttle action is observed and the effectiveness of control actions in terms of preventing rule conflicts or accidents (e.g.Gold et al., 2013;Hergeth et al., 2015;Kerschbaum et al., 2014;Merat et al., 2014;Petermann-Stock et al., 2013;Zeeb et al., 2015).Literature studies showed substantially reduced workload with automation (de Winter et al., 2014), while after effects of automation include significant effects on following distance which is often lower after using automation reflecting a trend to adopt following distances for manual driving similar to the automation (Skottke et al., 2014).Regarding steering performance a somewhat increased lateral path deviation and standard deviation of steering wheel angle are found after using automation (Skottke et al., 2014).However very limited evidence is available regarding effects of using automation on the performance quality and dynamics of evasive manoeuvres.In some TOR studies vehicles overshoot the target lane followed by oscillating or poorly damped stabilisation (e.g.Fig. 4 in Gold et al., 2013).However, similar overshoots are found in manual evasive manoeuvres (e.g.Katzourakis et al., 2014).Thus the current knowledge does not disclose the precision and safety margin with which drivers pass other (stationary) vehicles, and their ability to perform rapid lane changes and precisely stabilise the vehicle in the target lane without overshoot into other lanes.
With longer time budgets (e.g. 8 s) no or few accidents are observed thus creating a need for alternative metrics representing dynamics, precision and risk.Here we need so-called "surrogate safety metrics" reflecting the criticality of near accident conditions (see Gettman and Head, 2003;FHWA, 2008;Laureshyn et al., 2010;Tarko, 2012;Wu and Jovanis, 2012;Young et al., 2014).Hayward (1972) defined time to collision (TTC) as "the time required for two vehicles to collide if they continue at their present speed and on the same path".This definition is intuitive in cases where braking is needed to resolve critical scenarios, but sees complications in evasive manoeuvres where the projected path rapidly varies with steering actions.In deriving TTC, effects of steering and braking are often simplified or ignored and general guidelines describe both simplified as well as more complex definitions (SAE J2944).Hence this paper investigates the relevance of various TTC definitions in evasive manoeuvres.TTC particularly captures braking performance, but provides limited information on steering performance.Time to line crossing (TLC) or standard deviation of lateral position (SDLP), are useful metrics for slow lane and road departures (e.g.van Leeuwen et al., 2015), but are hardly relevant in rapid evasive manoeuvres where fast changes of TLC and SDLP occur in well performed manoeuvres, thus creating a need for alternative metrics capturing steering performance.
Hence we pursued the following objectives: • To establish a set of metrics quantifying driver performance in rapid evasive manoeuvres in terms of dynamics and risk.
• To quantify the effect of prior use of automation on the performance of rapid evasive manoeuvres following take-over requests.
A range of potentially relevant performance metrics was evaluated in manual driving and take-over conditions using existing driving simulator data.

Experimental data
We selected existing data enabling a comparison of evasive manoeuvres during take-over after using automation, with equivalent manoeuvres in manual driving.We focussed on SAE level 3 automation defined as "The sustained and operational design domain (ODD) specific performance by an automated driving system (ADS) of the entire dynamic driving task (DDT) with the expectation that the DDT fallback-ready user is receptive to ADS-issued requests to intervene, as well as to DDT performance-relevant system failures in other vehicle systems, and will respond appropriately" (SAE J3016, 2016).In selecting such data it emerged that many recent studies evaluate evasive manoeuvres in response to takeover requests, but unfortunately most studies do not include a manual driving baseline condition.The selected data is summarized in Table 1, and derives from experiments reported by Radlmayr et al. (2014) and Gold et al. (2013).Both studies investigated evasive manoeuvres when drivers encountered a blocked lane during highway driving in a high-end moving base driving simulator.Radlmayr et al. (2014) Participants drove 120 km/h on a three lane highway when an obstacle consisting of two stationary vehicles with flashing warning lights appeared at their current lane at a distance of 233 m representing a time budget of 7 s.The obstacle appeared suddenly to make sure the time budget would be the same for all conditions.Participants could prevent a collision by braking and/or performing a lane change.Each participant performed one trial including 4 events with intervals of about 7 min in a randomised order.In condition 1 participants were on the middle lane with traffic on the left and right lanes with a density of approximately 30 vehicles/km.There was no safe gap, but the blocking vehicles cleared the left and right lane roughly 2-3 s after the TOR, allowing a lane change (after braking initially).In conditions 2-4 no traffic was present, and participants were either on the right, left or middle lane.

Experimental conditions
The 48 participants (38 men) had a mean age of 33.5 years (SD = 9.0).A between participants design was applied, testing evasive manoeuvres in: group 1) manual driving performing the cognitively demanding n-back task, group 2) automated driving performing the nback task, and group 3) automated driving performing the visually demanding SuRT task with eyes off the road.Group 2 and 3 used a highly automated driving system and received a take-over request (TOR) consisting of a high-pitched tone in combination with an icon change in the instrument panel.Manual drivers (group 1) received an identical warning tone (no icon change).Upon the TOR the automation was deactivated resulting in 0.4 m/s 2 deceleration without deviations of lateral position, as all take-over events were located on straight course sections.Groups 1 and 2 performed the cognitive n-back task (e.g.Reimer et al., 2010) in the form of a two-back task prior to the take-over process.Group 2 was free to look around, not instructed to keep their eyes on the road and not instructed to monitor anything regarding the automation.Thus we classify group 2 as SAE level 3 automation, but in practice drivers almost continually gazed forward (eyes on the road).Group 3 performed the visual Surrogate Reference Task (ISO14198, 2012) on a screen below the mid console and did not monitor the road.In all conditions the cognitive demand of drivers was recorded and analysed using the Detection Response Task (e.g.Bengler et al., 2012;Conti et al., 2012).The DRT measures the reaction time between a tactile signal presented on the participant's neck via a vibrating node and the affirmation of noticing the signal by pushing a button.Gold et al. (2013) This experiment was performed in the same moving base driving simulator, with very similar conditions, and using the same automation system as in Radlmayr et al. (2014).The 32 participants (24 male) had a mean age of 27.6 years (SD = 8.7).Participants were driving in the right lane of a three-lane freeway.The take-over scenario was represented by an accident on the right lane.A leading vehicle obscured the accident, and when the TOR was prompted, the leading vehicle suddenly swerved to the middle lane and uncovered the crashed vehicles.No other traffic was present.A between participants design was applied testing manual driving without additional task (further referred to as group 4), and automated driving with eyes off the road while performing a SuRT task (group 5).The TOR consisted of a high pitched tone.In the manual condition no warning was given.In half of the participants the time budget was 7 s (233 m) and in the other half it was 5 s (167 m).One event occurred on average 14:26 (SD = 2:27) min after the start of each test.The manual condition was tested once per participant.The automated condition was tested twice and we now evaluated both tests, whereas Gold et al. (2013) only evaluated the first test.

Visual perception of the obstacle
Assuming a width ∼3 m the obstacle was initially visually perceived as 0.7 deg wide with an angular rate of 0.1 deg/s at 233 m or 1 deg with 0.2 deg/s at 167 m.At TTC = 5.5 s (which is 1.5 s after TOR with 7 s time budget) angular rate exceeded the 0.17 deg/s threshold at which drivers can give reasonable estimates of TTC (Hoffmann and Mortimer, 1994).Being ≥0.7 deg obstacle width is sufficient for TTC estimation even with monocular vision (Gray and Regan, 1998).

Analysis
The original papers provide a detailed analysis of reaction times including gaze reaction time, hands on wheel time, intervention time, and visual scanning of the road using direct sight and mirrors.This paper focuses on the quality and dynamics of the actual evasive manoeuvres.The analysis was restricted to a window from the TOR at 233 m (7 s time budget) or 167 m (5 s time budget) before the obstacle up to 120 m beyond the obstacle.A longer window would allow further study of stabilisation and lane keeping accuracy but participants reactivated the automation on average 130 m after passing the obstacle.Thus we limited the analysis to 120 m in all participants, and limited analysis up to the point of activating the automation if this was before 120 m.In all cases this window included passing the obstacle and stabilisation in the new lane.
Since this paper focuses on quality and dynamics of evasive manoeuvres, events where drivers did not perform an evasive manoeuvre were excluded from the analysis.Non effective manoeuvres resulting in collisions and road departures were included.Events where the obstacle was passed using the hard shoulder were excluded.Occasionally, drivers reactivated the automation after the lane-change but before passing the obstacle.These events were also excluded as they did not disclose the performance in manual evasive manoeuvres.We used the following geometric values: lane width = 3.5 m, vehicle width = 2 m, vehicle length = 4 m, and assumed the obstacle to block the entire lane width of 3.5 m.
A comprehensive set of performance metrics was derived from the original data, and correlations between metrics were evaluated to indicate in how far these metrics provide independent information about the quality of the take-over.

Steering & braking
The lane change direction (left/right) and the number of lanes shifted (0/1/2) was derived from the lateral position when passing the obstacle.Steering and braking response times were derived relative to the TOR when using automation or the warning and appearance of the obstacle when driving manually.The onset of relevant steering and braking actions was detected as described below.The intervention time was derived as the minimum of the steering response time and the brake response time.
The steering response time (RT steer ) was based on the point in time where the steering wheel angle exceeded 2°in the direction of the lane change (Gold et al., 2013).In some events the obstacle was successfully avoided with peak steering angles as low as 3°.To timely detect such steering actions, for cases with peak angles below 10°, the threshold was lowered to 20% of the peak steering angle but not below 1°.Some events contained early minor steering actions not leading to a lane change.These include corrective steering in manual driving and the placement of the hands on the wheel in TOR.To robustly detect the onset of evasive steering actions we selected the last onset of steering before reaching the maximum lateral velocity or entering a new lane.
The brake response time (RT brake ) was based on the first point in time where the brake pedal was depressed more than 10% of the available stroke.Vehicle deceleration was close to linear with 0.4 m/s 2 deceleration without braking, 2.3 m/s 2 with 10% braking, 8 m/s 2 with 40% braking and saturating ∼11 m/s 2 deceleration.ABS prevented brake lockup and ESC stabilised vehicle heading.Variations showed credible and similar results with brake detection thresholds between 0 and 10%, while thresholds of 15%, 20% and 30% resulted in several late and missed detections.Brake detections as a function of detection threshold reduced from n = 188 detections with 0% threshold, to n = 185, at 1%, n = 180 at 2%, n = 174 at 5%, n = 152 at 10%, n = 143 at 15%, n = 128 at 20%, n = 96 at 30% in the 243 events.In line with our earlier work the threshold of 10% was selected to focus on effective interventions rather than tentative braking.Even with a short reaction time of 1 s, continuous braking at 10% would result in 308 m stopping distance and be insufficient to prevent a collision with the obstacle which is located at 233 or 167 m.For less critical conditions we suggest to follow SAE J2994 (2015) which recommends a threshold of 1% when the foot is not originally on the destination pedal as a level which "can be reliably detected with contemporary sensors and distinguished from signal noise".
The magnitude of the steering action was represented by five metrics, namely the first peak of the lateral acceleration (Yacc peak1 ), the largest peak (Yacc max ), the peak steering wheel angle (steer max ), the peak yaw angle (yaw max ) representing heading, and the peak lateral velocity (Vy max ).In order to capture the aggressivity of the initial manoeuvre, Yacc peak1 , yaw max , steer max and Vy max were derived selecting the largest steering action in the direction of the lane change before reaching the obstacle.The magnitude of the braking action was represented by two metrics, the peak deceleration (Xacc min ) and the minimum velocity (V min ).In all conditions the initial velocity was ∼120 km/h and hence the velocity change was not analysed separately.The peak resultant acceleration Acc max was derived to capture usage of the available tyre grip by steering and braking.

Time to collision (TTC)
The criticality of the evasive manoeuvre was evaluated using the time to collision (TTC) defined as "the time required for two vehicles to collide if they continue at their present speed and on the same path" (Hayward, 1972;FHWA, 2008;SAE J2944, 2015.For stationary objects, TTC equals:  with obstacle distance dx, and vehicle velocity v.For larger distances Eq. (1) approximates the visually observed TTC v defined as: with visually perceived angle of the obstacle ϕ, and its time derivative ϕ '.The variable τ or inverse TTC v represents the relative visual expansion of the obstacle, which is commonly referred to as looming.As outlined in many papers, drivers are well able to estimate and respond to τ or TTC v and are not very precise in estimating distance or (relative) speed (e.g.Lee, 1976;Gray and Regan 2005;Hoffmann and Mortimer, 1996;Markkula et al., 2016).
The common TTC definition in Eq. ( 1) disregards acceleration.The Enhanced TTC (ETTC) takes the current acceleration into account, assuming constant acceleration rather than constant velocity (van der Horst, 1990; SAE J2944; Chen et al., 2016), and is sometimes referred to as ETTA.For a stationary object, ETTC equals: with vehicle velocity v, obstacle distance dx, and forward vehicle acceleration a. ETTC is defined only at instances where a > v 2 /(2 dx).At other instances the braking acceleration (-a) is sufficient to stop before reaching the obstacle.More detailed ETTC formulations taking into account lead vehicle acceleration and response delays can be found in Winner et al. (2016).
Minderhoud and Bovy ( 2001) described other "extended TTC" measures being Time Exposed TTC and Time Integrated TTC integrating critical TTC levels over time.Such an integration is relevant for evaluation of longer driving periods but was deemed irrelevant for the single events studied in this paper.
We evaluated TTC (Eq.( 1)) and ETTC (Eq.( 3)) towards the obstacle on the current lane as insufficient data was available for systematic evaluation of TTC towards other vehicles.Since we only consider stationary objects TTC equals time headway (in car following TTC and time headway are independent, see Vogel, 2003).At the TOR the TTC was equivalent to the time budget (7 or 5 s).From that moment TTC decreased until participants braked to a sufficient extent to postpone or prevent a collision.The minimum TTC is evaluated up to the collision free point, where drivers have steered sufficiently to pass the obstacle.Two definitions of the collision free point have been evaluated (see Fig. 1).In our earlier analyses the collision free point was based on the vehicle lateral position in the lane.We reanalysed all data, and the TTC L using lateral position was calculated up to the point when the entire vehicle front entered the new lane (Fig. 1).This definition disregards the vehicle heading which determines whether the current vehicle path will lead to a collision (e.g.van der Horst, 1990;SAE J2944, 2015.Hence, we also derived the heading based TTC H up to the point where the vehicle path no longer crossed the obstacle.At large distances this happens already with minor steering, and as illustrated in Fig. 1, TTC H will generally exceed the commonly used TTC L .ETTC L and ETTC H were derived as minima of ETTC up to the same collision free points as TTC L and TTC H .

Steering accuracy
Drivers often overshot the center of the new lane, which may be functional as it facilitates a rapid manoeuvre, but may also indicate imprecise control and may lead to lane and road departures (see Katzourakis et al., 2014).Two overshoot values were derived, the initial overshoot (overshoot 1 ) defined as the lateral position at the first point in time after the evasive manoeuvre with a zero lateral velocity, and the maximum overshoot (overshoot max ).Overshoot 1 represents the precision of the initial lane change, and overshoot max represents additional lateral movement and stabilisation in the new lane (Fig. 2).Overshoot 1 can be negative indicating undershoot of the center of the new lane while overshoot max is generally positive, indication movement beyond the center of the new lane.
Collisions with the obstacle were evaluated by means of the minimum clearance (distance) of the vehicle swept path with respect to the obstacle where negative values indicate collisions.Roadway departures towards the hard shoulder were evaluated by means of the minimum road side clearance (being the opposite of the maximum roadway departure).Thus for both obstacle clearance and road side clearance positive values indicate a safety margin.

Statistics
Main effects were evaluated using linear regression, creating a model describing the above performance metrics as: The experimental conditions were captured by four independent variables: Automation (0 = off & 1 = on), Group2 (1 = group 2 & 0 = all other groups), Traffic being present (0 or 1), and Repetition (0-3) being zero for the first event encountered by a participant.The variable Group2 was introduced to capture the effect of the cognitive nback task instead of the visually distracting SuRT when using automation.The variable Repetition captures a possible learning effect which proved significant in similar experiments (Körber et al., 2016;Gold and Bengler, 2014).The constant C 0 captures the baseline condition being manual driving without traffic in the first TOR encountered by the participant.
The regression coefficients C in Eq. ( 3) and their significance were estimated minimizing the root mean squared error of the linear regression versus the experimental results.The false discovery rate was controlled for with Benjamini and Hochberg's (1995) method.
Automation resulted in later interventions thus reducing the available time to perform the evasive manoeuvre (Table 3).To compensate for this effect, the TTC at the intervention time was entered into an additional linear regression.As mentioned in the text in the results section several effects of automation became insignificant in this additional regression, thus indicating that these effects are due to the limited remaining time, and not to an essentially different response after automation.
The frequencies of collisions, critical TTC (TTC < 1s), and critical obstacle clearance (< 0.25m) for manual and automated driving were compared using a two-sided Fishers exact test.

Results
Example results are shown in Figs.1-3.Correlations between the 19 performance metrics are presented in Table 2. Means and standard deviations per group are presented in the upper part of Table 3 and significance and coefficients of the regression are in the mid and lower parts of Table 3.
Fig. 4 shows distribution functions for all data, and Fig. 6 shows distributions for limited conditions.
Vehicle paths were similar with and without automation.Paths showed substantial variation in control strategies with steering only evasive manoeuvres, combined steering and braking, and in some cases a full stop before performing the evasive manoeuvre (Fig. 3).After passing the obstacle drivers either continued in the new lane, returned to their original lane, or occasionally stopped at the hard shoulder.

Comparison to normal evasive manoeuvres
The onset of braking occurred after ∼2.5 s with automation and significantly earlier with manual driving (see RT brake in Table 3 and Fig. 4.).At braking onset the TTC had a median value of 5.4 s in manual driving (Group 1), and 5.1 s with automation (Groups 2-3) with 7 s time budget.These TTC values are similar to naturalistic (near) accident data where braking mostly often occurred within a second after TTC decreased below 5 s (Markkula et al., 2012).Velocity was reduced from 120 km/h to an average of 75 km/h in groups 1-3, 113 km/h in group 4, and 86 km/h in group 5. Steering was mostly initiated after braking (see RT steer in Table 3 and Fig. 4.).The median TTC at steering onset was 5.7 s in manual driving (Group 1) and 5.1 s with automation (Groups 2-3) with 7 s time budget.These are comparable to naturalistic data showing median TTC at steering onset around 3 s at 75 km/h (Chen et al., 2015: Fig. 3), where the difference with our data can be explained by the fact that Chen reported minimal values over multiple events within each participant.Winner et al. (2016Winner et al. ( , page 1165) ) summarize that with TTC < ∼0.6 s evasion is no longer physically possible, at TTC < 1 s a driver is no longer capable of evasive action in practice, at TTC < 1.6 s evasive action is regarded as dangerous, and at TT-C = 2.5 s drivers "feel no danger" where the first thresholds is derived analytically and the further thresholds derive from driver performance and subjective evaluation by Kodaka et al. (2003).Smith et al. (2003) compared normal and critical lane changes in a track study.Approaching a slower lead vehicle, the lane change onset was detected around TTC = 4 s when drivers were instructed to pass at the "last second they normally would" (referred as normal steering), and around TTC = 2.7 s when drivers were instructed to pass at the "last second they possibly could to avoid colliding with the target" (referred as hard steering).For the same data Fig. 8 in Kiefer et al., 2003 shows lateral accelerations around 1.4 m/s 2 in normal steering and 2.5 m/s 2 (4 m/s 2 at 90th percentile) in hard steering.Considering the distribution of lateral accelerations in Fig. 4. our results include both normal and hard steering.
The above comparisons show that even with automation drivers have sufficient time for "normal" braking and evasive manoeuvres, in particular for the 7 s time budget without traffic.As illustrated in Fig. 4 the intervention time shows a large variance in manual and after automated driving.Early interventions may directly relate to the auditory warning and the appearing obstacle, somewhat later responses can still provide adequate handling of the situation using a satisficing control strategy (e.g.Summala, 2007), while a minority of responses is Fig. 2. ETTC versus TTC.From t = 0.3 s the vehicle decelerates as the automation is disabled leading to a slight increase of ETTC.Between t = 2.2 and 4 s braking with varying intensity creates two periods with sufficient deceleration to stop before reaching the obstacle − hence ETTC does not exist in these periods.After t = 4 s braking is discontinued and the obstacle is passed at 85 km/h with limited acceleration and convergence of TTC and ETTC.After t = 7.3 the heading is sufficiently adapted to pass the obstacle which determines TTC H and ETTC H .After t = 8.1 the vehicle front is fully on the new lane which determines TTC L and ETTC L .The top graph illustrates the initial overshoot (overshoot 1 marked with red circles) and maximum overshoot (overshoot max marked with orange crosses) where in this case both overshoots are positive.

Performance metrics
The 19 metrics show several high correlations indicating redundancy (Table 2).Correlations are very similar for manual driving (lower left of diagonal) and automated driving (upper right of diagonal).High correlations (Table 2) and similar effects (Table 3) are observed for Yacc peak1 , Yacc max and Vy max , and here we prefer Yacc max which is easily derived and interpreted.Likewise Xacc min , Acc max , and V min are highly correlated.Acc max is dominated by braking (.96 correlation to Xacc min ) rather than steering (no significant correlation to Yacc max ).Hence in terms of acceleration presenting Xacc min and Yacc max would suffice.Vehicle yaw and steering angle are highly correlated and here we prefer the yaw (heading) angle which shows more significant effects in the regression presumably since it is less sensitive to brief steering actions.Furthermore yaw directly represents the vehicle kinematics and is independent of the steering properties of specific vehicles.The two overshoot metrics are highly correlated but still seem to present independent information, while only overshoot 1 shows significant effects in the regression.The four TTC and ETTC metrics are highly correlated and here we prefer the simple and robust TTC L as motivated below and in the discussion.TTC H exceeded TTC L on average with 1.5 s and the minimum ETTC was very close to the minimum TTC (Table 3).During initial braking ETTC often became non-existent and varied in time.However, when approaching the obstacle accelerations were generally low and TTC and ETTC converged to similar or identical minima (see Fig. 2).The obstacle clearance shows moderate correlations to other metrics including TTC.Hence we recommend using obstacle clearance as additional surrogate safety metric in evasive manoeuvres.

Effects of automation
The regression indicated that the intervention time increased 0.64 s with automation and 0.67 s with traffic (both p < 0.0001) and decreased 0.16 s with each subsequent repetition (p=0.0008).The intervention time did not differ between group 2 driving with automation distracted by the cognitive n-back task with eyes on the road and group 3 which was visually distracted with the SuRT task with eyes off the road.Fig. 4 shows a substantial spread in reaction times, which as shown in the regression is partially explained by the repetition and the presence of other traffic.Time budget also explains part of the variance, where in group 4 & 5 the intervention time increased from 1.81 to 2.38 s with time budgets of 5 and 7 s respectively (p = 0.01).Hence we reanalysed the data selecting only the first event encountered by each participant, only events without traffic, and only events with 7 s time budget (n = 43).As illustrated in Fig. 6 this somewhat reduced the variance in reaction times, while effects of automation on reaction times and the four TTC metrics remained similar and remained significant (p < 0.03 for RT brake and p < 0.005 for RT steer , intervention time, TTC L , TTC H , ETTC L , ETT H ).
After automation somewhat stronger steering and braking actions were observed.Effects of automation on steering were significant for the first peak (p < 0.0143) and maximum lateral acceleration (p < 0.0033) and the peak lateral velocity (p < 0.0112).After compensation for the later intervention time (see methods) effects of automation on steering were no longer significant.With automation stronger braking actions with lower minimum velocities were found but these effects derived only from groups 4 & 5 which included smaller time budgets.The minimum velocity was lower with traffic (p < 0.0001) accompanied with stronger but later braking with traffic (both p < 0.0001).The overshoots showed marginal effects (< 0.2 m) with limited significance.A negative road clearance indicating road departure was found in 18% of events but this was not unsafe due to the presence of a hard shoulder.The road clearance showed no significant effects in the regression (Table 3), but showed several significant correlations (Table 2) indicating more road departures with late intervention times and low TTC.
The four TTC and ETTC metrics showed almost identical trends.As expected automation reduced TTC (p < 0.0001 for all 4 metrics).Using automation with n-back task eyes on road (group 2), all TTC and ETTC metrics were in between those for groups 1 (manual) and 3 (automation with SuRT, eyes off road), and the difference between group 2 and other conditions was significant (p < 0.0042 for all 4 TTC metrics).After compensation for the later intervention time (see methods) these effects of automation and group 2 remained significant.With automation 4 collisions with the obstacle occurred while no collisions occurred in manual driving, but this effect was not significant (p = 0.3).Here it shall be noted that in this paper we only analysed events where the driver performed an evasive manoeuvre.The 4 collisions resulted in TTC L = TTC H = ETTC L = ETTC H = 0 with negative obstacle clearance.These 4 collisions all occurred in group 3 with automation and SuRT task.Two of these collisions occurred after a full stop close to the obstacle, where participants subsequently passed the obstacle with low velocity and strong steering actions, and could hardly prevent an incident as the vehicle had no reverse gear.In 21% of the events TTC L was between 0 and 1 s, and in 1.7% TTC H was between 0 and 1 s (Fig. 4).TTC L was more often below 1 s with automation (p=0.02).In 14% of events the obstacle clearance was between 0 and 0.5 m, and in 4.5% obstacle clearance was between 0 and 0.25 m (Fig. 4).The clearance towards the obstacle was significantly reduced with traffic (p < 0.0025).The clearance towards the obstacle showed no significant effects of automation in the regression, but obstacle clearance was more often below 0.25 m with automation (p = 0.042).These low values for obstacle clearance and TTC suggest critical interactions, as will be addressed further in the discussion.

Surrogate safety metrics
In order to capture the criticality of evasive manoeuvres we evaluated four TTC (and ETTC) metrics, clearance towards the obstacle and the road side, two overshoot metrics, and peak accelerations.Both TTC and clearance towards the obstacle disclosed a substantial number of near miss events and are regarded as valuable surrogate safety metrics in evasive manoeuvres.The four TTC metrics were highly correlated and showed almost identical trends with automation.However, TTC H exceeded TTC L on average with 1.5 s which illustrates that in evasive manoeuvres TTC is highly sensitive to the applied definition of colliding paths.A quick scan of studies applying minimal TTC in human evasive, lane change or cut-in manoeuvres in highway conditions, showed that minimal TTC is generally reported without definition of colliding paths.An exception is formed by Hegeman (2008) who used lane position in overtaking as: "TTC is defined as the time between the moment the back left wheel of the overtaker has crossed the center line until the oncoming vehicle is at the same level as the instrumented vehicle".
TTC H and ETTC H take into account the effect of vehicle heading on vehicle path and are thereby in formal agreement with common definitions of TTC (Hayward, 1972;FHWA, 2008;SAE J2944, 2015ISO15623, 2013).At large distances a limited vehicle rotation can ensure that the vehicle path no longer intersects with the obstacle (Fig. 1) but a further lateral motion into the target lane will provide more certainty to safely pass the obstacle.TTC values below 1 s are considered to represent critical interactions (van der Horst 1992;Young et al., 2014;Hyden and Linderholm, 1984).Considering the more conservative TTC L 21% of events would be in the critical range between 0 and 1 s, while considering TTC H only 1.7% of events would be critical.TTC H requires precise heading information which will often not be available in real vehicles.Derivation of TTC H is complex on curved roads where heading angular velocity should be included.Van der Horst (1990) concluded that "paths calculated with constant angular velocity easily take very peculiar shapes and lead outside the road".This means that taking angular velocity into account in the derivation of TTC can lead to low TTC values for interactions which do not represent an actual risk.
The extended time to collision (ETTC) which takes into account acceleration was generally close to the conventional TTC since longitudinal accelerations were limited when passing the obstacle.However the time course of ETTC can disclose the effectiveness of braking and is thereby useful to analyse single events (Fig. 2).ETTC may also be relevant with braking or accelerating lead vehicles (Chen et al., 2016).As the current results show that the 4 TTC/ETTC metrics provide similar conclusions we recommend using TTC L which is robust and easily derived but may be somewhat conservative.
The clearance towards the obstacle represents the lateral passing distance and appears to be a useful surrogate safety metric representing lateral precision.In 14% of events the obstacle was passed within 0.5 m, and in 4.5% the clearance was below 0.25 m indicating near miss conditions.Knowledge is lacking on safe lateral clearance distances in evasive manoeuvres but a general reference is the standard deviation lateral position which somewhat increased from 0.18 m in manual driving to 0.22 m after using automation in a driving simulator study by Skottke et al. (2014).Obstacle clearance shows only moderate correlations with TTC and hence we recommend including both obstacle clearance and TTC L in evasive manoeuvres.
The overshoot and road clearance metrics displayed a huge variation in the current data where participants often used multiple lanes and the hard shoulder without causing an accident.In some cases other traffic led to evasive actions.Thus with the current data the overshoot and road clearance metrics were not informative of driver's ability to perform a rapid lane change and precisely stabilise the vehicle in the target lane without overshoot into other lanes.Common metrics like peak acceleration, lateral velocity, steering angle and minimum speed proved informative but also displayed a huge variance associated with varying strategies involving steering only evasive manoeuvres, combined steering and braking, and in some cases a full stop before performing the evasive manoeuvre.Simpler experiments with a prescribed target lane and speed would thus be more informative of driver's ability to precisely stabilise a vehicle in rapid evasive manoeuvres.
To further establish and validate the accident risk as a function of TTC, obstacle clearance and other surrogate safety metrics it would be desirable to analyse naturalistic driving data including evasive manoeuvres.Controlled experiments like those analysed in the current study could then use such validated surrogate safety metrics to predict effects of automation on accident risk, using statistical models (e.g.Markkula 2012;Zheng et al., 2014;Sheridan, 2013;Gold et al., 2015;Gold, 2016).

Effects of automation
We quantified the effect of prior use of automation on the performance of rapid evasive manoeuvres following take-over requests, where drivers acted as fallback to automation.After using automation, drivers performed evasive manoeuvres with similar braking and/or steering strategies as during manual driving.In line with earlier analyses and other publications, the initial steering or braking intervention was delayed after using automation compared to manual driving.This resulted in lower TTC values and stronger steering and braking actions.However, the steering precision of the evasive manoeuvres was hardly affected after using automation as evidenced by a similar clearance towards the obstacle, similar overshoots and similar excursions to the hard shoulder.
In Radlmayr et al. (2014) we found no significant differences between the cognitively demanding n-back task (condition 2, where drivers generally gazed at the road), and the visually demanding SuRT task (condition 3, eyes off road).The current regression with recalculated and additional metrics confirms this conclusion for the intervention time, but both TTC L and TTC H show significant effects with automation with n-back task (condition 2) being intermediate to manual driving with n-back task (condition 1) and automation with SuRT task (condition 3).Thus we conclude that while using automation, effects of cognitive distraction can be similar to visual distraction for the intervention time with effects on obstacle avoidance being larger with visual distraction.
The intervention time decreased by 0.16 s with each repetition (TOR), and the brake reaction time decreased with even 0.29 s accompanied with similar reductions of TTC.This indicates a learning effect, which may also occur in real world deployment.However, the observed learning mainly derives from groups 1-3 with 4 rather similar TOR per hour, which may lead to exaggerated learning.
Fig. 4 shows that the distribution functions of obstacle and road side clearance are very similar with and without automation.Both with manual driving and with automation a substantial number of near misses occurred.This suggests that the evasive manoeuvres studied are risky both during manual and after automated driving.The number of collisions is not significantly higher with automation, but the surrogate safety metrics indicate a somewhat increased risk with automation (p = 0.02 for TTC L < 1 s and p = 0.042 for obstacle clearance < 0.25 m) where low TTC L indicate late manoeuvres while low obstacle clearance indicates passing the obstacle at a small distance.It shall be noted that performance may be affected by limitations of driving simulator validity.Low obstacle clearances indicate imprecise lateral control.However this may well relate to visual fidelity of the simulator which limits the driver's ability to accurately perceive the vehicle external geometry and lateral position and the vehicle motion relative to the obstacle.The projection employed a dome with ∼4 m radius (Fig. 5), without stereo vision and depth perception due to head motion, thus rendering perception at small distances less reliable.A high- end motion base was used, but even with high-end motion and vision, driving simulators are known to provide reduced accuracy in stopping a vehicle compared to real vehicle testing.Thus further research with real vehicles is needed to validate the current simulator based results in particular for obstacle clearance.
Fig. 6.Distribution of performance metrics, selecting only events (n = 43) with 7 s time budget, without traffic, and in the first event encountered by each participant (all 243 events are in Fig. 4).

Fig. 1 .
Fig. 1.Typical vehicle path and derivation of TTC.The green rectangle (x = −37) represents the vehicle at the first instance where its heading no longer intersects with the obstacle (red block) and is used to derive TTC H .The blue rectangle (x = −14) represents the vehicle after a sufficient lateral motion to bring the full vehicle front to the new lane, and is used to derive TTC L .The thin green and blue lines project the linear vehicle paths resulting from the current heading.The red rectangle (x = −2) represents the vehicle position with minimum obstacle clearance.
brake .The number of samples for automated driving was n = 166 in general and n = 103 for RT brake .The Pearson's correlation between metrics X and Y, with meansX and Y was derived as:

*
Comparing group 2 to all others.

Fig. 3 .
Fig. 3. Vehicle path and velocity while driving on the left lane automated with SuRT task (group 3) where the red line indicates a collision after a safe stop.Similar paths are found in other conditions.

Fig. 4 .
Fig. 4. Distribution of performance metrics for all data (*leftmost TTC cases represent 4 collisions with automation with TTC L = TTC H = 0 and negative obstacle clearance, obstacle clearance reached up to 4 m reflecting a double lane change, RT steer reached up to 30 s for events with a full stop).

Table 1
Driving Tasks and Conditions.Selected numbers of participants and events differ from the original papers as motivated in the text.In total 243 events were selected.

Table 3
Performance metrics per group and results of the linear regression.