Passenger opinions of the perceived safety and interaction with automated shuttles A test ride study with ‘hidden’ safety steward

A necessary condition for the effective integration of automated vehicles in our daily lives is their acceptance by passengers inside and pedestrians and cyclists outside the automated vehicle. 119 respondents experienced an automated shuttle ride with a ‘hidden steward on board’ in a mixed traffic environment in Berlin-Schöneberg. A mixed-method approach was applied gathering qualitative interview data during the ride and quantitative questionnaire data after the ride. Responses were classified into three main categories: (1) Perceived safety, (2) interactions with automated shuttles in crossing situations, and (3) communication with automated shuttles. Respondents associated their perceptions of safety with the low speed, dynamic object and event identification, longitudinal and lateral control, pressing the emergency button inside the shuttle, their general trust in technology, sharing the shuttle with fellow travellers, the operation of the shuttle in a controlled environment, and the behaviour of other road users outside the shuttle. Respondents pressed the emergency button inside the automated shuttle on 28 out of 62 test rides in order to test its behavior. They further expected to be more cautious in crossing the road before an automated shuttle due to the lack of eye contact with the human driver and a lack of trust in the behavior of the automated shuttle, and expected road users testing the automated shuttle due to the conservative driving behavior of automated shuttles. We recommend future research into the hypothesis that the acceptance of automated shuttles will be associated with the perceived safety of and their effective and intuitive interaction and communication with both passengers and other road users.


Introduction
The past years have shown a dramatic upsurge of scientific publications in the field of automated driving. Ample studies focused on automated vehicle technology, and examined impacts on transport, mobility and society. The user acceptance of automated vehicles is gaining attention as exemplified by our multi-level model on automated vehicle acceptance (MAVA) that summarizes current trends of acceptance research on the basis of 124 empirical studies (Nordhoff et al., 2019a). The acceptance of automated vehicles does not only depend on occupants inside automated vehicles -the car drivers and passengers -but also on vulnerable road users such as pedestrians and cyclists outside automated vehicles. Managing safe and efficient interactions between automated vehicles and pedestrians and cyclists provides a necessary condition for the successful deployment and acceptance of automated vehicles by vehicle occupants inside and pedestrians and cyclists outside automated vehicles .
T Safety is one of the basic human needs and a key requirement of automated vehicle acceptance (Fagnant and Kockelman, 2015;Garidis et al., 2020;Nordhoff et al., 2019aNordhoff et al., , 2019bXu et al., 2018). Several passenger cars now provide SAE Level 2 automation, which requires the permanent supervision of human drivers (SAE International, 2018). Similarly, SAE Level 4 automated shuttles are being tested at low speeds in semi-public controlled environments under the supervision of safety stewards on board. Higher automation levels (SAE Level 3+) will gradually reduce the level of supervision, allowing the engagement in eyes-off road activities, and ultimately allowing driverless operation. This sacrifice of self-control, however, requires vehicle occupants to exhibit more trust in the automation, which increases the relevance of perceived safety for automated vehicle acceptance.
Several studies investigated perceived safety from the perspective of human drivers in private automated cars. Using an online survey to investigate perceived safety of SAE Level 5 1 private automated vehicles, Moody et al. (2020) revealed that perceived safety is a function of a person's awareness of automated vehicle technology, socio-demographic characteristics (e.g., age, gender, education, employment, income, car ownership and use, residential area), and country-level variables. Xu et al. (2018) investigated the influence of direct experience with SAE Level 3 cars and found that trust served as determinant of perceived safety. Vehicle occupants, however, also consider others in their assessment of safety and acceptance. In the interview study of Nordhoff et al. (2019aNordhoff et al. ( , 2019b respondents considered sharing an automated shuttle with fellow travellers an uncomfortable aspect of automated vehicle usage. Sanguinettiet al. (2019) posit that sharing a ride with fellow travellers in automated vehicles poses a risk to the physical and emotional safety of its passengers. Besides fellow occupants, occupants' automated vehicle acceptance may further depend on concerns towards the safety of pedestrians and cyclists as vulnerable road users.
Pedestrians and cyclists will have to interact with automated vehicles -whether private automated passenger cars or public automated shuttles -in complex and mixed traffic situations Hagenzieker et al., 2020). Little is known about how pedestrians and cyclists perceive the safety of and interact and communicate with automated vehicles Hagenzieker et al., 2020;Mahadevanet al., 2018;Merat et al., 2018;Palmeiro et al., 2018;Rasouli and Tsotsos, 2018;Vissers et al., 2016). Filling these gaps in research is important as vulnerable road users constitute more than half of all traffic deaths on the road (WHO, 2018). In the survey study of Pyrialakou et al. (in press), the awareness of automated vehicles, level of automation, attitudinal variables, mode-choice related and socio-economic factors were associated with perceptions of safety of travelling near an automated vehicle. Merat et al. (2018) revealed that the presence of road markings and the type of environment affected perceptions of safety of pedestrians and cyclists: Respondents indicated to feel less safe in a shared space environment without road markings, and felt safest in a university campus environment rather than in an urban environment. In their study, pedestrians felt less safe when interacting with automated shuttles in comparison to manually driven vehicles. In Penmetsa et al. (2019), 6% of respondents who interacted with private automated cars on open streets considered automated cars safer than human driven vehicles, 71% did not experience a difference between a human driver or experienced no negative interaction with an automated car, 12% considered automated vehicles more cautious or slower, or had difficulties anticipating their movements, and 7% indicated to have witnessed a negative interaction with an automated car.
The interaction between human drivers and vulnerable road users is often regulated by informal cues such as gestures, eye contact and braking (Ackermann et al., 2019;Hagenzieker et al., 2020). In driverless automated vehicles these common forms of communication become obsolete due to the absence of a human operator Merat et al., 2018Merat et al., , 2019. This may induce additional risks, such as when vulnerable road users become hesitant or over-reliant in their interactions with an automated vehicle due to a lack of clarity on whether the vehicle they encounter is automated or manually driven . It is also likely that automated vehicles are being tested or blocked intentionally or unintentionally by vulnerable road users, e.g., to see how they react in near collision situations or because it is expected that these vehicles will always stop (Ackermann et al., 2019;Hagenzieker et al., 2020;Liu et al., 2020;Madigan et al., 2019;Merat et al., 2018). These situations can be avoided if automated vehicles and vulnerable road users effectively interact with each other, communicating their intentions to one another and agreeing on future motion trajectories . It has been suggested that a communication framework between all actors interacting with automated vehicles can compensate for the absence of a physical driver (De Clercq et al., 2019;Malmsten Lundgren et al., 2016;Merat et al., 2018). External human machine interfaces can inform other road users on the automated vehicle's driving mode and intention, cooperation manoeuvres, and perception of the environment, providing safe and intuitive interactions with automated vehicles Schieben et al., 2018).

Research objectives
The above challenges were addressed investigating opinions of passengers experiencing an automated shuttle in a mixed traffic environment in Berlin-Schöneberg that was accompanied by a 'hidden' steward onboard. The research objectives were: (i) To explore the safety perceptions of passengers of automated shuttles and how passengers perceive the safety of pedestrians and cyclists interacting with automated shuttles as external vulnerable road users (ii) To explore how pedestrians and cyclists envision their interactions with automated shuttles (iii) To explore the communication needs of pedestrians and cyclists interacting with automated shuttles A mixed-method approach was adopted triangulating qualitative interview data that was collected during the ride with questionnaire data obtained after the ride. The interview and questionnaire addressed occupants' perceived safety from a vehicle, individual and environmental perspective, their envisioned interactions and communication needs with an automated shuttle in crossing scenarios from the perspective of pedestrians and cyclists.

Automated shuttle, route and procedure
The automated shuttle 'Emily' from Easymile (2nd generation EZ10) operated on a 1500 m route on the EUREF office campus in Berlin-Schöneberg, sharing the road with conventional vehicles, pedestrians, cyclists and electric scooters (Fig. 1A). It was electrically powered and controlled the steering, braking, and acceleration on the basis of a pre-programmed map using Lidar (Light Detection Fig. 1. A (top left) Automated shuttle 'Emily' by Easymile on the EUREF campus in Berlin B (top, right) Accompanied test ride with two respondents (front), the interviewer and steward hidden as minute writer C (middle, left) Information display with route information D (middle, right) Operator interface E (below, left) Door opening button (top), button to pull out wheelchair ramp (middle), button to connect the shuttle to an external control room (below) (not active when test rides were performed) F (below, right) Emergency stop button. and Ranging) and geo-positioning technology. It ran at an average speed of 10 km/h and at a maximum speed of 13 km/h. A safety steward had to be on-board the shuttle to supervise its operations and intervene when necessary. For example, obstacles on the trajectory of the shuttle (e.g., parked bicycles or cars) had to be removed or overtaken manually by the steward.
Respondents were accompanied in the automated shuttle by the interviewer and the 'hidden steward'. The steward was introduced to respondents as 'minute writer' (Fig. 1B) to hide the true purpose of the steward and improve the respondents' sense of autonomy inside the shuttle. At the beginning of the test ride, respondents were instructed to interact with the shuttle to get acquainted with its characteristics such as the display showing the route (Fig. 1C), the operator interface (Fig. 1D), and the buttons to open the door, pull out the wheelchair ramp and connect the shuttle to an external control room (Fig. 1E). Respondents were also allowed to press the emergency button inside the shuttle (Fig. 1F). When respondents pressed the emergency button, the interviewer asked them why they pressed the button. To simulate a more realistic public transport experience, respondents shared the ride with fellow travellers (n > 1), or took the ride alone (n = 1). After the accompanied test ride, respondents were asked to complete a questionnaire that consisted of the same set of questions that were asked during the test ride by the interviewer. Respondents were offered no financial compensation for participation in the study.

Questionnaire content
All responses to the questions can be found in Table S1 in the supplementary material.

Shuttle and service characteristics & usefulness & satisfaction
Q1 was designed to explore respondents' perceptions of the driving behavior of the automated shuttle on a scale from very passive (1) to very aggressive (5). Next, respondents were asked to rate specific aspects of the shuttle (i.e., air quality, size, cleanliness, comfort, speed, reliability, safety, brightness, number of seats, seating comfort, standing room, hand grips, interior and exterior design, accessibility, shuttle noise, practicality for everyday use) on a scale from very good (1) to very bad (6) (Q2). The operationalization of Q2 was based on the measurement of the service and vehicle characteristics in the study of Nordhoff et al. (2018). Q3 was self-constructed to identify additional service and vehicle features that had not been mentioned in Q2. Q4 was designed to rate the comfort of the trip on a scale from 1 (very comfortable) to 5 (very uncomfortable) (Q4), and to mention the specific aspects that were comfortable (Q5), and uncomfortable (Q6). In Q7, respondents were asked to rate the usefulness and satisfaction of the shuttle using the usefulness and satisfaction scale of Van der Laan et al. (1997).

Attitudinal questions
Questions Q8-Q11 were self-constructed to explore respondents' perceptions of their perceived safety as passengers when using the shuttle (Q8) and the safety of other external vulnerable road users (i.e., pedestrians and cyclists) when travelling with the shuttle who are outside the shuttle at the time of their journey (Q9). Q8-Q9 were measured on a scale from 1 (not at all worried) to 6 (extremely worried). Q10-Q11 were designed to assess respondents' perceptions of safety as a pedestrian (Q10) or cyclist (Q11) when sharing space with the driverless shuttle on a scale from 1 (strongly disagree) to 6 (strongly agree).
Q12-Q13 were designed to explore how respondents would cross the road as a cyclist (Q12) or pedestrian (Q13) in close proximity of the driverless shuttle, indicating whether they would wait in a convenient place to cross until there is an acceptable gap (Q12-1, Q13-1), until there is no traffic coming (Q12-2, Q13-2), whether they would cross the road before (Q12-3, Q13-3), or after the shuttle (Q12-4, Q13-4). Next, based on the operationalization of Merat et al. (2018), respondents were asked whether they would prefer to receive auditory (i.e., words, tones, signals), visual (i.e., lights, words) or neither auditory nor visual signals as pedestrian or cyclist to be informed about when the shuttle is stopping, starting to move, turning, has detected them, and how fast the shuttle is riding (Q14).
Q15 was designed to explore using a body diagram whether respondents felt any pain during the ride or abrupt, unpleasant movements somewhere in their body during the ride that did not show up until the ride, and if so, to indicate these sensations on the body diagram.

Indicators of acceptance
Q16-Q17 were designed to explore whether respondents could imagine to use automated shuttles feeding public transport on the first and last end on their daily trips. Q18 explored respondents' assessment of the impact (i.e., radical, moderate, not at all) of automated shuttles in public transport on their mobility behaviour, adjusting the construct 'mobility needs satisfaction' of Labeye et al. (2015) to the context of this study. In Q19, respondents were asked how the service with automated shuttles would have to change to make the use of automated shuttles attractive on their daily trips. The operationalization of Q19 was used from the WOB emobility cube questionnaire (WOB, 2016) and adjusted to the context of this study. The WOB emobility cube questionnaire was conducted by the Innovation Centre for Mobility and Societal Change (InnoZ) to assess the acceptance of electric vehicles. Q16-Q19 were open-ended questions.

Demographics
Questions Q20-Q31 asked the respondents to provide their personal information on their gender (Q20), age (Q21), residential situation (Q22), employment status (Q23), highest level of education (Q24), access to a valid driver license (Q25), frequency of walking more than 500 m per trip (Q26), cycling (Q27), moped or motorcycle (Q28), conventional car as driver or passenger (Q29), public transport on less than 100 km per trip (Q30), and public transport on more than 100 km per trip (Q31).
During the accompanied test rides, respondents were asked to provide answers to Q8-Q14 in general terms (without rating their responses on scales). We will not report the results from Q1-Q7 and Q15-Q19 as these questions are beyond the objectives of the present paper. Responses to all questions can be found in the digital annex.

Analysis of responses
Interview responses were analysed following principles of inductive category development. The data analysis was primarily performed by the first author of this study and two colleagues until consensus on the coded interview material was reached. Four main steps were executed.
First, the interviews were recorded and transcribed verbatim. The interview transcripts were scrutinized line-by-line by applying common steps of text analysis, such as underlining/highlighting in the text, writing notes, searching for keywords in the text, and jumping to different text passages (Mayring, 2000).
In the second stage, we read the transcripts again to develop the categories and subcategories out of the interview responses, refine and align them to the literature (e.g., Ackermann et al., 2018;Mahadevan et al., 2018;Petrovych et al., 2018;Nordhoff et al., 2019b).
In the third step, we quantified the qualitative data from the accompanied test rides by counting the number of times a subcategory was mentioned by respondents (i.e., quotes). As respondents shared the ride with other travellers at the same time, the clear assignment of a quote to a single respondent could not always be warranted. Therefore, instead of assigning a quote to a single respondent, we assigned every quote to the test ride (TR) which the respondent participated in. We decided to report subcategories only when mentioned at least ten times across all test rides to maintain a certain degree of objectivity in the data analysis. Subcategories that received less than ten quotes were reported as miscellaneous. Multiple mentions of a subcategory by the same respondent were merged. A maximum of one quote per respondent was allowed for each subcategory to prevent a dominance of a subcategory by a respondent. When a quote addressed more than one subcategory, it was assigned to each of these subcategories.
In the fourth step of the analysis, we illustrated the nature of each category with quotes following the principle of "prototypical and outlier illustrations" (Graham-Rowe et al., 2012, p. 144). We decided to select a maximum of four illustrative quotes per subcategory to preserve some objectivity in processing the interview material.
For the data obtained from the questionnaires, descriptive statistics (i.e., means, standard deviations) were calculated per questionnaire item.

Respondents
Individuals who (1) participated in former car-sharing or electro-mobility projects of the Innovation Centre for Mobility and Societal Change (InnoZ), and (2) who enrolled in the meetup group named "Taking a ride with an Automated Vehicle in Berlin" (www.meetup.com) organized by the first author of this study were invited to take part in the accompanied test ride. In total, 119 respondents participated in 62 accompanied test rides that lasted on average 37:08 min. An overview of respondents' sociodemographic profile is given in Table 1.

Results
The categories and sub-categories extracted in the accompanied test rides, their meaning and literature sources are shown in Table 2. In total, we identified three main categories and eighteen sub-categories.

Perceived safety
The first category -perceived safety -represents an assembly of eight factors that respondents associated with the perceived safety of traveling as passengers in automated shuttles. These were divided into vehicle-related, individual-related, and environmentrelated factors. The following sections will now address each sub-category separately.

Speed
Concerning their personal safety inside the shuttle, several respondents attributed their positive safety perceptions to the low speed of the automated shuttle: "Yes, it's slow, so it's safe." (TR16) "It seems very slow. Sure, the slower it is, the safer you feel on the one hand. There are two souls living in my chest, because the low speed gives you the feeling of safety, but on the other hand, you would also like to arrive quickly somewhere." (TR25) "No, because of the speed, the speed is very comforting." (TR30)

Dynamic object and event identification
Respondents also indicated that they felt safe due to the automated shuttle's ability to identify and react to road users and objects in the external environment.
"No, the system seems to respond to anything, if it is even responding to grass." (TR26) "Not with this driving behavior; it is doing well. It knows where the stop signs are as well." (TR43) "I'm more concerned about my arrival time, technically it works fine. It stops right away, it's safe, I have no fears that someone outside will be run over." (TR57)

Longitudinal and lateral control
Safety also seems to be a function of the smooth and passive lateral (i.e., steering) and longitudinal (i.e., acceleration and deceleration) control of the automated shuttle. Respondents noted:

Emergency button
Respondents pressed the emergency button on 28 of 62 test rides. When respondents were asked why they pressed the emergency button, they stated that they wanted to test the reactions of the automated shuttle. Several respondents expressed their views on pressing the emergency button as follows: "So if you have an emergency stop button somewhere, that's a safety for the people." (TR28) "Actually, it feels safe now. I do not feel like we're going to hit the wall right now. If the button here 2 works, why not." (TR37) "I like it that I press the button myself. I'm curious pressing it because I don't know what happens then. But it's good to have that. Also as an idea if something happens. Not just for getting out of the shuttle but also in cases of emergency. For anxious people." (TR47) Several respondents referred to the absence of physical supervision inside the automated shuttle.
"Here is no traffic, so I'm not really afraid. But still, that's the first association, there is no driver here." (TR10) "It actually feels really normal even though you do not see anyone steering or accelerating or doing anything." (TR18) "Does not bother me at all to be in a driverless shuttle. There are also subways that are autonomous." (TR57) Type of crossing behaviour: Crossing road (1) before or (2) after automated shuttle, (3) waiting until there is acceptable gap between automated shuttle and road user, or (4) until no traffic is coming Merat et al. (2018) Lack of eye contact: Being more concerned in crossing situation with automated shuttle than with human driver due to lack of eye contact Self-constructed Lack of trust: Being more concerned in crossing situation with automated shuttle than with human driver due to lack of trust in automated shuttle

Self-constructed
Testing automated shuttles: Crossing road in close distance to automated shuttle to test its capabilities "To be honest, it's still funny. But at the end of the day you still drive in a shuttle without a driver not operating on tracks." (TR59)

Trust in technology
Respondents indicated to feel safe due to their general trust in technology, including automated driving technology and the entities operating it.

Controlled environment
Several respondents indicated that their positive perceptions of safety are the result of the operation of the automated shuttle in a controlled environment.
"Not worried, the environment is very safe, it is not stressful at all." (TR14) "Not worried, but again it is a very limited environment and that's the issue I would say. Apart from that I don't feel unsafe, I did not put the emergency stop. The shuttle is doing very well and I like the performance, but it is very limited in scope, this is a limited scope environment." (TR43) "I do not feel insecure at all. This is also a closed campus. It cannot happen that much." (TR51) Consequently, respondents noted that their safety perceptions might change when the automated shuttle is driving at higher speeds in a mixed-traffic situation: "Yes, I feel safe, but I think it is not so easy to transfer this experience to something like: "Ok, it is going 50 km/h, there is traffic around it." (TR18) "At the moment I am not sure whether it is safe enough to drive on public roads. This is a very overfitted environment. For example, the situation that a car came in and it just stopped, that is not reliable. Would I trust this particular shuttle or the software in it? No. This was a very nice and smooth ride, but it was actually not enough for me to trust it on the road. It is a very limited test ground to say that the software is reliable." (TR43) "At a speed above 80 km/h, I wouldn't trust it. Because the traffic or the situation is too complex and there can be so much exceptions that can happen." (TR47)

Behavior of other road users
As shown in Fig. 2, respondents indicated that that they were less worried about their personal safety when travelling with the automated shuttle (Q8, M = 1.68, SD = 1.01) than about the safety of road users outside the shuttle (Q9, M = 2.57, SD = 1.28). Safety concerns related to a lack of knowledge on how the automated shuttle perceives and reacts to road users outside the shuttle.
"I think […] I am a little concerned for the people outside like the couple that was passing in front. The shuttle slowed down eventually, but they were looking at us and making eye contact, and then I got nervous for them, yes, and of course they got nervous because you don't know whether we are going to stop, we don't have control over it. I was nervous for them because they were nervous." (TR30) "Actually, more worried about mine. I'm more afraid that the car will not recognize them." (TR53) "Just because I do not exactly know how it is working, I would be more concerned about the pedestrians and cyclists outside the shuttle. Where I come from, New Mexico, I think we have the highest amount of crashes in the entire United States. A lot of people die all the time in car accidents. Friends have died, relatives have died, I do not trust other people driving. Like my only concern here is not trusting other people driving around me. It is absolutely insane." (TR61)

Type of crossing behavior
As shown by Fig. 3, respondents indicated to wait in a convenient place to cross until there is an acceptable gap between the shuttle and them as cyclist (Q11, M = 2.07, SD = 1.24), and pedestrian (M = 2.08, SD = 1.25).

Lack of eye contact
Several respondents indicated that they would be more cautious in crossing the road in front of an automated shuttle due to the lack of eye contact with the human driver.

Lack of trust
Several respondents reported to be more cautious in the crossing situation with an automated shuttle due to a lack of trust in the behaviour of the automated shuttle.
"I would not -as I do now with a regular car -cross the road just in front of the shuttle, hoping that the shuttle would stop […] because I'm not sure how fast it will react until it perceives something. As new as the technology is, I would be a little more cautious than with conventional vehicles." (TR07) "I would simply wait in the initial phase until it really stops before I cross the street. It is also a question of trust." (TR11) "I will wait for it to pass […]. When I see an autonomous car, I still have some worry about it. I don't know who will take over the responsibility when I got in there? […] I can't say that I am 100% trusting this technology right now." (TR27)

Testing automated shuttles
Several respondents anticipated that the automated shuttle will be tested by other road users due to its defensive driving behavior.

Predictability of shuttle behaviour
In addition to receiving information about travel, respondents emphasized the need to be able to intuitively understand the current and future actions of the automated shuttle.

Information about shuttle behaviour
As shown in Fig. 4, respondents reported to prefer visuals (lights) and auditory tone and signals (26%, 34%) when the shuttle is stopping (48%) and turning (44%).

"Yep, maybe like an LED screen that displays a stop sign." (TR51) "Maybe with a light. If I am crossing and I didn't see the shuttle, my first reaction would be to look at it in panic. […] Either ring the bell or show a red light that can tell you 'Ok, it is stopping' so I don't have to panic or run or do something." (TR51)
Respondents would also like to receive visuals (lights), followed by auditory tones and signals to be informed when the shuttle has detected them (40%, 16%). The reception of visuals (lights) could replace the eye contact with human drivers: "It would be nice to see that the shuttle sees me and stops. Maybe some lights; something that has the same function like eye contact." (TR16) "For me, some visual sign would be needed. […] It is like getting an eye contact with the driver." (TR18) "It would be nice to know that I am visible, maybe with a Bluetooth that within a certain area of the shuttle you know you've been detected.
[…] Maybe it is the shuttle signalling to me or one of my personal devices. Let's say there is a poplight saying: 'Yes, you are visible.'" (TR42) Auditory tones and signals (54%) and visual information (lights) (23%) were preferred by the respondents when the automated shuttle starts moving.
"Maybe a bit more information than the turning signal. For example, some lights in the front when it starts moving." (TR10) Respondents did not want to receive any (47%), or visual information (words) (27%) to be informed on how fast the automated shuttle is riding.

"I really do not want to communicate that much. We live […] in a time of overstimulation. Less is more […]. If the door is closing and there is a light then, that's maybe good to prevent that I will be stuck in the door. […]
There should not be unnecessary information; this would be annoying." (TR20)

Auditory shuttle information
Respondents had diverse opinions about the ring of the automated shuttle to inform outside road users when it starts driving or moving around the corner. The ring of the automated shuttle was considered annoying, disturbing and useless.
"Just earlier the bell rang around 10 times. This is a bit annoying, but once is okay." (TR34) "It must be driving people nuts with the rings 6 . If this would run every ten minutes, I would either wear earplugs or go out and break the speaker.

[…] With this speed, you don't need any rings. It's basically like you have a better chance of being hit by a garbage can. The bell is fine but it seems not to have a cool-off period. So it's ringing and then if it gets stuck, like with the grass, it rings to say: 'All right, I am turning.' And then it's like: 'This is bothering me, this is bothering me, this is bothering me.' It's ringing again and again and again.
And to me ringing when you are at a speed of 1 km/h, is a bit ridiculous, unless you are trying to say: 'Move, move, move." (TR35) Others considered the auditory information useful.
"The horn is fine, you know that it is aware of you, that there is a cyclist or a person coming, so it detects it, it knows it should not run over that person." (TR43) "It seems to work with the rings. This dog, who did not know any better, and the cyclist who deliberately cut us were now the only ones who ever stopped the shuttle. Insofar it feels safe and less stressful compared to a ride on the M29 7 where not only the driver is stressed but also the passengers." (TR51)

Discussion
The aim of the accompanied test ride study was to obtain insights into the perceived safety of passengers of automated shuttles, and pedestrians and cyclists interacting with automated shuttles as external road users. The data that was gathered in this study can be summarized into three main categories: (1) Perceived safety, (2) interaction with automated shuttles in crossing situations, and (3) communication with automated shuttles.

Vehicle-related
Respondents reported to feel safe due to the operation of the automated shuttle at low speeds, the ability of the automated shuttle to identify and react to other road users and traffic objects in the external environment, and the smooth and passive longitudinal and lateral vehicle control. Several respondents pointed out that they would not feel comfortable driving at higher (car-like) speeds. This can be due to the capabilities and behavior of the automated shuttle, which are naturally adapted to the operation at lower speeds, or to a lack of trust in the behavior of the automated shuttle (see Section 4.1.2). Our findings correspond with prior research on the determinants of perceived safety and trust, which associated vehicle characteristics (e.g., motion and speed) with perceptions of safety and trust (Aarts and Van Schagen, 2006;Charalambous et al., 2015;Mahmud et al., 2019;Smiley and Rudin-Brown, 2020;Van Winsum and Heino, 1996). We recommend future research to examine the relationship between vehicle motion characteristics and perceptions of safety. Key challenges will include enabling intuitive interactions with other road users, in particular at higher speeds. This will presumably lead to "smooth" driving styles also contributing to improvements in motion comfort. Here it shall be noted that the current test rides resulted in a positive evaluation of motion comfort (rated 2.6 on a scale from 1 = very good, to 6 = very badsee the digital annex).
The emergency button was used on 28 from 62 test rides. Respondents reported to press the button to see how the automated shuttle behaved. This suggests that passengers "test" the shuttle. It is somewhat surprising that a relatively large number of respondents felt the need to put it to the test, considering the ubiquitousness of emergency stop buttons in current transport systems, where misuse is discouraged socially and financially. The urge to test could be an indicator of distrust or curiosity, but can also indicate that respondents perceived some level of responsibility for the vehicle due to the absence of physical human supervision. The absence of supervision inside the automated shuttle may have enhanced their feelings of perceived responsibility, and suggests that they indeed believed that the automated shuttle was driverless. The curiosity may also be rooted in the novelty factor of the vehicle, and may have been encouraged further by the experimental setting (i.e., the interviewer of this study invited the respondents to press the button when desired). Future research should investigate whether pressing the emergency button could be considered an objective indicator of perceived safety or trust in response to studies calling for the use of more objective (vehicle) data to measure acceptance (Nordhoff et al., 2018), and how the use of the emergency button changes with the operation of automated shuttles in more realistic and complex driving conditions.

Individual-related
Our study has shown that respondents attributed their perceptions of safety to their general trust in technology. The relationship between trust and perceived safety has been documented in the literature. Zoellick et al. (2019b) modelled trust in automated vehicles as a function of the perceived safety of automated vehicles. In contrast, Kerschbaum et al. (2015) posited that trust is a condition for the belief that automated cars are safe.
Respondents considered sharing the ride with other fellow travellers an uncomfortable aspect of automated shuttle use. This concurs with our multi-level model on automated vehicle acceptance, called MAVA (Nordhoff et al., 2019a), which has shown that sharing the ride with individuals outside one's own immediate social circle is an important factor in automated vehicle acceptance. Our automated shuttle provided two opposing rows with three seats of limited width, which could magnify privacy issues. This may be addressed with larger-sized vehicles simulating the 'cocoon effect' of the private car. The shuttle interior can be redesigned, for example, preventing the opposite seating of passengers, dividing the space into smaller cubicles, providing larger-width seats and armrests. High speed internet could allow passengers to hide behind their digital devices rather than having to interact with their fellow passengers.

Environment-related
Respondents reported to be more worried about the safety of road users outside the automated shuttle than about their personal safety as passengers and attributed their perceptions to a lack of knowledge on how the automated shuttle perceives and reacts to road users outside the shuttle. Respondents are probably aware of the risky behaviour of vulnerable road users and their high involvement in fatal accidents (Dey and Terken, 2017;Hulse et al., 2018). Our study has also shown that respondents attributed their perceptions of safety to the operation of the shuttle in a controlled environment. More research is necessary that exposes automated shuttles to more dynamic and complex traffic situations to get a more accurate estimate of respondents' perceptions of safety.

Interaction with automated shuttles in crossing situations
Our results show that respondents reported to be more cautious in crossing the road in front of the automated shuttle than in front of conventional human drivers, indicating to wait in a convenient place until there is an acceptable gap to cross the road. They attributed their more cautious crossing behavior to the lack of eye contact with the human driver and trust in the automated shuttle. These findings correspond with Rothenbücher et al. (2016) who reported that respondents mentioned an uncertainty about the automated vehicle's behavior in a crossing decision, and with Pillai (2017) who revealed that respondents had difficulties to cross the road in front of an automated vehicle in a virtual-reality environment due to the lack of confirmation from the driver. Rad et al. (2020) found that respondents who trust in automated vehicle technology were more likely to cross the road before the automated vehicle.
Our study has further shown that respondents indicated to rely on the movements of the automated shuttle (e.g., speed, distance) to form their crossing decision. The literature widely supports the use of eye contact between pedestrians and human drivers as communication cue (Chang et al., 2016;Guéguen et al., 2015;Ren et al., 2016;Ŝucha, 2014). Recent studies (AlAdawy et al., 2019;Rad et al., 2020), however, propose that instead of making eye contact with an approaching vehicle when making their crossing decisions, pedestrians might be more likely to rely on vehicle kinematics (e.g., vehicle speed, distance to approaching vehicle, turn signals, brake lights) (Fridman et al., 2017;Ackermann et al., 2019). We encourage future research to examine the hypothesis that pedestrians look in the direction of the car approaching, and force the car to slow down without necessarily establishing eye contact with the driver. Future research should also establish the causality of this relationship, and investigate the perspective of the human driver. Do human drivers recognize the eye contact of pedestrians, and if so, how do they interpret and respond to it? Do human drivers slow down because they recognize a pedestrian and anticipate that the pedestrian wants to cross the road without establishing eye contact, or do they make eye contact with the pedestrian and then slow down?
Respondents also expected that automated shuttles will be tested by other road users due to their conservative driving behavior, which is consistent with Millard-Ball (2018) who posited that autonomous vehicles are programmed to obey the rules of the road, giving pedestrians the right of way to cross the road. Similarly, Madigan et al. (2019) who analysed video data on the interactions between automated shuttles and external road users found incidences of road users testing an automated shuttle. In the study of Rad et al. (2020), 81.7% of respondents expected the automated vehicle to stop for them at zebra crossings, and even 25% of respondents everywhere. Manufacturers, vehicle operators, and the media should educate the public that the testing of automated vehicles could jeopardize the safety and efficiency benefits of vehicle automation. Another strategy is to program automated vehicles to behave more human-like (i.e., less conservative and more aggressive without mitigating safety benefits) to prevent that road users change their crossing behavior when interacting with automated vehicles. Infrastructure could also be put in place that separates other road users and automated vehicles. Finally, the sensors of automated vehicles could also identify road users disrespecting the traffic rules and inform the corresponding authorities (e.g., police) sanctioning this type of activity.

Information about travel
Our study has shown a preference of individuals to receive information about travel. This finding corresponds with our interview study (Nordhoff et al., 2019a(Nordhoff et al., , 2019b, which has shown that respondents considered the provision of travel information on routes and interchanges a convenient aspect of automated vehicle usage. Smartphone apps and displays inside the vehicle or in close proximity to the vehicle's stop could provide individuals with the corresponding information.

Predictability of shuttle behavior
Respondents emphasized the importance of having an intuitive understanding of the current and future actions of the automated shuttle both as passengers and pedestrians and cyclists. This is in line with Ackermann et al. (2019) who found that respondents preferred the communication with an automated vehicle to be intuitively comprehensible and similar to pedestrians' current communication system. The desire to receive information about the behavioral intent of the automated shuttle as passengers might indicate that respondents have an initial need to verify that they can trust the system. The usefulness of communicating the intent and current state of the automated vehicle to pedestrians has been corroborated Rad et al., 2020;Reig et al., 2018). Khastgir et al. (2018) showed that informing respondents on system capabilities increased their trust. We suggest to install displays inside automated shuttles to provide passengers with information on road users' movements outside the environment as well as on the current and future actions of the automated shuttle. The additional use of smartphone apps is also advisable given the high saturation and daily usage rates of smartphones. Chater et al. (2019), however, remind us that the communication between automated vehicles and road users has to work in safety-critical environments, in real time and with low-bandwidth communication, implying that smartphone apps can't be the sole medium of communication. Finally, in line with the reflections of Fridman et al. (2017, p. 3-5), the need to employ "external communication signals in automated vehicles intended for public roadways beyond those already used in non-automated vehicles" should be critically discussed to address the still "open research question as to whether new external displays are necessary for communication of intent". We suggest that it might be more effective if automated shuttles behave like conventional vehicles when interacting with road users so that the proportion of new communication to be learned by road users is limited.

Information about shuttle behavior
Respondents favoured auditory information in situations when the automated shuttle was starting to move. This corresponds with Merat et al. (2018) where respondents interacted with an automated shuttle in La Rochelle and Trikala. Several respondents also considered the reception of auditory information annoying and useless, which suggests that sound is not appreciated for non-urgent and frequent messages. Visual information was preferred when the automated shuttle was stopping or turning, which is a finding that reflects the preferences of respondents who encountered an automated shuttle in Lausanne . Our respondents also reported to favor visual information when the automated shuttle detected them, which is in correspondence with the respondents testing an automated shuttle in Lausanne, but not consistent with respondents testing an automated shuttle in La Rochelle and Trikala . These findings suggest that the reception of visual and auditory information and a combination thereof to communicate with automated shuttles is appreciated differently by individuals across countries. We recommend that future studies examine the cultural differences of interacting with automated shuttles as external road users, and how their interaction behavior determines their requirements to communicate with automated shuttles.
Our study further revealed that only a small proportion of respondents valued the reception of words as visual information cues to be informed when the automated shuttle was stopping, turning, starting to move, or whether it had detected them. Mahadevan, Somanath, and Sharlin (2018) found that an interface, which included a visual cue sent through an Android phone signalling the respondent to be seen, was considered the most effective interface for communicating with an automated vehicle. Bazilinskyy et al. (2019) revealed that electronic human-machine-interfaces with simple textual instructions and icons to communicate intent to road users obtained the highest clarity ratings by respondents. Following the recommendation of Bazilinskyy et al. (2019), we recommend future research to test electronic Human Machine Interfaces in real traffic with high visual demands to investigate the conditions under which individuals prefer textual information. Future research should also consider the situational aspects in which Human Machine Interfaces are presented to respondents since it can be expected that results depend on the context (e.g., whether interfaces are presented on computer screen without context, on a picture or simulation of the vehicle, or in the real world).

Study strengths & limitations
A first strength of our study is that respondents had an extensive physical experience with the automated shuttle that lasted on average around 37 min. per ride. In previous studies, respondents experienced automated shuttles for a limited amount of time (e.g., 15 min per ride) (Nordhoff et al., 2018;Zoellick et al., 2019a, b), or were asked to imagine the use of automated shuttles (Nordhoff et al., 2018). We recommend future research to survey respondents before and after their rides with automated shuttles to control for the influence of experience.
Second, despite improvements in the performance of on-road automated vehicles (Favarò et al., 2018), human safety drivers or operators are often still needed to monitor the operation of automated vehicles to take over vehicle control when requested (Van Brummelen et al., 2018;Wang and Li, 2019). Consequently, in studies investigating automated vehicle acceptance, respondents were accompanied by human drivers or operators on board the vehicle. This may have inflated their safety perceptions. In our study, respondents were accompanied by a 'hidden steward' on board the shuttle to increase their sense of autonomy and simulate the experience of taking a ride in a driverless shuttle.
Third, our study contributes to the discussion on the suitability of the research methods to evaluate the interactions between automated vehicles and other road users (Dey et al., 2018). Our research study applied a mixed-method approach to examine the interactions between automated shuttles and pedestrians and cyclists from the perspective of passengers who were interviewed on an individual-and group-basis during and surveyed after their ride in the automated shuttle. Sharing the ride with fellow travellers has the advantage that respondents learn from, build upon and contrast their ideas (Pudāne et al., 2019). The triangulation of qualitative, in-depth data with questionnaire data elucidated motives behind the ratings of the questionnaire items, and revealed new themes and patterns that were not covered by the questionnaire items. Furthermore, the shuttle is outfitted with windows on all sides (Zoellick et al., 2019a). We argue that this research setting along with the extensive test experience allowed our respondents to accurately envision the interactions with automated shuttles as external road users and to evaluate their perceived safety. We recommend future research to replicate such methods to evaluate suitability given the lack of standardized methods to assess interactions with automated vehicles .
A first limitation is that respondents were asked to evaluate a hypothetical scenario: Respondents did not physically interact with the automated shuttle as pedestrians or cyclists but were passengers, which may bias results. It is possible that respondents had overly negative attitudes towards interacting with automated shuttles as pedestrians and cyclists, which may have been fostered by the media reporting accidents with automated vehicles and vulnerable road users, or because they are aware of the risky behaviour of vulnerable road users when interacting with conventional traffic (Hulse et al., 2018;King et al., 2009). It is also plausible that the ring that respondents heard during the test ride inside the shuttle did affect their preference to communicate with the shuttle as pedestrians and cyclists. For example, respondents may have a higher preference for the reception of visual rather than auditory information when interacting with automated shuttles as pedestrians and cyclists because they were negatively affected by the ring as passengers inside the shuttle.
Second, it can be argued that more important than the exposure time for the development of an accurate mental model and acceptance as well as perceptions of safety is the exposure to traffic scenarios that permit the testing of the system performance and reliability. Consequently, we encourage future research to assess acceptance of automated shuttles in more complex and risky traffic scenarios that bear the risk of failure.
Third, the present study did not account for how sharing the ride with fellow travellers influenced the response behaviour of our respondents. While it was of the impression of the interviewer of this study that group behaviour effects were not present, the presence of other travellers should be taken into account when interpreting the results.
Finally, the automated shuttle in the present study was not truly driverless but was operated by a 'hidden' steward on board. Future research should try to omit obvious physical supervision and deploy automated shuttles that are supervised from an external control room or without any type of supervision if possible.

Funding source
The PhD research of the first author of the present study was financially supported by the Innovation Centre of Mobility and Society in Berlin.