Using a Virtual Reality Walking Simulator to Investigate Pedestrian Behavior

Gap crossing, an interceptive behavior, requires moving oneself in relation to a gap between two moving vehicles^1,2,3,4. Gap crossing involves perceiving oncoming vehicles and controlling movement in relation to moving traffic. This requires actions to be precisely coupled with perceived information. Many previous studies have examined perceptual judgment and gap-crossing behavior using artificial roads, roadside simulators, and screen projection virtual environments^5,6. However, previous road-crossing literature has an incomplete understanding of this behavior, and the ecological validity of these studies has been questioned^7,8,9.

This protocol presents a research paradigm for studying gap crossing behavior in virtual reality, thus maximizing ecological validity. A walking simulator is used to examine the perception and actions of gap crossing behavior. The simulator provides a safe walking environment for participants, and the actual walking in the simulated environment allows researchers to fully capture the reciprocal relationship between perception and action. Individuals who actually cross a road are known to judge the time gap more accurately than those who only verbally decide to cross¹⁰. The virtual environment is ecologically valid and allows researchers to easily change task-related variables by altering the program’s parameters.

In this study, a participant’s initial starting location is manipulated to assess the velocity control while approaching the gap. This protocol allows the investigation of pedestrian locomotion control while intercepting a gap. Analyzing a participant’s velocity changing over time allows a functional interpretation of velocity adjustments while he or she approaches a gap.

In addition, the spatial and temporal characteristics of intercepted objects specify how a person can move. In a gap crossing environment, changing of the gap size (inter-vehicle distances) and vehicle size should affect how a pedestrian’s locomotion also changes. Accordingly, manipulating the gap characteristics will likely cause velocity adjustments in the participant’s approaching behavior. Thus, manipulating gap characteristics (i.e., gap size and vehicle size) provides valuable information for understanding crossing behavior changes according to various gap characteristics. This study examines how children and young adults regulate their velocity when crossing gaps in various crossing environments. The speed regulation profile can be assessed for various gap crossing environments with different starting locations, inter-vehicle distances, and vehicle sizes.

This experimental protocol involves human subjects. The procedure was approved by the Kunsan National University Research Board.

1. Preparation of equipment

NOTE: The equipment includes the following: a personal computer (PC, 3.3 GHz with 8 GM) with a mouse, keyboard, and monitor; Walking Simulator software installed on the desktop PC; a customized treadmill (width: 0.67 m, length: 1.26 m, height: 1.10 m) equipped with handrails, a belt, and a magnetic encoder with a USB cable; and an Oculus Rift virtual reality device (DK1, U.S., 1280 x 800 pixels). The equipment also includes a customized manual treadmill. The treadmill turns via the walking motions of the participants and does not use an internal motor.

1. Prepare sufficient space for the treadmill and a nearby desk for the PC. A photograph of the experimental setup is shown in Figure 1A.
2. Connect the equipment as shown in Figure 2.
   1. Connect treadmill’s magnetic encoder to the PC via a USB port.
   2. Connect the treadmill to a power source.
   3. Connect the headset to the PC via DVI/HDMI and USB ports.

2. Preparation of walking simulator configurations

1. Access the walking simulator directory on the PC and open the “Config” directory.
   NOTE: Each configuration is saved as a text file in the “Config” directory with file names of “config001”, “config002”, etc. Here, 001, 002, etc. are the configuration numbers. Steps 2.2–2.8 describe how to create the configuration files so that they are readable by the simulator software. A schematic of a two-vehicle crossing situation showing customizable initial distances is shown in Figure 3. An example configuration file with proper formatting is shown in Figure 4. Section headings of the configuration file use square brackets (e.g., “[WALKER]”).
2. Complete the section [WALKER] containing the parameter regarding the starting point of the participants.
   1. Set the parameter “Distance”, which indicates the starting distance of the participant from the starting point in meters (m).
3. Complete the section [CAR] containing parameters regarding the first vehicle.
   1. Set the parameter “Type” (which indicates the type of vehicle) to “1” for sedan, “2” for bus, or “0” to remove the vehicle.
   2. Set the parameter “Speed” (which indicates the vehicle speed) to the desired value in km/h.
   3. Set the parameter “Distance” (which indicates initial distance of the vehicle from the crossing point) to the desired value in meters.
4. Complete the section [SECONDCAR] containing the parameters related to the second vehicle. Parameters are identical to those of [CAR].
   NOTE: In two-vehicle studies, the gap is defined as the empty space between the two vehicles. The gap size, defined as the length of time during which the gap is along the participant’s walking path, is a function of the “Distance”, “Speed”, and “Type” parameters of [CAR] and [SECONDCAR].
5. Complete the section [NEXTCAR] containing parameters related to additional vehicles. The parameters are identical to those of [CAR].
   NOTE: This option can be used to investigate pedestrian behavior within continuous traffic flow. This option is not discussed in the representative results section.
6. Complete the section [ROAD], containing the parameter for lane selection. Set the parameter “lane” to “1” to use the lane closer to pedestrian’s starting position, or “2” for the lane further away. [OBSTACLE] indicates the parameters that configure a vehicle traveling in the second lane at the same speed as the first vehicle.
   NOTE: When using the closer lane as the primary lane, this option can be used to place additional vehicles on the farther lane going in the same direction. Hence, it can be used to study the impedance of the view of a vehicle by a parallel vehicle. This section has parameters “Type” and “Distance” with the same definitions described above. This option is not discussed in the representative results section. All results shown involve two vehicles driving in the lane closer to the pedestrian.
7. Complete the section [SAVE], which contains the parameter related to sampling frequency. Set the parameter “numberpersecond” to the desired value in Hz.
8. Save the configuration file and exit.
9. Repeat sections 2.2–2.8 for all desired configurations and prepare data sheets with the list of configurations (in a randomized order) to be used in the experiment.
10. Prepare three configuration files to be used in the practice trials.
    NOTE: The first practice configuration should have no vehicles (i.e., all “Type” parameters set to “0”). The second and third practice configuration files should have vehicles. The third configuration should have lenient crossing conditions. The same configuration may be used for the second and third practice trials, depending on the experimental design.

3. Participation screening and preparation

1. Recruit participants with normal or corrected-to-normal vision.
   NOTE: All participants should be free of any conditions that prevent normal walking. They should be free of any dizziness while walking, and they should not have any history of serious traffic accidents.
2. Ask the participant to sign a written, informed consent form before each experiment.
3. Prepare an audio recording with verbal instructions of the task and play the recording to the participant.
   NOTE: The verbal instructions should narrate the basic procedure described below and give any specific prompts required by the experimental design.
4. Encourage the participant to ask any questions about the experiment.
5. Lead the participant to stand on the treadmill when ready.
6. Harness the stabilizing belt to the participant’s waist. Instruct the participant to hold the handrails at all times during the experiment.

4. Running the practice trials

1. Instruct the participant to practice walking on the treadmill, with the belt on, while holding the handrails.
2. Begin the walking simulator program by double-clicking the executable simulator program once the participant is able to walk on the treadmill comfortably.
   NOTE: The black and white cartoon crosswalk shown in Figure 1B is displayed between crossing trials. At this point, it should be shown on the PC screen.
3. Instruct the participant to wear the headset. Give assistance as needed. Check for both comfort and stability with respect to head turns.
4. Calibrate the headset so that the black and white cartoon crosswalk is properly aligned with participant’s view.
   NOTE: Sections 4.5–4.7 describe three practice trials, which are designed to gradually allow the participant to become accustomed to the simulator environment. If the participant fails any trial due to misunderstanding of the instructions, up to two more extra trials should be performed until the participant understands the instructions. Extra trials are not performed in cases of failure to cross for reasons other than misunderstanding the rules (e.g., if a collision occurs).
5. Begin the first practice trial.
   NOTE: The first practice trial should be without any vehicles for the participant to become accustomed to walking in the virtual reality setting.
   1. Inform the participant that the first practice trial will occur without any vehicles.
   2. Instruct the participant to look straight ahead.
   3. Enter the first practice trial’s configuration number in the text box on the bottom of the screen.
   4. Click the “Start” button at the bottom of the screen.
      NOTE: The program should display the realistic setting depicted in Figure 1C on the screen.
   5. Inform the participant to get ready when hearing “Ready” and to begin walking when hearing “Go”. Give the verbal cues “Ready” and “Go”.
6. Second practice trial
   NOTE: The second practice trial should introduce the vehicles without walking. The direction of the virtual reality view shifts as the participant’s head is turned.
   1. Instruct the participant in this trial, at the verbal cue “Go”, to look to the left and simultaneously take a small step forward, but not to walk forward any further. The participant should instead watch the vehicles pass by.
   2. Type the second trial’s configuration number into the text box and click “Start” by providing the verbal cues.
      NOTE: The vehicles begin moving as the participant begins moving.
7. Third practice trial
   NOTE: The third practice trial should be similar to the experimental configurations, but with lenient crossing conditions.
   1. Inform the participant that 1) the third practice trial will involve two vehicles coming from the left side, and 2) he/she should attempt to cross the road between the two vehicles.
   2. Enter the third practice trial number in the text box by providing the verbal cue.
   3. Click the “Start” button and begin the trial by providing the verbal cues.

5. Virtual walking experiment

1. Confirm that the participant understands the experimental task and is able to perform it.
2. When the participant is ready, type in the first configuration number from the data sheet on the text box and click “Start”.
3. Perform the simulation as done in the final practice trial.
   NOTE: At the end of each crossing trial, the program displays “S”, “F”, or “C”, depending on whether the result is a successful crossing (i.e., the participant crosses to the other side of the street with no collisions), no crossing (participant does not cross to the other side), or a collision (participant has contact with a vehicle), respectively.
4. Record the result next to the configuration number on the data sheet.
5. Repeat for all configurations on the data sheet and complete the experiment.

6. Data export and analysis

1. Retrieve the data files for analysis. The walking simulator software saves each run as a spreadsheet file in the “Data” folder.
2. Analyze data with the preferred tools. The output data records the positions and velocities of the walker and the vehicles as a time series. Use this data to analyze participant movements and the dependence on traffic conditions.

The walking simulator can be used to examine a pedestrian’s crossing behavior while manipulating the initial distance from curb to interception point and the gap characteristics (i.e., gap and vehicle sizes). The virtual environment method allows the manipulation of gap characteristics to understand how dynamically changing crossing environments affect children’s and young adults’ road-crossing behaviors.

A quantified velocity profile and crossing position within the gap used to compare the crossing behavior of various pedestrian groups. We evalutated time of intercept (TOI) as the instantaneous effect of speed adjustment on participants' position within the gap. These representative results use data from 16 young adults (mean age = 22.75 years, SD = 2.56) and 16 children (mean age = 12.18 years, SD = 0.83). Generally, 12 year-old children undergo developmental changes in the ability to coordinate movements with moving objects^{3,4,11,12,13,14}, so varying the initial distance provided an opportunity to compare functional adjustment of approaching velocity in children vs. young adults. The participants were recruited via a university social media posting. Of the recruited participants, two young adults experienced motion sickness, in which the experiments were immediately stopped, and they were excluded from the study.

The success rate was 98.95% among children and 99.48% among young adults. Only successful trials were included in the analysis. To access the velocity data, a 3 x 2 x 2 x 4 (initial distance [near, intermediate, far]; gap size [3 s, 4 s]; vehicle size [car, bus]; time [3.5 s, 2.5 s, 1.5 s, 0.5 s]) repeated measures ANOVA was performed using initial distance, gap size, vehicle size, and time as within factor variables. Timing data was analyzed by performing a 3 x 2 x 2 (initial distance [near, intermediate, far]; gap size [3 s, 4 s]; vehicle size [car, bus]) repeated measures ANOVA with initial distance, gap size, and vehicle size as within factor variables. To estimate effect size, the partial eta squared (η²_p) was used. For all pairwise post-hoc analyses, least square means were used.

Effects of initial distance
Tested first was the hypothesis that manipulation of the initial distance from the curb to interception point would affect the approach velocity of participants. The systematic change in initial distance affected both young adults’ and children’s velocity adjustments: F(2, 30) = 29.62, p < 0.0001, η² _p = .66; and F(2, 30) = 207.32, p < 0.0001, η²_p = .93, respectively.

For young adults, the initial distance and time interaction was significant: F(6, 90) = 11.88, p < 0.0001, η² _p = 0.44. A simple effects test showed a significant effect of time for: near initial distance, F(3, 45) = 140.34, p < 0.0001, η²_p = 0.90; intermediate initial distance, F(3, 45) = 29.93, p < 0.0001, η² _p = 0.67; and far initial distance, F(3, 45) = 184.46, p < 0.0001, η²_p = 0.93. It was found from the post-hoc analysis that young adults increased in speed throughout the approach (p < 0.0001). However, when the initial distance was short, participants slowed down (p < 0.0001) at the beginning of trials and sped up continuously. This represents the functional adjustment. The mean velocities during approach are plotted across age groups (Figure 5).

For children, initial distance and time interaction was also significant: F(6, 90) = 53.51, p < 0.0001, η²_p = 0.78. This interaction effect was captured by the three-way interaction. The vehicle size, initial distance, and time interaction was significant: F(6, 90) = 2.12, p < 0.05, η²_p = 0.12. The results indicate that children’s velocity changes induced by the initial distance were affected by vehicle size.

Effects of vehicle size in children
Tested next was the hypothesis that manipulation of vehicle size would affect the velocity profiles and crossing time of children and young adults. It was found that in children, vehicle size affected the velocity profiles and crossing position induced by the initial distance.

In children, vehicle size, initial distance, and time interaction was significant: F(6, 90) = 2.12, p < 0.05, η²_p = 0.12. Further analysis revealed that, between the cars, the initial distance x time interaction was significant, F(6, 90) = 33.55, p < 0.0001, η²_p = 0.69. A simple effects test showed a significant effect of time for near initial distance, F(3, 45) = 132.54, p < 0.0001, η²_p = 0.90; intermediate initial distance, F(3, 45) = 173.83, p < 0.0001, η²_p = 0.92; and far initial distance, F(3, 45) = 272.78, p < 0.0001, η²_p = 0.95. Post-hoc analysis showed that children sped up throughout the approach (p < .0001); however, when they crossed between the cars, they slowed down at the beginning of the approach for the near initial distance (p < 0.0002),

However, when children crossed between the buses, the initial distance and time interaction was also significant: F(6, 90) = 18.70, p < 0.0001, η²_p = 0.55. A simple effects test showed a significant effect of time for the near initial distance: F(3, 45) = 124.41, p < 0.0001, η²_p = 0.89; intermediate initial distance, F(3, 45) = 132.79, p < 0.0001, η²_p = 0.90; and far initial distance, F(3, 45) = 331.16, p < 0.0001, η²_p = 0.96. Post-hoc analysis showed that when children crossed between the buses, their speeds neither increased nor decreased at the beginning of the approach for the near initial distance. The mean velocities during approach are plotted across age groups in Figure 6.

Evidently, vehicle size influenced children’s crossing behavior as induced by initial distance. The children’s crossing times deviated systematically from the gap center depending on the initial distance at which they crossed between the small vehicles. However, children did not deviate based on the initial distance when they crossed between the large vehicles.

The vehicle size also significantly affected the children’s crossing position within the gap induced by initial distance. The vehicle size and initial distance interaction was significant: F(2, 30) = 18.13, p < 0.0001, η²_p = 0.55. A simple effects test showed a significant effect of initial distance between cars, F(2, 30) = 62.30, p < 0.0001, η²_p = 0.81, and between buses, F(2, 30) = 6.15, p < 0.005, η²_p = 0.30. It was found that children's times of intercept increased significantly (p < 0.0001) as the initial distance increased from near to far initial distances. However, when crossing between buses, children's times of interception were not significantly different between near and intermediate initial distances. The mean crossing position during approach are plotted across age groups (Figure 7).

Interaction effects of vehicle size and gap size in children
Finally, the interaction effects of vehicle size and gap size in children were examined. The vehicle size and gap size interaction was significant: F(1, 15) = 4.26, p < 0.05, η²_p = 0.22. A simple effects test showed a significant effect of gap size between the cars: F(1, 15) = 7.42, p < .02, η²_p = 0.33; and between the buses, F(1, 15) = 35.93, p < 0.001, η²_p = 0.71. Post-hoc analysis showed that when crossing between the cars, children crossed the gap significantly further ahead of the gap center in the 4 s gap than the 3 s gap (p < 0.01). When crossing between the buses, children also crossed the gap significantly earlier in the 4 s gap than the 3 s gap (p < 0.0001). Children crossed the gap further ahead of the gap center in the 4 s gap than the 3 s gap, regardless of vehicle size (Table 1).

Figure 1: Images depicting the walking simulation experiment. (A) Photograph of a participant walking on the treadmill and an experimenter viewing the walking simulator program. (B) Image of the cartoon crosswalk displayed before the configuration is loaded. (C) Image of the realistic virtual environment in which the simulation takes place. Please click here to view a larger version of this figure.

Figure 2: Experimental setup diagram. The components of the experimental setup and their connections are illustrated. Please click here to view a larger version of this figure.

Figure 3: Diagram of the crossing situation. Distance parameters that can be configured for each experiment are shown. Please click here to view a larger version of this figure.

Figure 4: Configuration file example. Example of a properly formatted configuration text file for the simulation program. Please click here to view a larger version of this figure.

Figure 5: Velocity dependence on initial distance. Mean velocities for each initial distance in children and young adults (near, intermediate, and far defined as 3.5 m, 4.5 m, and 5.5 m from the interception point) as a function of time before reaching the interception point. The approaching velocity was averaged into 1 s intervals (-3.5 s, -2.5 s, -1.5 s, and -0.5 s), counting backwards from the interception point. Asterisks represent statistically significant inter-mean differences for initial distances at each timepoint. One asterisk represents one inter-mean difference, and two asterisks represent two or more inter-mean differences. Error bars indicate SD. This figure has been reprinted with permission from Chung et al.¹⁵. Please click here to view a larger version of this figure.

Figure 6: Children’s velocity dependence on initial distance based on two different vehicle sizes. Children’s mean velocity profiles before reaching the interception point for each initial distance are plotted for cars (top) and buses (bottom). The approach velocity was averaged into 1 s intervals, counting backwards from the interception point. Asterisks represent statistically significant inter-mean differences for initial distances at each timepoint. One asterisk represents one inter-mean difference, and two asterisks represent two or more inter-mean differences. Error bars indicate SD. This figure was reprinted with permission from Chung et al.¹⁵. Please click here to view a larger version of this figure.

Figure 7: Effect of vehicle size on children’s TOI. The children group’s mean TOI for each initial distance is shown as a function of vehicle size (car, bus). TOI refers to the temporal distance relative to the gap center at the moment of crossing, such that 0.2 s refers to 1.6 m when the vehicle speed is 30 km/h (8.3 m/s). Asterisks represent statistically significant inter-mean differences for vehicles at each initial distances. One asterisk represents one inter-mean difference, and two asterisks represent two or more inter-mean differences. Error bars indicate SD. This figure was reprinted with permission from Chung et al.¹⁵. Please click here to view a larger version of this figure.

Table 1: Interaction effects of vehicle size and gap size in children. Children’s mean TOI as function of vehicle size and gap size Note. Values are given in means(Standard deviations). . Please click here to download this table.

Previous studies have used simulators with projected screens^16,17, but this protocol improves ecological validity via a fully immersive virtual view (i.e., 360 degrees). In addition, requiring participants to walk on a treadmill enables the examination of how children and young adults calibrate their actions to a changing environment. This experimental design’s virtual scene changes simultaneously with participant motions, and the vehicles arrive at the pedestrian’s crossing line at a specific point in time. This prevents participants from delaying their crossing times due to decisions or preparations to move. In this study, participants are already in motion when attempting to cross the road⁶, so researchers can clearly access the control of locomotion while crossing.

Critical steps include properly setting the parameters to reflect the experimental design, stopping the experiment when motion sickness occurs, and performing the practice trials so that the participants are comfortable with the treadmill environment. A wide range of traffic flows beyond those discussed in the results is configurable with the current software. The software may also be easily extended to include a wider range of crossing situations (i.e., by adding more lanes or more vehicle types).

The protocol allows the investigation of how children and young adults regulate their locomotion according to dynamically changing environments. Specifically, systematically varying the initial starting location allows the examination of velocity adjustments in children and young adults. The protocol also permits the determination of whether changes in gap characteristics lead to specific velocity control patterns in interceptive actions. The results demonstrate that varying initial distances and gap characteristics is important for identifying systematic crossing behavior adaptations that reflect the perception/action type of control in crossing roads. The results indicate interaction effects of initial distance and vehicle size in children; specifically, their velocity adjustments while approaching the interception were affected by gap characteristics.

In contrast to previous findings on the weak effects of vehicle size on adults’ crossing behaviors, this study found that children poorly adjusted their approach velocities according to the initial distance when facing a large vehicle from a close distance. The results suggest that the ability to finely tune motor movements using visual information in complex interception tasks is subject to developmental changes. However, future research should differentiate vehicle types and sizes by using various sizes of the same vehicle type. This setup would allow a more accurate answer for which visual information is used to control crossing actions in a dynamic environment.

Furthermore, manipulating gap size and vehicle size together did not answer to which properties of the dynamic gap environment directly influence movement modulation. The findings suggest that children underestimate a vehicle’s arrival time and attempt to cross more quickly in front of large vehicles. Notably, children cross the gaps between buses earlier than expected in the 4 s gap. This may be due to a LV’s closer distance in the 4 s gap. One limitation of this design is that the gap size’s effects are confounded by the effects of a vehicle’s outer edges. Future experimental designs may alter gap size without altering a vehicle’s outer edges.

Compared to previous virtual reality research, this experiment’s design offers a safe environment to investigate crossing behavior. However, the apparatus causes motion sickness in some participants. The literature on motion sickness reveals a relationship between motion sickness and postural control, so people who have poor balance control should be excluded^18,19,20. Additionally, participants hold the handrails during walking, and this may interrupt a natural walking motion, which may be a limitation of the method. In sum, this study contributes to the understanding of children’s road crossing behavior in relation to a gap’s temporal and spatial characteristics.