Effects of increasing visual load on aurally and visually guided target acquisition in a virtual environment
Introduction
Recent advances in digital signal-processing technology and the development of electromagnetic position trackers have enabled the construction of virtual three-dimensional (3D) audiospatial displays. These displays have been used to aid visual target acquisition in many modern workstations including aircraft cockpits and training simulators (Bronkhorst et al., 1996; Begault and Pittman, 1996).
A number of studies have investigated the use of 3D auditory displays to reduce the workload and enhance the acquisition of visual targets (Nelson et al., 1998; Perrott et al., 1996). For example, in Perrott et al. (1996), participants sat at the centre of a geodesic sphere and detected a visual target presented at one of 264 different locations. Results demonstrated that the addition of a 3D virtual auditory cue produced a significant reduction in target acquisition time. Similarly, Nelson et al. (1998) demonstrated the beneficial effects of 3D virtually localised auditory cues on performance and perceived workload in a visual target acquisition task. In this study, visual targets were presented on a head-mounted-display (HMD) in three auditory cue conditions that differed in the amount of information provided: (1) localised auditory cues; (2) non-localised auditory cues; or (3) no auditory cues. The addition of localised auditory cues led to a significant improvement in target acquisition performance and to significant reductions in workload ratings as compared to conditions in which auditory information was either non-localised or absent.
Other studies have focused on possible multisensory integration by including situations in which 3D auditory cues were presented together with a visual cue (Flanagan et al., 1998; Bronkhorst et al., 1996). For example, Bronkhorst et al. (1996) tested the effectiveness of a 3D virtual auditory cue presented in isolation or together with a visual cue in a flight simulation experiment in which participants had to locate and track a target aircraft as quickly as possible. The results of their experiment indicated that a 3D auditory cue could be as effective as a visual cue.
More recently, Flanagan et al. (1998) administered a target acquisition paradigm in which participants were required to locate a target presented outside the initial field of view on a HMD. The target location was cued by 3D auditory cues presented in isolation or together with a visual cue. Their results suggested that both visual and 3D auditory spatial cues reduced acquisition time dramatically as compared to unaided search.
Although all of these studies provided clear evidence that 3D auditory cues have beneficial effects on target detection performance, they did not test the effectiveness of 3D auditory cues depending on visual workload. However, a study by Bolia et al. (1999) did investigate this issue. In their study, participants searched, with or without the aid of a 3D auditory cue, for a visual target presented in isolation or in the presence of a variable number of visual distractors. Results indicated that the addition of the 3D auditory cue significantly decreased search time without a corresponding increase in error rate. Further, although search time linearly increased depending on the number of distractors, it was always faster than search time obtained in the control condition in which no cue was present.
Thus, various studies have examined the use of 3D auditory cues during visual target acquisition. Included among these are studies that have gone beyond the use of a single cue modality to address multisensory integration. Other studies have investigated the effects of using only 3D auditory cues to alleviate visual workload when visual search in a cluttered environment was performed. However, as of yet the effects of multisensory cues on target acquisition performance in situations of variable visual load have not been investigated. We believe that the use of multisensory cues and visual workload are interrelated issues that should be addressed together.
Consequently, the aim of the present study was to determine whether cues presented in different modalities might differentially affect target acquisition performance depending on the level of visual workload. To this end, we examined the acquisition of a visual target in a task in which the target was signalled by auditory and visual cues that could be presented either together or in isolation. The visual workload was varied by introducing a secondary visual task. Thus, the experimental question was whether in conditions of high visual load there would be a differential effect of cue modality on target acquisition performance. It could be hypothesised that overloading vision by means of a visual secondary task should decrease the efficacy of a visual cue. This is because visual resources might be mainly engaged in performing the secondary task. Therefore, it is reasonable to expect target acquisition performance using only a visual cue to be more affected by the introduction of a secondary task than performance using only an auditory cue or both a visual and an auditory cue.
Section snippets
Participants
All 8 participants were volunteers and naïve as to the purpose of the experiment. Their ages ranged from 19 to 41 years with a mean of 26 years. All reported normal or corrected to normal vision and normal auditory functioning.
Materials
A computer equipped with a Pentium III processor was used to present the visual cues and to record the time taken by participants to locate the target. Head orientation was monitored by a three-degrees-of-freedom head tracker (Intertrax2, Intersense) that sampled head
Results
The main factor type of cue was significant, , . Acquisition time was 2.850 s for the combined cue condition, 3.153 s for the visual cue condition, 4.396 s for the auditory cue condition, and 6.618 s for the absent cue condition (see Fig. 2). The main factor type of visual workload task was also significant , . Acquisition time was 3.886 s for the control visual workload task, 3.957 s for the passive visual workload task, and 4.920 s for the active visual
General discussion
The aim of the present study was to investigate the effects of cues presented in different modalities (i.e. vision and audition) during a target acquisition task involving different level of visual load.
There were three main findings of the present study. First, the combination of the auditory and the visual cues led to better performance in target acquisition than when the two cues were presented in isolation. Second, regardless of the level of visual workload both cues produced faster
Implication for design
Data obtained in the present research suggest that, although 3D auditory cues can be effectively used to aid visual target acquisition, they probably require a certain amount of attentional resources to extract and resolve the spatial information they carry. Thus the use of 3D auditory cues in situations in which attentional resources are required for the execution of other concurrent tasks is not the best means of aiding target acquisition. However, presenting auditory and visual cues that
Acknowledgements
A preliminary report of the work described herein was presented at the 10th Annual Meeting of the Cognitive Neuroscience Society (March 30th–April 1st, 2003, New York City). The authors would like to thank Andy Haswell and Philip Roberts for providing essential hardware support throughout all the phases of this investigation. Francesco Pavani and Massimo Turatto are also thanked for their useful comments on previous versions of this manuscript. Andrea Caria was supported by a Leverhulme Trust
References (14)
- et al.
Saccade latency toward auditory targets depends on the relative position of the sound source with respect to the eyes
Vision Res.
(1995) - et al.
Three-dimensional audio versus head-down traffic alert and collision avoidance system displays
Int. J. Aviat. Psychol.
(1996) - et al.
Aurally aided visual search in three-dimensional space
Hum. Factors
(1999) - et al.
Application of a three-dimensional auditory display in a flight task
Hum. Factors
(1996) - et al.
Aurally and visually guided visual search in a virtual environment
Hum. Factors
(1998) - et al.
Free-field equivalent localization of virtual audio
J. Audio Eng. Soc.
(2001) - et al.
Reflexive and voluntary orienting of visual attention: time course of activation and resistance to interruption
J. Exp. Psychol.: Hum. Percept. Performance
(1989)
Cited by (10)
Primary or secondary tasks? Dual-task interference between cyclist hazard perception and cadence control using cross-modal sensory aids with rider assistance bike computers
2017, Applied ErgonomicsCitation Excerpt :This may imply that tactile signals cause the information to be automatically processed (Koch et al., 2009); thus, tactile signals may allow more cognitive resources to attend to primary tasks (Huxhold et al., 2006). Auditory cues are found to contribute to rider distraction, similar to the results of Pierno et al. (2005), whereas tactile cues are found to improve rider ability to detect hazards. As a secondary indicator alert (e.g., for RPM), auditory cues are the worst choice for delivery mode integration and the worst component.
Adapting Visual Complexity Based on Electrodermal Activity Improves Working Memory Performance in Virtual Reality
2023, Proceedings of the ACM on Human-Computer InteractionVisual distractors disrupt audiovisual integration regardless of stimulus complexity
2017, Frontiers in Integrative NeuroscienceConsidering older adults' perceptual capabilities in the design process
2015, The Cambridge Handbook of Applied Perception ResearchDo whole-body vibrations affect spatial hearing?
2014, Ergonomics