Elsevier

Applied Ergonomics

Volume 36, Issue 3, May 2005, Pages 335-343
Applied Ergonomics

Effects of increasing visual load on aurally and visually guided target acquisition in a virtual environment

https://doi.org/10.1016/j.apergo.2004.11.002Get rights and content

Abstract

The aim of the present study is to investigate interactions between vision and audition during a target acquisition task performed in a virtual environment. We measured the time taken to locate a visual target (acquisition time) signalled by auditory and/or visual cues in conditions of variable visual load. Visual load was increased by introducing a secondary visual task. The auditory cue was constructed using virtual three-dimensional (3D) sound techniques. The visual cue was constructed in the form of a 3D updating arrow. The results suggested that both auditory and visual cues reduced acquisition time as compared to an uncued condition. Whereas the visual cue elicited faster acquisition time than the auditory cue, the combination of the two cues produced the fastest acquisition time. The introduction of secondary visual task differentially affected acquisition time depending on cue modality. In conditions of high visual load, acquiring a target signalled by the auditory cue led to slower and more error-prone performance than acquiring a target signalled by either the visual cue alone or by both the visual and auditory cues.

Introduction

Recent advances in digital signal-processing technology and the development of electromagnetic position trackers have enabled the construction of virtual three-dimensional (3D) audiospatial displays. These displays have been used to aid visual target acquisition in many modern workstations including aircraft cockpits and training simulators (Bronkhorst et al., 1996; Begault and Pittman, 1996).

A number of studies have investigated the use of 3D auditory displays to reduce the workload and enhance the acquisition of visual targets (Nelson et al., 1998; Perrott et al., 1996). For example, in Perrott et al. (1996), participants sat at the centre of a geodesic sphere and detected a visual target presented at one of 264 different locations. Results demonstrated that the addition of a 3D virtual auditory cue produced a significant reduction in target acquisition time. Similarly, Nelson et al. (1998) demonstrated the beneficial effects of 3D virtually localised auditory cues on performance and perceived workload in a visual target acquisition task. In this study, visual targets were presented on a head-mounted-display (HMD) in three auditory cue conditions that differed in the amount of information provided: (1) localised auditory cues; (2) non-localised auditory cues; or (3) no auditory cues. The addition of localised auditory cues led to a significant improvement in target acquisition performance and to significant reductions in workload ratings as compared to conditions in which auditory information was either non-localised or absent.

Other studies have focused on possible multisensory integration by including situations in which 3D auditory cues were presented together with a visual cue (Flanagan et al., 1998; Bronkhorst et al., 1996). For example, Bronkhorst et al. (1996) tested the effectiveness of a 3D virtual auditory cue presented in isolation or together with a visual cue in a flight simulation experiment in which participants had to locate and track a target aircraft as quickly as possible. The results of their experiment indicated that a 3D auditory cue could be as effective as a visual cue.

More recently, Flanagan et al. (1998) administered a target acquisition paradigm in which participants were required to locate a target presented outside the initial field of view on a HMD. The target location was cued by 3D auditory cues presented in isolation or together with a visual cue. Their results suggested that both visual and 3D auditory spatial cues reduced acquisition time dramatically as compared to unaided search.

Although all of these studies provided clear evidence that 3D auditory cues have beneficial effects on target detection performance, they did not test the effectiveness of 3D auditory cues depending on visual workload. However, a study by Bolia et al. (1999) did investigate this issue. In their study, participants searched, with or without the aid of a 3D auditory cue, for a visual target presented in isolation or in the presence of a variable number of visual distractors. Results indicated that the addition of the 3D auditory cue significantly decreased search time without a corresponding increase in error rate. Further, although search time linearly increased depending on the number of distractors, it was always faster than search time obtained in the control condition in which no cue was present.

Thus, various studies have examined the use of 3D auditory cues during visual target acquisition. Included among these are studies that have gone beyond the use of a single cue modality to address multisensory integration. Other studies have investigated the effects of using only 3D auditory cues to alleviate visual workload when visual search in a cluttered environment was performed. However, as of yet the effects of multisensory cues on target acquisition performance in situations of variable visual load have not been investigated. We believe that the use of multisensory cues and visual workload are interrelated issues that should be addressed together.

Consequently, the aim of the present study was to determine whether cues presented in different modalities might differentially affect target acquisition performance depending on the level of visual workload. To this end, we examined the acquisition of a visual target in a task in which the target was signalled by auditory and visual cues that could be presented either together or in isolation. The visual workload was varied by introducing a secondary visual task. Thus, the experimental question was whether in conditions of high visual load there would be a differential effect of cue modality on target acquisition performance. It could be hypothesised that overloading vision by means of a visual secondary task should decrease the efficacy of a visual cue. This is because visual resources might be mainly engaged in performing the secondary task. Therefore, it is reasonable to expect target acquisition performance using only a visual cue to be more affected by the introduction of a secondary task than performance using only an auditory cue or both a visual and an auditory cue.

Section snippets

Participants

All 8 participants were volunteers and naïve as to the purpose of the experiment. Their ages ranged from 19 to 41 years with a mean of 26 years. All reported normal or corrected to normal vision and normal auditory functioning.

Materials

A computer equipped with a Pentium III processor was used to present the visual cues and to record the time taken by participants to locate the target. Head orientation was monitored by a three-degrees-of-freedom head tracker (Intertrax2, Intersense) that sampled head

Results

The main factor type of cue was significant, F(3,21)=185.12, p<0.0001. Acquisition time was 2.850 s for the combined cue condition, 3.153 s for the visual cue condition, 4.396 s for the auditory cue condition, and 6.618 s for the absent cue condition (see Fig. 2). The main factor type of visual workload task was also significant F(2,14)=26.713, p<0.0001. Acquisition time was 3.886 s for the control visual workload task, 3.957 s for the passive visual workload task, and 4.920 s for the active visual

General discussion

The aim of the present study was to investigate the effects of cues presented in different modalities (i.e. vision and audition) during a target acquisition task involving different level of visual load.

There were three main findings of the present study. First, the combination of the auditory and the visual cues led to better performance in target acquisition than when the two cues were presented in isolation. Second, regardless of the level of visual workload both cues produced faster

Implication for design

Data obtained in the present research suggest that, although 3D auditory cues can be effectively used to aid visual target acquisition, they probably require a certain amount of attentional resources to extract and resolve the spatial information they carry. Thus the use of 3D auditory cues in situations in which attentional resources are required for the execution of other concurrent tasks is not the best means of aiding target acquisition. However, presenting auditory and visual cues that

Acknowledgements

A preliminary report of the work described herein was presented at the 10th Annual Meeting of the Cognitive Neuroscience Society (March 30th–April 1st, 2003, New York City). The authors would like to thank Andy Haswell and Philip Roberts for providing essential hardware support throughout all the phases of this investigation. Francesco Pavani and Massimo Turatto are also thanked for their useful comments on previous versions of this manuscript. Andrea Caria was supported by a Leverhulme Trust

References (14)

  • D. Zambarbieri et al.

    Saccade latency toward auditory targets depends on the relative position of the sound source with respect to the eyes

    Vision Res.

    (1995)
  • D.R. Begault et al.

    Three-dimensional audio versus head-down traffic alert and collision avoidance system displays

    Int. J. Aviat. Psychol.

    (1996)
  • R.S. Bolia et al.

    Aurally aided visual search in three-dimensional space

    Hum. Factors

    (1999)
  • A.W. Bronkhorst et al.

    Application of a three-dimensional auditory display in a flight task

    Hum. Factors

    (1996)
  • P. Flanagan et al.

    Aurally and visually guided visual search in a virtual environment

    Hum. Factors

    (1998)
  • R.L. Martin et al.

    Free-field equivalent localization of virtual audio

    J. Audio Eng. Soc.

    (2001)
  • H.J. Müller et al.

    Reflexive and voluntary orienting of visual attention: time course of activation and resistance to interruption

    J. Exp. Psychol.: Hum. Percept. Performance

    (1989)
There are more references available in the full text version of this article.

Cited by (10)

View all citing articles on Scopus
View full text