Visual, auditory and tactile stimuli compete for early sensory processing capacities within but not between senses

Highlights

• We studied stimulus interactions within and across senses in the EEG.

• Stimulus streams elicited visual, auditory and tactile steady-state responses.

• Competing stimuli were briefly presented in the same or in a different sense.

• Across senses, only competitors in the same sense led to reduced processing.

• Competition exerted modality-specific influences on early sensory processing.

Abstract

We investigated whether unattended visual, auditory and tactile stimuli compete for capacity-limited early sensory processing across senses. In three experiments, we probed competitive audio-visual, visuo-tactile and audio-tactile stimulus interactions. To this end, continuous visual, auditory and tactile stimulus streams (‘reference’ stimuli) were frequency-tagged to elicit steady-state responses (SSRs). These electrophysiological oscillatory brain responses indexed ongoing stimulus processing in corresponding senses. To induce competition, we introduced transient frequency-tagged stimuli in same and/or different senses (‘competitors’) during reference presentation. Participants performed a separate visual discrimination task at central fixation to control for attentional biases of sensory processing. A comparison of reference-driven SSR amplitudes between competitor-present and competitor-absent periods revealed reduced amplitudes when a competitor was presented in the same sensory modality as the reference. Reduced amplitudes indicated the competitor's suppressive influence on reference stimulus processing. Crucially, no such suppression was found when a competitor was presented in a different than the reference modality. These results strongly suggest that early sensory competition is exclusively modality-specific and does not extend across senses. We discuss consequences of these findings for modeling the neural mechanisms underlying intermodal attention.

Introduction

Imagine standing in a particularly crowded place while listening to someone on your phone. People are walking all over the place; talking out loud; perhaps even bumping into you. You will have to try hard to focus on the caller's voice. This situation illustrates how in everyday life limited neural processing capacities force the human brain to actively select a specific source of information among a manifold of unrelated sensory events (Broadbent, 1952; Kastner and Ungerleider, 2000; Neisser and Becklen, 1975). In our example a distinction can be made between selecting the voice over background noise – a selection within the auditory modality – and selecting auditory information over visual and tactile information – a selection between sensory modalities.

The neural mechanisms underlying attentional selection within a sensory modality have been formalized in the biased competition model (Kastner et al., 1998, Moran and Desimone, 1985). Biased competition rests on two central assumptions: (I) Two or more concurrent stimuli enter a competition for limited processing capacity that leads to mutual suppression. Although primarily established in visual attention research, modality-specific inter-stimulus competition can also be inferred from findings of suppressive effects between auditory (Kawase et al., 2012, Ross et al., 2012) and tactile stimuli (Severens et al., 2010). (II) Selective attention to one stimulus releases it from mutual suppression and biases the competition in favor of the selected stimulus' processing. Following these assumptions, inter-stimulus competition poses a necessary prerequisite for the attentional bias in modality-specific sensory processing.

Attentional selection between sensory modalities, i.e. ‘intermodal’ attention (Alho et al., 1992, Boulter, 1977, Eimer and Schröger, 1998, Porcu et al., 2013), is less well understood. It stands to question, whether the attentional mechanisms that constitute the biased competition framework also account for intermodal attention. Recent neuroimaging studies have investigated crossmodal² interactions in early sensory processing while participants attended to one sensory modality (Johnson and Zatorre, 2005, Langner et al., 2011, Laurienti et al., 2002, Shomstein and Yantis, 2004). A consistent finding of these studies was that neural activity that corresponded to the processing of input from unattended modalities decreased. This reduction might have been the consequence of a reallocation of processing capacities from unattended to attended sensory modalities. Importantly, such a push–pull mechanism necessarily implies (I) that sensory modalities share common processing capacities and (II) that sensory modalities compete for these common capacities.

However, in the studies described above, participants always attended to stimulation in at least one sensory modality. These situations must have imposed a strong bias on any crossmodal competition when we assume that a biased-competition-like mechanism governs intermodal attention. Participants' attention to visual stimulation, for example, imposed a strong bias towards visual processing while suppressing auditory and/or tactile processing. As a consequence, it has not been addressed to date whether (and how) visual, auditory and tactile processing interacts when neither stimulation is attended. An unbiased crossmodal competition that influences sensory stimulus processing per se has yet to be observed. As laid out above, such crossmodal competitive stimulus interactions would be the vital foundations of a biased-competition-like account of intermodal attention.

The present study aimed to test for crossmodal competition in the absence of an attentional bias. To this end, we conducted three experiments, all employing similar paradigms, yet, each featuring a unique combination of stimuli from two sensory modalities: visual and auditory stimuli in Experiment 1, visual and tactile stimuli in Experiment 2 and tactile and auditory stimuli in Experiment 3. In each experiment we frequency-tagged sensory stimulus streams (‘reference stimuli’) that were presented for several seconds. Frequency-tagged stimuli elicited oscillatory brain responses, phase-locked to stimulation that indexed the ongoing sensory processing in corresponding sensory modalities. These so-called steady-state responses (SSRs; Regan, 1989) have been shown to decrease in amplitude when a competing stimulus was presented in the same sensory modality in vision (Fuchs et al., 2008, Keitel et al., 2010), audition (Kawase et al., 2012, Ross et al., 2012) and touch (Severens et al., 2010). During reference stimulus presentation, we therefore introduced frequency-tagged ‘competitors’, i.e. stimuli of the same and/or different sensory modality, to induce competition. We compared amplitude changes of reference-driven SSRs between competitor-absent and competitor-present periods.

In all three experiments, participants were engaged in a visual discrimination task at central fixation. Participants were instructed to count brief contractions of the fixation cross while ignoring elongations. The task was designed to withdraw participant's attention from task-irrelevant peripheral visual, auditory and tactile reference stimuli and competitors in order to prevent attentional biases of inter-stimulus competition.

We hypothesized that, if, on the one hand, stimuli of different sensory modalities entered a crossmodal competition we would observe effects of suppression. SSR amplitudes during the competitor-present period would be lower than during the competitor-absent period. On the other hand, if no suppression occurred, SSR amplitudes would remain constant. Additionally, in line with previous studies on intra-modal competition, we expected reduced SSR amplitudes during competitor-present periods to indicate suppression between stimuli within senses.

Consistently, across all combinations of stimuli, we found that suppression only occurred between stimuli of the same sensory modality but not between stimuli of different sensory modalities. Therefore, while well in line with biased competition governing processing within senses, our results challenge the notion of a biased competition for common processing capacities between senses.