Multiple brain signatures of integration in the comprehension of degraded speech

Abstract

When listening to speech under adverse conditions, expectancies resulting from semantic context can have a strong impact on comprehension. Here we ask how minimal variations in semantic context (cloze probability) affect the unfolding comprehension of acoustically degraded speech. Three main results are observed in the brain electric response. First, auditory evoked responses to a degraded sentence's onset (N100) correlate with participants' comprehension scores, but are generally more vigorous for more degraded sentences. Second, a pronounced N400 in response to low-cloze sentence-final words, reflecting the integration effort of words into context, increases linearly with improving speech intelligibility. Conversely, transient enhancement in Gamma band power (γ, ~ 40–70 Hz) during high-cloze sentence-final words (~ 600 ms) reflects top-down-facilitated integration. This γ-band effect also varies parametrically with signal quality. Third, a negative correlation of N100 amplitude at sentence onset and the later γ-band response is found in moderately degraded speech. This reflects two partly distinct neural strategies when dealing with moderately degraded speech; a more “bottom-up,” resource-allocating, and effortful versus a more “top-down,” associative and facilitatory strategy. Results also emphasize the non-redundant contributions of phase-locked (evoked) and non-phase-locked (induced) oscillatory brain dynamics in auditory EEG.

Introduction

The influence of semantic context in listening to, analysing and comprehending speech signals is well known. However, the mechanisms behind context-aided speech comprehension have remained surprisingly opaque. This holds especially true for speech comprehension in noisy environments where semantic context is thought to be highly relevant (e.g. Miller et al., 1951, Stickney and Assmann, 2001).

Increasing efforts are made to study the functional neuroanatomy behind this complex sensory–perceptive–cognitive process of speech comprehension in a more integrative approach. Various recent psycholinguistic (Shinn-Cunningham and Wang, 2008, van Linden and Vroomen, 2007) and neuroscientific (Davis et al., 2007, Davis and Johnsrude, 2007, Hannemann et al., 2007, Obleser et al., 2007, Sivonen et al., 2006, van Atteveldt et al., 2007) studies of speech-receptive function were devoted to those factors in speech communication that lie beyond the bottom-up mechanisms of the auditory pathways in analysing the signal itself. Besides semantic meaning in the narrow sense, a whole family of influences facilitate and disambiguate speech comprehension, especially under acoustically compromised conditions, amongst them talker familiarity (e.g., Eisner and McQueen, 2005); prior experience and knowledge (e.g., Hannemann et al., 2007); and prosodic and emotional cues (e.g., Gandour et al., 2004). The main focus of this investigation is on the potential relationship of early, sensory-driven processes, as reflected in early EEG components (N100) and later cognitive–linguistic integration processes, as reflected in the sentence-final N400 and the Gamma-band response.

In the present study, we aimed at narrowing down the possible top-down influences and constrained the sentence context to a minimal and constant one. We employed the well-studied measure of cloze probability. Originally developed as a measure of text readability (Taylor, 1953), it is an empirical measure for lexical expectancy of a given word in a context. Variations of it have been adopted into important studies of semantic processing, mostly in N400 electroencephalography (EEG) designs (Connolly et al., 1995, Federmeier et al., 2007, Gunter et al., 2000, Kutas and Hillyard, 1984). A stronger N400 in response to the sentence-final key word is elicited by a low-cloze sentence such as “she weighs the flour” compared to an (otherwise identical) high-cloze version such as “she sifts the flour”. This N400 effect is commonly interpreted to reflect higher cognitive effort to integrate the low-cloze (higher lexical competition) elements of the sentence. A previous study by Aydelott and colleagues had utilised degradation of the sentence context (but not the target word) by 1-kHz-low-pass filtering, while presenting the sentence word by word with 200-ms intervals (Aydelott et al., 2006). The study delivered important initial confirmation that acoustic degradation of the context influences the magnitude of the integration-related N400: No proper N400-like response to incongruent target words built up when the preceding sentence material had been strongly low-pass filtered. However, the current study goes a step further by using a parametric (i.e., more than twofold) variation of the signal degradation; by utilizing naturally spoken sentences; and by taking a comprehensive look at various EEG signatures (i.e., the evoked or phase-locked N100 and N400 potentials, and the induced or non-phase-locked Gamma-band response, see next paragraph).

Especially at the point of sentence meaning integration (i.e., the sentence-final keyword), we expect to observe distinct influences of semantic expectancy constraints on degraded speech in the stimulus-correlated EEG: The N400, as described above, is unanimously interpreted as a signature of integration effort; we therefore expect an enhanced N400 at low-expectancy words. As outlined below, we also expect facilitation of integration (an enhancement for high-expectancy words) being reflected in a modulation of Gamma-band (γ, > 30 Hz) oscillations.

Apart of the N400 literature, a whole line of evidence points to an increased effect (i.e., higher amplitudes in response to highly predictable sentence endings) for Gamma-band (γ, > 30 Hz) oscillations. Many studies on the top-down formation of percepts have focussed on this most prominent electrophysiological signature in the visual domain (e.g., Gruber et al., 2002, Tallon-Baudry and Bertrand, 1999), in multisensory integration (e.g., Schneider et al., 2008), but also in the auditory (e.g., Lenz et al., 2007), and the speech domain (Hannemann et al., 2007, Shahin et al., 2009). All of these indicate that facilitation of percept formation (“gestalt”) through memory representations (“adaptive resonance,” Grossberg, 2009) is accompanied by comparably focal bursts of enhanced Gamma-band synchrony (for review see e.g., Fries, 2009). This synchrony surfaces as power enhancements (compared to selected baseline periods) in the Gamma-band range when analysing the EEG data for non-phase-locked oscillations.

In order to allow for a precise spectro-temporal control over the speech degradation levels applied, we chose noise-band vocoding (e.g., Shannon et al., 1995) as a manipulation technique. It allows for exact control over the spectral detail being presented (and hence the speech intelligibility) in arbitrary levels (for review see Shannon et al., 2004). It was originally designed to mimic the spectro-temporal characteristics of the input a cochlear implant carrier would receive (Faulkner et al., 2001). Published evidence on the EEG responses to noise-vocoding in normal-hearing listeners has been lacking so far. Unlike fMRI, EEG offers to potentially reveal (and decouple the time courses of) partly differential mechanisms that underlie speech comprehension, such as more “bottom-up,” effortful resource-allocation versus rather “top-down” associative facilitation.

The current experiment utilises a simple 2 × 3 factorial design: first, simple sentences were varied in the strength of the semantic expectancy coupling within that sentence (cloze probability). Second, these sentences were subjected to multiple levels of speech degradation in order to disentangle the (possibly interactive) mechanisms of degraded signal quality (a well-controlled sensory–acoustic parameter) and cloze probability (a well-controlled cognitive–linguistic parameter) on deflections of the auditory event-related potential.

In an early time range (approximately 100 ms after sound onset), effects of the degree of speech degradation are expected on the N100 component which has been shown to index early abstraction and percept-formation stages (Krumbholz et al., 2003, Naatanen, 2001, Obleser et al., 2006), especially in the context of an active comprehension task (for task effects on the N100/N100m see Bonte et al., 2009, Obleser et al., 2004a, Poeppel et al., 1996). Also based on a recent study finding enhanced N100 amplitudes in response to degraded sound (Miettinen et al., 2010), we expect the following: the more thorough the degradation of the signal, the more neural effort is likely be allocated to encoding the acoustic signal and mapping it onto known phonological and lexical categories, leading to an enhanced N100 amplitude measured on the scalp.

For the typical locus of semantic integration effort (N400, 300–500 ms post onset of the critical word in the sentence context), we expect an enhanced N400 to the sentence-final word in low-cloze probability sentences; however, the influence of various levels of speech degradation on the build-up of the sentence context is expected to modulate the N400 (cf. Aydelott et al., 2006).

Lastly, we have good reason to expect enhanced synchronisation in the Gamma band frequency range for facilitated integration of meaning for high-cloze more than for low-cloze sentences, but its dependency on acoustic signal quality and its possible interaction with other signatures of the speech-evoked EEG will be of particular interest.