Automatic processing of grammar in the human brain as revealed by the mismatch negativity

Abstract

The Mismatch Negativity (MMN), a neurophysiological indicator of cognitive processing, was used to investigate grammatical processes in the absence of focused attention to language. Subjects instructed to watch a silent video film and to ignore speech stimuli heard grammatical and ungrammatical spoken word strings that were physically identical up to a divergence point where they differed between each other by a minimal acoustic event, the presence or the absence of a final -s sound. The sentence we come was presented as a rare deviant stimulus against the background of frequently occurring ungrammatical strings, and, in a different experiment, the ungrammatical string *we comes was the deviant in the reverse design. To control for effects related to differences between the critical words, come and comes, control conditions were used in which the same words were presented out of linguistic context. At 100–150 ms after the divergence point, the ungrammatical deviant stimulus elicited a larger MMN than the correct sentence at left-anterior recording sites. This difference was not seen under the out-of-context conditions. In the time range 100–400 ms after stimulus divergence, a spatiotemporal pattern of grammatically related effects was documented by statistically significant interactions of the word and context variables. Minimum-Norm Current Estimates of the cortical sources of the grammaticality effects revealed a main source in the left frontal cortex. We use a neurobiological model of serial order processing to provide a tentative explanation for the data.

Introduction

Language consists of discrete building blocks, language sounds (phonemes) and meaningful units (morphemes and words), plus mechanisms establishing serial order among phonemes and words. In the present study, we used the Mismatch Negativity (MMN), a well-known component of the event-related neurophysiological brain response, to investigate the mechanisms underlying the processing of the serial order of words in grammatical and ungrammatical word strings.

The MMN is elicited by infrequent deviant stimuli occasionally replacing frequently occurring “standard” stimuli. It typically reaches its maximal amplitude at frontal and central sites of the scalp, with its main generators being located in the auditory cortex of each hemisphere. Reliable MMNs can be obtained even after hundreds of presentations of the same deviant stimulus, given that it occurs at a low probability (<20%) among frequent standard stimuli. Importantly, MMN elicitation is largely independent of the direction of the subjects' attention. Even if they ignore these stimuli, being engaged in other activities, infrequent deviants elicit an MMN, suggesting that it is generated by an automatic, preattentive process Näätänen and Winkler, 1999, Picton et al., 2000a.

Recently, it was shown that the MMN reflects the presence of experience-dependent cortical memory traces for acoustic stimuli in the human brain (Näätänen, 2000). If a complex sound sequence has frequently been encountered in the past and a memory trace must therefore have formed for it due to associative learning, the MMN is enhanced. Thus, although the MMN can be elicited by both learned and new stimuli, its amplitude and possibly scalp topography is significantly modulated depending on the existence of cortical memory traces. This is best illustrated in the realm of language: The MMN was found to be larger for sounds from the subjects' native language compared with foreign language sounds that lack a corresponding phoneme in their language Dehaene-Lambertz, 1997, Näätänen et al., 1997, Winkler et al., 1999, Näätänen, 2001. Syllables were found to elicit larger MMNs when being part of a spoken meaningful word of the subjects' language compared with when the same syllables were part of meaningless pseudo-words Pulvermüller et al., 2001, Shtyrov and Pulvermüller, 2002b. Thus, it appears that the existence of brain-internal memory traces for language elements, such as phonemes and words, is reflected by the MMN response Näätänen, 2001, Pulvermüller, 2001. Since the MMN has proven to be an eminently useful tool in the cognitive neuroscience of language, its application to questions of serial order appears to be promising.

Although MMN studies could reveal brain correlates of language units, phonemes, and words, one of the most burning questions remains how the serial order of these units is processed in the human brain. Earlier investigations into the neurophysiology of syntactic processing Neville et al., 1991, Osterhout and Holcomb, 1992, Friederici et al., 1993, Hagoort et al., 1993 have led to important insights into the workings of the brain-internal grammar, but there is still no model accounting for the observed phenomena at the neurobiological level.

In animals, neurophysiological studies have revealed neuronal sequence detectors specifically responding to the serial order of elementary events Reichardt and Varju, 1959, Barlow and Levick, 1965, Hubel, 1995. A sequence detector is a neuronal element, a neuron or larger neuronal ensemble, which specifically responds to a defined sequence of elementary sensory events (Kleene, 1956), but not to the same events appearing in a different order. Sequence detectors similar to those for elementary visual events may process information about the serial order of words in sentences Pulvermüller, 2000, Pulvermüller, 2002. A grammatical sequence detector would be connected to two word category representations, A and B, so that, whenever a stimulus word from category A is followed by a member of category B, the sequence detector would become active. The neural representations of words regularly following each other in grammatical sentences would therefore be connected through a sequence detector, but not so the representations of words or morphemes that would form an ungrammatical and uncommon sequence (Fig. 1).

Critical predictions from this minimal neurobiological grammar model are the following. Since the sequence detector would connect the neuronal representations of words from categories A and B, any presentation of an A member would therefore prime B members through connections via the sequence detector. This syntactically related priming effect should become visible in the brain responses to grammatical and ungrammatical strings. Language-related priming effects are known to reduce negative-going components of the event-related potential (e.g., N2 and N4, Bentin et al., 1985, Holcomb and Neville, 1990, and, therefore, a specific prediction can be made of the behavior of an MMN that might reflect the expected syntactically related priming effect. If a word appears in a grammatically correct sequence (e.g., come after we), its memory trace is primed, and, therefore, the MMN it elicits should be smaller than that in an out-of-context condition. In contrast, the memory trace of a word placed in an ungrammatical and unusual context (e.g., comes after we) would not be primed and should, therefore, elicit a pronounced MMN similar to the one elicited by the word standing alone. Importantly, this model predicts that words in a grammatical string elicit a reduced MMN compared with the same word out of syntactic context. In contrast, if two syllables form a word and therefore have a common memory trace in the brain, the MMN to the word-final syllable is larger than that elicited in a meaningless pseudo-word context (Pulvermüller et al., 2001). This may be due to the potentiation of activity related to the ignition of the word-related memory trace, which is absent for the meaningless syllable sequence.

A second prediction is that, after the activation of the word representation, the sequence detector should become active as well. This subsequent activation of a sequence detector should be specific to grammatically correct word chains, that is, it should not occur if the stimulus word sequence is ungrammatical. In sum, the predictions are (1) that grammatical context reduces the MMN elicited by a word and (2) that a neurophysiological phenomenon interpretable as an index of the activation of a sequence detector follows the initial word-elicited MMN.

When investigating the neurobiological basis of grammar, an important methodological issue must be taken into account. A grammatical and an ungrammatical sentence necessarily differ with respect to words of which they are composed. Brain responses have been found to reliably differ between words with different properties. Words that differ with respect to physical, semantic, or other psycholinguistic features can induce event-related responses with different latencies and topographies (e.g., Osterhout et al., 1997a, Skrandies, 1998, Assadollahi and Pulvermüller, 2001, Pulvermüller, 2001. When comparing correct and incorrect sentences in a brain imaging study, it is therefore never possible to determine with certainty whether any physiological difference that may arise relates to grammar or, instead, to physical or psycholinguistic differences between word stimuli. There may be systematic differences between sets of grammatical and ungrammatical sentences—even minimal ones, such as the presence vs the absence of the suffix -s or a slight but systematic variation in intonation—that could, in principle, cause pronounced physiological effects possibly overriding or contaminating any difference between grammatical and ungrammatical strings per se. To rule out this problem, rigorous control conditions are necessary that investigate the physiological consequences of differences between the words that constitute grammatical and ungrammatical strings. Therefore, we investigated brain responses to words in grammatical and ungrammatical contexts and, in addition, to the same words presented out of linguistic context. Using the same words out of any linguistic context as controls allows us to distinguish brain indicators of grammatical context from those of other properties of the stimulus words, be they physical, phonological, lexical, or semantic in nature (Pulvermüller, 1999). In this controlled design, the critical effects should become manifest as significant interactions indicating that the context (grammatical vs ungrammatical) affects the neurophysiological difference that may be present between words (interaction Context × Word).

To avoid the strategic differences with which subjects may intentionally approach grammatical and ungrammatical strings in linguistic tasks, we chose to avoid such tasks. Instead, we asked our subjects not to attend to the language stimuli at all, but to focus their attention elsewhere, while stimulus words were presented acoustically and neurophysiological responses were recorded.

In four experiments, we examined neurophysiological brain responses to sequences of spoken English words that agree with or violate grammatical rules of the English language. As control conditions, we used the same words presented out of linguistic context. The study demonstrates for the first time that the brain detects grammatical agreement violations even if subjects are instructed to direct their attention away from the language input.