Neurodynamics of sentence interpretation: ERP evidence from French

Abstract

Sentence interpretation was examined with event-related brain potentials (ERPs). The ERPs were recorded while participants listened to French sentences containing a subject-modifying relative clause (SRC). These were either correct, semantically incorrect, syntactically incorrect, or doubly (syntactically and semantically) incorrect. The semantic anomaly realized as a selectional restriction violation was associated with an N400. The syntactic anomaly realized as a phrase structure violation in the SRC elicited a frontal negativity between 150 and 600 ms. This negativity was more pronounced in the left than in the right hemisphere in the early time window (150–300 ms). In a later time window (300–600 ms), it was more broadly distributed including anterior and posterior regions, but with a maximum over the anterior recording sites. Finally, a centro-parietal late positivity (P600) was found between 600 and 1000 ms. While syntactic and semantic information in the double violation condition did not interact between 150 and 300 ms, they did interact between 300 and 600 ms. This finding supports serial models of sentence processing that postulate an initial autonomous stage of phrase structure building and a late stage of interaction.

Introduction

Sentence interpretation requires rapid integration of multiple sources of linguistic information such as phonological, morphological, syntactic, and semantic information. The issue of how the neural generators underlying the processing of these sources of information are coordinated in time (i.e., in a serial, parallel, or interactive fashion) is still a matter of considerable debate in the modeling of the brain–language relationship. With respect to the syntax–semantics interplay, one class of models, which are serial in nature, postulates that syntactic analysis is autonomous and not initially influenced by lexical-semantic or sentential-semantic information (De Vincenzi, 1991, Frazier, 1978, Frazier, 1987a, Frazier, 1987b, Gorrell, 1995). Lexical-semantic aspects only come into play during a second stage of processing. The alternative class of models claims that structural and lexical-semantic information interact continuously during parsing. Consequently, initial structure building processes can be influenced by lexical-semantic knowledge (Marslen-Wilson and Tyler, 1980, McClelland et al., 1989). This theoretical controversy has proven difficult to resolve on the basis of behavioral data, partly because off-line measures and even reaction times often represent the end product of a process rather than its on-line dynamics. The crucial timing information is now available with neurophysiological methods such as event-related brain potentials (ERPs). The millisecond time-course resolution of ERPs allows us to trace the possible interplay between different sources of linguistic information.

On the basis of the available ERP data, neurochronometric models of sentence processing have been developed with the goal of accounting for the neurodynamics of the language processes. One class of model is serial, with a first phase that is purely syntactic in nature (syntax-first model). The three-phase model formulated by Friederici, 1995, Friederici, 2002 basically relies on the assumption that lexical-semantic integration of a given word can only take place when the syntactic word category information of this word is compatible with the prior syntactic structure. This model predicts a differential time course for different types of syntactic information. Phrase structure information relevant for the build-up of local syntactic phrase structure (Phase 1) should precede the processing of thematic information, be it signaled by lexical-semantic or morpho-syntactic information relevant for thematic role assignment (Phase 2). In this model, interaction between semantic and syntactic information should only take place during the final integration phase (Phase 3). The other class of model assumes parallel-interactive processes throughout. The Unification Model proposed by Hagoort, 2003b, Hagoort, 2005 assumes that each bit of information is issued immediately to guide the interpretation process once it becomes available. It is important to note that other neurobiologically inspired models of the human language have been proposed (Hickok and Poeppel, 2000, Hickok and Poeppel, 2004, Ullman, 2001). However, these models focus mainly on the topographical aspects of the language processes (single word processing: Hickok and Poeppel, 2000, Hickok and Poeppel, 2004; sentence processing: Ullman, 2001). They therefore do not allow us to formulate predictions on the timing of a possible interaction of syntactic and semantic processes.

A distinction between semantic processing and syntactic processing during sentence comprehension has been reported in a number of ERP studies. Semantic processes are reflected in a centro-posterior bilateral negativity around 400 ms called the N400 (Kutas and Hillyard, 1980, Kutas and Federmeier, 2000). The N400 has been reported in different languages including Dutch (Brown and Hagoort, 1993, Gunter et al., 1992), French (Astésano et al., 2004, Besson and Macar, 1987), and German (Friederici et al., 1993, Münte et al., 1993, Rösler et al., 1993). The N400 component has also been shown to correlate with morpho-syntactic violations in German (Frisch and Schlesewsky, 2001, Hopf et al., 1998).

Syntactic processes have been associated with an early and a late ERP component, namely a left anterior negativity (LAN) present between 300 and 500 ms (Friederici et al., 1996, Friederici et al., 2004), and a late centro-posterior positivity (P600) present around 600 ms post-stimulus onset (Friederici, 2002, Hagoort et al., 1993, Osterhout et al., 1994). Note that the latency of the LAN correlates with the location of the word category identification point in the critical word; an early left anterior negativity (ELAN) with a latency of about 100–300 ms is observed in cases in which the identification point is early (Friederici et al., 1993, Hahne and Friederici, 1999, Hahne and Friederici, 2002). The ELAN and P600 have been attributed to two functionally different stages of syntax processing, i.e., an initial, automatized structure building process (ELAN) and late, controlled processes of syntactic reanalysis and repair (P600; Friederici et al., 1996, Hahne and Friederici, 1999). Furthermore, it has been shown that the processing of word category violations that disturb reference processing (i.e., a reference's specification of the noun in a subject-modifying relative clause or SRC) was associated with a negative shift between 300 and 600 ms with a maximum over the anterior region (Isel et al., 2000, Isel et al., 2004). Isel et al. (2004) proposed that this negative shift might reflect difficulties with reference processing. Recently, Rossi et al. (2005) replicated the reference-related negativity effect in German using similar violations, namely a syntactic word category violation in a SRC as in the French studies by Isel et al. [e.g., Der Junge im singt ein Lied (The boy in-the sings a song)].

As mentioned above, one of the questions of central interest addressed by most of the recent behavioral as well as neurochronometric models of sentence processing is whether and when syntactic information and semantic information interact in time. A critical test for approaching this issue is given by doubly (syntactically and semantically) incorrect sentences. To date, only few ERP studies have investigated the syntax–semantics interplay by crossing phrase structure violations with semantic violations (selectional restriction). Most of them have been conducted in Germanic languages. The originality of the current study is to test the effect of combined phrase structure violations in subject-modifying relative clause (SRC) which lead to a difficulty in reference processing and lexical-semantic violations employing spoken French sentences.

Studies that have investigated combined morpho-syntactic and semantic violations cluster into those that found an N400–P600 pattern (Ainsworth-Darnell et al., 1998, Gunter et al., 1997, Hagoort, 2003a, Osterhout and Nicol, 1999) and those that found an E/LAN–P600 pattern (Friederici et al., 1999b, Frisch et al., 2004, Hahne and Friederici, 2002). Ainsworth-Darnell et al. (1998) reported an N400–P600 waveform complex for visually presented doubly anomalous English sentences. In the aforementioned study, the absence of a LAN could be due to the fact that the critical words did not constitute an outright structural violation, but only a violation of a structural preference. Osterhout and Nicol (1999) reported an N400–P600 waveform complex for the visual processing of English sentences containing both a syntactically anomalous verb tense violation and a semantic selectional restriction violation. In Dutch, Hagoort (2003a) reported a larger N400 component for the visual processing of doubly anomalous sentences containing a syntactic gender violation and a semantic violation than for the processing of sentences in a pure semantic condition. This is in contrast to Gunter et al. (1997) who had reported an approximate summation of an N400 and a P600 for the visual processing of doubly anomalous Dutch sentences containing a syntactic gender violation and a lexical-semantic violation (Experiment 1). However, in a second experiment, the double violation elicited a LAN followed by an N400 and a P600 that was reduced relative to the single syntactic violation. Gunter et al. (1997) concluded that lexical-semantic constraints affect late syntactic analyses. A similar pattern, namely a LAN, an N400 and a reduced P600, was observed for German sentences including a syntactic gender violation and a semantic anomaly (Gunter et al., 2000).

Previous ERP studies in German have systematically revealed an ELAN–P600 pattern in response to the processing of doubly (semantically and syntactically) anomalous sentences (Friederici et al., 1999b, Hahne and Friederici, 2002). An N400 was not found in these studies unless subjects were instructed to ignore the syntactic error (Hahne and Friederici, 2002), but even then, the N400 was preceded by the ELAN. The biphasic ELAN–P600 complex was even observed when the semantic information present in the verb stem preceded the word category information present in the suffix [veredel-t (refined) versus Veredel-ung (refinement); Friederici et al., 2004]. This finding suggests a functional primacy of local phrase structure processes over semantic processes. The data from the latter experiment indicate that the semantics given in the stem is not processed until word category information is available to license integration.

However, Van den Brink and Hagoort (2004) reported contradictory ERP data as they showed a LAN that was time-locked to the moment of word category violation. Van den Brink and Hagoort (2004; see Hagoort, 2003b for a discussion) used Dutch sentences containing a critical word in which the earliest moment at which a listener was able to detect that the critical word did not have the appropriate word category (i.e., a verb instead of a noun) lay at the onset of the word-final syllable of the critical word. Note, however, that it is not surprising that an N400 was elicited before a LAN in the Van den Brink and Hagoort study as the word category interpretation point was not only at 300 ms into the verb (klieder-de/messed), but the first part of the stimulus (i.e., the verb stem without the inflection (-de)) constituted a noun (klieder/mess). Thus, for the double violation condition whose prior syntactic context requires a noun, the first part of the stimulus (klieder/mess) makes a good syntactic match, but its semantic mismatch elicited an N400. Only when the inflection (-de) becomes available can the parser detect a syntactic mismatch, leading to a late LAN following the N400. Frisch et al. (2004) investigated double violations containing a phrase structure violation and an argument structure violation in the visual modality (Experiment 1) and in the auditory modality (Experiment 2). In both experiments, an N400–P600 pattern was found in association with the processing of an argument structure violation alone. For the processing of double violations, in contrast, no N400 was observed. These findings are in agreement with Friederici (2002) Friederici's model (2002), which postulates that verb argument information is only processed after initial phrase structure building. The N400 component, which has also been shown to be elicited as a function of unsuccessful thematic role assignment (Friederici et al., 1999b, Frisch and Schlesewsky, 2001), is absent when the critical word's syntactic word category is incorrect. Thus semantic and thematic processes only take place once correct word category information is available.

The present study aimed to trace a possible syntax–semantics interaction that might occur during the interpretation of spoken sentences in French. The contribution the current investigation makes to the existing literature is to extend the use of the violation paradigm in French using another syntactic violation, namely a word category violation that leads to difficulties in reference processing. Previous studies have suggested that difficulties in reference processing might be associated with a frontal negativity that starts early and lasts long (sentence processing: Isel et al., 2000, Isel et al., 2004, Isel, 2005, Rossi et al., 2005; sentence processing in discourse: Van Berkum et al., 1999). In contrast to earlier studies in German or Dutch, the crucial phrase structure violation and the semantic violation in the current study appeared in sentences in which the sentence-final verb is the critical word, and the subject is modified by a relative clause (see Table 1). While the semantic violation is realized in the main clause as a mismatch between the subject noun and the sentence-final verb, the syntactic violation is realized in the relative clause as a mismatch between the preposition and the next word. Two electrophysiological experiments using a violation paradigm with an off-line task of grammaticality judgment were carried out. ERPs as well as off-line behavioral measures of accuracy were recorded while native speakers of French listened to semantically and/or syntactically correct and incorrect French subject-relative clauses. Our goal in the present study was to gain further cross-linguistic ERP evidence in order to test alternative predictions from the above-mentioned neurochronometric models. According to a serial model of sentence processing (Friederici, 1995, Friederici, 2002), initial structure building precedes semantic processes. Therefore, no interaction of the factor Syntax and Semantics is expected in the ELAN time window (150–300 ms). An interaction is expected, however, in later time windows (300–600 ms and/or 600–1000 ms). Alternatively, if initial structure building processes can be influenced by lexical-semantic knowledge as postulated by the Unification Model (Hagoort, 2003b), then an interaction of the factor Syntax and Semantics should be observed as early as the first time window of 150–300 ms.