The time-course of prediction in incremental sentence processing: Evidence from anticipatory eye movements

doi:10.1016/S0749-596X(03)00023-8

Journal of Memory and Language

Volume 49, Issue 1, July 2003, Pages 133-156

https://doi.org/10.1016/S0749-596X(03)00023-8 Get rights and content

Abstract

Three eye-tracking experiments using the ‘visual-world’ paradigm are described that explore the basis by which thematic dependencies can be evaluated in advance of linguistic input that unambiguously signals those dependencies. Following Altmann and Kamide (1999), who found that selectional information conveyed by a verb can be used to anticipate an upcoming Theme, we attempt to draw here a more precise picture of the basis for such anticipatory processing. Our data from two studies in English and one in Japanese suggest that (a) verb-based information is not limited to anticipating the immediately following (grammatical) object, but can also anticipate later occurring objects (e.g., Goals), (b) in combination with information conveyed by the verb, a pre-verbal argument (Agent) can constrain the anticipation of a subsequent Theme, and (c) in a head-final construction such as that typically found in Japanese, both syntactic and semantic constraints extracted from pre-verbal arguments can enable the anticipation, in effect, of a further forthcoming argument in the absence of their head (the verb). We suggest that such processing is the hallmark of an incremental processor that is able to draw on different sources of information (some non-linguistic) at the earliest possible opportunity to establish the fullest possible interpretation of the input at each moment in time.

Section snippets

Experiment 1

In this experiment, we consider the processing of 3-place verbs, as in

(1)
The woman will spread the butter on the bread.
(2)
The woman will slide the butter to the man.

The verbs spread and slide subcategorize for up to two post-verbal arguments. At issue is whether the processor anticipates, either at the verb or during the first post-verbal argument (the Theme), information pertaining to the second post-verbal argument (the Goal). This study is analogous to the Altmann (1999) experiment, except that,

Discussion

The data demonstrate that during the immediately post-verbal referring expression (during ‘the butter_’—between the onset of the determiner and the onset of the subsequent preposition), there are more anticipatory eye movements towards the appropriate Goal object than towards the inappropriate Goal object. Given the 200 ms generally assumed necessary for programming and initiating an eye movement (e.g., Matin, Shao, & Boff, 1993; Saslow, 1967), these anticipatory eye movements must have been

Experiment 2

Experiment 1 and the Altmann and Kamide (1999) study explored whether one or other of a verb’s internal arguments can be anticipated on the basis of information accessed at the verb. Experiment 2 aims to extend these basic findings: In principle, the anticipatory effects reported in Altmann and Kamide (1999) and observed also in Experiment 1 here could have been driven solely by lexical information retrieved at the verb. Alternatively, information from the verb could be combined with

Experiment 3

In Japanese, all arguments of the verb appear prior to the verb. For example, consider the following example:

(7)
syoojo-ga neko-ni sakana-o yatta.
girl-nominative cat-dative fish-accusative gave.
The girl gave the fish to the cat.

In (7), all three of the verb’s arguments come before the verb, and as is typical with head-final languages, each argument is case-marked (in Japanese, the case-marking is not a nominal inflection but rather a post-nominal particle). If verb’s are the driving force behind

General discussion

The primary issue in the present paper has been whether preceding linguistic input can provide the basis for the processor to anticipate upcoming input, and if so, what kinds of information provided by that preceding input provide that basis. Previously, we have shown that selectional restrictions can be used to predict the semantic properties of whatever will be referred to post-verbally (with the assumption that the processor predicts that something will be referred to post-verbally on the

Acknowledgements

We are grateful also to Vicki White, Lynne Weighall, and Matt Smelt-Webb for helping us recruit the participants in the experiments reported here. Also, we appreciate very helpful comments from Rick Lewis, and two anonymous reviewers given to earlier versions of the draft. The present research was supported by grants to GTMA from the Medical Research Council (G9628472N and G0000224), the Economic and Social Research Council (R000222798), and the Royal Society.

References (41)

G.T.M Altmann
Thematic role assignment in context
Journal of Memory and Language
(1999)
G.T.M Altmann et al.
Late closure in context
Journal of Memory and Language
(1998)
G.T.M Altmann et al.
Incremental interpretation at verbs: Restricting the domain of subsequent reference
Cognition
(1999)
G.T.M Altmann et al.
Interaction with context during human sentence processing
Cognition
(1988)
J.E Boland et al.
Verb argument structure in parsing and interpretation: Evidence from wh-questions
Journal of Memory and Language
(1995)
C.G Chambers et al.
Circumscribing referential domains during real time language comprehension
Journal of Memory and Language
(2002)
R.M Cooper
The control of eye fixation by the meaning of spoken language: A new methodology for the real-time investigation of speech perception, memory, and language processing
Cognitive Psychology
(1974)
J.L Elman
Finding structure in time
Cognitive Science
(1990)
K Rayner et al.
The interaction of syntax and semantics during sentence processing: eye movements in the analysis of semantically biased sentences
Journal of Verbal Learning and Verbal Behavior
(1983)
L.K Tyler et al.
The on-line effects of semantic context on syntactic processing
Journal of Verbal Learning and Verbal Behaviour
(1977)

G.T.M Altmann et al.

Effects of syntax in human sentence parsing: Evidence against a structure-based proposal mechanism

Journal of Experimental Psychology: Learning, Memory and Cognition

(1994)

G.T.M Altmann

The ascent of Babel

(1997)

Androutsopoulos, I., & Dale, R. (2000). Selectional restrictions in HPSG’. Proceedings of the 18th International...

M Bader et al.

German verb-final clauses and sentence processing: Evidence for immediate attachment

Bertolucci, B. (1972). Last Tango in Paris. United...

J.E Boland et al.

Verb argument structure in parsing and interpretation. Evidence from wh-questions

Journal of Memory and Language

(1990)

N Chomsky

Aspects of the theory of syntax

(1965)

N Chomsky

Lectures on government and binding

(1981)

N Chomsky

The minimalist program

(1995)

B Hemforth

Kognitives parsing: Repräsentation und verarbeitung sprachlichen wissens

(1993)

Cited by (659)

Category Locality Theory: A unified account of locality effects in sentence comprehension
2024, Cognition
In real-time sentence comprehension, the comprehender is often required to establish syntactic dependencies between words that are linearly distant. Major models of sentence comprehension assume that longer dependencies are more difficult to process because of working memory limitations. While the expected effect of distance on reading times (locality effect) has been robustly observed in certain constructions, such as relative clauses in English, its generalizability to a wider range of constructions has been empirically questioned. The current study proposes a new metric of syntactic distance that capitalizes on the flexible constituency of Combinatory Categorial Grammar (CCG), and argues that it offers a unified account of the locality effects. It is shown that this metric correctly predicts both the presence of the locality effect in English relative clauses and its absence in verb-final languages, without assuming language- or dependency-specific differences in the sensitivity to the locality effect. It is further shown that the CCG-based distance is a significant predictor of the self-paced reading times from an English corpus, even when other known predictors such as dependency-based locality and surprisal are taken into account. These results suggest that human sentence comprehension involves rapid integration of input words into efficiently compressed syntactic representations, and CCG is a plausible theory of the grammar that subserves this process.
Multimodal measures of sentence comprehension in agrammatism
2023, Cortex
Agrammatic or asyntactic comprehension is a common language impairment in aphasia. We considered three possible hypotheses about the underlying cause of this deficit, namely problems in syntactic processing, over-reliance on semantics, and a deficit in cognitive control. We tested four individuals showing asyntactic comprehension on their comprehension of syntax-semantics conflict sentences (e.g., The robber handcuffed the cop), where semantic cues pushed towards a different interpretation from syntax. Two of the four participants performed above chance on such sentences indicating that not all agrammatic individuals are impaired in structure-based interpretation. We collected additional eyetracking measures from the other two participants, who performed at chance on the conflict sentences. These measures suggested distinct underlying processing profiles in the two individuals. Cognitive assessments further suggested that one participant might have performed poorly due to a linguistic cognitive control impairment while the other had difficulty due to over-reliance on semantics. Together, the results highlight the importance of multimodal measures for teasing apart aphasic individuals’ underlying deficits. They corroborate findings from neurotypical adults by showing that semantics can strongly influence comprehension and that cognitive control could be relevant for choosing between competing sentence interpretations. They extend previous findings by demonstrating variability between individuals with aphasia—cognitive control might be especially relevant for patients who are not overly reliant on semantics. Clinically, the identification of distinct underlying problems in different individuals suggests that different treatment paths might be warranted for cases who might look similar on behavioral assessments.
Inhibition of lexical representations after violated semantic predictions
2023, Cognition
There is a consensus that humans predict upcoming words during sentence processing. Prediction makes language comprehension fast and efficient if this anticipatory processing is accurate. However, often times, predictions are not correct. There is a lack of research investigating the cognitive operations at play when predictions are violated. According to several proposals, such violations lead to an inhibition of the predicted word to facilitate the integration of the unexpected word. Across four experiments, we have tested whether predicted words are indeed inhibited when listeners encounter unexpected stimuli, and whether the linguistic status (word or sound) and semantic congruency of a word (plausible or implausible) influences this purported inhibitory process. Using a Cross-Modal Lexical Priming paradigm, we showed that when predictions are violated, the activation of the predicted word is inhibited, resulting in increased reaction times. These inhibitory effects appear to be language specific, in that they are only observed after unexpected words, as opposed to non-linguistic sounds (tones). However, contrary to a long-held assumption in the field of sentence processing, inhibitory effects are not modulated by the semantic congruency of the unexpected word (i.e., whether the unexpected word is plausible within the sentence context). Indeed, in the current study, any linguistic information that violated listeners' semantic prediction resulted in the inhibition of the predicted word. Thus, the current findings are more compatible with a view in which unexpected linguistic events that are meaningful engage inhibitory processes with the specific purpose of inhibiting the predicted, though out-of-date, word.
When a look is enough: Neurophysiological correlates of referential speaker gaze in situated comprehension
2023, Cognition
Behavioral studies have shown that speaker gaze to objects in a co-present scene can influence listeners’ expectations about how the utterance will unfold. These findings have recently been supported by ERP studies that linked the underlying mechanisms of the integration of speaker gaze with an utterance meaning representation to multiple ERP components. This leads to the question, however, as to whether speaker gaze should be considered part of the communicative signal itself, such that the referential information conveyed by gaze can help listeners not only form expectations but also to confirm referential expectations induced by the prior linguistic context. In the current study, we investigated this question by conducting an ERP experiment (N=24, Age:[19,31]), in which referential expectations were established by linguistic context together with several depicted objects in the scene. Those expectations then could be confirmed by subsequent speaker gaze that preceded the referential expression. Participants were presented with a centrally positioned face performing gaze actions aligned to utterances comparing two out of three displayed objects, with the task to judge whether the sentence was true given the provided scene. We manipulated the gaze cue to be either Present (toward the subsequently named object) or Absent preceding contextually Expected or Unexpected referring nouns. The results provided strong evidence for gaze as being treated as an integral part of the communicative signal: While in the absence of gaze, effects of phonological verification (PMN), word meaning retrieval (N400) and sentence meaning integration/evaluation (P600) were found on the unexpected noun, in the presence of gaze effects of retrieval (N400) and integration/evaluation (P300) were solely found in response to the pre-referent gaze cue when it was directed toward the unexpected referent with attenuated effects on the following referring noun.
Two pathways in vocabulary development: Large-scale differences in noun and verb semantic structure
2023, Cognitive Psychology
In adults, nouns and verbs have varied and multilevel semantic interrelationships. In children, evidence suggests that nouns and verbs also have semantic interrelationships, though the timing of the emergence of these relationships and their precise impact on later noun and verb learning are not clear. In this work, we ask whether noun and verb semantic knowledge in 16–30-month-old children tend to be semantically isolated from one another or semantically interacting from the onset of vocabulary development. Early word learning patterns were quantified using network science. We measured the semantic network structure for nouns and verbs in 3,804 16–30-month-old children at several levels of granularity using a large, open dataset of vocabulary checklist data. In a cross-sectional approach in Experiment 1, early nouns and verbs exhibited stronger network relationships with other nouns and verbs than expected across multiple network levels. Using a longitudinal approach in Experiment 2, we examined patterns of normative vocabulary development over time. Initial noun and verb learning was supported by strong semantic connections to other nouns, whereas later-learned words exhibited strong connections to verbs. Overall, these two experiments suggest that nouns and verbs demonstrate early semantic interactions and that these interactions impact later word learning. Early verb and noun learning is affected by the emergence of noun and verb semantic networks during early lexical development.
The predictive processing of number information in subregular verb morphology in a first and second language
2023, Applied Psycholinguistics

View all citing articles on Scopus

View full text

The time-course of prediction in incremental sentence processing: Evidence from anticipatory eye movements

Abstract

Section snippets

Experiment 1

Discussion

Experiment 2

Experiment 3

General discussion

Acknowledgements

Journal of Memory and Language

Journal of Memory and Language

Cognition

Cognition

Journal of Memory and Language

Journal of Memory and Language

Cognitive Psychology

Cognitive Science

Journal of Verbal Learning and Verbal Behavior

Journal of Verbal Learning and Verbal Behaviour

Effects of syntax in human sentence parsing: Evidence against a structure-based proposal mechanism

Journal of Experimental Psychology: Learning, Memory and Cognition

The ascent of Babel

German verb-final clauses and sentence processing: Evidence for immediate attachment

Verb argument structure in parsing and interpretation. Evidence from wh-questions

Journal of Memory and Language

Aspects of the theory of syntax

Lectures on government and binding

The minimalist program

Kognitives parsing: Repräsentation und verarbeitung sprachlichen wissens