Experience and sentence processing: Statistical learning and relative clause comprehension
Introduction
George Miller’s (1956) landmark description of the nature of short term memory was a characterization of both its limits (7 ± 2 units) and the modulation of these limits through learning, in that the units were chunks, the size of which could change through a person’s experience with the material being processed. In discussions of computational capacity since that time, different research paradigms have tended to vary in their attention to the claim of capacity limits vs. the claim that capacity changes through learning. For example, within adult sentence comprehension, many accounts have invoked capacity limits to explain people’s difficulties in relative clause comprehension (e.g., Gibson, 1998, Just and Carpenter, 1992, Lewis et al., 2006). All of these accounts have noted that experience could affect processing abilities, but the focus in these accounts has been on showing how a characterization of capacity limits explains certain aspects of sentence comprehension performance, rather than on investigating how capacity (and presumably performance) could change with learning over time.
In another sub-field of sentence processing, constraint-based accounts of ambiguity resolution have invoked comprehenders’ detailed knowledge of the language (knowledge of verb biases, discourse plausibility, etc.) in explaining differences in processing difficulty across sentences (see MacDonald & Seidenberg, 2006, for review), but here too there has been little formal attempt to address the learning that must underlie the acquisition and use of the probabilistic constraints hypothesized to shape comprehension performance. This paper begins to address the lack of research on the role of learning in sentence processing. We take relative clauses as our domain of investigation, following prior work (MacDonald & Christiansen, 2002) in which we investigated experience-based changes in relative clause processing in a computational model.
Relative clauses are a good choice for investigating the role of learning in part because these structures have been central to sentence processing research since Miller and Chomsky (1963) observed that certain types are very difficult to comprehend. Examples such as (1) are termed subject relatives because the noun modified by the relative clause (the head noun), reporter, is the subject of the relative clause verb attacked. Reordering some of the words in (1) yields example (2); sentences of this type are termed object relatives because the head noun reporter is the object of the verb attacked. The contrast between the relatively easy subject relatives and the much more difficult object relatives has formed the basis for investigations in virtually every area of psycholinguistics, including studies of memory use in language comprehension (Gibson, 1998, Gordon et al., 2001, King and Just, 1991, Lewis et al., 2006), impairment after brain injury (Dickey & Thompson, 2004), comprehension changes in cognitive aging (Wingfield, Peelle, & Grossman, 2003), typical and atypical child language development (Booth et al., 2000, Friedmann and Novogrodsky, 2007, Kidd et al., 2007), and individual differences in adults Just and Carpenter, 1992, King and Just, 1991.
- 1.
Subject relative clause: The reporter that attacked the senator admitted the error.
- 2.
Object relative clause: The reporter that the senator attacked admitted the error.
Despite this extensive use of relative clause materials in language research, there is remarkably little agreement about what makes object relatives harder than subject relatives in English. Some researchers have suggested that object relatives are harder because the sentence’s meaning is more complicated in object relatives than in subject relatives, in that the head noun (such as reporter in the object relative (2)) is simultaneously the object of the relative clause verb attacked and the subject of the main clause verb admitted, whereas the head noun is the subject of both verbs in subject relatives (Bever, 1970, MacWhinney and Pléh, 1988). Other researchers have pointed to processing difficulty as the sentence unfolds. Gibson and colleagues (Gibson, 1998, Grodner and Gibson, 2005, Warren and Gibson, 2002) have emphasized the locality of the thematic role assignments and the working memory burden of maintaining noun phrases in memory before they can be assigned thematic roles and integrated into the sentence. The word order of object relatives requires longer maintenance of unanalyzed noun phrases than in subject relatives. Others have suggested that these unintegrated noun phrases in the object relative clause interfere with each other in working memory (Gordon et al., 2004, Gordon et al., 2001, Lewis and Nakayama, 2002, Lewis et al., 2006).
A few accounts have emphasized the role of experience in object relative processing. Gennari and MacDonald (2008) argued that object relatives are much easier with inanimate head nouns as in (3) than with animate head nouns as in (2) because the animacy information is relevant to resolving ambiguities in the object relative sentences (see also Gordon et al., 2001, Mak et al., 2002, Traxler et al., 2002, Warren and Gibson, 2002). Gennari and MacDonald linked the processes of object relative interpretation to constraint-based ambiguity resolution, in which comprehenders use information concerning the most likely interpretation to guide the interpretation of temporary ambiguities in object relative clause structures. On this view, comprehenders are exposed to the distributional patterns of noun animacy that tend to occur in relative clauses (which are themselves shaped by various constraints on the language production system, Gennari & MacDonald, submitted for publication), they encode these regularities via statistical learning, and they use this knowledge to guide their interpretation of new input. Similarly, Reali and Christiansen, 2007a, Reali and Christiansen, 2007b showed that comprehenders’ knowledge of typical pronoun usage patterns in relative clauses predicted processing difficulty.
- 3.
Object relative, inanimate head: The article that the senator attacked was retracted.
To date, the most explicit description of statistical learning and ambiguity resolution in relative clauses comes from the claims and computational modeling in MacDonald and Christiansen (2002), who examined effects of learning about relative clause structures themselves, independent of animacy or other lexical content. They argued that statistical learning would have different effects on comprehension of subject and object relative clauses in English and drew an analogy between sentence comprehension processes and the Frequency × Regularity interaction in word recognition (e.g., Seidenberg, 1985). This phenomenon refers to the ambiguity in English spelling patterns, such as the fact that the letter sequence int sometimes is pronounced /int/ (as in mint) and at other times is pronounced /aynt/ (as in pint). The effect of this ambiguity on reading speed and accuracy varies with both a word’s frequency in the language and the number of other words that share the same spelling-sound correspondence (its regularity or consistency). Specifically, reading is faster and more accurate for regular words (ones with many “neighbors” with the same spelling-sound relationship, as in hint, mint, lint, dint, etc.) than for irregular ones like pint. Moreover, this effect of regularity is much larger for low frequency words than for high frequency words in the language. Regular words receive a benefit from the neighbors, which provide practice on similar spelling-sound relationships, so that computing the pronunciation of the rare regular word dint, for example, is affected not only by experience with dint itself, but also by experience with similar words hint, mint, sprint, etc. Irregular words such as pint, however, have few neighbors with similar spelling-sound computations, and thus ease of computing pronunciations for these words is strongly dependent on specific experience with the words themselves. Thus very common irregular words, such as have, are read quickly and on par with regular words of similar frequency, while rare irregular words are read more slowly than regularly spelled words in the same frequency range.
Several researchers have suggested that sentence ambiguity resolution also has aspects of the frequency × regularity interaction, that certain sentence types are more “regular” than others, meaning that there is a more consistent mapping between their surface form (the word order) and their meaning. On this view, sentence processing should also exhibit Frequency × Regularity interactions, such that interpretation of less regular sentence types (those with idiosyncratic syntax-meaning mappings) depend heavily on specific experience (frequency) of that exact structure (Juliano and Tanenhaus, 1993, Pearlmutter and MacDonald, 1995). MacDonald and Christiansen suggested that object relatives were an example of irregular sentences, while subject relatives are more regular, in that they adhere closely to the overwhelmingly frequent S–V–O (subject–verb–object) word order in English. Moreover, the pattern of thematic role assignments also follows the overwhelmingly common English pattern of the immediately preverbal noun being the agent of the action and the post-verbal noun being the patient. Readers’ processing of subject relatives thus benefits from their many encounters with sentence “neighbors”: simple transitive sentences that share the S–V–O word order and thematic role assignments of subject relatives. Object relatives, however, follow an irregular O–S–V word order and patient-agent order of thematic role assignment that are extremely rare in English. Object relatives thus have essentially no sentence “neighbors” in terms of word order and thematic role assignments. As a result, processing of object relatives benefits almost exclusively from direct experience with object relatives themselves, which are far lower in frequency than simple transitive sentences in English (Rohde, 2002, Roland et al., 2007).
MacDonald and Christiansen used the Frequency × Regularity analogy to reinterpret previous research suggesting that working memory capacity limits comprehenders’ interpretation of object relative clauses. King and Just (1991) and Just and Carpenter (1992) argued that individual differences in speed and accuracy of comprehending object relative clauses was predicted by performance on Daneman and Carpenter’s Reading Span task, that those who scored high on this measure could comprehend object relatives faster or better than those with poorer reading span scores. Whereas King and Just (1991) and Just and Carpenter (1992) emphasized capacity limits in this case, arguing that the working memory capacity of lower span readers was not sufficient to process object relatives sufficiently, MacDonald and Christiansen emphasized modulation of performance through learning. Their argument was again inspired by results in the word recognition domain. Seidenberg (1985) had suggested that the exact nature of the frequency by regularity interaction might vary across individuals. He examined readers with different levels of reading skill and argued that highly skilled readers had a large band of irregular words that could be read as quickly as regular words. By contrast, the less skilled readers read irregular words more slowly than regular words for all but the highest frequency irregulars. Seidenberg attributed this effect to differential effects of reading experience on regular vs. irregular forms—effectively a Frequency × Regularity × Experience interaction. The highly skilled group, who presumably read more than the low skill group, had more experience with both regular and irregular words, but this extra experience was much more helpful for the irregular words than the regular ones.
MacDonald and Christiansen argued that King and Just (1991) relative clause results could also be an example of a Frequency × Regularity × Experience interaction. They suggested that people who scored well on the reading span task were those who read more and thus had more experience than those with lower reading span scores; this extra experience was hypothesized to be both the source of the high-span group’s good reading span performance and their better comprehension of object relatives. More specifically, variations in reading experience were hypothesized to change the nature of people’s Frequency × Regularity interaction for relative clauses. Amount of reading experience was predicted to have little influence on subject relative processing, because even inexperienced readers have encountered the regular S–V–O word order often enough in simple transitive sentences to be fairly adept at its processing. However, reduced experience should impair the processing of object relatives, because the latter’s irregular O–S–V word order makes processing them highly dependent upon direct experience with object relatives themselves.
MacDonald and Christiansen tested these hypotheses through computational simulations in which connectionist networks were provided with differing amounts of experience on corpora generated by a probabilistic context-free grammar and a small vocabulary. The corpora included simple intransitive S–V sentences, simple S–V–O transitive sentences, and sentences with subject or object relatives, some with multiple embeddings. Importantly, subject and object relatives occurred with equal probability, each accounting for about 2.5% of the sentences in the experience corpora. MacDonald and Christiansen assessed network performance after different amounts of experience by measuring processing accuracy for novel test sentences. They predicted that subject relative processing would be facilitated by extensive exposure to simple transitive sentences in the corpora, which shared much of the S–V–O word order of subject relatives, and thus amount of experience would have little effect on subject relative processing. In contrast, the processing of irregular object relatives was predicted to be largely dependent on exposure to object relatives themselves, so that there should be a large effect of experience on object relative processing. These predictions were confirmed, lending support to the claim that the King and Just (1991) effects of reading span could have arisen from variation in reading experience.
These modeling results are suggestive, but they do not provide a direct test of the role of experience on relative clause comprehension. The current study therefore aims to investigate whether manipulations of people’s experience with relative clause constructions will result in the same experience-based patterns of performance suggested by the model. We designed a study to manipulate readers’ experience with relative clause constructions in four experimental sessions spaced over 3–4 weeks. Half of the participants were assigned to a Relative Clause experience condition and received reading experience with an equal number of subject and object relative clauses. The other half, termed the Control experience group, received experience with other complex sentences. All participants were administered the reading span task twice. The first administration was prior to the experience manipulation and was done in part to assure that the Relative Clause and Control experience groups were matched on reading span, a known correlate of verbal ability (Daneman & Carpenter, 1980). The second administration followed the experience manipulation and assessed the effect of experience on reading span scores. Reading performance on sentences involving subject and object relative clauses was also assessed before and after the experience manipulations, using a self-paced reading task similar to that used by King and Just. We predicted that the reading patterns of Relative Clause experience participants would resemble those of MacDonald and Christiansen’s SRN models—namely, equivalent exposure to subject and object relatives would facilitate reading times on object far more than on subject relatives. In other words, we predicted that readers would initially show a strong effect of relative clause type in reading times, but that the size of this effect would diminish after exposure to an equal number of object and subject relative sentences—a Testing Session (pre- vs. post-test) × Relative Clause Type interaction in reading times in critical regions. No such result is expected for the control experience group, so that across both groups of subjects, we expect an Experience Group × Testing Session × Relative Clause Type interaction. These results would support claims for the importance of reading experience—permitting statistical learning about key properties of relative clauses—in adult relative clause processing.
Section snippets
Participants
Participants were 97 undergraduates from the University of Wisconsin-Madison. They received either monetary compensation or course credit for their participation in four testing sessions. All were native speakers of English.
Reading span task
Because each participant performed the reading span task twice (once in Session 1 and once in Session 4), two lists were constructed, each composed of 70 unrelated sentences ranging in length from 11 to 17 words. None of the 70 sentence-final words was repeated within a
Results
For the following analyses, four participants (all in the Relative Clause experience group) were excluded due to experimenter or equipment error in some task, and ten (six Control experience and four Relative Clause experience) were excluded because they failed to return for all four experimental sessions. We also excluded those participants whose mean comprehension accuracy across experimental items and fillers was 75% or below on either the pre- or the post-test, removing data for 19
Discussion
The reading time results show a strong effect of experience in processing relative clauses. Importantly, the effect was asymmetrical, in that equal amounts of experience with subject and object relatives had a greater effect on the object relatives than on the subject relatives, reducing differences in reading times between the two conditions from pre-test to post-test. This pattern is consistent with the experience-based predictions in our account and is notable for its generality in several
General discussion
This work has investigated the role of experience, or learning, in interpreting complex relative clauses. More specifically, we have tested the claim that the effect of a given amount of sentence comprehension experience will vary with the nature of the sentences that are being comprehended and their relationship to previously experienced sentences. We hypothesized that subject relative structures, which share basic word order similarities with common simple transitive sentences, would receive
Acknowledgments
This research was supported by NIH Grants P50-MH64445, R01-HD47425, and the Graduate School of the University of Wisconsin-Madison. We thank Thomas Farmer, Richard Lewis, Jennifer Misyak, Florencia Reali, Mark Seidenberg, and Tessa Warren for their comments and helpful discussion.
References (85)
- et al.
Priming ditransitive structures in comprehension
Cognitive Psychology
(2007) - et al.
Acquisition of gender-like noun subclasses in an artificial language: The contribution of phonological markers to learning
Journal of Memory and Language
(1993) - et al.
Do thematic roles leave traces of their places?
Cognition
(2003) - et al.
Skill and working memory
- et al.
Toward a connectionist model of recursion in human linguistic performance
Cognitive Science
(1999) - et al.
Individual differences in working memory and reading
Journal of Verbal Learning and Verbal Behavior
(1980) - et al.
The resolution and recovery of filler-gap dependencies in aphasia: Evidence from on-line anomaly detection
Brain and Language
(2004) Finding structure in time
Cognitive Science
(1990)- et al.
The independence of syntactic processing
Journal of Memory and Language
(1986) - et al.
Is the movement deficit in syntactic SLI related to traces or to thematic role transfer?
Brain and Language
(2007)