Sentence comprehension and morphological cues in aphasia: What eye-tracking reveals about integration and prediction

Abstract

Comprehension of non-canonical sentences can be difficult for individuals with aphasia (IWA). It is still unclear to which extent morphological cues like case marking or verb inflection may influence IWA's performance or even help to override deficits in sentence comprehension. Until now, studies have mainly used offline methods to draw inferences about syntactic deficits and, so far, only a few studies have looked at online syntactic processing in aphasia. We investigated sentence processing in German-speaking IWA by combining an offline (sentence-picture matching) and an online (eye-tracking in the visual-world paradigm) method. Our goal was to determine whether IWA are capable of using inflectional morphology (number-agreement markers on verbs and case markers in noun phrases) as a cue to sentence interpretation. We report results of two visual-world experiments using German reversible SVO and OVS sentences. In each study, there were eight IWA and 20 age-matched controls. Experiment 1 targeted the role of unambiguous case morphology, while Experiment 2 looked at processing of number-agreement cues at the verb in case-ambiguous sentences. IWA showed deficits in using both types of morphological markers as a cue to non-canonical sentence interpretation and the results indicate that in aphasia, processing of case-marking cues is more vulnerable as compared to verb-agreement morphology. We ascribe this finding to the higher cue reliability of agreement cues, which renders them more resistant against impairments in aphasia. However, the online data revealed that IWA are in principle capable of successfully computing morphological cues, but the integration of morphological information is delayed as compared to age-matched controls. Furthermore, we found striking differences between controls and IWA regarding subject-before-object parsing predictions. While in case-unambiguous sentences IWA showed evidence for early subject-before-object parsing commitments, they exhibited no straightforward subject-first prediction in case-ambiguous sentences, although controls did so for ambiguous structures. IWA delayed their parsing decisions in case-ambiguous sentences until unambiguous morphological information, such as a subject-verb-number-agreement cue, was available. We attribute the results for IWA to deficits in predictive processes based on morpho-syntactic cues during sentence comprehension. The results indicate that IWA adopt a wait-and-see strategy and initiate prediction of upcoming syntactic structure only when unambiguous case or agreement cues are available.

Introduction

Many individuals with aphasia (IWA) show impairments in auditory sentence comprehension whenever reliance on syntactic structure is necessary in order to derive the correct sentence interpretation (Caramazza & Zurif, 1976; for an overview see Caplan, 2006). Often, severe comprehension deficits can be observed for semantically reversible non-canonical sentences such as object-verb-subject sentences (OVS), passives, object clefts and object relative clauses, which are derived by movement operations. In contrast, IWA perform better with canonical structures like subject-verb-object sentences (SVO), subject clefts and subject relative clauses (e.g., Cho-Reyes and Thompson, 2012, Grodzinsky, 2000, Mitchum and Berndt, 2008). However, different syntactic structures may be affected to varying degrees across individual IWA and intra-individual patterns do not always yield significant canonicity effects (Berndt et al., 1996, Caplan et al., 2007, Caramazza et al., 2001, Luzzatti et al., 2011). Nevertheless, IWA's sentence comprehension abilities are significantly worse than controls' and this effect is frequently more pronounced for non-canonical structures.

Although traditionally sentence comprehension deficits have been associated with Broca's aphasia, there is overwhelming evidence that impairments in sentence comprehension, particularly deficits in assigning thematic roles correctly, can occur across all aphasic syndromes (Caplan et al., 1985, Caramazza and Miceli, 1991, Dronkers et al., 2004, Luzzatti et al., 2011).

The answer to one important question remains unclear: to what extent do different morphological cues (for example, case marking or verb inflection) influence IWA's performance in non-canonical structures? Such cues might equally well hinder or help override sentence processing deficits (Burchert, De Bleser, & Sonntag, 2003).

As languages differ in the extent to which morphological cues are overtly realized and, thus, may constitute cues to sentence meaning, it is important to study the interplay of grammatical morphology and syntactic processing with reference to language-specific properties. In the case of studies involving English-speaking IWA, only limited conclusions can be drawn about the interplay of morphology and syntax from IWA's performance, because in English many morphological markers are not realized overtly. Therefore, morphological cues like case markers on nouns and person or number-agreement morphemes on verbs provide only limited information towards the meaning of a sentence. For most sentence structures, English heavily relies on a strict subject-verb-object word order principle.

In contrast, languages with rich grammatical morphology are less restricted in their word order and they provide overt morphological cues to sentence meaning. In German, for example, word order is less constrained than in English and sentences may deviate from canonical SVO order. Moreover, German has a rich case-marking system realized on nouns and determiners and, although some case syncretisms and ambiguities exist, the grammatical function and thematic role of many noun phrases (NPs) can be inferred from their case markings.¹ In addition, German verbs are inflected for person and number in agreement with the subject. Thus, it is possible to deduce theta-role assignments from the verb's inflectional cue even in sentences with case-ambiguous NPs. However, these overt morphological cues to theta-role-assignment, case and agreement, have different properties in terms of their reliability. Agreement cues on verbs are more reliable than case cues on NPs, because case markings can be ambiguous due to case syncretisms. Agreement cues, on the other hand, are not ambiguous when they mark third person singular or plural.

In this study, we capitalize on these morpho-syntactic properties of the German language in order to investigate processing of case morphology and verb inflection cues in German-speaking IWA and age-matched neurologically unimpaired controls. So far, studies have mostly used offline methods to investigate sentence processing in aphasia and only a few studies have investigated this issue from the perspective of online processing (for example, Dickey et al., 2007, Hanne et al., 2011, Meyer et al., 2012). Therefore, this study focuses on online sentence processing in addition to offline comprehension.

Studies looking at morpho-syntactic processing in aphasia in languages with overt morphology indicate that IWA show deficits in processing case markers as a cue to sentence comprehension (Friedmann and Shapiro, 2003, Kljajevic and Murasugi, 2010, MacWhinney et al., 1991, Smith and Bates, 1987, Smith and Mimica, 1984, Yarbay Duman et al., 2011).

Yet, some studies found preserved abilities in IWA (De Bleser et al., 1988, Lukatela et al., 1988, Milekić et al., 1995). However, these studies included tasks like grammaticality judgement or constituent ordering, which do not explicitly target IWA's ability to use case markers as a cue for identifying theta-role relations in non-canonical sentences.

In contrast, across languages, IWA have been shown to be impaired in tasks targeting specifically this ability. For example, Kljajevic and Murasugi (2010) used a figurine-pointing task to study Croatian-speaking IWA's ability to rely on case morphology as a cue to thematic role identification in reversible subject- and object-extracted sentences. IWA's performance on this task was impaired, although the degree of impairment was different across aphasia syndromes. Using an enactment task, Smith and Mimica (1984) and Smith and Bates (1987) also observed impairments in Serbo-Croatian-speaking IWA's processing of case inflections. Impairments in correctly processing case markers in an enactment task have also been observed in Hungarian and Turkish-speaking IWA (MacWhinney et al., 1991). The results for Turkish-speaking IWA have been replicated in another study involving a sentence-picture matching task (Yarbay Duman et al. 2011). Sentence-picture matching has also been used by Friedmann, Reznick, Dolinski-Nuger, and Soboleva (2010) in order to investigate processing of case cues and theta-role assignment in Russian-speaking IWA. Consistent with the previous studies, the authors found no facilitatory effect of case-marking cues on IWA's sentence comprehension. Similar results were obtained for the use of case markers in Hebrew-speaking IWA (Friedmann & Shapiro, 2003).

Impaired abilities in using case marking as a cue for establishing thematic relations have also been found for German-speaking IWA (Burchert et al., 2003, Heeschen, 1980, Swoboda-Moll et al., 2002). In the study by Burchert et al. (2003), agrammatic IWA performed significantly worse in comprehending reversible non-canonical (OVS, object relatives) as compared to canonical sentences (SVO, subject relatives), even when unambiguous case markers provided explicit cues to theta-role assignments. Moreover, although individual patterns across IWA were heterogenous, ranging between above chance (but still impaired) and chance performance, none of the IWA performed below chance on OVS sentences. Such a performance would be expected if case markers in object-first sentences were completely ignored. If IWA did not process any case morphology at all, they would be expected to apply a linear agent-first strategy in identifying thematic roles in OVS sentences. This way, a reversible OVS sentence would be interpreted as if it was an SVO structure, leading to an interpretation with reversed theta-roles. Thus, performance would be expected to be constantly below chance in tasks like enactment or sentence-picture matching with distractor pictures showing role reversals.

As mentioned earlier, most studies on sentence comprehension and morphological cue processing in aphasia have used offline methods and, so far, there is only one study which investigated the effect of case-marking cues on sentence comprehension in aphasia online. Hanne et al. (2011) conducted an eye-tracking study in order to investigate theta-role assignment and sensitivity to case-marking cues in German-speaking IWA. Similar to the results of Burchert et al. (2003), IWA were impaired in using the case-marking cues to correctly interpret OVS sentences, for which performance was at chance level. The eye-tracking data, however, revealed that IWA's online sentence processing and integration of the morphological cue was not completely impaired but delayed, as correct offline responses were associated with normal-like fixation patterns. However, fixation patterns diverged from those of controls for IWA's incorrect responses.

Taken together, studies targeting the use of case markers as a cue to thematic-role assignment have found impairments in aphasic sentence comprehension irrespective of the presence or absence of morphological cues. However, although these studies indicate that, across different languages, IWA are severely impaired in using case cues, the frequent observation of chance and sometimes even above chance performance instead of below chance performance for non-canonical case-marked sentences indicates that some retained sensitivity to morphology could be present in aphasia (cf. Burchert et al., 2003, Friedmann and Shapiro, 2003). This view is also corroborated by the online results in Hanne et al. (2011), which point to a processing-based deficit in terms of a delay in integration of word order and case-marking cues rather than a complete insensitivity to morphological markers.

When it comes to the processing of verb inflection as a cue to sentence comprehension, studies using tasks like grammaticality judgement or word-monitoring have reported preserved sensitivity to inflectional morphology in Italian-, English- and German-speaking IWA (Devescovi et al., 1997, Friederici et al., 1992, Wulfeck and Bates, 1991). However, as mentioned above, these tasks do not explicitly tap whether theta-roles have successfully been assigned for sentence comprehension.

Other studies have used tasks which allow us to draw more conclusions about the use of inflectional markers as a cue to assigning thematic roles (Bates et al., 1987, Burchert et al., 2003, Nicol et al., 1996, Smith and Bates, 1987). Using an enactment task, Bates et al. (1987) found that German-speaking IWA were impaired in processing agreement markers on verbs as a cue for theta-role assignment.² Similarly, in the study by Smith and Bates (1987), Serbo-Croatian-speaking IWA's sentence comprehension was impaired when verb agreement cues alone indicated the agent–theme relations of a sentence. Their ability to use the inflectional cue for assigning theta-roles depended on the presence of additional converging cues like case markers. In the study by Nicol et al. (1996), English- and French-speaking IWA failed to use the number-marking inflection on the verb in order to correctly perform a sentence-picture matching task.

Burchert et al. (2003) studied IWA's use of verb inflection as a cue to theta-role assignment in German case-ambiguous sentences using sentence-picture matching. Consistent with Smith and Bates (1987), they found no positive effect of number-agreement cues on IWA's comprehension performance, and, moreover, IWA appeared to be more impaired in using agreement as compared to case-marking cues (the use of which, nevertheless, was impaired, see above). However, similar to the findings for case-marking cues, analyses at an individual level revealed that IWA performed heterogeneously as two out of the seven IWA could benefit from the presence of verb agreement cues.

In sum, although some studies reported retained sensitivity to subject-verb-agreement cues in tasks like grammaticality judgement, there is no evidence for beneficial effects of verb morphology on sentence comprehension in aphasia in tasks that more explicitly tap theta-role assignment. Importantly, no study has, to our knowledge, investigated online processing of subject-verb agreement cues for sentence comprehension in aphasia.

German word order is rather free and grammatical roles as well as thematic relations must often be derived from the case-marking cues contained in NPs. For example, in simple declarative sentences, the first NP may be the subject as well as the object of the sentence. In sentences containing NPs that are unambiguously marked for case, word order (SVO or OVS, respectively) is signaled by the nominative or accusative marker at NP1 (see (1)). However, due to case syncretisms, a sentence like (2) is globally ambiguous with respect to word order and the SVO as well as the OVS reading is possible. Yet, case-ambiguous sentences like (3) may be disambiguated at the verb by subject-verb-agreement cues.

• (1) a. SVO:

  • Der Arzt schubst den Dieb.

  • the_NOM doctor pushes the_ACC thief

  • ‘The doctor pushes the thief.’

  • b. OVS:

  • Den Arzt schubst der Dieb.

  • the_ACC doctor pushes the_NOM thief

  • ‘The thief pushes the doctor.’

• (2) Ambiguous:

  • Das Kind schubst die Tante.

  • the_NOM/ACC child pushes the_NOM/ACC aunt

  • ‘The child pushes the aunt./The aunt pushes the child.’

• (3) a. Verb-agreement based disambiguation towards SVO:

  • Das Kind schubst die Tanten.

  • the_NOM/ACC child_singular pushes_{3rd–pers–singular} the_NOM/ACC aunts_plural

  • ‘The child pushes the aunts.’

  • b. Verb-agreement based disambiguation towards OVS:

  • Das Kind schubsen die Tanten.

  • the_NOM/ACC child_singular push_{3rd–pers–plural} the_NOM/ACC aunts_plural

  • ‘The aunts push the child.’

Overall, as revealed by reading time studies, there is a strong subject-before-object preference for German sentences in which the first NP is ambiguous between nominative and accusative case (Gorrell, 2000, Hemforth and Konieczny, 2000) and processing of OVS sentences with unambiguous case marking like (1-b) is associated with increased reading times as compared to SVO sentences. It has been suggested that the subject-first bias cannot solely be explained in terms of the frequency of particular structures, but rather reflects the application of language-specific grammatical principles in order to predict upcoming structural positions (Bornkessel, Schlesewsky, & Friederici, 2002). However, although SVO is the preferred order, unambiguous morphological cues indicating a non-canonical ordering of arguments are processed incrementally, leading to rapid revision of the assumed SVO template in locally case-ambiguous OVS sentences. Yet, reanalysis is costly, as reflected by increased end-of-sentence reading times (Bader and Meng, 1999, Schlesewsky et al., 2000, Schriefers et al., 1995) and findings of an early negativity together with a P600-component in ERPs (Frisch et al., 2002, Matzke et al., 2002, Mecklinger et al., 1995).

Given the presence of case ambiguities on the one hand and the incremental nature of processing of unambiguous morphological cues on the other hand, Schlesewsky and Bornkessel (2004) have suggested a two-pathway processing system for theta-role assignment in German sentence comprehension. When sentences are case ambiguous, listeners rely on positional order information in terms of a subject-first bias. However, when there is unambiguous case-marking information, a morphological pathway is pursued and theta-role assignment is based solely on morphological information (Bornkessel et al., 2004, Bornkessel et al., 2003).

Besides reading time and ERP studies, processing of morphological cues in German has also been studied using the visual-world paradigm (Kamide et al., 2003, Knoeferle, 2007, Knoeferle et al., 2005). The visual-world paradigm provides an excellent method to investigate language comprehension in real time (Boland, 2005). By recording the eye-movements on visually presented scenes or objects during spoken language processing, studies applying the paradigm reveal the rapid online processing mechanisms used during syntactic parsing and sentence comprehension. These processes may remain concealed in experiments looking at offline comprehension or end-of-sentence reaction times in isolation. Thus, although offline results in reading time studies suggest that processing of OVS sentences is associated with higher processing load, visual-world studies have revealed that, in German declaratives, listeners incrementally integrate unambiguous case-marking information (Kamide et al., 2003). Moreover, visual-world data has provided evidence that in case of a sentence-initial NP unambiguously marked for accusative case, the preferred SVO template is rapidly revised and listeners immediately pursue an OVS parse after processing of the verb. Applying the visual-world paradigm, Knoeferle (2007) also found that identification of non-canonical structure in case-unambiguous sentences is directly time-locked to processing of the case-marking cue in NP1, even before the verb has been processed. For sentences with a case-ambiguous first NP, on the other hand, participants preferred an SVO interpretation. However, just as in unambiguous sentences, listeners rapidly reanalyzed their subject-first expectation immediately after morphological information later in the sentence (e.g., the case marker at NP2) indicated an OVS structure. Such disambiguation is directly time-locked to processing of the respective lexical or morphological information, as revealed by reading times and ERP patterns (Knoeferle et al., 2008, Matzke et al., 2002).

When it comes to processing of verb inflection cues, studies have looked at wh-questions with a case-ambiguous wh-NP (i.e., a case-ambiguous wh-question-word together with a case-ambiguous noun), which is disambiguated by a number-agreement cue at the verb (beim Graben et al., 2000, Meng and Bader, 2000). These studies revealed that, in general, there is a subject-first bias for such sentences. However, the inflectional cue at the verb triggers immediate reanalysis towards an object-first structure when it is not in agreement with the number of the initial NP. According to Meng and Bader (2000), in wh-questions, this reanalysis is associated with higher processing costs as compared to reanalysis triggered by case cues. However, in German, object wh-questions occur more frequently than object-initial declaratives and such differences in input frequency may affect sentence processing. Therefore, the finding of different costs of reanalysis in questions may not be directly extended to declaratives. Furthermore, to our knowledge, so far no study has used the visual-world paradigm to investigate online processing of inflectional agreement markers on verbs as a cue to theta-role assignment in unimpaired sentence comprehension in German. Thus, it is still an open question whether in German declaratives, reanalysis triggered by verb-agreement cues is indeed more costly than reanalysis triggered by case markers.

In summary, for processing of case-ambiguous German main clauses, studies have found a strong bias for an SVO interpretation of the sentence. However, morphological cues indicating non-canonical word order that become available later in the sentence (for example, case markers at NP2) are incrementally integrated into the current parse and the structure is rapidly revised towards an OVS order. Moreover, studies investigating processing of wh-questions revealed that, when no unambiguous case cues are available, verb inflection cues trigger rapid revision towards a non-subject-initial word order. Nevertheless, the costs of reanalysis are reflected in increased processing time, as revealed by studies measuring offline data such as reading times or end-of-sentence response times in additional tasks like sentence-picture matching. For sentences containing unambiguous morphological cues at the initial NP1, listeners override their subject-first bias even more quickly, due to rapid cue integration. However, just as in case-ambiguous sentences, end-of-sentence response times are still increased for case-unambiguous OVS sentences.

In recent years, prediction and integration processes have gained a lot of attention in sentence comprehension research (e.g., Gibson, 2000, Levy, 2008, Lewis and Vasishth, 2005). It is commonly assumed that when hearing a sentence like (1-a), listeners treat the sentence-initial NP as the subject of the sentence and predict a verb in third person singular. Once the listener encounters the transitive verb, a post-verbal object-NP is predicted. By contrast, in a sentence like (1-b), the unambiguous accusative cue at NP1 leads to prediction of an OVS structure with a transitive verb. On encountering the verb, listeners predict a post-verbal subject-NP. For sentences like (3-a) and (3-b), in the absence of unambiguous case cues, listeners base their predictions on canonical assumptions and presume that NP1 is the subject of the sentence. Due to its number marking, they predict a verb in third person singular. In a sentence like (3-a) this prediction is confirmed at the verb and the verb's agreement cue can be integrated properly and a post-verbal object-NP is predicted. In contrast, in a sentence like (3-b), a garden-path arises because the predicted verb morphology does not confine with the agreement cue in plural. In order to integrate this cue, listeners revise their parsing decision towards an OVS structure and finally predict a post-verbal subject-NP. Little is known about such purely morpho-syntactic predictive abilities related to sentence processing in aphasia. There is evidence that IWA are impaired in generating predictions about upcoming arguments (Mack, Ji, & Thompson, 2013), however, this study focused on lexical-semantic prediction based on the meaning of the verb in a sentence. No study so far has investigated prediction and integration based on morpho-syntactic cues in aphasia. We will return to these issues in the general discussion and discuss the impact of our findings on assumptions about predictive abilities in aphasia.