Elsevier

Brain and Language

Volume 125, Issue 3, June 2013, Pages 272-282
Brain and Language

Regular Article
Good, bad and ugly word stress – fMRI evidence for foot structure driven processing of prosodic violations

https://doi.org/10.1016/j.bandl.2013.02.012Get rights and content

Abstract

Using a stress violation paradigm, we investigated whether metrical feet constrain the way prosodic patterns are processed and evaluated.

Processing of correctly versus incorrectly stressed words was associated with activation in left posterior angular and retrosplenial cortex, indicating the recognition of an expected and familiar pattern, whereas the inverse contrast yielded enhanced bilateral activation in the superior temporal gyrus, reflecting higher costs in auditory (re-)analysis. More fine-grained analyses of severe versus mild stress violations revealed activations of the left superior temporal and left anterior angular gyrus whereas the opposite contrast led to frontal activations including Broca’s area and its right-hemisphere homologue, suggesting that detection of mild violations lead to increased effort in working memory and deeper phonological processing.

Our results provide first evidence that different incorrect stress patterns are processed in a qualitatively different way and that the underlying foot structure seems to determine potential stress positions in German words.

Highlights

► Correct word stress show activation of left post. AG and RSC indicating recognition of expected pattern. ► Incorrect word stress show bilateral STG activation reflecting higher costs in auditory analysis. ► Severe stress violation revealed activations of the left STG and anterior AG. ► Mild stress violation led to BA 44/45 activation and its homologue, more WM load and deeper phonol processing. ► Stress violations in German are processed in qualitatively different way depending on foot structure.

Introduction

How do we know on which syllable a given word should be stressed? For some languages including Russian (Idsardi, 1992) it has been proposed that the position of primary stress is completely arbitrary and thus subject to idiosyncratic lexical specifications. For other languages like Polish (Hayes, 1995), simple rules are assumed such as “always put primary stress on the penultimate syllable”. For a third group of languages including Dutch, English, and German (Trommelen, Zonneveld, & Jessen, 1999), more complex rules have been proposed, typically referring to a hierarchical prosodic structure consisting of syllables which are parsed into so-called metrical feet (for more details see below). However, empirical evidence for sublexical prosodic structures is still sparse.

To date, the factors that influence stress assignment in German are still under debate. There are some accounts assuming that word stress positions other than penultimate stress are unpredictable and have to be specified in the lexicon (Levelt, 1999, Wiese, 1996), whereas others propose that stress positions can be calculated on the basis of heavy and light syllables (Féry, 1998, Giegerich, 1985, Janßen, 2003, Vennemann, 1990). There is accumulating evidence for prosodic structures at the sub-word level suggesting that syllables are parsed into feet and that it is this prosodic structure which – at least partly – drives the assignment of word stress in German (Domahs et al., 2008, Janßen and Domahs, 2008, Roettger et al., 2012, Tappeiner et al., 2007). In the present study, we aim at investigating whether sublexical prosodic structures play a role in German stress assignment and which brain areas are involved in prosodic parsing.

Before the details of the current study will be outlined, we will give a brief summary of recent advances in research on prosodic properties (i) by describing the prosodic properties of German words in more detail, (ii) summarizing previous psycho- and neurolinguistic findings on the processing of prosodic information, and (iii) arguing how a stress violation paradigm can be informative for a better understanding of the neural basis of prosodic word processing.

In the metrical description of phonological words, an intermediate foot level has been adopted between the level of syllables and words in order to account for the property that strong and weak syllables tend to alternate (Liberman and Prince, 1977, Nespor and Vogel, 1986, Selkirk, 1984). The concept of the foot relates to entities consisting of maximally two syllables of which either the first or the second one is strong. The grouping of syllables into feet and, more particularly, the grouping of feet into prosodic words determines the potential position of primary stress in a word, in which the strong syllable of a strong foot bears primary stress (see Fig. 1).

Despite the fact that there seems to be a statistical dominance of words stressed on the penultimate syllable in Modern Standard German (e.g., Eisenberg, 1991, Wiese, 1996), the primary stress position of monomorphemic words is variable and can be assigned to one of the last three syllables of a prosodic word (Three-Syllable Window, Vennemann, 1990). In accounts which assume a metrical structure (e.g., Domahs et al., 2008, Féry, 1998, Giegerich, 1985, Janßen, 2003, Vennemann, 1990), German is characterized as being a trochaic language in which feet are built from the right edge of the word to the left. In words including more than one foot, typically the rightmost foot is strong and receives primary stress. Yet, it is controversial whether German stress assignment is sensitive to the quantity of syllables, i.e., whether heavy final or penultimate syllables attract stress (as assumed by Domahs et al., 2008, Féry, 1998, Giegerich, 1985, Janßen, 2003, Vennemann, 1990, but see Eisenberg, 1991, Levelt, 1999, Wiese, 1996). In quantity-sensitive languages, closed syllables or syllables containing long vowels are considered to be heavy, whereas light syllables are open and contain only a short vowel. In quantity-insensitive accounts on stress assignment, the weight of syllables does not contribute to the position of word stress.

In some phonological and psycholinguistic accounts, German is considered to be insensitive to syllable weight (e.g., Eisenberg 1992; Kaltenbacher, 1994, Levelt, 1999, Levelt et al., 1999, Wiese, 1996). Since most of the native words are bisyllabic and end in a reduced syllable, which cannot bear primary stress, the statistically predominant stress position is the penultimate syllable. Thus, it is suggested that only penultimate stress can be considered as the regular (default) pattern, whereas for words with final and antepenultimate stress the stress position has to be determined lexically.

However, an alternative class of approaches assumes that syllable weight is relevant for German stress assignment (e.g., Alber, 1997, Féry, 1998, Janßen, 2003, Giegerich, 1985, Knaus and Domahs, 2009, Vennemann, 1990). In particular, it has been suggested that the final position tends to be stressed in words with a heavy final syllable (e.g., Vitaˈmin), but typically remains unstressed in words with a light final syllable (e.g., Aˈgenda). In the latter case, the penultimate syllable receives primary stress. According to these approaches, only final and prefinal stress is predictable by the weight of the final syllable, whereas antepenultimate stress (e.g., ˈLexikon) is assumed to be prespecified in the lexicon (Knaus and Domahs, 2009, Vennemann, 1990).

In terms of foot structure, it is suggested that German words with final or antepenultimate stress consist of two prosodic feet: the final syllable forms a non-branching foot (σ)F and the first two syllables build a trochee (‘σσ)F–resulting in very similar structures like ((vi.ta)Fw(ˈmin)Fs)ω and ((ˈa.na)Fs(nas)Fw)ω; see Fig. 1). In words with final stress, the final foot bears word stress, while in words with antepenultimate stress, the first foot does. In contrast to words with final and antepenultimate stress, we assume that words with penultimate stress consist of only one final trochee, whereas the first light syllable is left unparsed (bi(ˈki.ni)Fs)ω; see Fig. 1).

While the role of prosody in general is well-established in psycholinguistic studies of word processing, it remains still unknown whether feet are cognitively valid concepts. Previous research mostly elucidated the importance of prosody in the identification and segmentation of linguistic units such as words or phrases (Cutler and Clifton, 1984, Cutler and Norris, 1988, Grosjean and Gee, 1987, Friedrich et al., 2004, Knaus et al., 2007, Mattys, 2000, Nazzi et al., 1998, Steinhauer et al., 1999). The most crucial finding for Germanic languages such as English, German, or Dutch is that in most cases initial strong syllables mark the beginnings of words and are therefore used for the segmentation of words out of the speech stream (the so-called Metrical Segmentation Strategy, as proposed in the seminal work by Cutler and Norris (1988)). Furthermore, stressed syllables play a greater role for word recognition than unstressed syllables (Domahs et al., 2008, Gerken, 1994, Grosjean and Gee, 1987).

More specific empirical evidence for the sensitivity of stress to syllable weight in German comes from production tasks using pseudo-words (Janßen, 2003, Janßen and Domahs, 2008, Roettger et al., 2012, Tappeiner et al., 2007). In these tasks, trisyllabic pseudo-words varying in syllable structure were stressed predominantly on the penultimate syllable only if words ended with an open final syllable. In contrast, words with a closed final syllable were realized either with antepenultimate or final stress. Thus, the distribution of stress patterns suggested that the former case involves a single foot (final trochee) whereas the latter involves two feet (final monosyllabic foot preceded by a trochee), both of which may attract primary stress (see Fig. 1).

ERP studies investigating the processing of different stress patterns revealed either a fronto-central negativity effect (N325) for deviations from strong-initial words by weak-initial words (Boecker, Bastiaansen, Vroomen, Brunia, & De Gelder, 1999) or no ERP differences between correct and incorrect stress patterns, but an overall parietal P200-effect for initially unstressed words (Friedrich, Alter, & Kotz, 2001). Thus, these studies provide evidence for processing differences between the so-called default stress on the penultimate syllable and the exceptional stress on the final syllable. Note that these results were not obtained in stress violation paradigms.

A metrical violation paradigm has been investigated by Magne et al. (2007) as well as by Marie, Magne, and Besson (2011). In these EEG-studies, French participants were presented with trisyllabic words embedded in a sentence context with congruent and incongruent (i.e., lengthened penultimate syllable) metrics and were asked to explicitly judge the syllable length of the critical words. As a result, the authors obtained a biphasic pattern which they interpreted as an N400 followed by a late positive component which was interpreted as an instance of a P300 or P600 (Magne et al., 2007) or P600 component (Marie et al., 2011)). The authors suggested that metrical information may be used in the process of lexical retrieval and word comprehension. The positivity effect was interpreted to reflect the processing of unexpected phonetic parameters in the condition of manipulated words. Although Magne et al. (2007) did not manipulate the prosodic prominence relation at the level of word stress, their data suggest that metrical violations also affect the processing of words.

Late positive components were also obtained in a series of ERP-studies on German word stress processing performed by Knaus et al., 2007, Domahs et al., 2008, Domahs, Knaus, Orzechowska, and Wiese (2012) and Domahs, Genc, Knaus, Wiese, and Kabak (in press). The late positive component these authors reported for specific types of stress violations was interpreted as a variant of a P300 component based on the fact that this component is task-dependent (as this effect does only occur in explicit tasks (Knaus et al., 2007)) and modulated by the degree of deviation from the expected form. In particular, in the study reported by Domahs et al. (2008), participants were confronted with correctly and incorrectly stressed trisyllabic words that either led to a change in prosodic structure or not. Trisyllabic German words consisting of two feet and being stressed on either the antepenultimate syllable (APU, (ˈLe.xi)Fs(kon)Fw) or final syllable (F, (Vi.ta)Fw(ˈmin)Fs) were contrasted with words consisting of one single foot stressed on the penultimate syllable (PU, Bi(ˈki.ni)Fs). A comparison between ERPs associated with different types of incorrect probes revealed that foot structure violations (incorrect forms leading to restructuring of feet (e.g., PU changed into F as in Bi(ˈki.ni)Fs to (Biki)Fw(ˈni:)Fs), henceforth called severe violations, produced P300 effects, whereas stress violations maintaining foot structure (e.g., APU changed into F as in (ˈLe.xi)Fs(kon)Fw) to (Le.xi)Fw(ˈkon)Fs)), henceforth called mild violations, did not. So-called severe violations are more clearly deviant and therefore easier to reject than mild violations. The experiment also demonstrated that binary foot structures are a crucial part of the German prosodic system and that stress information is processed in the perception of words as soon as information about prominence relations between the first two incoming syllables is available.

In addition to studies on word stress processing, a series of studies investigated rhythmical processing in sentences with either regular or irregular alternations of strong and weak syllables. Since rhythmical processing is related to word stress processing, these studies are briefly summarized in the following. Rothermich et al., 2010, Rothermich et al., 2012, and Schmidt-Kassow and Kotz, 2008, Schmidt-Kassow and Kotz, 2009 observed biphasic ERP patterns associated with the processing of metrical irregularities. In sentences with otherwise regular meter (isochronous occurrence of strong syllables), a deviation from the regular meter led to an early negativity effect followed by a late positive component. Depending on the task, metrical manipulations either modulated an N400 component that occurred in a semantic task (e.g. Rothermich, Schmidt-Kassow, & Kotz, 2012), or yielded an early negativity (e.g. Rothermich, Schmidt-Kassow, Schwartze, & Kotz, 2010). With respect to the positivity effect, the authors found similar components for syntactic and metrical violations and therefore interpreted the late positivity as a P600 indicating that processes of reanalysis may be involved in explicit judgments of metrical violations. In an ERP-study on rhythmically marked structures involving either stress clash or a sequence of adjacent unstressed syllables (stress lapse), Bohn, Knaus, Wiese, and Domahs (2013) found that the acceptance of stress shifts is influenced by the overall rhythmical preference for alternations of strong and weak syllables. Stress shifts within words are rhythmically licensed if they avoid stress clashes, but not if they lead to stress lapse. In this experiment, sentences with stress clash evoked an early negativity and sentences with stress lapse an N400 due to deviations from default stress. Furthermore, the evaluation of metrical structures evoked a late positive component for the condition with adjacent weak syllables but not for the clash condition.

So far, there are only two neuroimaging studies directly addressing the processing of word stress. Aleman et al. (2005) observed cortical areas in the left precentral gyrus, the left superior parietal lobule, and in the posterior part of the left superior temporal gyrus extending into the sulcus to be more active in a word stress task compared to a semantic control condition. However, when word stress processing was compared to a more similar phonological task (vowel processing), a bilateral fronto-temporal network comprising the superior temporal gyri extending into the sulci as well as the inferior frontal gyri was found to be activated (Klein et al., 2011, Domaha and Knaus, 2012). These findings corroborate the assumption that phonological processing in general is subserved by a network comprising posterior-frontal and superior temporal areas (Vigneau et al., 2006). However, there is no neuroimaging study yet which examines the influence of metrical structure on word stress processing. Geiser, Zaehle, Jancke, and Meyer (2008) investigated neural correlates of speech rhythm, in particular the processing of isochronous and non-isochronous pseudosentences. Brain areas involved in the processing of speech rhythm were the bilateral supplementary motor area (SMA) extending into the cingulate gyrus as well as the insulae and the right inferior frontal gyrus. In an explicit task (judgment of isochronicity) a contrast between isochronously and non-isochronously spoken sentences revealed more activity in the right superior temporal gyrus extending into the supramarginal gyrus and the parietal operculum. In an implicit task (judgment of sentence mode: statement vs. question) more activity was found in the left supramarginal gyrus and the superior temporal gyrus extending into the parietal operculum.

Previous research suggested that the language parser seems to be sensitive to foot structure such that mild violations maintaining foot structure show less pronounced positivity effects than severe violations involving restructuring of feet (e.g., Domahs et al., 2008). However, these findings reflect effects of foot structure violations rather indirectly as the P300 may not be related to the kind of linguistic analysis per se but might rather reflect the ease with which an incorrect form can be judged as incorrect: Violations with structural change can be rejected easily and violations without structural change cannot. The main objectives of the present study are to explore processing differences between different types of violation more directly using an explicit evaluation paradigm and to extend our understanding of the underlying brain mechanisms in prosodic tasks. In subtle changes of prosodic structures (i.e. mild violations), quantitatively or qualitatively distinct mechanisms may be involved compared to severe violations. Furthermore, we wanted to examine whether brain regions active during word stress processing are similar to those found in rhythmic processing of sentences (Geiser et al., 2008).

The main rationale is as follows: If systematic behavioral variation related to different types of stress violation simply reflects the ease of processing, this should be documented by stronger activation within the same brain regions. However, if this systematic variation is due to qualitatively different kinds of processing, then different brain areas should be engaged in the processing of different types of stress violation.

In metrical theory, it has been proposed that words with distinct primary stress positions vary in terms of their underlying metrical structure. Fig. 1 illustrates how the syllables of words with final, penultimate, or antepenultimate stress can be parsed into feet. To examine effects induced by stress violations per se and by different stress types, we recorded brain responses to correctly and incorrectly stressed probes with final target stress (F, e.g. Vitaˈmin ‘vitamin’), prefinal target stress (penultimate, PU, e.g. Biˈkini ‘bikini’), and antepenultimate target stress (APU, e.g. ˈAnanas ‘pineapple’).

In sum, the current study was setup to systematically investigate the mental processing of different types of word stress violations. In an initial step, we aimed at identifying more general activation differences dependent on whether word stress was correct or violated. In a second step, comparisons between mild and severe word stress violations also allow for more specific inferences to be drawn. Based on our former considerations we hypothesized that the influence of foot structure should be reflected in different activation patterns for mild stress violations maintaining foot structure (APU replaced by F and vice versa) on the one hand and severe violations (APU and F replaced by PU and vice versa) on the other.

Section snippets

Participants

Nineteen right-handed native German-speaking healthy male

Behavioral results

Overall, participants produced 9.5% errors. Specific analyses of error rates focused on hypothesis-driven comparisons between different violation conditions. Fig. 2 shows mean error rates as a function of violation condition: APU-F and F-APU are mild violations with foot structure conservation, while APU-PU, PU-APU, PU-F, and F-PU are severe violations with the need for foot structure reanalysis (see Fig. 1).

For each target stress pattern, paired t-tests were calculated showing that structure

Discussion

The current study was set off to investigate two main questions regarding the mental processing of different types of word stress violation. First, our goal was to identify general activation differences between the perception and evaluation of correct and incorrect word stress. Second, we studied more fine-grained contrasts between mild and severe word stress violations to test whether the kind of violation of prosodic structure has specific effects on processing and explicit judgment. Based

Acknowledgments

The research presented here was funded by the German Science Foundation Grants “DO 1433/1-1” supporting Frank Domahs and Walter Huber and “WI 853/7-2” supporting Ulrike Domahs. We are grateful to Steffen Schulz for his help in data collection.

References (84)

  • J. Kong et al.

    The neural substrate of arithmetic operations and procedure complexity

    Cognitive Brain Research

    (2005)
  • S.A. Kotz et al.

    Cortical speech processing unplugged: A timely subcortico-cortical framework

    Trends in Cognitive Sciences

    (2010)
  • W.J. Levelt

    Models of word production

    Trends in Cognitive Sciences

    (1999)
  • P.A. Lewis et al.

    Brain activation patterns during measurement of sub- and supra-second intervals

    Neuropsychologia

    (2003)
  • M. Meyer et al.

    Neurocognition of auditory sentence comprehension: Event related fMRI reveals sensitivity to syntactic violations and task demands

    Cognitive Brain Research

    (2000)
  • M. Meyer et al.

    Brain activity varies with modulation of dynamic pitch variance in sentence melody

    Brain and Language

    (2004)
  • K. Rothermich et al.

    Rhythm’s gonna get you: Regular meter facilitates semantic sentence processing

    Neuropsychologia

    (2012)
  • A. Schirmer et al.

    Gender differences in the activation of inferior frontal cortex during emotional speech perception

    NeuroImage

    (2004)
  • S. Soto-Faraco et al.

    Segmental and suprasegmental mismatch in lexical access

    Journal of Memory and Language

    (2001)
  • E. Svoboda et al.

    The functional neuroanatomy of autobiographical memory: A meta-analysis

    Neuropsychologia

    (2006)
  • Y. Tong et al.

    Neural circuitry underlying sentence-level linguistic prosody

    NeuroImage

    (2005)
  • T. Twomey et al.

    Top-down modulation of ventral occipito-temporal responses during visual word recognition

    NeuroImage

    (2011)
  • S.D. Vann et al.

    Fos expression in the rostral thalamic nuclei and associated cortical regions in response to different spatial memory tests

    Neuroscience

    (2000)
  • M. Vigneau et al.

    Meta-analyzing left hemisphere language areas: Phonology, semantics, and sentence processing

    NeuroImage

    (2006)
  • S. Wehrum et al.

    Can you hear a difference? Neuronal correlates of melodic deviance processing in children

    Brain Research

    (2011)
  • D. Wildgruber et al.

    Cerebral processing of linguistic and emotional prosody: fMRI studies

    Progress in Brain Research

    (2006)
  • L. Zago et al.

    Neural correlates of simple and complex mental calculation

    NeuroImage

    (2001)
  • Alber, B. (1997). Quantity sensitivity as the result of constraint interaction. In G.E. Booij & J.v.d. Weijer (Eds.),...
  • A. Aleman et al.

    The functional neuroanatomy of metrical stress evaluation of perceived and imagined spoken words

    Cerebral Cortex

    (2005)
  • R.H. Baayen et al.

    The CELEX lexical database

    (1995)
  • K.B.E. Boecker et al.

    An ERP correlate of metrical stress in spoken word recognition

    Psychophysiology

    (1999)
  • K. Bohn et al.

    The influence of rhythmic (ir)regularities on speech processing: Evidence from an ERP study on German phrases

    Neuropsychologia

    (2013)
  • J.R. Booth et al.

    Development of brain mechanisms for processing orthographic and phonologic representations

    Journal of Cognitive Neuroscience

    (2011)
  • J.A. Church et al.

    Manipulation of length and lexicality localizes the functional neuroanatomy of phonological processing in adult readers

    Journal of Cognitive Neuroscience

    (2011)
  • J.A. Church et al.

    A developmental fMRI study of reading and repetition reveals changes in phonological and visual mechanisms over age

    Cerebral Cortex

    (2008)
  • A. Cutler et al.

    The use of prosodic information in word recognition

  • A. Cutler et al.

    The role of strong syllables in segmentation for lexical access

    Journal of Experimental Psychology: Human Perception and Performance

    (1988)
  • A. Cutler et al.

    Voornaam is not (really) a homophone: Lexical prosody and lexical access in Dutch

    Language and Speech

    (2001)
  • M.H. Davis et al.

    A complementary systems account of word learning: Neural and behavioural evidence

    Philosophical Transactions of the Royal Society B: Biological Sciences

    (2009)
  • G. Dogil

    Understanding prosody

  • Domahs, U., Genc, S., Wiese, R., Knaus, J. & Kabak, B. (in press). Processing (un)predictable word stress: ERP evidence...
  • Domahs, U., Knaus, J., Orzechowsky, P. & Wiese, R. (2012). Stress 'deafness' in a language with fixed word stress: an...
  • Cited by (17)

    • Expectations about word stress modulate neural activity in speech-sensitive cortical areas

      2020, Neuropsychologia
      Citation Excerpt :

      This is an especially interesting question for further brain imaging studies, because it is unclear if the stress pattern of words is specified or not in the mental lexicon of listeners of fixed-stress languages. Concerning the neurobiological background of words stress processing, the superior temporal lobe (STG/STS regions) has been previously found to be active (Aleman et al., 2005; Domahs et al., 2013; Heisterueber et al., 2014; Kandylaki et al., 2017; Klein et al., 2011). In these previous studies, several other brain regions have been found active, including the IFG, SMA, areas in the parietal lobe (angular gyrus, superior parietal gyrus, parietal lobule), and frontal lobe (precentral, postcentral, and middle frontal gyrus).

    • White matter pathways for prosodic structure building: A case study

      2018, Brain and Language
      Citation Excerpt :

      The right-lateralization of prosody is less clear-cut in studies with natural language material (e.g., Perkins, Baran, & Gandour, 1996; Tang, Hamilton, & Chang, 2017), prosody production (Kellmeyer et al., 2013; Peschke, Ziegler, Eisenberger, & Baumgaertner, 2012), and tasks that go beyond the processing of low-level acoustic-prosodic cues such as pitch contour (for reviews showing bilateral involvement, see Baum & Pell, 1999; Belyk & Brown, 2014; Paulmann, 2016; Witteman et al., 2011). This indicates the inevitable interaction of prosodic information with concurrent syntactic (den Ouden, Dickey, Anderson, & Christianson, 2016) or lexical-semantic processes (Domahs, Klein, Huber, & Domahs, 2013; Gandour et al., 2004; van Lancker, 1980) that are hard to separate during natural language comprehension. The present study focuses on sentence-level prosodic structure building, i.e., the gradual emergence of a (hierarchical) representation of prosodic constituency that aligns with syntactic structure.

    View all citing articles on Scopus
    View full text