Embodied language in first- and second-language speakers: Neural correlates of processing motor verbs

Highlights

• L2 speakers display motor-related neural activations to simple motor verbs.

• L2 motor-related activations occur with both cognate and non-cognate verbs.

• The same results are obtained for native (L1) speakers.

• L2 semantic representations are rich enough to lead to embodiment effects.

Abstract

The involvement of neural motor and sensory systems in the processing of language has so far mainly been studied in native (L1) speakers. In an fMRI experiment, we investigated whether non-native (L2) semantic representations are rich enough to allow for activation in motor and somatosensory brain areas. German learners of Dutch and a control group of Dutch native speakers made lexical decisions about visually presented Dutch motor and non-motor verbs. Region-of-interest (ROI) and whole-brain analyses indicated that L2 speakers, like L1 speakers, showed significantly increased activation for simple motor compared to non-motor verbs in motor and somatosensory regions. This effect was not restricted to Dutch–German cognate verbs, but was also present for non-cognate verbs. These results indicate that L2 semantic representations are rich enough for motor-related activations to develop in motor and somatosensory areas.

Introduction

During the last decade, the role of sensory and motor neural systems in the construction of linguistic meaning has received a great deal of attention. It has been postulated that the processing of action- and perception-related language leads to activation of the same brain areas as action and perception themselves. In other words, language is grounded in bodily action and perception, or ‘embodied’ (Barsalou, 2008; but see Mahon & Caramazza, 2008). Most studies on language embodiment have focused on native (L1) speakers. In contrast, non-native (L2) speakers have hardly been investigated in this regard (see below). It has been claimed that L2 semantic representations are less developed than L1 semantic representations (Finkbeiner, Forster, Nicol, & Nakamura, 2004 – for details, see below). When we extrapolate this to the question of language embodiment in L2 speakers, this implies that activation in action- and perception-related brain areas for L2 words may be absent or reduced compared to L1 words.

To our knowledge, the present study is the first functional magnetic resonance imaging (fMRI) study to investigate whether evidence of embodied grounding of language can be found with L2 speakers (but see Buchweitz, Shinkareva, Mason, Mitchell, & Just, 2012, for some evidence indirectly related to embodied cognition in bilinguals). Our results will extend insights into embodiment effects to a new domain, i.e. L2 processing, and thus also shed more light on how non-native language is processed. For this, we will study both morphologically simple and complex verbs. In the remainder of this paper, the term ‘embodiment effects’ will be used as a shorthand for the occurrence of activations in action- and/or somatosensory-related brain areas in response to words with a motor-related meaning. With the term embodiment effects, we do not imply any commitment as to the source of these activations, i.e. whether they are a necessary part of semantic representations or a by-product of such representations (i.e. epiphenomenal).

Motor and/or premotor activations are reported in a variety of fMRI studies on action or motor verbs. Hauk, Johnsrude, and Pulvermüller (2004) investigated neural correlates of passively read face-, arm- and leg-related motor verbs (e.g., lick, pick and kick) and the corresponding actions executed by participants. When comparing motor verbs to baseline, they found somatotopically organized activation of motor and premotor cortex, and partial overlap of these with activations for face, arm and leg actions. These motor-related activations were interpreted as reflecting meaning representations of motor verbs. Somatotopic organization of activations in motor regions was also found by Raposo, Moss, Stamatakis, and Tyler (2009) when participants listened to arm- (e.g., grab) and leg-related (e.g., trample) motor verbs. Again, these activations partially overlapped with activations for arm and leg actions, respectively (but see also their results on idiomatic sentences in that same study; see also Kemmerer, Castillo, Talavage, Patterson, & Wiley, 2008, for (partial) support for somatotopically organized activations). No such somatotopic organization was found by Postle, McMahon, Ashton, Meredith, and de Zubicaray (2008), although they did report pre-SMA activation when passively read motor verbs (leg-, arm- and mouth-related motor verbs taken together) were compared with non-motor nouns. This activation was interpreted as reflecting the retrieval of motor programs, with motor verbs serving as instructional cues.

Other regions commonly reported in neuroimaging studies on motor verbs are located in the parietal lobe, for example the anterior inferior parietal cortex (aIPC, associated with abstract somatosensory knowledge of actions – see Binder, Desai, Graves, & Conant, 2009), or the parietal operculum (secondary somatosensory cortex, associated with finger stimulation – see Ruben et al., 2001). Noppeney, Josephs, Kiebel, Friston, and Price (2005) found activations in the aIPC for semantic decisions to motor verbs relative to non-motor words. In a visual lexical decision fMRI study, Rüschemeyer, Brass, and Friederici (2007) reported not only activations in left precentral gyrus and central sulcus as well as bilateral postcentral gyrus in response to German simple hand-related motor verbs (e.g., werfen ‘throw’) versus non-motor verbs (e.g., denken ‘think’), but also in left parietal operculum (S2). In contrast, a comparison of these same motor and non-motor verbs embedded as stems in complex verbs with a non-motor meaning (e.g., werfen in entwerfen ‘design’ – denken in bedenken ‘consider’) revealed no activations in sensorimotor (i.e. (pre)motor or somatosensory) areas.

All these studies focused on the processing of motor verbs by L1 speakers (see Willems & Casasanto, 2011, for an overview). It is unclear, though, whether L2 speakers display the same kind of embodiment effects as L1 speakers, as hardly any study has addressed this issue. A number of fMRI studies have looked into semantic processing in bilinguals (Chee et al., 2001, Illes et al., 1999, Isel et al., 2010, Rüschemeyer et al., 2006, Wartenburger et al., 2003). However, none of these systematically manipulated motor-relatedness, for example by including a contrast between motor and non-motor words or between different types of motor words, thus precluding any conclusions as to the embodied nature of L2 semantic representations.

The only study in which language embodiment effects in L2 speakers were investigated is a behavioral study using a picture–verb matching task (Bergen, Lau, Narayan, Stojanovic, & Wheeler, 2010). For both L1 and advanced L2 participants, judgment times were longer when the word and the picture referred to different actions performed with the same effector (mouth, hand or foot; e.g., run-kick) than when they referred to different actions performed with different effectors (e.g., run-drink). This suggests that words and pictures led to activation of the same sensorimotor circuits, causing interference when the same effector was involved.

Although most models of bilingual word processing do not speak to the presence or absence of embodiment effects in L2 speakers, their descriptions of L1 versus L2 processing allow us to derive hypotheses concerning embodiment effects in L2 speakers. Some models argue for shared semantic representations in L1 and L2, whereas other models claim that L2 semantic representations are less detailed than L1 semantic representations. The first position is taken by the Revised Hierarchical Model (RHM, Kroll & Stewart, 1994) and the extended Bilingual Interactive Activation (BIA+) Model (Dijkstra & Van Heuven, 2002). In both models, access to word meanings is delayed in L2 compared to L1, but there is no difference in the semantic information accessed in L1 and L2, as semantic representations are shared across languages. Therefore, these models should predict no differences between L1 and L2 speakers with respect to embodiment effects: The same sensorimotor regions would be involved in L1 and L2 speakers, and the difference in speed of access to semantic representations would be too slight to be picked up by a method with such a limited temporal resolution as fMRI.

The second position, i.e. less detailed L2 semantic representations, is taken by the Sense Model (Finkbeiner et al., 2004). This model argues for (partially) overlapping distributed semantic representations for L1 and L2 words. L2 words are supposed to have “less rich” semantic representations, i.e. they may be associated with fewer senses than L1 words. Therefore, according to this model, embodiment effects might be reduced or even absent in L2 speakers.

Embodiment effects are mostly used to shed light on the question of language embodiment per se. However, they can also be used as a tool to investigate whether morphologically complex words are decomposed into their constituent parts or processed holistically during comprehension. As far as we know, this approach has so far only been used in Rüschemeyer et al.’s (2007) study on German as L1. In this study, morphologically complex derivations (i.e. words in which a stem is combined with a morpheme that changes the meaning of the stem) were included which were abstract and opaque: Their meaning as a whole was not related to the (motor-related) meaning of their parts. For example, the meaning of entwerfen (‘design’) is not semantically related to the meaning of its motor-related stem werfen (‘throw’). Finding embodiment effects for these complex verbs would indicate that the meaning of the motor-related stems was accessed, i.e. that the opaque complex verbs were decomposed into prefix and (motor-related) stem. However, the results showed a significant interaction between complexity and motor-relatedness, with embodiment effects with simple motor versus non-motor verbs, but not with complex verbs with motor versus non-motor stem. This suggests that opaque complex verbs were processed holistically.

The processing of morphologically complex derivations has been studied extensively in behavioral experiments in L1 speakers. Often, morphological priming/lexical decision experiments are used to compare opaque complex words such as ‘restrain’ with transparent complex verbs such as ‘reheat’ (whose meaning as a whole is related to the meaning of their constituent parts). In such experiments, first, a prime word is presented (e.g., a complex verb), followed by the presentation of a target word (e.g., the stem of the complex verb prime). A lexical decision has to be made to the target word. With supraliminal, i.e. non-masked priming designs (in which the prime word is shown long enough to be perceived consciously), many studies find a dissociation between priming for transparent versus opaque conditions: Transparent conditions show facilitatory priming, while this is not the case for opaque conditions. This has been found for visual priming (visually presented primes and targets; English: Feldman and Soltano, 1999, Feldman et al., 2004, Rastle et al., 2000; Serbian: Feldman, Barac-Cikoja, & Kostić, 2002; but see Smolka, Komlósi, & Rösler, 2009 (German)) and cross-modal priming (auditorily presented primes and visual targets; English: Feldman et al., 2004, Gonnerman et al., 2007, Marslen-Wilson et al., 1994; French: Longtin, Segui, & Hallé, 2003; Dutch: Zwitserlood, Bolwiender, & Drews, 2005; but see Luttmann, Zwitserlood, & Bolte, 2011 (German)).¹ These results suggest that transparent complex verbs are decomposed during word recognition, whereas opaque complex verbs are processed holistically.

To our knowledge, supraliminal (non-masked) priming/lexical decision studies on derivations have not yet been reported with L2 speakers. However, there are a few studies on L2 morphological processing in which other methods (such as masked priming) and/or other materials (such as inflections, e.g., walk-walked, or compounds, e.g., sleepwalk) were used. These studies have produced conflicting results. Some studies focusing on derivations have found evidence for more reliance of L2 speakers on holistic processing (Clahsen and Neubauer, 2010, Silva and Clahsen, 2008), whereas others have found no difference between L1 and L2 speakers (Diependaele et al., 2011, Portin and Laine, 2001). Similarly, some studies on inflections have found evidence for more reliance of L2 speakers on holistic processing (Neubauer and Clahsen, 2009, Silva and Clahsen, 2008), whereas others have found more L2 reliance on decomposition (Lehtonen and Laine, 2003, Portin and Laine, 2001). Lemhöfer, Koester, and Schreuder (2011)’s study on noun compounds also found evidence for more L2 reliance on decomposition. Any direct comparison between these studies is complicated by differences between the tasks (masked priming vs. unprimed lexical decision) and/or materials (inflections vs. derivations vs. compounds), and by different combinations of languages used as L1 and L2, since differences in morphological processing in the L2 may be due to differences in (the morphological richness of) the L1 of the participants (see Basnight-Brown et al., 2007, Portin et al., 2008). Also, the contrast between transparent and opaque complex words was not specifically targeted by the studies discussed (except by Diependaele et al., 2011, in which masked priming was used). Therefore, it is difficult to reach conclusions with regard to the processing of opaque or transparent derived verbs by L2 speakers.

In this fMRI study, we focus on German advanced L2 speakers of Dutch. German and Dutch are highly related languages with a large number of cognates, i.e. words with similar form and meaning in the two languages. Cognates have been found to elicit faster reaction times than non-cognates in word recognition studies, the so-called cognate facilitation effect (e.g., Dijkstra, Miwa, Brummelhuis, Sappelli, & Baayen, 2010). This effect indicates that, in L2 processing, there might be transfer from L1 to L2 through cognates: As semantic representations may be less firmly established in L2, they may be more easily accessible with cognates than non-cognates. To maximize our chances of finding embodiment effects in L2, we included cognate verbs. These were contrasted with non-cognate verbs, allowing us to find out whether potential embodiment effects with motor verbs are primarily mediated by L1–L2 transfer due to cognate status, or whether they are present for cognate and non-cognate verbs alike.

We were interested in two issues. Firstly, we wanted to find out whether, like L1 speakers, L2 speakers show embodiment effects with morphologically simple verbs. For this, we investigated whether Dutch simple motor verbs induce more activation in S2 and/or (pre)motor areas than Dutch simple non-motor verbs in German L2 speakers of Dutch and in Dutch L1 speakers. These simple verbs were either cognates (e.g., nemen (German: nehmen) ‘take’) or non-cognates (e.g., gooien (German: werfen) ‘throw’), to investigate the role of form overlap between L1–L2 translation pairs.

Secondly, we wanted to know whether L2 speakers decompose opaque complex verbs (in contrast with L1 speakers) or process them holistically (like L1 speakers). To investigate this, we compared the processing of Dutch opaque complex verbs with a motor stem with Dutch opaque complex verbs with a non-motor stem in German L2 speakers of Dutch and in Dutch L1 speakers. If opaque complex verbs with a motor stem (whose meaning as a whole is not motor-related) were found to induce more activation in S2 and/or (pre)motor areas than opaque complex verbs with a non-motor stem in L2 speakers, this would suggest that the meaning of the motor stem was accessed, i.e. that opaque complex verbs are decomposed. If no increased activation of S2 and/or (pre)motor areas was found, this would suggest that opaque complex verbs are processed holistically.

To investigate these questions, a 2 (Complexity: Simple vs. Complex) × 2 (Cognate Status: Cognate vs. Non-Cognate) × 2 (Motor-Relatedness: Motor vs. Non-Motor) design would have been ideal. However, the close relation between Dutch and German made it impossible to find enough non-cognate opaque complex verbs, leading to two empty cells in the design (see Table 1). Therefore, we opted for a combination of two 2×2 subdesigns. In the first one, the Simple Verb subdesign, only morphologically simple verbs were examined, with Cognate Status (Cognate vs. Non-Cognate) and Motor-Relatedness (Motor vs. Non-Motor) as factors. In the second one, the Cognate Verb subdesign, only cognate verbs were included, with Complexity (Simple vs. Complex) and Motor-Relatedness (Motor vs. Non-Motor) as factors. Adding the factor of Language (L1 vs. L2 speakers) enabled us to compare the two language groups, turning the subdesigns into 2×2×2 designs.

The Simple Verb subdesign allowed us to investigate whether motor-related activations (i.e. higher activations for motor than for non-motor words) can be found with simple verbs not only in L1, but also in L2. We hypothesized that, since the L2 speakers were advanced speakers of Dutch, such activations would indeed become apparent. However, if L2 speakers’ semantic representations are less rich than those of L1 speakers (Finkbeiner et al., 2004), motor-related activations in L2 speakers might be reduced compared to L1 speakers.

With the Cognate Verb subdesign, we first aimed at a replication of the interaction between Complexity and Motor-Relatedness reported by Rüschemeyer et al. (2007) for L1 speakers. For L2 speakers, this subdesign allowed us to investigate whether L2 speakers show a similar pattern as L1 speakers. In line with Rüschemeyer et al., we expected L1 speakers to show motor-related activation for simple verbs, but not for opaque complex verbs. The absence of motor-related activation for opaque complex verbs would be an indication that these verbs are not decomposed. As mentioned above, evidence on L2 morphological processing is mixed. If L2 speakers decompose opaque complex verbs, we should find motor-related activation for both simple and complex verbs. If they process opaque complex verbs holistically, we should find the same results as for L1 speakers, i.e. motor-related activation for simple motor verbs and no motor-related activation for opaque complex verbs containing motor stems.