Context-dependent interpretation of words: Evidence for interactive neural processes
Introduction
Neuroimaging methods have been extensively used to study how the brain represents and processes the meanings of words (e.g., Thompson-Schill, 2003, Martin and Chao, 2001, Damasio et al., 2004). Most research has addressed the brain areas and circuits that are activated in processing different types of isolated words. For example, words for animals and artefacts (e.g., cow, knife) produce systematically different patterns of activation (Chao et al., 1999, Moore and Price, 1999, Perani et al., 1999b). This research indicates that word meanings are represented in a distributed network involving sensory, motor, individual-specific, and other types of information (Martin and Chao, 2001).
An interesting property of words, however, is that their meanings are highly context-dependent. In fact, most English words are ambiguous: they have multiple meanings that vary in how much they overlap. Many words have multiple semantically unrelated meanings (e.g., watch: a time piece, to look; rose: a flower, past tense of rise); others have multiple semantically related senses (e.g., twist an ankle vs. twist the truth); and some have both (e.g., one of the meanings of rose is the name of both a flower and a related color). Even the meaning of a seemingly unambiguous word such as piano depends on the context in which it occurs: moving a piano brings to mind different concepts than playing a piano; the fact that cats have fur is relevant to understanding pet the cat whereas having claws is relevant to scratched by the cat (Tabossi, 1988). Thus, meanings are not fixed entries or lists of attributes; they are dynamically computed each time a word is encountered. Determining the meanings of words requires combinatorial processing: using different sources of information (prior knowledge, context) to converge on an interpretation. This fundamental aspect of language processing poses a complex problem insofar as each word’s meaning depends in part on the meanings of others words whose meanings are themselves also context-dependent in varying degrees.
Our goal in this study was to examine the brain mechanisms and circuits underlying such context-dependent combinatorial processes. We examined a common type of lexical ambiguity that allowed us to assess the effects of different contexts on comprehending the same word. Most content words in English such as hammer or bowl can be used as either nouns or verbs and thus require contextual information to be correctly interpreted as object (he wants the hammer) or action (he wants to hammer). The alternative meanings are from different grammatical and conceptual categories (noun-object, verb-action) and thus clearly disambiguated by minimal contexts such as the and to. Behavioral studies have shown that elements of both common meanings of such words are transiently activated, even in strongly disambiguating contexts such as I bought a bowl. For example ambiguous words automatically prime target words that are semantically related to either meaning early in processing (Swinney, 1979, Tanenhaus et al., 1979, Federmeier et al., 2000). Selection of the contextually appropriate meaning via top–down contextual influences then occurs within about 200 ms (see Simpson, 1994 for review). fMRI does not have sufficient temporal resolution to examine rapid changes over this short time window. However, it does provide a way to examine how a word’s context modulates brain activity.
Subjects in an event-related design were presented with phrases referring to tools and manipulable objects and actions performed with such objects. Highly ambiguous words such as hammer or bowl were presented in a noun context (the hammer, the bowl) or in a verb context (to hammer, to bowl). These conditions were compared to similar phrases containing words that are minimally ambiguous because they have a single dominant interpretation (e.g., the dagger, to sharpen; see Table 1). Whereas combination with the context was required to interpret the high-ambiguity words as an object or an action, it was not required for low-ambiguity words. Hence the comparison between high- and low-ambiguity conditions provides a way to isolate ambiguity-specific activity, including multiple semantic associations and the combinatorial processes by which context disambiguates interpretation. Moreover, the comparison between the noun and verb contexts of the same words provides a way to isolate the brain activity that is modulated by linguistic context, a factor previously unexplored in fMRI studies of ambiguity. Unlike previous sentential studies of ambiguity (e.g., Rodd et al., 2005), the use of minimal two-word combinations from a restricted semantic domain reduces both variability in meaning driven activity and the influence of other factors such as working memory demands and other types of linguistic analyses (e.g., thematic role assignment).
Current word processing models suggest that reading or producing a word consistently implicates a distributed network of brain regions associated with distinct functions. Beyond brain regions decoding orthographic or auditory word forms (see Hickok and Poeppel, 2004, Price, 2000), lexical semantic sensory–motor attributes are stored and activated in distributed networks of cortical regions organized around sensory–motor systems (Martin and Chao, 2001, Damasio et al., 2004, Pulvermüller, 1999, Pulvermüller, 2001). These semantic attributes are claimed to be subsequently integrated for further manipulation in frontal areas such as left inferior frontal gyrus (LIFG), an area associated with a variety of semantic and integrative functions (Thompson-Schill, 2003, Gabrieli et al., 1998, Martin and Chao, 2001, Hagoort, 2005). Because the roles of LIFG and the semantic sensory–motor areas associated with our stimuli have been extensively investigated, we build upon previous research to further explore the relationship between these regions.
Words referring to tools and manipulable objects such as hammer automatically activate sensory–motor aspects of tool use in regions also implicated in perceiving, imagining, executing and planning actions with tools (Martin and Chao, 2001, Chao and Martin, 2000, Kellenbach et al., 2003, Beauchamp et al., 2002, Johnson-Frey et al., 2005). Such regions include two areas encoding motor schemas for interacting with objects – the intraparietal sulcus (IPS) and ventral premotor cortex (PMv) (cf. Binkofski et al., 1998, Grafton et al., 1997, Gerlach et al., 2002, Noppeney et al., 2005) – and the posterior middle temporal gyrus (PMTG), an area anterior to motion perception area V5/MT, which is sensitive to motion aspects of tool use and actions (Chao et al., 1999). Although PMTG has also been argued to perform more general semantic processes in sentence comprehension (e.g., Kuperberg et al., 2003, Baumgaertner et al., 2002), systematic investigations comparing a variety of visual and word stimuli have consistently showed its engagement in processing stimuli implying motions such as actions and tools (Kable et al., 2002, Kable et al., 2005, Tyler et al., 2003, Beauchamp et al., 2002, Beauchamp et al., 2003). PMTG has also been shown to play a causal role in representations of actions as demonstrated by lesion studies (Tranel et al., 2003) and to be more sensitive to verbs than nouns as verbs tend to imply motion more than ordinary nouns (Perani et al., 1999a, Damasio et al., 2001, Kable et al., 2002; see also Fiez et al., 1996, Tranel et al., 2005).
LIFG in turn has been associated with at least two integrative functions. Sentence processing studies have proposed that LIFG, particularly its posterior portion, processes syntactic structures and serial-order based representations (Caplan et al., 1998, Dapretto and Bookheimer, 1999, Friederici et al., 2003, Keller et al., 2001). Semantic processing studies in contrast have proposed that the anterior portion of LIFG is responsible for the controlled retrieval and selection of appropriate semantic information among competing alternatives on the bases of contextual information (Thompson-Schill et al., 1997, Wagner et al., 2001). The strength of the response in LIFG is sensitive to the number of competing alternatives and the amount of semantic information it receives (see Badre and Wagner, 2002, Thompson-Schill et al., 2005 for reviews). Demands on LIFG thus vary as a function of whether task-relevant semantic knowledge can be accessed through bottom–up retrieval. When automatic access is insufficient due to the presence of prepotent competing representations, LIFG’s selection or regulatory processes play a central role in guiding the processing of meaning stored in posterior cortex (Badre and Wagner, 2002).
This account of the role of LIFG in semantic processes is consistent with previous studies using stimuli similar to those used in the present research. Highly ambiguous words and sentence structures have been shown to elicit a stronger response in LIFG than less ambiguous words and structures, due to competition between alternatives, which requires the inhibition of inappropriate interpretations (Mason et al., 2003, Rodd et al., 2005, Chan et al., 2004). Moreover, the process of settling on an action meaning (e.g., sharpen) also elicits a stronger neural response in LIFG than settling on a noun meaning (e.g., knife; Perani et al., 1999a, Damasio and Tranel, 1993, Shapiro et al., 2005). Verbs involve more complex selection or retrieval processes than nouns because verbs carry additional morphological, syntactic and semantic information (concerning, e.g., the kinds of nouns that occur with them; Tyler et al., 2004, Thompson-Schill et al., 2005).
Given these facts, we examined two alternative hypotheses about the relation between semantic sensory–motor areas and processes in LIFG. The activation–selection hypothesis posits that ambiguous words such as hammer will automatically elicit semantic attributes associated with both common meanings regardless of context, consistent with earlier behavioral studies (Simpson, 1994). Regulatory or selection mechanisms in LIFG would then strengthen contextually appropriate information and inhibit inappropriate information, thus determining the appropriate object or action interpretation. This view predicts that PMTG, IPS and PMv should show an ambiguity effect (more semantic activation for high-ambiguity words than low-ambiguity words) but not a context effect (greater semantic activation for words in verb contexts compared to the same words in noun contexts). Although PMTG may be more responsive to low-ambiguity verbs compared to low-ambiguity nouns due to its previous sensitivity to actions compared to objects (Perani et al., 1999a, Damasio et al., 2001, Kable et al., 2002, Kable et al., 2005), PMTG should not show a context effect for high-ambiguity items. In contrast, LIFG should exhibit both an ambiguity effect (because it receives greater input from high-ambiguity words) and a context effect (because of its role in selection and controlled processes and its greater sensitivity to verbs than nouns). Thus, this hypothesis suggests a feedforward, activate-decide model in which areas representing sensory–motor semantics “propose” and LIFG “disposes.”
The interactive hypothesis in contrast suggests that mechanisms in LIFG take place within a process involving possible collaboration or feedback interactions between areas. Specifically, activation in semantic sensory–motor areas may also be modulated by contextual information (in addition to ambiguity) and thus differ for to hammer vs. the hammer, as expected for LIFG. Of the semantic areas discussed so far, PMTG is more likely to show a context effect given its previous sensitivity to actions compared to objects (Perani et al., 1999a, Damasio et al., 2001, Kable et al., 2002). The interactive hypothesis is thus consistent with low-ambiguity verb phrases such as to sharpen producing greater activity in PMTG than low-ambiguity noun phrases such as the dagger solely in virtue of their respective lexical meanings (action vs. object). The critical prediction, however, is that the same pattern should be obtained when the action interpretation can only be determined via combinatorial processing, as in the high-ambiguity condition. Thus, to hammer (and other stimuli of this type) should produce greater activation than the hammer in PMTG. This outcome would suggest that contextual information strengthens the activation of action-related motion attributes in PMTG and that the contextually appropriate interpretation is determined via feedback or interaction between PMTG and LIFG, rather than via a strictly feedforward activation–selection process.
Section snippets
Materials
Forty high-ambiguity words were each matched for frequency and character length with two low-ambiguity words that have dominant uses as noun and verb respectively (see Table 1). The high-ambiguity words were equibiased, i.e., they had similarly frequent object and action interpretations (noun and verb uses) in English (according to the 20 million words Cobuild corpus, cf. Sinclair, 1995). Log10 frequencies reported were computed from the total frequency in the corpus. The mean log frequency for
Behavioral responses
92% of yes–no responses to the questions in each trial were answered correctly, confirming that participants paid attention to the task and read the phrases for meaning. There was no significant main effect of ambiguity or context in the proportion of correct responses in a repeated measures ANOVA (ambiguity: F(1,16) = 0.71, p = 0.41; context: F(1,16) = 0.07, p = 0.79) although there was a small interaction (F(1,16) = 6.9, p < 0.02). This was due to the fact that correct responses were higher for
Discussion
The results indicate that LIFG and areas associated with semantic attributes including IPS, PMv and PMTG were all sensitive to the ambiguity manipulation: high-ambiguity words such as bowl and hammer in any syntactic context elicited greater brain activity than low-ambiguity words such as dagger. This difference can be explained by the fact that ambiguous words automatically activate elements of more than one meaning (Simpson, 1994). Stimulus phrases containing ambiguous words activated motor
Acknowledgments
This study was supported by NIH grants RO1-MH064498, RO1-HD29891 and P50-MH64445 and was conducted at the Waisman Laboratory for Brain Imaging and Behavior (University of Wisconsin-Madison). The authors are particularly grateful to Olufunsho Faseyitan and Cristopher Jordan, whose technical expertise and friendly attitude greatly facilitated this research.
References (78)
- et al.
Event-related fMRI reveals cortical sites involved in contextual sentence integration
NeuroImage
(2002) - et al.
Functional interactions of the inferior frontal cortex during the processing of words and word-like stimuli
Neuron
(2001) - et al.
Neural systems for word meaning modulated by semantic ambiguity
NeuroImage
(2004) - et al.
Representation of manipulable man-made objects in the dorsal stream
NeuroImage
(2000) - et al.
Dynamic statistical parametric mapping: combining fMRI and MEG for high-resolution imaging of cortical activity
Neuron
(2000) - et al.
Neural correlates of naming actions and of naming spatial relations
NeuroImage
(2001) - et al.
Neural systems behind word and concept retrieval
Cognition
(2004) - et al.
Form and content: dissociating syntax and semantics in sentence comprehension
Neuron
(1999) - et al.
Modulation of inferotemporal cortex activation during verbal working memory maintenance
Neuron
(2006) - et al.
The brain basis of syntactic processes: functional imaging and lesion studies
NeuroImage
(2003)
Premotor cortex activation during observation and naming of familiar tools
NeuroImage
On Broca, brain, and binding: a new framework
Trends Cogn. Sci.
Somatotopic representation of action words in human motor and premotor cortex
Neuron
Dorsal and ventral streams: a framework for understanding aspects of the functional anatomy of language
Cognition
Anatomical variability of the lateral frontal lobe surface: implication for intersubject variability in language Neuroimage
NeuroImage
Semantic memory and the brain: structure and processes
Curr. Opin. Neurobiol.
The neural areas that control the retrieval and selection of semantics
Neuropsychologia
Action selectivity in parietal and temporal cortex
Cogn. Brain Res.
Visual word recognition: the first half second
NeuroImage
Lateralization of ventral and dorsal auditory-language pathways in the human brain
NeuroImage
Word and picture matching: a PET study of semantic category effects
Neuropsychologia
Using event-related fMRI to assess delay-period activity during performance of spatial and nonspatial working memory tasks
Brain Res. Brain Res. Protoc.
Brain reflections of words and their meanings
Trends Cogn. Sci.
Making sense of semantic ambiguity: semantic competition in lexical access
J. Mem. Lang.
Dissociating neural correlates for nouns and verbs
NeuroImage
Lexical access during sentence comprehension: (Re) consideration of context effects
J. Verbal Learn. Verbal Behav.
Evidence for multiple stages in the processing of ambiguous words in syntactic contexts
J. Verbal Learn. Verbal Behav.
Neuroimaging studies of semantic memory: inferring “how” from “where”
Neuropsychologia
The frontal lobes and the regulation of mental activity
Curr. Opin. Neurobiol.
Effects of noun–verb homonymy on the neural correlates of naming concrete entities and actions
Brain Lang.
Objects and their actions: evidence for a neurally distributed semantic system
NeuroImage
Neural processing of nouns and verbs: the role of inflexional morphology
Neuropsychologia
Recovering meaning: left prefrontal cortex guides controlled semantic retrieval
Neuron
Modeling for intergroup comparisons of imaging data
NeuroImage
A trial-based experimental design for fMRI
NeuroImage
Semantic retrieval, mnemonic control, and prefrontal cortex
Behav. Cogn. Neurosci. Rev.
Parallel visual motion processing stream for manipulable objects and human movements
Neuron
FMRI responses to video and point-light displays of moving humans and manipulable objects
J. Cogn. Neurosci.
Human anterior intraparietal area subserves prehension: a combined lesion and functional MRI activation study
Neurology
Cited by (85)
Representation of motion concepts in occipitotemporal cortex: fMRI activation, decoding and connectivity analyses
2022, NeuroImageCitation Excerpt :Others hold a weaker embodiment view, arguing that regions directly involved in perception do not necessarily engage in semantic processing, but areas close to them are recruited (Barsalou, 2003; Bedny et al., 2008; Kable et al., 2002; Martin and Chao, 2001). For understanding motion concepts, studies using univariate contrasts have found mixed evidence, with some reporting V5 involvement (Assmus et al., 2007; Glenberg and Kaschak, 2002; Revill et al., 2008; Rueschemeyer et al., 2010; Saygin et al., 2010) while others do not (Bedny et al., 2008; Gennari et al., 2007; Noppeney et al., 2005). In our study, we did not find any effect in LOTC3 in the univariate analyses but the more sensitive MVPA analyses did reveal motion effects, suggesting that motion perception regions are functionally involved in semantic processing.
Brain activation patterns associated with paragraph learning in persons with multiple sclerosis: The MEMREHAB trial
2020, International Journal of PsychophysiologyMetaphor processing is supramodal semantic processing: The role of the bilateral lateral temporal regions in multimodal communication
2020, Brain and LanguageCitation Excerpt :Moreover, a greater bilateral region centered in the STS is recruited in multimodal communication of iconic gestures and speech when the noise affecting the affiliate speech comprehensibility is reduced (Holle et al., 2010), suggesting that this area goes beyond simple multimodal input integration but is also implicated in the semantics of this multimodal information especially during under increased complexity requiring greater semantic memory access. Indeed, the lateral temporal areas have been repeatedly implicated in unimodal linguistic comprehension and inferencing (Gennari, MacDonald, Postle, & Seidenberg, 2007; Kim, Yoon, Kim, Lee, & Kang, 2012; Turken & Dronkers, 2011) nonlinguistic semantic tasks such as those requiring more complex categorizations or identification (van der Linden, van Turennout, & Indefrey, 2009; Wei et al., 2012), and conventionalization of metaphors (Cardillo et al., 2012). Adding to the growing list, they have also been implicated in social cognitive processing, such as, joint attention tasks compared to attending to social situations (Redcay et al., 2010), inferring intentions of a possible action than processing the action itself (Iacoboni et al., 2005), and determining intentions and beliefs (Lotze et al., 2006; Saxe, Xiao, Kovacsc, Perrett, & Kanwisher, 2004; Schultz, Imamizu, Kawato, & Frith, 2004; Zaitchik et al., 2010).