When vocal processing gets emotional: On the role of social orientation in relevance detection by the human amygdala
Introduction
Sensitivity to the emotional state of others is a prerequisite for successful navigation within social groups (Darwin, 1872, Baron-Cohen, 1992). It allows individuals to predict the behavior of interaction partners and thus to flexibly employ strategies that increase the probability of obtaining the desired interaction outcome (e.g., marriage, business contract). Among the different nonverbal channels that humans use for emotional communication, the vocal channel is especially important because it can carry over large distances and is independent of sight. Moreover, it can be used to communicate emotions in a non-linguistic context in form of emotionally charged exclamations, such as laughs or sighs, or as prosody in the context of speech. The brain structures that support the processing of emotional exclamations or prosody have been investigated in a number of patient and neuroimaging studies (for a review see Schirmer and Kotz, 2006). Not surprisingly, this research implicates areas involved in various aspects of cognition ranging from perception to semantic memory and language. Specifically, vocal processing seems to be mediated by a pathway that extends from primary and secondary auditory cortex to regions along the bilateral superior temporal sulcus (STS). The STS has been proposed to include voice selective areas (Belin et al., 2004) that are specifically engaged in the processing of vocal as compared to non-vocal, environmental sounds (e.g., tool sounds). Interestingly, activity in these regions appears to be enhanced if vocal sounds are emotional as demonstrated by studies that compared emotional and neutral tone of voice (e.g., Grandjean et al., 2005, Kotz et al., 2003) or that correlated emotional intensity with fMRI signal changes (e.g., Ethofer et al., 2006, Wiethoff et al., 2008). This suggests that the STS is involved in the analysis of vocal emotional information. Moreover, in line with primate work demonstrating projections from this region to the frontal lobes (Petrides and Pandya, 1988), it has been proposed that the results of vocal processing at the level of the STS serve as input to higher order cognitive processes mediated by the frontal lobes (Schirmer and Kotz, 2006). Accordingly, there is evidence that vocal emotional judgments activate the right inferior frontal gyrus (IFG) relative to acoustic or linguistic judgments (Buchanan et al., 2000, Gandour et al., 2003, Wildgruber et al., 2002, Wildgruber et al., 2004, Wildgruber et al., 2005; but see Mitchell et al., 2003). As right IFG involvement seems to depend on listeners actively evaluating the perceived stimuli on an emotional dimension, one may conclude that the right IFG plays a special role in the conscious evaluation of vocal emotional exclamations or prosody. Increased activity in the left IFG has been observed in response to emotional words spoken with incongruous as compared to congruous emotional prosody, suggesting that this area is implicated in the integration of verbal and vocal cues (Schirmer et al., 2004, Mitchell, 2006). In addition to these more cognitive processes, emotional vocalizations have also been shown to activate structures associated with the elicitation of emotional or feeling states. For example, emotional vocalizations were found to elicit activity in the bilateral orbitofrontal cortex (OFC; Wildgruber et al., 2002, Wildgruber et al., 2004, Sander et al., 2005), a structure which presumably represents the reinforcement value of a given stimulus (i.e., its rewarding or punishing nature) and mediates emotional responses via connections to the amygdala (Kringelbach and Rolls, 2004). In line with this, some investigations on the processing of emotional exclamations (Fecteau et al., 2007; Phillips et al., 1998, Sander and Scheich, 2001; but see Meyer et al., 2005) and emotional prosody (Sander et al., 2005; but see Ethofer et al., 2006, Kotz et al., 2003, Mitchell et al., 2003, Morris et al., 1999, Wiethoff et al., 2008) also reported amygdala activation. However, these reports frequently relied on a more liberal statistical threshold and are, particularly in the case of prosody, relatively sparse.
The minimal support for the involvement of OFC and amygdala in vocal emotional processing stands in contrast to the more consistent evidence that implicates these areas in the processing of other types of emotional stimuli. For example, emotional pictures (e.g., Britton et al., 2006) odors (e.g., Gottfried et al., 2003) or facial expressions (e.g., Vuileumier et al., 2001, Vuilleumier et al., 2004; for a review see Adolphs, 2002) reliably activate the amygdala and the OFC. One possible explanation for the discrepancy between this work and the research on vocal processing may be that the vocal material used in the latter studies is simply less emotionally provoking and hence less likely to recruit emotion centers in the brain. In line with this argument, emotional exclamations (e.g., laughing, crying) – which are presumed to be more emotionally charged than the prosodic expressions that accompany speech – are more likely to activate the amygdala (Fecteau et al., 2007; Phillips et al., 1998, Sander and Scheich, 2001). Additionally, interindividual differences in the emotional response to prosodic signals may contribute to inconsistencies in the reported activations. If not taken into account, such interindividual differences may obscure amygdala and OFC contribution to stimulus processing especially when the emotionality of a given stimulus is relatively low.
Support for a role of interindividual differences in vocal and more specifically in prosodic emotional processing comes from a number of studies. For example, as for other channels of communication (e.g., Hamspon et al., 2006), women have been shown to recognize emotional-prosodic cues faster and more accurately then men (Hall, 1978; Dromey et al., 2005, Schirmer and Kotz, 2003; but see Fecteau et al., 2005). Furthermore, women are more likely to differentiate between emotional and neutral prosody when these prosodic stimuli are unattended (Schirmer et al., 2005). Women are also more likely to integrate unattended prosodic cues into language processing as demonstrated by facilitated processing of words that correspond in valence to the speaker's emotional prosody (e.g., “success” spoken with a happy tone; Schirmer and Kotz, 2003, Schirmer et al., 2004; but see Kotz and Paulmann, 2007). These findings have been interpreted in light of women's role in the caring for offspring and their presumably greater affiliative tendencies (Babchuk et al., 1985, Hamspon et al., 2006, Taylor et al., 2000). Accordingly, it has been proposed that the emotional quality of vocal expressions is more relevant to women than to men and hence women tend to be more proficient in using vocal cues.
Additionally, interindividual differences in emotion recognition have been revealed by cross-cultural comparisons (e.g., Yuki, Maddux, Masuda, 2007). With respect to the present study, work by Kitayama and colleagues (Kitayama and Ishii, 2002, Ishii et al., 2003) on emotional-prosodic processing is of importance. The authors presented positive and negative words spoken with either a congruous or an incongruous emotional prosody to Asian and Western listeners. In different tasks, participants were asked to judge verbal content while ignoring prosody or to judge prosody while ignoring verbal content. Asian listeners from Japan and the Philippines had greater difficulty ignoring prosody, whereas the reversed pattern was observed for North American listeners. The authors concluded from these results that Asian and Western listeners differ in the sensitivity to vocal emotional signals. Moreover, the authors speculated that because Asian listeners come from a collectivistic cultural background, which places greater emphasis on an individual's ability to accurately interpret subtle emotional signals, these signals are of greater relevance. As a consequence, listeners with a collectivistic cultural background may process vocal emotional information in a more automatized fashion than listeners with an individualistic background.
These processing differences and the differences that have been identified between male and female listeners are interesting in the context of equivocal neuroimaging findings concerning the role of amygdala and OFC during vocal emotional processing. Based on the above evidence it seems that in order to study the role of these structures it might be useful to consider the individual and to identify variables that drive his or her emotional response to vocal signals. Given that current proposals to explain gender and cultural differences in vocal emotional processing evoke the relevance of vocal expressions for the individual, we speculated that a personality trait that captures this relevance would be a variable of interest. From among the dimensions that psychologists use to characterize an individual's personality (e.g., extraversion, neuroticism), we identified “social orientation” as being most clearly linked to the relevance of social signals such as vocal expressions. Social orientation reflects an individual's interest in social exchange and the extent to which he or she cares about other individuals (Andsager et al., 2006, Soares et al., 2005). Psychologists assess social orientation by means of questionnaires which may include true/false questions such as “I feel responsible for individuals beyond my immediate family” or “If I see a person cry I feel the need to comfort him or her”. Generally, women score higher than men on measures of social orientation (Fahrenberg et al., 1994). Furthermore, as collectivism emphasizes the importance of the group over the individual (Hofstede, 1980), individuals from a collectivistic culture typically score higher on measures of social orientation than do individuals from an individualistic culture (Hofstede, 1980, Conway et al., 2005). Taken together this evidence supports the idea that interindividual differences in social orientation may underlie previously observed gender and cultural differences in vocal emotional processing. Thus, one may hypothesize that social orientation modulates the relevance of vocal signals and thereby influences the processing of and emotional response to these signals.
In the present fMRI study, we sought to determine the relationship between social orientation and vocal emotional processing. Specifically, we used an auditory oddball paradigm which has been tested in a prior event-related potential (ERP) study and shown to be sensitive to interindividual differences (Schirmer et al., 2005). In this paradigm, participants watch a silent movie with subtitles while an auditory oddball sequence is presented over loudspeakers. This sequence consists of spoken syllables that rarely and unpredictably convey a different emotional tone. In one block, frequent syllables are spoken with a neutral prosody and rare ones with an angry or happy prosody, whereas in another block, frequent syllables are spoken with an angry or happy prosody and rare ones are neutral. In accordance with other auditory oddball studies (for a review see Näätänen et al., 2005), a prior ERP study using this paradigm found a mismatch negativity (MMN) when subtracting frequent from rare stimuli. Moreover, the MMN was larger for rare stimuli spoken with an emotional as compared to neutral prosody. These results suggest that auditory change detection, as reflected by the MMN (Näätänen et al., 2005), is enhanced for emotional as compared to neutral events (De Baene et al., 2004, Schirmer et al., 2005).
Using the paradigm described above in the present fMRI study, we predicted that rare changes in tone of voice would activate the primary and secondary auditory cortex. This prediction was derived from prior work that investigated auditory oddball effects using fMRI or optical imaging (Doeller et al., 2003, Tse and Penney, 2007, Tse et al., 2006, Tervaniemi et al., 2006). Oddball effects in this area have been linked to the updating of a sensory memory trace with new, incoming information. Such a process would be expected regardless of whether the incoming information is neutral or emotional. Additionally, we hypothesized that social orientation would enhance the processing of rare emotional as compared to rare neutral prosody. If this were true for perceptual or cognitive processing aspects, we expected to see a correlation between social orientation and fMRI signal changes in regions along the vocal processing pathway in the STS or in the inferior frontal cortex, respectively. Moreover, in line with prior studies, we expected such effects to be lateralized to the right hemisphere (e.g., Fecteau et al., 2007; Wiethoff in press; for a review see Schirmer and Kotz, 2006). If social orientation modulates a listener's emotional response to vocal stimuli, correlations between social orientation and fMRI signal changes were expected to show in the OFC and the amygdala. Given the amygdala's presumed role in appraising the relevance of a given stimulus (Sander et al., 2003) finding a correlation between social orientation and amygdala activity would support the proposition that social orientation modulates an individual's emotional response to social signals by enhancing the relevance of these signals.
Section snippets
Participants
Nineteen right handed native German speakers participated in the present experiment. Participants were undergraduate or postgraduate students with no known psychiatric conditions. To prevent potential confounds due to gender differences in social orientation, only one gender was included in the study. Moreover, as the norms provided with the social orientation scale employed in the present study indicated greater variability in the social orientation scores of males as compared to females (
Rare versus Frequent Vocal Expressions
To investigate the brain correlates associated with the detection of change in speaker tone of voice, we contrasted rare with frequent events by collapsing over emotional and neutral prosody (see Table 1 and Fig. 1). This contrast showed that rare vocalizations elicited larger activity in the primary and secondary auditory cortex as compared to frequent vocalizations (x − 62, y − 28, z 15; x 61, y − 25, z 9). Separate contrasts for emotional and neutral prosody indicated that these areas were
Discussion
The present study investigated the role of social orientation for the processing of emotional prosody by means of an auditory oddball paradigm. To assess whether our results correspond to previous auditory oddball studies using fMRI, we contrasted rare and frequent events across emotional conditions. In accord with previous work (e.g., Doeller et al., 2003, Rinne et al., 2005), rare acoustic change was associated with activity in the primary and secondary auditory cortex. These areas also
Acknowledgments
The data were collected at the Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany. The authors would like to thank Trevor B. Penney, D. Yves von Cramon, and Barbara Ryan for valuable comments on earlier versions of the manuscript.
References (88)
- et al.
Empirical analyses of BOLD fMRI statistics. II. Spatially smoothed data collected under null-hypothesis and experimental conditions
NeuroImage
(1997) - et al.
Lateralization of amygdala activation: a systematic review of functional neuroimaging studies
Cogn. Brain Res.
(2004) - et al.
Sex differences in the recognition of infant facial expressions of emotion – the primary caretaker hypothesis
Ethol. Sociobiol.
(1985) - et al.
Thinking the voice: neural correlates of voice perception
Trends Cogn. Sci.
(2004) - et al.
Neural correlates of social and non-social emotions
Neuroimage
(2006) - et al.
Recognition of emotional prosody and verbal components of spoken language: an fMRI study
Cogn. Brain Res.
(2000) - et al.
Roughness perception in sounds: behavioral and ERP evidence
Biol. Psychol.
(2004) - et al.
Prefrontal cortex involvement in preattentive auditory deviance detection: neuroimaging and electrophysiological evidence
NeuroImage
(2003) - et al.
Recognition of affective prosody by speakers of English as a first or foreign language
Speech Commun.
(2005) - et al.
An electrophysiological and behavioral investigation of involuntary attention towards auditory frequency, duration and intensity changes
Cogn. Brain Res.
(2002)
Deconvolution of impulse response in event-related BOLD fMRI
NeuroImage
Generalisability, random effects and population inference
NeuroImage
When emotional prosody and semantics dance cheek to cheek: ERP evidence
Brain Res.
On the lateralization of emotional prosody: An event-related functional imaging study
Brain Lang.
The functional neuroanatomy of the human orbitofrontal cortex: evidence from neuroimaging and neuropsychology
Prog. Neurobiol.
Mismatch negativity and late discriminative negativity in sleeping human newborns
Neurosci. Lett.
Distinct fMRI responses to laughter, speech, and sounds along the human peri-sylvian cortex
Cogn. Brain Res.
The neural response to emotional prosody, as revealed by functional magnetic resonance imaging
Neuropsychologia
Saying it with feeling: neural responses to emotional vocalizations
Neuropsychologia
Superior temporal and inferior frontal cortices are activated by infrequent sound duration decrements: an fMRI study
NeuroImage
Auditory perception of laughing and crying activates the human amygdala regardless of attentional state
Cogn. Brain Res.
Emotion and attention interactions in social cognition: brain regions involved in processing anger prosody
NeuroImage
Beyond the right hemisphere: brain mechanisms mediating vocal emotional processing
Trends Cogn. Sci.
Gender differences in the activation of inferior frontal cortex during emotional speech perception
NeuroImage
Dissociation of neural representation of intensity and affective evaluation in human gustation
Neuron
Erotic and disgust-inducing pictures – Differences in the hemodynamic responses of the brain
Biol. Psychol.
Image matching as a diffusion process: an analogy with Maxwell's demons
Med. Imag. Anal.
Event-related optical imaging reveals the temporal dynamics of right temporal and frontal cortex activation in pre-attentive change detection
NeuroImage
Valence, gender and lateralization of function brain anatomy of emotion: A meta-analysis of findings from neuroimaging
Neuroimage
Both of us disgusted in my insula: the common neural basis of seeing and feeling disgust
Neuron
Cerebral processing of emotional prosody — influence of acoustic parameters and arousal
Neuroimage
Identification of emotional intonation evaluated by fMRI
NeuroImage
Dynamic brain activation during processing of emotional intonation: influence of acoustic parameters, emotional valence, and sex
NeuroImage
Analysis of fMRI time-series revisited—Again
NeuroImage
Are the windows to the soul the same in the East and West? Cultural differences in using the eyes and mouth as cues to recognize emotions in Japan and the United States
J. Exp. Soc. Psychol.
Functional neuroimaging of the olfactory system in humans
Int. J. Psychophysiol.
Empirical analyses of BOLD fMRI statistics. I. Spatially unsmoothed data collected under null-hypothesis conditions
NeuroImage
Structure and function of auditory cortex: music and speech
Trends Cogn. Sci.
Recognizing emotion from facial expressions: psychological and neurological mechanisms
Behav. Cogn. Neurosci. Rev.
Amygdalo-cortical projections in the monkey (Macaca fascicularis)
J. Comp. Neurol.
Lesions of the human amygdala impair enhanced perception of emotionally salient events
Nature
Perceived similarity of exemplar traits and behavior: effects on message evaluation
Communic. Res.
The theory of mind hypothesis of autism: history and prospects of the idea
The Psychologist
The amygdala and emotional memory
Nature
Cited by (55)
Amygdala function in emotion, cognition, and behavior
2022, Handbook of Clinical NeurologyMapping the interconnected neural systems underlying motivation and emotion: A key step toward understanding the human affectome
2020, Neuroscience and Biobehavioral ReviewsBrain correlates of emotional prosodic change detection in autism spectrum disorder
2020, NeuroImage: ClinicalThe sound of emotions-Towards a unifying neural network perspective of affective sound processing
2016, Neuroscience and Biobehavioral Reviews