Abstract
For decades the study of social perception was largely compartmentalized by type of social cue: race, gender, emotion, eye gaze, body language, facial expression etc. This was partly due to good scientific practice (e.g., controlling for extraneous variability), and partly due to assumptions that each type of social cue was functionally distinct from others. Herein, we present a functional forecast approach to understanding compound social cue processing that emphasizes the importance of shared social affordances across various cues (see too Adams et al. In The science of social vision, ed. R.B. Adams Jr., N. Ambady, K. Nakayama, and S. Shimojo. Oxford: Oxford University Press, 2010b; Adams and Nelson In The handbook of social neuroscience, ed. J. Decety and J. Cacioppo. Oxford: Oxford University Press, 2011; Weisbuch and Adams Social and Personality Psychology Compass 6: 499–514, 2012). We review the traditional theories of emotion and face processing that argued for dissociable and noninteracting pathways (e.g., for specific emotional expressions, gaze, identity cues), as well as more recent evidence for combinatorial processing of social cues. We argue here that early, and presumably reflexive, visual integration of such cues is necessary for adaptive behavioral responding to others. In support of this claim, we review contemporary work that reveals a flexible visual system, one that readily incorporates meaningful contextual influences in even nonsocial visual processing, thereby establishing the functional and neuroanatomical bases necessary for compound social cue integration. Finally, we explicate three likely mechanisms driving such integration. Together, this work implicates a role for cognitive penetrability in visual perceptual abilities that have often been (and in some cases still are) ascribed to direct encapsulated perceptual processes.
Similar content being viewed by others
Notes
Please see this Slate article on how drivers of certain car makes are perceived:
References
Ackerman, J.M., J.R. Shapiro, S.L. Neuberg, D.T. Kenrick, D.V. Becker, V. Griskevicius, et al. 2006. They all look the same to me (unless they’re angry): from out-group homogeneity to out-group heterogeneity. Psychological Science 17: 836–840.
Adams Jr., R.B., and R.E. Kleck. 2003. Perceived gaze direction and the processing of facial displays of emotion. Psychological Science 14: 644–647.
Adams Jr., R.B., and R.E. Kleck. 2005. Effects of direct and averted gaze on the perception of facially communicated emotion. Emotion 5: 3–11.
Adams Jr., R.B., and A.J. Nelson. 2011. Intersecting identities and expressions: the compound nature of social perception (pp. 394–403). In The handbook of social neuroscience, ed. J. Decety and J. Cacioppo. Oxford: Oxford University Press.
Adams Jr., R.B., H.L. Gordon, A.A. Baird, N. Ambady, and R.E. Kleck. 2003. Effects of gaze on amygdala sensitivity to anger and fear faces. Science 300: 1536.
Adams Jr., R.B., N. Ambady, C.N. Macrae, and R.E. Kleck. 2006. Emotional expressions forecast approach-avoidance behavior. Motivation and Emotion 30: 177–186.
Adams Jr., R.B., N. Ambady, K. Nakayama, and S. Shimojo (eds.). 2010a. The science of social vision. New York: Oxford University Press.
Adams Jr., R.B., R.G. Franklin Jr., A.J. Nelson, and M.T. Stevenson. 2010b. Compound social cues in face processing (pp. 90–107). In The science of social vision, ed. R.B. Adams Jr., N. Ambady, K. Nakayama, and S. Shimojo. Oxford: Oxford University Press.
Adams Jr., R.B., K. Pauker, and M. Weisbuch. 2010c. Looking the other way: the role of gaze direction in the cross-race memory effect. Journal of Experimental Social Psychology 46: 478–481.
Adams, R.B., Jr., R.G. Franklin Jr., K. Kveraga, N. Ambady, R.E. Kleck, P.J. Whalen, N. Hadjikhani, and A.J. Nelson. 2012. Amygdala responses to averted versus direct gaze fear vary as a function of presentation speed. Social Cognitive and Affective Neuroscience 7: 568–577.
Aguirre, G.K., J.A. Detre, D.C. Alsop, and M. D’Esposito. 1996. The parahippocampus subserves topographical learning in man. Cerebral Cortex 6: 823–829.
Ambady, N., D. LaPlante, T. Nguyen, R. Rosenthal, and W. Levinson. 2002. Surgeon’s tone of voice: a clue to malpractice history. Surgery 132: 5–9.
Aminoff, E., N. Gronau, and M. Bar. 2007. The parahippocampal cortex mediates spatial and nonspatial associations. Cerebral Cortex 17: 1493–1503.
Bar, M., and E. Aminoff. 2003. Cortical analysis of visual context. Neuron 38: 347–358.
Bar, M., E. Aminoff, and D.L. Schacter. 2008. Scenes unseen: the parahippocampal cortex intrinsically subserves contextual associations, not scenes or places per se. Journal of Neuroscience 28: 8539–8544.
Bar, M., K.S. Kassam, A.S. Ghuman, J. Boshyan, A.M. Schmid, A.M. Dale, M.S. Hamalainen, K. Marinkovic, D.L. Schacter, B.R. Rosen, and E. Halgren. 2006. Top-down facilitation of visual recognition. Proceedings of the National Academy of Sciences of the United States of America 103: 449–454.
Baron-Cohen, S. 1995. Theory of mind and face-processing: how do they interact in development and psychopathology. In Developmental psychopathology, vol. 1, ed. D. Cicchetti and D. Cohen, 343–356. New York: Wiley.
Becker, D.V., D.T. Kenrick, S.L. Neuberg, K.C. Blackwell, and D.M. Smith. 2007. The confounded nature of angry men and happy women. Journal of Personality and Social Psychology 92: 179–190.
Bernstein, M.J., S.G. Young, and K. Hugenberg. 2007. The cross-category effect: mere social categorization is sufficient to elicit an own-group bias in face recognition. Psychological Science 18: 706–712.
Brodmann, K. (1909/2006). Localization in the cerebral cortex. Springer.
Brothers, L., and B. Ring. 1993. Mesial temporal neurons in the macaque monkey with responses selective for aspects of social stimuli. Behavioural Brain Research 57: 53–61.
Bruce, V., and A. Young. 1986. Understanding face recognition. British Journal of Psychology 77: 305–327.
Buckner, R., and F. Krienen. 2013. The evolution of distributed association networks in the human brain. Trends in Cognitive Sciences 17: 648–665.
Buzsáki, G. 2006. Rhythms of the Brain. Oxford University Press.
Calder, A.J., and A.W. Young. 2005. Understanding the recognition of facial identity and facial expression. Nature Reviews Neuroscience 6: 641–651.
Calder, A.J., A.M. Burton, P. Miller, A.W. Young, and S. Akamatsu. 2001. A principal component analysis of facial expressions. Vision Research 41: 1179–1208.
Chaumon, M., K. Kveraga, L.F. Barrett, and M. Bar. 2013. Visual predictions in the orbitofrontal cortex rely on associative content. Cerebral Cortex. doi:10.1093/cercor/bht146.
Chiao, J.Y., R.B. Adams Jr., P.U. Tse, W.T. Lowenthal, J.A. Richeson, and N. Ambady. 2008. Knowing who’s boss: fMRI and ERP investigations of social dominance perception. Group Processes & Intergroup Relations 11: 201–214.
Cohen, L., et al. 2002. Language-specific tuning of visual cortex? Functional properties of the visual word form area. Brain 125: 1054–1069.
Darwin, C. (1872; 1965; 1998). The expression of the emotions in man and animals. Chicago: University of Chicago Press.
Davidson, R.J., and K. Hugdahl (eds.). 1995. Brain asymmetry. Cambridge: MIT Press.
de Gelder, B., and M. Tamietto. 2010. Faces, bodies, social vision as agent vision and social consciousness (pp. 51–74). In The science of social vision, ed. R.B. Adams, N. Ambady, K. Nakayama, and S. Shimojo. New York: Oxford University Press.
de Gelder, B., G. Pourtois, M. van Raamsdonk, J. Vroomen, and L. Weiskrantz. 2001. Unseen stimuli modulate conscious visual experience: evidence from inter-hemispheric summation. Neuroreport 12: 385–391.
de Gelder, B., J.S. Morris, and R.J. Dolan. 2005. Unconscious fear influences emotional awareness of faces and voices. Proceedings of the National Academy of Sciences 102: 18682–18687.
Downing, P.E., Y. Jiang, M. Shuman, and N. Kanwisher. 2001. A cortical area selective for visual processing of the human body. Science 293: 2470–2473.
Dunbar, R.I.M. 1998. The social brain hypothesis. Evolutionary Anthropology 6: 178–190.
Epstein, R., and N. Kanwisher. 1998. A cortical representation of the local visual environment. Nature 392: 598–601.
Fodor, J.A. 1983. The modularity of mind. Cambridge: MIT Press.
Freeman, J.B., and N. Ambady. 2011. A dynamic interactive theory of person construal. Psychological Review 118: 247–279.
Freeman, J.B., N.O. Rule, R.B. Adams Jr., and N. Ambady. 2010. The neural basis of categorical face perception: graded representations of face gender in fusiform and orbitofrontal cortices. Cerebral Cortex 20: 1314–1322.
Frischen, A., A.P. Bayliss, and S.P. Tipper. 2007. Gaze cueing of attention: visual attention, social cognition, and individual differences. Psychological Bulletin 133: 694–724.
Friston, K.J., L. Harrison, and W. Penny. 2003. Dynamic causal modelling. NeuroImage 19: 1273–1302.
Ghuman, A.S., J.R. McDaniel, and A. Martin. 2010. Face adaptation without a face. Current Biology 20: 32–36.
Gibson, J.J. 1979. The ecological approach to visual perception. Boston: Houghton-Mifflin.
Gifford, R. 1991. Mapping nonverbal behavior on the interpersonal circle. Journal of Personality and Social Psychology 61: 279–288.
Goodale, M.A., and A.D. Milner. 1992. Separate visual pathways for perception and action. Trends in Neuroscience 15(1): 20–25.
Gregory, R.L. 1970. The intelligent eye. Weidenfeld and Nicolson, London.
Hadjikhani, N., K. Kveraga, P. Naik, and S.P. Ahlfors. 2009. Early (M170) activation of face-specific cortex by face-like objects. Neuroreport 20(4): 403–407.
Haxby, J.V., E.A. Hoffman, and M.I. Gobbini. 2000. The distributed human neural system for face perception. Trends in Cognitive Sciences 4: 223–233.
Hendry, S.H., and C. Reid. 2000. The koniocellular pathway in primate vision. Annual Review of Neuroscience 23: 127–153.
Hess, U., R.B. Adams Jr., and R. Kleck. 2004. Facial appearance, gender, and emotion expression. Emotion 4: 378–388.
Hess, U., R.B. Adams Jr., and R. Kleck. 2005. Who may frown and who should smile? Dominance, affiliation, and the display of happiness and anger. Cognition & Emotion 19: 515–536.
Hess, U., R.B. Adams Jr., K. Grammer, and R.E. Kleck. 2009. Sex and emotion expression: are angry women more like men? Journal of Vision 9: 1–8.
Hildebrandt, K.A., and H.E. Fitzgerald. 1981. Mothers’ responses to infant physical appearance. Infant Mental Health Journal 2: 56–61.
Hubel, D.H., and T.N. Wiesel. 1959. Receptive fields of single neurones in the cat’s striate cortex. Journal of Physiology 148: 574–591.
Hugenberg, K. 2005. Social categorization and the perception of facial affect: target race moderates the response latency advantage for happy faces. Emotion 5: 267–276.
Isbell, L.A. 2006. Snakes as agents of evolutionary change in primate brain. Journal of Human Evolution 51(1): 1–35. doi:10.1016/j.jhevol.2005.12.012.
Jayakumar, J., R. Sujata, B. Dreher, P.R. Martin, and T.R. Vidyasagar. 2013. Multiple pathways carry signals from short-wavelength-sensitive (‘blue’) cones to the middle temporal area of the macaque. Journal of Physiology 591(1): 339–352.
Jones, B.C., L.M. DeBruine, J.C. Main, A.C. Little, L.L. Welling, D.R. Feinberg, and B.P. Tiddeman. 2010. Facial cues of dominance modulate the short-term gaze-cuing effect in human observers. Proceedings of the Royal Society B: Biological Sciences 277: 617–624.
Kaas, J. (2000) TITLE. In Cognitive Neuroscience, A reader. Ed. Gazzaniga.
Kanwisher, N. 2010. Functional specificity in the human brain: a window into the functional architecture of the mind. Proceedings of the National Academy of Sciences 107: 11163–11170.
Kanwisher, N.G., J. McDermott, and M.M. Chun. 1997. The fusiform face area: a module in human extrastriate cortex specialized for face perception. Journal of Neuroscience 17: 4302–4311.
Keltner, D., P. Ekman, G.C. Gonzaga, and J. Beer. 2003. Facial expression of emotion. In Handbook of affective sciences, ed. R.J. Davidson, K.R. Scherer, and H.H. Goldsmith, 415–431. New York: Oxford University Press.
Knutson, B. 1996. Facial expressions of emotion influence interpersonal trait inferences. Journal of Nonverbal Behavior 20: 165–182.
Króliczak, G., P. Heard, M.A. Goodale, and R.L. Gregory. 2006. Dissociation of perception and action unmasked by the hollow-face illusion. Brain Research 1080: 9–16.
Kveraga, K., J. Boshyan, and M. Bar. 2007a. Magnocellular projections as the trigger of top-down facilitation in recognition. Journal of Neuroscience 27: 13232–13240.
Kveraga, K., A.S. Ghuman, and M. Bar. 2007b. Top-down predictions in the cognitive brain. Brain and Cognition 65(2): 145–168.
Kveraga, K., J. Boshyan, and M. Bar. 2009. The proactive brain: using memory-based predictions in visual recognition. In Object categorization: computer and human vision perspectives, ed. S. Dickinson, M. Tarr, A. Leonardis, and B. Schiele. Cambridge: Cambridge University Press.
Kveraga, K., A.S. Ghuman, K.S. Kassam, E. Aminoff, M.S. Hamalainen, M. Chaumon, et al. 2011. Early onset of neural synchronization in the contextual associations network. Proceedings of the National Academy of Sciences 108(8): 3389–3394.
Le Gal, P.M., and V. Bruce. 2002. Evaluating the independence of sex and expression in judgments of faces. Perception and Psychophysics 2: 230–243.
Le Gal, P.M., and V. Bruce. 2002. Evaluating the independence of sex and expression in judgments of faces. Perception and Psychophysics 64: 230–243.
LeDoux, J.E. 1996. The emotional brain. New York: Simon & Schuster.
Maclin, O.H., and R.S. Malpass. 2001. Racial categorization of faces: the ambiguous race face effect. Psychology, Public Policy, and Law 7: 98–118.
Maior, R.S., et al. 2011. Superior colliculus lesions impair threat responsiveness in infant capuchin monkeys. Neuroscience Letters 504(3): 257–260.
Marsh, A.A., R.B. Adams Jr., and R.E. Kleck. 2005. Why do fear and anger look the way they do? Form and social function in facial expressions. Personality and Social Psychology Bulletin 31: 73.
McKone, E., J.L. Brewer, S. MacPherson, G. Rhodes, and W.G. Hayward. 2007. Familiar other-race faces show normal holistic processing and are robust to perceptual stress. Perception 36: 224.
Meeren, H.J.M., C.C.R.J. van Heijnsbergen, and B. de Gelder. 2005. Rapid perceptual integration of facial expression and emotional body language. Proceedings of the National Academy of Sciences 102: 16518–16523.
Michel, C., B. Rossion, J. Han, C. Chung, and R. Caldara. 2006. Holistic processing is finely tuned for faces of one’s own race. Psychological Science 17: 608–615.
Mogg, K., M. Garner, and B.P. Bradley. 2007. Anxiety and orienting of gaze to angry and fearful faces. Biological Psychology 76: 163–169.
Morand, S., G. Thut, R. Grave de Peralta, S. Clarke, A. Khateb, T. Landis, and C.M. Michel. 2000. Electrophysiological evidence for fast visual processing through the human koniocellular pathway when stimuli move. Cerebral Cortex 10: 817–825.
Morris, J.S., A. Öhman, and R.J. Dolan. 1998. Conscious and unconscious emotional learning in the human amygdala. Nature 393: 467–470.
Nakayama, N. Vision going social (2010) (pp. xv-xxix). In The science of social vision, eds. R.B. Adams Jr., N. Ambady, K. Nakayama & S. Shimojo Oxford University Press.
Öhman, A. 2005. The role of the amygdala in human fear: automatic detection of threat. Psychoneuroendocrinology, 30: 953–958.
O’Toole, A.J., et al. 1998. The perception of face gender: the role of stimulus structure in recognition and classification. Memory & Cognition 26: 146–160.
Pauker, K., N. Ambady, M. Weisbuch, S.R. Sommers, R.B. Adams Jr., and Z. Ivcevic. 2009. Not so black and white: memory for ambiguous group members. Journal of Personality and Social Psychology 96: 795–810.
Pessoa, L., and R. Adolphs. 2010. Emotion processing and the amygdala: from a “low road” to “many roads” of evaluating biological significance. Nature Reviews Neuroscience 11: 773–783.
Peters, J., I. Daum, E. Gizewski, M. Forsting, and B. Suchan. 2009. Associations evoked during memory encoding recruit the context-network. Hippocampus 19: 141–151.
Pineda, J.A., G. Sebestyen, and C. Nava. 1994. Face recognition as a function of social attention in non-human primates: an ERP study. Brain Research. Cognitive Brain Research 2: 1–12.
Redican, W.K. 1982. An evolutionary perspective on human facial displays. In Emotion in the human face, 2nd ed, ed. P. Ekman, 212–280. Cambridge: Cambridge Univ. Press.
Richeson, J.A., A.R. Todd, S. Trawalter, and A.A. Baird. 2008. Eye-gaze direction modulates race-related amygdala activity. Group Processes and Intergroup Relations 11: 233–246.
Rule, N.O., N. Ambady, R.B. Adams Jr., and C.N. Macrae. 2007. Us and them: memory advantages in perceptually ambiguous groups. Psychonomic Bulletin & Review 14: 687–692.
Sapir, E. 1927. Speech as a personality trait. American Journal of Sociology 32: 892–905.
Schaefer, E.S., and R. Plutchik. 1966. Interrelationships of emotions, traits, and diagnostic constructs. Psychological Reports 18: 399–410.
Shepherd, S.V., R.O. Deaner, and M.L. Platt. 2006. Social status gates social attention in monkeys. Current Biology 16: R119–R120.
Shriver, E., S. Young, K. Hugenberg, M. Bernstein, and J. Lanter. 2008. Class, race, and the face: social context modulates the cross-race effect in face recognition. Personality and Social Psychology Bulletin 34: 260–278.
Siegel, S. 2011. Cognitive penetrability and perceptual justification. Noûs 46: 201–222.
Tamietto, M., and B. de Gelder. 2010. Neural bases of the non-conscious perception of emotional signals. Nature Reviews Neuroscience 11: 697–709.
Tamietto, M., L. Weiskrantz, G. Geminiani, and B. de Gelder (2007). The Medium and the Message: Non-conscious processing of emotions from facial expressions and body language in blindsight. Paper presented at the Cognitive Neuroscience Society Annual Meeting, New York, NY.
Tanaka, J.W., M. Kiefer, and C.M. Bukach. 2004. A holistic account of the own-race effect in face recognition: evidence from a cross-cultural study. Cognition 93: B1–B9.
Townsend, J.T., and M.J. Wenger. 2004. A theory of interactive parallel processing: new capacity measures and predictions for a response time inequality series. Psychological Review 111: 1003–1035.
Trawalter, S., A. Todd, A.A. Baird, and J.A. Richeson. 2008. Attending to threat: race-based patterns of selective attention. Journal of Experimental Social Psychology 44: 1322–1327.
Tudusciuc, O., and A. Nieder. 2010. Comparison of length judgments and the Müller-Lyer illusion in monkeys and humans. Experimental Brain Research 207: 221–231.
Turk, D.J., T.C. Handy, and M.S. Gazzaniga. 2005. Can perceptual expertise account for the own-race bias in face recognition? A split-brain study. Cognitive Neuropsychology 22: 877–883.
Vuilleumier, P., J.L. Armony, J. Driver, and R.J. Dolan. 2003. Distinct spatial frequency sensitivities for processing faces and emotional expressions. Nature Neuroscience 6: 624–631.
Weisbuch, M., and R.B. Adams Jr. 2012. The functional forecast model of emotional expression processing. Social and Personality Psychology Compass 6: 499–514.
Weisbuch, M., K. Pauker, S. Lamer, and R.B. Adams, Jr. (under review). Social Stratification and Reflexive Gaze-Following.
Wiesenfeld, A.R., and R. Klorman. 1978. The mother’s psychophysiological reactions to contrasting affective expressions by her own and an unfamiliar infant. Developmental Psychology 14(3): 294–304.
Wiggins, J.S., and R. Broughton. 1991. A geometric taxonomy of personality scales. European Journal of Personality 5: 343–365.
Xu, X., J.M. Ichida, J.D. Allison, J.D. Boyd, and A.B. Bonds. 2001. A comparison of koniocellular, magnocellular and parvocellular receptive field properties in the lateral geniculate nucleus of the owl monkey (Aotus trivirgatus). Journal of Physiology 531: 203–218.
Zebrowitz. 1997. Reading faces: window to the soul? Boulder: Westview Press.
Zebrowitz, L., M. Kikuchi, and J.M. Fellous. 2010. Facial resemblance to emotions: group differences, impression effects, and race stereotypes. Journal of Personality and Social Psychology 98: 175–189.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Adams, R.B., Kveraga, K. Social Vision: Functional Forecasting and the Integration of Compound Social Cues. Rev.Phil.Psych. 6, 591–610 (2015). https://doi.org/10.1007/s13164-015-0256-1
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13164-015-0256-1