Elsevier

NeuroImage

Volume 59, Issue 1, 2 January 2012, Pages 4-13
NeuroImage

Review
Attention, biological motion, and action recognition

https://doi.org/10.1016/j.neuroimage.2011.05.044Get rights and content

Abstract

Interacting with others in the environment requires that we perceive and recognize their movements and actions. Neuroimaging and neuropsychological studies have indicated that a number of brain regions, particularly the superior temporal sulcus, are involved in a number of processes essential for action recognition, including the processing of biological motion and processing the intentions of actions. We review the behavioral and neuroimaging evidence suggesting that while some aspects of action recognition might be rapid and effective, they are not necessarily automatic. Attention is particularly important when visual information about actions is degraded or ambiguous, or if competing information is present. We present evidence indicating that neural responses associated with the processing of biological motion are strongly modulated by attention. In addition, behavioral and neuroimaging evidence shows that drawing inferences from the actions of others is attentionally demanding. The role of attention in action observation has implications for everyday social interactions and workplace applications that depend on observing, understanding and interpreting actions.

Research highlights

► We review the role of attention in action observation. ► Attention is needed when displays of human actions are degraded, or contain noise or competing stimuli. ► The neuroergonomic implications of these findings are discussed.

Introduction

Identifying and understanding the movements and actions of others are fundamental properties of visual perception and cognition that are important for effective communication and interaction with other individuals in the environment. The movements of living organisms such as people or animals—both whole-body motion as well as partial movements by hands, head, eye, etc.—are typically referred to as biological motion (Blake and Shiffrar, 2007, Johansson, 1973). This form of motion is central to our perception of the dynamic natural environment and for inferring intent from the actions of others.

In addition to allowing for smooth and efficient interaction with others, biological motion perception is also important in many everyday and work activities. For example, rapidly detecting and identifying the actions and movements of other people is a critical need in search and rescue and surveillance tasks performed by remotely located operators monitoring video or infrared images from closed circuit TV (Tickner et al., 1972) or uninhabited air and ground vehicles (Cooke et al., 2006). Such systems are widely used in various environments such as hazardous or hostile areas, prisons, airports, train terminals, highways, and busy city streets. Surveillance images typically show people or vehicles in motion and engaged in various activities. Such information can be used, either by human monitors (Blake and Shiffrar, 2007) or automated systems (Cohen et al., 2008), to detect individuals with hostile intent or to determine the potential for danger in crowd-control situations. Identifying the neural bases of biological motion and action observation is therefore a key neuroergonomic issue, with implications for both theory and practice (Parasuraman and Rizzo, 2007). Studies using functional MRI (fMRI), event-related potentials (ERPs), and other cognitive neuroscience methods can further our understanding of human behavior in biological motion and action observation/understanding tasks and point to ways in which systems can be better designed or people trained to improve performance.

Section snippets

Is the processing of biological motion automatic?

Biological motion is often conceived of as a relatively primitive, automatic function that requires little or no help from conscious attention, consistent with its ecological significance and importance for survival. For example, two day-old human infants who have barely developed their visual attention skills prefer to look at point-light displays of biological motion than at other moving objects (Simion et al., 2008). The same is true of newborn chicks hatched in complete darkness and with no

Evidence for the role of attention in the perception of biological motion and the understanding of actions

Given its significance for survival, it makes sense for biological motion to proceed on a largely automatic basis. The findings with the newly born across species also suggest that many organisms may be genetically predisposed to process biological motion. Yet, despite its robustness, biological motion may be influenced by attention and other top-down factors. Such factors may especially come into play in conditions where different biological motion stimuli are ambiguous, degraded, or overlap

From biological motion perception to action recognition and understanding

Multiple brain regions that allow us to perceive and understand the actions of other people have been defined in detail using human neuroimaging, human scalp and monkey single-unit electrophysiology, and human lesion studies. From these studies we have been able to establish a network of regions that interact to complete a number of steps in the analysis of observed actions (Fig. 2). At least three such steps can be identified. First, the features defining an action must be detected and

Detecting the features that define actions

One of the important characteristics of human actions is that they consist of a specific form (a hand, a mouth, or a whole body) that moves in a specific way. These two features are not strictly independent, as the range of motion of an effector is determined by factors such as the joint-articulation and the elasticity of muscles and skin. However, it appears that early in the visual processing of actions, form and motion features might be processed somewhat independently in the brain.

Forming action-specific representations

The neural populations within the pITS region do start to encode detected actions into specific action representations. Several studies using fMRI adaptation have indicated that MT+ and EBA show an attenuated response to repeated presentation of actions relative to novel presentations (Grossman et al., 2010, Kable and Chatterjee, 2006, Wiggett and Downing, 2011). Some of this reduction in response might be due to adaptation of either the form or the motion, rather than the integrated

Understanding intentions from observed actions

While regions in lateral temporal cortex detect and analyze observed actions, understanding the meaning of those actions, including the intentions of the actor, appears to involve regions in inferior parietal and inferior frontal cortex. Within parietal cortex, the inferior parietal lobule (IPL) and the neighboring anterior intraparietal sulcus (aIPS) play a key role in action understanding. The aIPS and IPL regions are primarily concerned with processing the goals of observed actions, rather

The spatiotemporal dynamics of action understanding

While the brain regions that form a network that allows us to understand the actions of other people have been well defined, we have only a preliminary understanding of how these regions interact in time. Studies using ERPs have begun to examine the timing of the response to human actions in lateral temporal cortex. More recently, several studies have combined fMRI with source-localized electrophysiological activity in order to examine in detail the timing of activity in specific parts of the

Perceiving biological motion requires attention when competing objects are present

In the previous section we described the cortical networks that are activated during the detection and identification of human actions, and the related brain regions that are involved when such information is used for making inferences about the intentions of an actor. The relevant neuroimaging studies typically required their participants to either passively observe biological motion (or actions) or had them perform some active task with these stimuli. The use of an active task condition

Dividing attention can impair biological motion

The effects of selective attention on the fMRI and EEG response of the STS reveal that the detection and encoding of human actions are not automatic processes. Instead, these findings suggest that integrating form and motion in order to make a representation of actions requires the engagement of an attentional selection mechanism. However, this selection mechanism might be most necessary when actions are observed in busy, cluttered environments, in which other stimuli compete for our focus.

Detecting the intentions of others can be attentionally demanding

The previous sections have highlighted the role that selective and divided attention can play in the perception and neural processing of biological motion. The reviewed studies have shown that when biological motion has to compete with other objects, or if stimuli require integration over space or time, attention is required. So far, however, we have not discussed the role that attention might play in detecting or understanding the intentions behind observed actions. If attention is necessary

Why is attention needed for the processing of biological motion?

The studies reviewed here provide evidence that the perception and neural processing of biological motion can be shown to require attention under certain circumstances. In particular, attention plays an important role in the processing of biological motion and observed actions when stimuli are degraded, ambiguous, or cluttered. Such evidence suggests that the mechanisms that underlie the processing of biological motion, and action observation in general, have some capacity limitation, but that

Discussion

In this paper we have reviewed neuroimaging evidence pointing to the role of attention in identifying and understanding the movements and actions of others. Understanding the neural bases of these functions and their modulation by attention are important for refining theories of biological motion (Blake and Shiffrar, 2007), which in turn can inform theories of how people communicate with each other in social exchanges (Allison et al., 2000). Such an understanding can also inform neuroergonomic

Conclusions

The STS plays a key role in the detection and identification of the movements and actions of others. Evidence from fMRI and ERP studies indicates that, based on the integration of form and motion signals (from pITS areas), the STS sends representations of actions to inferior parietal and inferior frontal regions. Contrary to prevailing views of biological motion as being a bottom-up, automatic process, neural responses to human actions in the STS and inter-related cortical regions are strongly

Acknowledgments

This study was supported by the Army Research Laboratory Contract DAAD-19-01-C-0065, Office of Naval Research Multidisciplinary University Research Initiative (MURI) Grant N000141010198, Air Force Office of Sponsored Research/Air Force Research Laboratory Grant FA9550-10-1-0385, and the Center of Excellence in Neuroergonomics, Technology, and Cognition (CENTEC).

References (74)

  • S.H. Johnson-Frey et al.

    Actions or hand-object interactions? Human inferior frontal cortex and action observation

    Neuron

    (2003)
  • D. Jokisch et al.

    Structural encoding and recognition of biological motion: evidence from event-related potentials and source analysis

    Behav. Brain Res.

    (2005)
  • N. Lavie

    Distracted or confused: Selective attention under load

    Trends Cogn. Sci.

    (2005)
  • M.V. Peelen et al.

    Patterns of fMRI activity dissociate overlapping functional brain areas that respond to biological motion

    Neuron

    (2006)
  • R. Saxe et al.

    Making sense of another mind: the role of the right temporo-parietal junction

    Neuropsychologia

    (2005)
  • L. Shmuelof et al.

    Dissociation between ventral and dorsal fMRI activation during object and action recognition

    Neuron

    (2005)
  • I.M. Thornton et al.

    Incidental processing of biological motion

    Curr. Biol.

    (2004)
  • K.J. Wheaton et al.

    Viewing the motion of human body parts activates different regions of premotor, temporal, and parietal cortex

    Neuroimage

    (2004)
  • W. Baccus et al.

    Early integration of form and motion in the neural response to biological motion

    Neuroreport

    (2009)
  • M.S. Beauchamp et al.

    FMRI responses to video and point-light displays of moving humans and manipulable objects

    J. Cogn. Neurosci.

    (2003)
  • R. Blake et al.

    Perception of human motion

    Annu. Rev. Psychol.

    (2007)
  • E. Bonda et al.

    Specific involvement of human parietal systems and the amygdala in the perception of biological motion

    J. Neurosci.

    (1996)
  • G. Buccino et al.

    Action observation activates premotor and parietal areas in a somatotopic manner: an fMRI study

    Eur. J. Neurosci.

    (2001)
  • A. Casile et al.

    Neuronal encoding of human kinematic invariants during action observation

    Cereb. Cortex

    (2010)
  • V.P. Clark et al.

    Selective attention to face identity and color studied with fMRI

    Hum. Brain Mapp.

    (1997)
  • M. Corbetta et al.

    Control of goal-directed and stimulus-driven attention in the brain

    Nat. Rev. Neurosci.

    (2002)
  • C.J. Cohen et al.

    A surveillance system for the recognition of intent within individuals and crowds

  • R. Desimone et al.

    Neural mechanisms of selective visual attention

    Annu. Rev. Neurosci.

    (1995)
  • P.E. Downing et al.

    A cortical area selective for visual processing of the human body

    Science

    (2001)
  • B.A. Eriksen et al.

    Effects of noise letters upon the identification of a target letter in a non-search task

    Percept. Psychophys.

    (1974)
  • M.A. Giese et al.

    Neural mechanisms for the recognition of biological movements

    Nat. Rev. Neurosci.

    (2002)
  • E.D. Grossman et al.

    fMR-adaptation reveals invariant coding of biological motion on the human STS

    Front. Hum. Neurosci.

    (2010)
  • E.D. Grossman et al.

    Learning to see biological motion: brain activity parallels behavior

    J. Cogn. Neurosci.

    (2004)
  • A.F. Hamilton et al.

    Goal representation in human anterior intraparietal sulcus

    J. Neurosci.

    (2006)
  • A.F. Hamilton et al.

    Action outcomes are represented in human inferior frontoparietal cortex

    Cereb. Cortex

    (2008)
  • M. Hirai et al.

    The relative importance of spatial versus temporal structure in the perception of biological motion: an event-related potential study

    Cognition

    (2006)
  • Cited by (107)

    View all citing articles on Scopus
    View full text