ReviewAttention, biological motion, and action recognition
Research highlights
► We review the role of attention in action observation. ► Attention is needed when displays of human actions are degraded, or contain noise or competing stimuli. ► The neuroergonomic implications of these findings are discussed.
Introduction
Identifying and understanding the movements and actions of others are fundamental properties of visual perception and cognition that are important for effective communication and interaction with other individuals in the environment. The movements of living organisms such as people or animals—both whole-body motion as well as partial movements by hands, head, eye, etc.—are typically referred to as biological motion (Blake and Shiffrar, 2007, Johansson, 1973). This form of motion is central to our perception of the dynamic natural environment and for inferring intent from the actions of others.
In addition to allowing for smooth and efficient interaction with others, biological motion perception is also important in many everyday and work activities. For example, rapidly detecting and identifying the actions and movements of other people is a critical need in search and rescue and surveillance tasks performed by remotely located operators monitoring video or infrared images from closed circuit TV (Tickner et al., 1972) or uninhabited air and ground vehicles (Cooke et al., 2006). Such systems are widely used in various environments such as hazardous or hostile areas, prisons, airports, train terminals, highways, and busy city streets. Surveillance images typically show people or vehicles in motion and engaged in various activities. Such information can be used, either by human monitors (Blake and Shiffrar, 2007) or automated systems (Cohen et al., 2008), to detect individuals with hostile intent or to determine the potential for danger in crowd-control situations. Identifying the neural bases of biological motion and action observation is therefore a key neuroergonomic issue, with implications for both theory and practice (Parasuraman and Rizzo, 2007). Studies using functional MRI (fMRI), event-related potentials (ERPs), and other cognitive neuroscience methods can further our understanding of human behavior in biological motion and action observation/understanding tasks and point to ways in which systems can be better designed or people trained to improve performance.
Section snippets
Is the processing of biological motion automatic?
Biological motion is often conceived of as a relatively primitive, automatic function that requires little or no help from conscious attention, consistent with its ecological significance and importance for survival. For example, two day-old human infants who have barely developed their visual attention skills prefer to look at point-light displays of biological motion than at other moving objects (Simion et al., 2008). The same is true of newborn chicks hatched in complete darkness and with no
Evidence for the role of attention in the perception of biological motion and the understanding of actions
Given its significance for survival, it makes sense for biological motion to proceed on a largely automatic basis. The findings with the newly born across species also suggest that many organisms may be genetically predisposed to process biological motion. Yet, despite its robustness, biological motion may be influenced by attention and other top-down factors. Such factors may especially come into play in conditions where different biological motion stimuli are ambiguous, degraded, or overlap
From biological motion perception to action recognition and understanding
Multiple brain regions that allow us to perceive and understand the actions of other people have been defined in detail using human neuroimaging, human scalp and monkey single-unit electrophysiology, and human lesion studies. From these studies we have been able to establish a network of regions that interact to complete a number of steps in the analysis of observed actions (Fig. 2). At least three such steps can be identified. First, the features defining an action must be detected and
Detecting the features that define actions
One of the important characteristics of human actions is that they consist of a specific form (a hand, a mouth, or a whole body) that moves in a specific way. These two features are not strictly independent, as the range of motion of an effector is determined by factors such as the joint-articulation and the elasticity of muscles and skin. However, it appears that early in the visual processing of actions, form and motion features might be processed somewhat independently in the brain.
Forming action-specific representations
The neural populations within the pITS region do start to encode detected actions into specific action representations. Several studies using fMRI adaptation have indicated that MT+ and EBA show an attenuated response to repeated presentation of actions relative to novel presentations (Grossman et al., 2010, Kable and Chatterjee, 2006, Wiggett and Downing, 2011). Some of this reduction in response might be due to adaptation of either the form or the motion, rather than the integrated
Understanding intentions from observed actions
While regions in lateral temporal cortex detect and analyze observed actions, understanding the meaning of those actions, including the intentions of the actor, appears to involve regions in inferior parietal and inferior frontal cortex. Within parietal cortex, the inferior parietal lobule (IPL) and the neighboring anterior intraparietal sulcus (aIPS) play a key role in action understanding. The aIPS and IPL regions are primarily concerned with processing the goals of observed actions, rather
The spatiotemporal dynamics of action understanding
While the brain regions that form a network that allows us to understand the actions of other people have been well defined, we have only a preliminary understanding of how these regions interact in time. Studies using ERPs have begun to examine the timing of the response to human actions in lateral temporal cortex. More recently, several studies have combined fMRI with source-localized electrophysiological activity in order to examine in detail the timing of activity in specific parts of the
Perceiving biological motion requires attention when competing objects are present
In the previous section we described the cortical networks that are activated during the detection and identification of human actions, and the related brain regions that are involved when such information is used for making inferences about the intentions of an actor. The relevant neuroimaging studies typically required their participants to either passively observe biological motion (or actions) or had them perform some active task with these stimuli. The use of an active task condition
Dividing attention can impair biological motion
The effects of selective attention on the fMRI and EEG response of the STS reveal that the detection and encoding of human actions are not automatic processes. Instead, these findings suggest that integrating form and motion in order to make a representation of actions requires the engagement of an attentional selection mechanism. However, this selection mechanism might be most necessary when actions are observed in busy, cluttered environments, in which other stimuli compete for our focus.
Detecting the intentions of others can be attentionally demanding
The previous sections have highlighted the role that selective and divided attention can play in the perception and neural processing of biological motion. The reviewed studies have shown that when biological motion has to compete with other objects, or if stimuli require integration over space or time, attention is required. So far, however, we have not discussed the role that attention might play in detecting or understanding the intentions behind observed actions. If attention is necessary
Why is attention needed for the processing of biological motion?
The studies reviewed here provide evidence that the perception and neural processing of biological motion can be shown to require attention under certain circumstances. In particular, attention plays an important role in the processing of biological motion and observed actions when stimuli are degraded, ambiguous, or cluttered. Such evidence suggests that the mechanisms that underlie the processing of biological motion, and action observation in general, have some capacity limitation, but that
Discussion
In this paper we have reviewed neuroimaging evidence pointing to the role of attention in identifying and understanding the movements and actions of others. Understanding the neural bases of these functions and their modulation by attention are important for refining theories of biological motion (Blake and Shiffrar, 2007), which in turn can inform theories of how people communicate with each other in social exchanges (Allison et al., 2000). Such an understanding can also inform neuroergonomic
Conclusions
The STS plays a key role in the detection and identification of the movements and actions of others. Evidence from fMRI and ERP studies indicates that, based on the integration of form and motion signals (from pITS areas), the STS sends representations of actions to inferior parietal and inferior frontal regions. Contrary to prevailing views of biological motion as being a bottom-up, automatic process, neural responses to human actions in the STS and inter-related cortical regions are strongly
Acknowledgments
This study was supported by the Army Research Laboratory Contract DAAD-19-01-C-0065, Office of Naval Research Multidisciplinary University Research Initiative (MURI) Grant N000141010198, Air Force Office of Sponsored Research/Air Force Research Laboratory Grant FA9550-10-1-0385, and the Center of Excellence in Neuroergonomics, Technology, and Cognition (CENTEC).
References (74)
- et al.
Social perception from visual cues: role of the STS region
Trends Cogn. Sci.
(2000) - et al.
Parallel visual motion processing streams for manipulable objects and human movements
Neuron
(2002) - et al.
Listening to a walking human activates the temporal biological motion area
Neuroimage
(2005) - et al.
fMRI adaptation reveals mirror neurons in human inferior parietal cortex
Curr. Biol.
(2008) - et al.
Selective attention modulates inferior frontal gyrus activity during action observation
Neuroimage
(2008) - et al.
Complementary systems for understanding action intentions
Curr. Biol.
(2008) - et al.
Practice under changing priorities: an approach to the training of complex skills
Acta Psychol.
(1989) - et al.
Repetitive TMS over posterior STS disrupts perception of biological motion
Vision Res.
(2005) - et al.
Brain areas active during visual perception of biological motion
Neuron
(2002) - et al.
Neural representations of perceived bodily actions using a categorical frame of reference
Neuropsychologia
(2006)
Actions or hand-object interactions? Human inferior frontal cortex and action observation
Neuron
Structural encoding and recognition of biological motion: evidence from event-related potentials and source analysis
Behav. Brain Res.
Distracted or confused: Selective attention under load
Trends Cogn. Sci.
Patterns of fMRI activity dissociate overlapping functional brain areas that respond to biological motion
Neuron
Making sense of another mind: the role of the right temporo-parietal junction
Neuropsychologia
Dissociation between ventral and dorsal fMRI activation during object and action recognition
Neuron
Incidental processing of biological motion
Curr. Biol.
Viewing the motion of human body parts activates different regions of premotor, temporal, and parietal cortex
Neuroimage
Early integration of form and motion in the neural response to biological motion
Neuroreport
FMRI responses to video and point-light displays of moving humans and manipulable objects
J. Cogn. Neurosci.
Perception of human motion
Annu. Rev. Psychol.
Specific involvement of human parietal systems and the amygdala in the perception of biological motion
J. Neurosci.
Action observation activates premotor and parietal areas in a somatotopic manner: an fMRI study
Eur. J. Neurosci.
Neuronal encoding of human kinematic invariants during action observation
Cereb. Cortex
Selective attention to face identity and color studied with fMRI
Hum. Brain Mapp.
Control of goal-directed and stimulus-driven attention in the brain
Nat. Rev. Neurosci.
A surveillance system for the recognition of intent within individuals and crowds
Neural mechanisms of selective visual attention
Annu. Rev. Neurosci.
A cortical area selective for visual processing of the human body
Science
Effects of noise letters upon the identification of a target letter in a non-search task
Percept. Psychophys.
Neural mechanisms for the recognition of biological movements
Nat. Rev. Neurosci.
fMR-adaptation reveals invariant coding of biological motion on the human STS
Front. Hum. Neurosci.
Learning to see biological motion: brain activity parallels behavior
J. Cogn. Neurosci.
Goal representation in human anterior intraparietal sulcus
J. Neurosci.
Action outcomes are represented in human inferior frontoparietal cortex
Cereb. Cortex
The relative importance of spatial versus temporal structure in the perception of biological motion: an event-related potential study
Cognition
Cited by (107)
Exploring biological motion perception in two-stream convolutional neural networks
2021, Vision ResearchAttentional bias towards social interactions during viewing of naturalistic scenes
2023, Quarterly Journal of Experimental Psychology