Mental blocks: fMRI reveals top-down modulation of early visual cortex when obstacles interfere with grasp planning
Highlights
► Slow event-related fMRI design isolates visuomotor planning network. ► Visual cortex encoding of obstacles is suppressed during grasp planning. ► Concurrent IPS activity suggests top-down parietal control of visual cortex activity. ► Suppressing obstacle representations provides for an efficient avoidance mechanism. ► Unifies understanding of PPC involvement in action planning and directing attention.
Introduction
Even after the visuomotor system has solved the difficult problem of selecting a specific target from the many that occupy our cluttered environment, non-target objects can function as obstacles that significantly alter the trajectory of the movement (Chapman and Goodale, 2008, Chapman and Goodale, 2010a, Chapman and Goodale, 2010b, Mon-Williams et al., 2001, Tresilian, 1998, Tresilian, 1999). This suggests the brain must flexibly encode objects to both attract and repel movements depending on whether the object is a potential target or a potential obstacle. Studies of patients with damage to the dorsal visual stream (Goodale & Milner, 1992) have implicated the posterior parietal cortex as being critical for obstacle encoding (McIntosh et al., 2004a, McIntosh et al., 2004b, Rice et al., 2008, Rice et al., 2006a, Schindler et al., 2004). Specifically, optic ataxic patients with damage to dorsal stream structures show significantly less deviation away from obstacles than normal participants (Schindler et al., 2004). Using functional magnetic resonance imaging (fMRI), we sought to identify whether the dorsal stream is also involved in encoding obstacles in normal individuals.
Studying real actions in the MRI environment is difficult due to the spatial constraints and the artifacts introduced by hand motion (Culham, 2006). To overcome these difficulties we adapted an obstacle task that interferes with grasping movements (which are easier to perform with less space, Tresilian, 1998) for use in a slow event-related paradigm. fMRI paradigms using delay periods have isolated planning responses in eye-movement tasks (Curtis et al., 2005, Curtis and Connolly, 2008, Curtis and D’Esposito, 2006, Curtis et al., 2004, Ikkai and Curtis, 2008) and have demonstrated preparatory activity in attentional cueing paradigms (Bressler et al., 2008, Kastner et al., 1999, Serences et al., 2004, Sylvester et al., 2008). Importantly, separating instructions with delay periods has also been used to isolate movement planning responses in reaching tasks (Beurze et al., 2007, Beurze et al., 2009). In addition to the fMRI experiment, we ran a behavioral task using the same apparatus outside of the MRI scanner to confirm that this setup would result in obstacle interference effects consistent with the previous literature (Tresilian, 1998).
Interestingly, tasks requiring attentional direction recruit a network of brain areas that are similar to those active in movement planning. Specifically, a dorsal parietal–frontal network has been shown to control the locus of attention (Beck and Kastner, 2009, Corbetta et al., 2008, Corbetta and Shulman, 2002, Kastner and Ungerleider, 2000, Riddoch et al., 2010, Serences and Yantis, 2007), with the intraparietal sulcus (IPS) playing a central role. This IPS activity aligns with parietal areas of activation when participants perform actions (Andersen and Buneo, 2002, Andersen and Cui, 2009, Astafiev et al., 2003, Beurze et al., 2007, Beurze et al., 2009, Culham et al., 2006, Culham and Valyear, 2006, Curtis and Connolly, 2008) as well as with the lesion sites of patients in the obstacle studies described above (e.g. McIntosh et al., 2004a, McIntosh et al., 2004b, Schindler et al., 2004). Recent studies using fMRI (Mevorach, Shalev, Allen, & Humphreys, 2009), transcranial magnetic stimulation (TMS, Mevorach et al., 2006b, Mevorach et al., 2009a, Silvanto et al., 2009), and both techniques together (Mevorach et al., 2010, Ruff et al., 2008) have shown that the IPS exerts top-down control over early visual areas. One particularly relevant study demonstrated that applying TMS over the left IPS interfered with participants’ ability to ignore salient information and reduced distractor suppression in early visual areas (Mevorach et al., 2010).
These previous studies, however, have relied on response-irrelevant spatial cues (i.e. arrows or verbal instructions to attend to one side of space or stimulus feature) to indicate which parts of a display are to be attended and which are to be ignored. Although this type of cueing is sufficient to produce IPS activation and the corresponding modulation of visual cortex (enhancement of attended locations and suppression of unattended locations), it is far removed from real-world settings where the demands of attention are directly linked to our interactions with the environment. That is, in real-world tasks – such as reaching for objects in cluttered environments – our goal-directed actions dictate which objects should be selected as targets and which should be ignored as non-targets. Furthermore, when non-targets are physical objects that could impede a desired movement, they require avoidance, which some researchers have argued may be implemented by inhibiting activity at obstacle locations (Howard and Tipper, 1997, Tipper et al., 1997, Welsh and Elliott, 2004). With this in mind, we were specifically interested in what would happen to the coding of non-target objects in visual cortex when those objects were potential obstacles as a consequence of their position with respect to the path of a planned movement. Thus, rather than relying on an arbitrary instruction to indicate whether or not a non-target object should be ignored (or avoided), we used an action task in which the requirements of the movement naturally determined how a non-target object should be treated. In this real-world scenario where non-target objects must be avoided in order to successfully complete the required response, we predicted that coding of those objects in visual cortex would be suppressed, similar to what happens with irrelevant stimuli or unattended visual locations in more cognitive tasks. In addition, given the previous patient literature, the overlap in attention and action networks, and the shared visual properties of distractors and obstacles, we predicted that the IPS would play a critical role in obstacle encoding, perhaps providing a source for the signals that modulate the activity associated with non-target objects in early visual cortex.
Section snippets
Participants
Fifteen (8 males, mean age 26.2) right-handed (determined by questionnaire, Oldfield, 1971) participants were scanned using blood-oxygenation-level-dependent functional magnetic resonance imaging (BOLD fMRI, Kwong et al., 1992, Ogawa et al., 1992). Informed consent was obtained in accordance with procedures approved by the University of Western Ontario's (London, Ontario, Canada) Health Sciences Research Ethics Board. All participants were naive with respect to the experimental hypothesis and
Behavioral experiment
To characterize the effect of obstacle position and wrist posture on movements, we calculated reaction time (time from OPTOTRAK record onset to first frame of movement onset) movement time (time from first frame of movement to last frame of movement) and peak grip aperture (the maximum distance between the index- and thumb-tip attained during the movement) for each participant on each trial. These measures were then entered into a two-factor Wrist-Posture × Obstacle-Position (2 × 3)
Summary
We adapted a reach-to-grasp obstacle paradigm (Tresilian, 1998) for use in an fMRI experiment to examine the encoding of obstacles for grasp movements in normal individuals. Using our adapted setup outside of the MRI environment we also replicated the behavioral finding that obstacles interfering with a grasping movement result in smaller openings of the hand (see Fig. 3). For the fMRI experiment, we employed a slow event-related paradigm that allowed us to isolate brain areas that were
Conclusion
The visuomotor planning network we defined in the current experiment functions as an elegant feedback loop. Initially, all objects in the workspace are encoded and compete for action-selection. When the specific movement is instructed during the plan phase, the left IPS encodes the task-relevant level of interference of an obstacle, and the activity in early visual areas is suppressed when interference is high. The top-down suppression of activity in early visual cortex is an efficient way of
Acknowledgements
The authors would like to thank Adam McLean and Jennifer Milne for their help with data collection and analysis. We would also like to thank Dr. Haito Yang and Jim Ladich for technical and hardware support. This work was supported by an NSERC grant to MG (Grant# 6313-2007 RGPIN) and a CIHR grant to JC (Grant# MOP 84293).
References (79)
- et al.
Intention, action planning, and decision making in parietal–frontal circuits
Neuron
(2009) - et al.
Attentional landscapes in reaching and grasping
Vision Research
(2010) - et al.
Top-down and bottom-up mechanisms in biasing competition in the human brain
Vision Research
(2009) Mechanisms of selection for the control of hand action
Trends in Cognitive Science
(1999)- et al.
Ventral and dorsal stream contributions to the online control of immediate and delayed grasping: A TMS approach
Neuropsychologia
(2009) - et al.
The reorienting system of the human brain: From environment to theory of mind
Neuron
(2008) - et al.
The role of parietal cortex in visuomotor control: What have we learned from neuroimaging?
Neuropsychologia
(2006) - et al.
Neuroimaging of cognitive functions in human parietal cortex
Current Opinion in Neurobiology
(2001) - et al.
Human parietal cortex in action
Current Opinion in Neurobiology
(2006) - et al.
Persistent activity in the prefrontal cortex during working memory
Trends in Cognitive Science
(2003)
Causal connectivity between the human anterior intraparietal area and premotor cortex during grasp
Current Biology
Attention narrows position tuning of population responses in V1
Current Biology
Separate visual pathways for perception and action
Trends in Neurosciences
Increased activity in human visual cortex during directed attention in the absence of visual stimulation
Neuron
The neural basis of biased competition in human visual cortex
Neuropsychologia
Preserved obstacle avoidance during reaching in patients with left visual neglect
Neuropsychologia
Effects of saliency, not global dominance, in patients with left parietal damage
Neuropsychologia
The assessment and analysis of handedness: The Edinburgh inventory
Neuropsychologia
Delay abolishes the obstacle avoidance deficit in unilateral optic ataxia
Neuropsychologia
Selective attention in reaching: When is an object not a distractor?
Trends in Cognitive Science
Intentional maps in posterior parietal cortex
Annual Review of Neuroscience
Functional organization of human intraparietal and frontal cortex for attending, looking, and pointing
The Journal of Neuroscience
Context-specific grasp movement representation in the macaque anterior intraparietal area
The Journal of Neuroscience
Comparing natural and constrained movements: New insights into the visuomotor control of grasping
PLoS One
Integration of target and effector information in the human brain during reach planning
Journal of Neurophysiology
Spatial and effector processing in the human parietofrontal network for reaches and saccades
Journal of Neurophysiology
Attention, intention, and priority in the parietal lobe
Annual Review of Neuroscience
Top-down control of human visual cortex by frontal and parietal cortex in anticipatory visual spatial attention
The Journal of Neuroscience
Functional magnetic resonance imaging reveals the neural substrates of arm transport and grip formation in reach-to-grasp actions in humans
The Journal of Neuroscience
Missing in action: The effect of obstacle position and size on avoidance while reaching
Experimental Brain Research
Obstacle avoidance during online corrections
Journal of Vision
Seeing all the obstacles in your way: The effect of visual feedback and visual feedback schedule on obstacle avoidance while reaching
Experimental Brain Research
Cortical mechanisms of action selection: The affordance competition hypothesis
Philosophical Transactions of the Royal Society of London B: Biological Sciences
Neural mechanisms for interacting with a world full of action choices
Annual Review of Neuroscience
Control of goal-directed and stimulus-driven attention in the brain
Nature Reviews. Neuroscience
Functional neuroimaging: Experimental design and analysis
Visually guided grasping produces fMRI activation in dorsal but not ventral stream brain areas
Experimental Brain Research
Canceling planned action: An FMRI study of countermanding saccades
Cerebral Cortex
Saccade preparation signals in the human frontal and parietal cortices
Journal of Neurophysiology
Cited by (34)
Action inhibition and affordances associated with a non-target object: An integrative review
2020, Neuroscience and Biobehavioral ReviewsCitation Excerpt :However, this interpretation of the finding has been challenged. According to the account of obstacle avoidance, the reach path veers away from the non-target in order to minimize the risk of colliding with the obstructing object (e.g., Tresilian, 1998; Schindler et al., 2004; Chapman et al., 2011). This latter explanation does not necessarily require any response inhibition related to the non-target object.
The target as an obstacle: Grasping an object at different heights
2018, Human Movement ScienceCitation Excerpt :It scales with the final distance between the digits on the to be grasped object but is influenced by other aspects as well. The MGA is for instance affected by the location of nearby obstacles (Chapman, Gallivan, Culham, & Goodale, 2011; Jackson, Jackson, & Rosicky, 1995; Mon-Williams, Tresilian, Coppard, & Carson, 2001; Rice et al., 2006; Tresilian, 1998; Tresilian, Mon-Williams, Coppard, & Carson, 2005; Voudouris, Smeets, & Brenner, 2012), target object orientation (Cicerale, Ambron, Lingnau, & Rumiati, 2014; Paulun, Gegenfurtner, Goodale, & Fleming, 2016) and target object shape (Borchers, Verheij, Smeets, & Himmelbach, 2014; Cuijpers, Smeets, & Brenner, 2004; Eloka & Franz, 2011; Hu, Eagleson, & Goodale, 1999; Verheij, Brenner, & Smeets, 2014; Zaal & Bootsma, 1993). Several of the latter authors proposed that the effect of object shape can be explained in terms of the target object being considered an obstacle at positions other than the goal positions for the digits.
Walking combined with reach-to-grasp while crossing obstacles at different distances
2018, Gait and PostureTime-resolved decoding of planned delayed and immediate prehension movements
2018, CortexCitation Excerpt :This seems to suggest transient neural representations of movement plans in parieto-frontal regions, at least at the level of multivariate movement decoding (as opposed to univariate signal amplitude). Previous studies using delayed movements (Chapman, Gallivan, Culham, & Goodale, 2011; Curtis, Rao, & D’Esposito, 2004; Gallivan et al., 2011b; Lindner et al., 2010; Toni et al., 2001) suggested that planning is a sustained neural process that begins with an instructing cue and persists throughout the entire delay until the trigger cue. We consider it likely that the nature of the planning-related activity during the delay (transient vs sustained) varies as a function of the task demands (Mauritz & Wise, 1986).
The Sander parallelogram illusion dissociates action and perception despite control for the litany of past confounds
2018, CortexCitation Excerpt :Grasp kinematics are influenced by objects that are in close proximity to either the target object or to the hand's path to the target (Mon-Williams & McIntosh, 2000; Mon-Williams, Tresilian, Coppard, & Carson, 2001; Saling, Alberts, Stelmach, & Bloedel, 1998; Tresilian, 1998; Voudouris, Smeets, & Brenner, 2012). Furthermore, neuropsychological and neuroimaging work implicate a role for the sensorimotor networks of the posterior parietal cortex, including the dorsal stream, in mediating obstacle avoidance when reaching to targets or reaching out to pick them up (Chapman, Gallivan, Culhama, & Goodale, 2011; Rice et al., 2008; Schindler et al., 2004; Striemer, Chapman, & Goodale, 2009). Importantly, obstacle avoidance mechanisms operate even when the obstacles are pictorial (e.g., Haffenden & Goodale, 2000; Jax & Rosenbaum, 2007).