Predictions to motion stimuli in human early visual cortex: Effects of motion displacement on motion predictability
Introduction
The human visual system is constantly exposed to visual motion information; objects within our visual field move, and the observer moves in his surroundings. Yet, one perceives the external world as stable and coherent. Imagine a new object moving into one's visual field. How does the visual system distinguish this novel motion information from the already present abundance of other motion signals? Recent studies suggest that the human brain treats novel or unpredictable input differently from previous detected or predictable input (Alink et al., 2010, de-Wit et al., 2010, Wacongne et al., 2012). These studies report a relative increase in neural activity for novel stimuli. In line with these findings, recent studies have linked enhanced BOLD responses during the presentation of moving random dot stimuli at the locations where novel dots entered the stimulus area (i.e., the motion trailing edge) to mechanisms of motion prediction (Maloney et al., 2014, Schellekens et al., 2014, Wang et al., 2014). Possibly, the human brain pro-actively predicts its input, rather than passively waiting for it. Predictive coding describes such a mechanism where predictions are estimated for all sensory input (Friston, 2005, Lee and Mumford, 2003, Rao and Ballard, 1999). Could it be that early visual cortex makes predictions, while encoding motion information?
In a previous study, we showed that fMRI BOLD activity obtained from early visual cortex (i.e., V1, V2, and V3) was relatively enhanced near a motion stimulus's trailing edge relative to the leading edge (Schellekens et al., 2014). We were able to show that these effects were not likely caused by classical receptive field effects, biases in motion direction sensitivity, flexible retinotopy, or visuo-spatial attention. Instead, predictive coding offered the most plausible explanation for the differences in BOLD activity. However, it remains unclear if early visual cortex actually creates predictions for all visual input, or if predictive coding is effectuated by simpler mechanisms (Srinivasan et al., 1982). The current study investigates to what extent fMRI BOLD responses reflect estimated predictions to moving random dot stimuli in early visual cortex.
During the presentation of moving random dot patterns, there are several factors that likely account for the measured BOLD signals. Firstly, classical receptive field (RF) effects could still contribute to enhanced BOLD signals directly near the motion trailing edge (Maloney et al., 2014). At the trailing edge, dots might appear in the middle of a neuron's receptive field (RF), which might cause strong transient responses (Borg-Graham et al., 1998, Gieselmann and Thiele, 2008). Moving dots more distant from the trailing edge gradually pass over neurons' RFs, which might not cause strong transient responses. Consequently, BOLD signal enhancements directly near the trailing edge can be explained by a model of classical RF effects (Fig. 1A). Secondly, a predictive coding effect is expected to cause a gradual signal decrease along motion trajectories. Predictive coding describes the estimation of input predictions used to inhibit the effective input signal, which ideally causes neurons to only signal the prediction error (Den Ouden et al., 2010, Egner et al., 2010, Rao and Ballard, 1999). For randomly positioned dots, the prediction error would be largest near the trailing edge where individual dots have not been previously detected, resulting in large BOLD signals. However, the prediction error and therefore the BOLD signal would gradually decrease as neurons code for portions of the stimulus that are further away from the trailing edge (Abler et al., 2006, Schellekens et al., 2014). Variance in the BOLD signal could therefore be accounted for by a linear prediction model where the amplitude of the BOLD signal is inversely correlated with the distance from trailing edge (Fig. 1B). Thirdly, the integration of prior visual input in order to form input predictions might have a particular effect on the BOLD amplitude. Information integration could be facilitated by the retinotopic structure of early visual cortex (Hubel and Wiesel, 1968, Wandell and Winawer, 2011): adjacent locations in visual space are represented by adjacent locations on the cortex. Predictions could, thus, be created through short-distance neuronal interactions, possibly through horizontal corticocortical connections (Angelucci et al., 2002, Gilbert and Wiesel, 1989). However, such scenario has an interesting consequence, for the cortical representation of the visual field is not fully continuous. Left and right visual field maps are represented in right and left hemispheres, respectively. Additionally, extrastriate visual areas V2 and V3 are divided in ventral and dorsal parts, representing upper and lower visual fields. These separations might prohibit horizontal connections to bridge the horizontal and vertical meridians. The location of cortical boundaries could therefore contribute to the changes in the BOLD signal (Fig. 1C).
Varying displacement characteristics of moving dots could expose how predictive coding is realized in visual cortex. The displacement of dots is characterized by motion direction, duration, and velocity. The motion direction determines where the trailing edge resides and where neural activity is largest due to classical RF effects and large prediction errors. Additionally, a prediction-based mechanism would be sensitive to the actual displacement of dots within motion stimuli. Longer motion durations allow the visual system to increase prediction accuracy, while the spatial extent of estimated motion predictions would scale with motion velocity. Therefore, different motion displacement parameters would affect the slope of the signal decrease for prediction-based mechanisms. When prediction accuracy increases, prediction errors and therefore the BOLD amplitudes decrease. However, prediction errors are expected to remain large directly at the trailing edge, where novel randomly positioned dots keep appearing. Thus, an increase in prediction accuracy would result in a steeper signal decrease with increasing distance from trailing edge. The opposite would be true for a decrease in prediction accuracy.
In the current study, we investigated previously reported novelty effects for moving random dot stimuli with respect to predictive coding type mechanisms. We hypothesize that the human visual system estimates predictions to encode motion stimuli. Accordingly, we expect BOLD amplitudes to decrease as a function of distance from the trailing edge. We additionally expect classical RF effects and horizontal connections to contribute to changes in the BOLD signal (Fig. 1D). Furthermore, differences in motion displacement should alter the slope of the BOLD signal decrease with increasing distance from the trailing edge, if human early visual cortex actually predicts the translation of individual dots.
Section snippets
Subjects
Twelve healthy volunteers (mean age = 23, female = 5) were recruited from the Utrecht University. All participants gave written informed consent before entering the study. The protocol was approved by the local ethics committee of the University Medical Center Utrecht, in accordance with the Declaration of Helsinki (World Medical Association, 2013).
Scan protocol
Scanning was performed on a 7 Tesla Philips Achieva scanner (Philips Healthcare, Best, Netherlands) with a 32-channel receive headcoil (Nova Medical,
Signal components
We showed moving random dot patterns with different motion durations and velocities to investigate underlying mechanisms of elevated BOLD responses at a motion stimulus' trailing edge. We selected 3 components, which likely contribute to the amplitude of the BOLD response, when viewing a moving random dot pattern: classical receptive field effects (RF), linear prediction (LP), and cortical boundaries (CB). Visual inspection of the BOLD amplitudes averaged across all conditions and corrected for
Discussion
In this study, we conducted an in-depth investigation on previously reported BOLD signal enhancements near trailing edges of motion stimuli. We hypothesized that human early visual cortex (i.e., V1, V2, and V3) creates input predictions to process visual motion, plausibly through horizontal connections. We additionally expected contributions of classical receptive field (RF) effects to the BOLD signal directly at the motion trailing edge. The results show that BOLD signals in early visual
Acknowledgements
This work was supported by a grant from the Dutch Organization for Scientific Research (NWO VENI 863.09.008).
References (48)
- et al.
Prediction error as a linear function of reward probability is coded in human nucleus accumbens
NeuroImage
(2006) - et al.
Motion and position coding
Vis. Res.
(2007) - et al.
Canonical microcircuits for predictive coding
Neuron
(2012) - et al.
Principles of visual motion detection
Trends Neurosci.
(1989) - et al.
A common network of functional areas for attention and eye movements
Neuron
(1998) - et al.
Cascaded effects of spatial adaptation in the early visual system
Neuron
(2014) - et al.
Separate modulations of human V1 associated with spatial attention and task structure
Neuron
(2006) - et al.
Lower thresholds of motion for gratings as a function of eccentricity and contrast
Vis. Res.
(1985) - et al.
Adaptation: from single cells to BOLD signals
Trends Neurosci.
(2006) - et al.
Determinants of motion response anisotropies in human early visual cortex: the role of configuration and eccentricity
NeuroImage
(2014)