Using the perceptual confirmation-bias to study learning and feedback in fovea and periphery

When temporally integrating information, humans are often, but not always, biased to overweigh early evidence (“primacy effect”). We recently showed how these observations could be explained by assuming that the brain performs approximate inference in a hierarchical model in which expectations influence sensory inferences. Here, we use this framework to ask two related questions: (1) Does perceptual decision-making adapt to the rate at which new visual information is presented, or is its assumption fixed, learnt over long times? (2) Does the strength of feedback differ near the fovea compared to larger eccentricities, as in a recently proposed hypothesis? In a first experiment, we compared the strength of the perceptual primacy effect in two conditions which only differed in the duration for which each stimulus frame was presented. We found that the primacy effect differed when measured in physical time, but stayed the same in “frame-time”, indicating that the brain had adapted to the rate at which it received independent information. Second, we compared the strength of the primacy effect near the fovea and in the periphery, and did not find a significant difference, suggesting an equal strength of feedback near the fovea and in the near periphery.


Introduction
Empirical studies show a wide variety in how subjects weigh evidence over time to make a perceptual choice. Sometimes subjects weigh early evidence more ("primacy effect") (Nienborg & Cumming, 2009;Kiani, Hanks, & Shadlen, 2008), sometimes they weigh each piece of evidence equally (optimally) (Wyart, Gardelle, Scholl, & Summerfield, 2012;Brunton, Botvinick, & Brody, 2013), and sometimes they weigh later evidence more into their decision ("recency effect") (Drugowitsch, Wyart, Devauchelle, & Koechlin, 2016) (Figure 1a). In a previous study (Lange, Chattoraj, Beck, Yates, & Haefner, 2018) we could explain these apparently conflicting results by making two contributions. First, given the fact that the brain's decisions are always based on intermediate sensory inferences rather than the sensory observations (e.g. on the retina) directly, we showed that this implied a partitioning of total information available in the stimulus into "sensory information" (the information in each observation about the sensory inference) and "category information" (the information in each inference is about the correct choice). The different relative proportions of sensory and category information were predictive of the experimental findings, with primacy effects (decreasing psychophysical kernels, or PKs) occuring for low sensory and high category information. Second, we showed that approximate inference in a hierarchical generative model predicts these different temporal biases depending on relative sensory and category information (Figure 1c and stimulus in 1d). For our present study, we concentrate on the regime of low sensory information and high category information. In this regime, our model makes testable predictions for how the strength of the primacy effect should depend on (1) the assumptions of the brain about the rate at which it receives independent information, and (2) on the strength of the feedback connections with which prior expectations influence sensory inferences in our model.

Study 1: Does the brain adapt its assumptions about the temporal correlations in the stimulus to the stimuli in different tasks?
On a given trial, if the evidence in frame f is e f and the (correct) categorical identity of the stimulus is a binary variable C ∈ {−1, +1}, then evidence in favor of C = +1 after F independent frames is: p(e f |C = +1). In the case of the brain, a decision-making area computing a belief about the correct choice only has access to a sensory representation of the stimulus, which we call x, not to the outside stimulus e directly (Figure 1b and c). As in the Sequential Probability Ratio Test (Gold & Shadlen, 2007), we assume the brain approximately computes beliefs about the correct choice as whereLLO f is the decision-making area's estimate of the log-likelihood ratio implied by evidence frame f , using the 126 This work is licensed under the Creative Commons Attribution 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/3.0 approximate sensory representation x (for details see (Lange et al., 2018)). Presumably the decision-making area updates its belief about C continuously, reflecting the rate of the brain's underlying computations, and without any knowledge about experimenter-controlled frame-by-frame changes in the externally presented stimulus. We therefore introduced a parameter n U for the number of online belief updates by the decision-making area per stimulus frame. As a result we have to amortize the per-frame updates over n U steps, updating n U times per frame; i.e.
This amortization coefficient 1/n U reflects a fundamental complication in applying the sequential ratio test (SRT) to continuous inputs from the real world: while the SRT is the correct solution for truly independent pieces of evidence, stimuli in the outside world are usually correlated over time, whether they are consecutive views of the same scene, or consecutive newspaper articles on the same topic. In agreement with intuition, our model of approximate hierarchical inference predicts that the stronger the assumption that external stimuli (in our case the images on the screen) are independent pieces of information, the stronger the primacy effect, and vice versa. In the context of our modeled experiment, a stronger independence assumption means that the brain assumes a smaller number of sensory updates n U per independent piece of external evidence. Optimal performance requires that it learns the correct n U . Whether the brain can do so in a context-dependent during perceptual decision-making, or whether it uses an approximate n U learnt during natural vision, is the question addressed by our first experiment.

Visual Discrimination Task
Following (Lange et al., 2018), the stimulus in our task consisted of ten visual frames. Each frame consisted of band-pass-filtered noise with excess orientation power either in the −45deg or the +45deg orientation (Nienborg & Cumming, 2014) (Figure 1d). Here, the excess orientation power, parameterized by 0 ≤ κ < ∞ (parametrizing a von-Mises filter function in orientation space), determines the uncertainty over orientation for each frame (sensory information). The stimulus was presented as an annulus spanning 2.088 degrees around the fixation marker in order to minimize the effect of small fixational eye movements (inner circular mask spans 0.432 degrees around fixation, see Figure 1d). Subjects reached threshold performance using a 2-1 staircase decreasing κ. We compared two variations of this experiment: in one, the duration of each evidence frame was 42 msecs, and in the other case it was 166 msecs.
Findings Figure 1f-g shows our model predictions for three different stimulus frame durations (n U = 5, 10, 20) measured in how many online updates the brain performs for a single stimulus frame, e f . Figure 1f shows the case in which the brain has learnt the correct amortization factor, 1/τ = 1/n U : the strength of the primacy effect (slope of the weights as a function of independent stimulus frame) is always the same. Figure 1g shows the results assuming the brain uses the same amortization constant no matter for how long each independent frame was presented (1/τ = 1/n U ): the slope is steeper for longer than for shorter frames. Figure 2a,b,e,f,g show our empirical data from 10 subjects (9 naive). When comparing the weights as a function of the independent frame number, the average slope is the same in both condition (Figure 2a+b) as in agreement with the hypothesis that the brain has changed its amortization constant by a factor of 4 to account for the different frame durations in both cases. If, however, the brain had kept its amortization constant the same across both conditions, the weights should have declined with the same slope when measured in physical time -in clear contradiction of the data (Figure 2a and e). Finally, we note that the clear primacy effect even over the relatively short stimulus duration of 420ms (Figure 2e+f) provides additional evidence (beyond that already in (Lange et al., 2018)) against an internal integration-to-bound process as an explanation for the observed primacy effect.

Study 2: Is the strength of the feedback signals different near fovea and in the periphery?
A key driver of the primacy effect in our model and -we propose -in the brain, is the fact that sensory inferences incorporate top-down prior expectations. If this was not the case, then even under the assumption of approximate inference, our model would never predict a primacy effect and the predicted slope would be zero. Recently, it was proposed (Zhaoping & Ackermann, 2018;Zhaoping, 2017) that the strength of feedback is greater in and near the fovea than in the periphery. We tested this hypothesis by comparing the strength of the primacy effect (quantified by the slope of the weights) for two stimuli that varied in eccentricity but were matched in all other aspects relevant for the model. In particular, we scaled the area that the stimulus occupied on the screen with the cortical magnification factor (Strasburger, Rentschler, & Jüttner, 2011). We equivalently scaled the spatial frequency content of the stimulus in order to account for the dependence of the spatial frequency preferences of cortical neurons on eccentricity (Strasburger et al., 2011). These manipulations ensured that the number of cortical visual neurons processing the the stimuli in the two conditions stayed roughly the same since their number may affect the quality of the sensory representation and hence the strength of the primacy effect.

Visual Discrimination Task
We had 7 naive subjects perform the same visual discrimination task as in Study 1 above and described in (Lange et al., 2018) with the following parameters. Independent stimulus frames of duration 83ms were presented in both conditions. In the condition 1, the stimulus had the same size as before, an annulus of size 2.088 degrees around the fixation marker (inner circular mask spans 0.4352 degrees around fixation). In condition 2, the stimulus was a larger annulus spanning 3.24 degrees and the inner circular mask spans 2.088 degrees around the fixation cross (Figure 1d+e). Spatial frequency peaked at 0.1194 cycles/pixel for the small stimulus, compared to 0.0980 cycles/pixel for the large stimulus.
Findings Figure 1h shows our model predictions for the change in the strength of top-down feedback signals from decision-making area to the sensory representation. We confirm in our model simulations that an increase in feedback strength implies a stronger primacy effect. Hence, if the feedback signals near fovea are stronger than in the periphery we would expect to see a stronger primacy effect for trials with stimuli near the fovea than for trials with stimuli in the periphery. Figure 2c-d shows our data from 7 naive subjects across the two conditions indicating no significant difference between the slopes of the weighting profiles for the two types of stimuli. This suggests that the strength of the feedback signals in the periphery are similar to that near fovea at over the range of eccentricities tested by our experiment. e f . The category is denoted by C and the sensory representation in the brain is x f . c) Approximate inference model: Red lines show information flow with our assumption that a posterior over x f is represented and evidence integration happens in a decision area representing C. Strong feedback from C to x gives primacy weighting. d) Example band-passed grating stimulus where all frames contain noisy information about one category used in Study 1 and fovea stimulus for Study 2. e) Same as (d) used as periphery stimulus in Study 2. f) Model Simulations: When the amortization factor τ matches number of belief updates per frame of evidence n U , then there is no change in slope of weights (strength of primacy) with change in in n U . g) If τ = n U , then primacy strengthens with increase in number of belief updates n U . h) Stronger top-down feedback from C to x gives stronger primacy weighting. performance across subjects is approximately the same for the two conditions of Study 1. h) Same observation as in (g) but for Study 2.