Neural Network Mechanisms Underlying Confirmation Bias in Stimulus Estimation

Perception is influenced by past choices. For example, an intermittent categorical choice biases the estimation of average motion direction across two stimuli (confirmation bias). To shed light on the underlying neural mechanisms, we develop a ring attractor model that integrates stimulus direction and represents a continuous estimate of the average stimulus in the phase of an activity bump. Depending on the relative strength of sensory input compared to the intrinsic network dynamics, the model can account for qualitatively distinct decision behaviors (uniform temporal weighting, and “recency” regime). We studied two potential mechanisms underlying confirmation bias and found that they predict different modulations of the estimation curve: (i) applying an urgency signal after the first stimulus leads to a shift modulation, (ii) a feature-based attention signal that boosts stimuli that are consistent with the intermittent choice leads to a gain modulation, the main feature observed in human behavior. Our work suggests bump attractor dynamics together with feature-based attention as a potential underlying mechanism of confirmation bias in stimulus estimation tasks.


Introduction
Perceptual decision making often involves making categorical judgments based on estimations of continuous stimulus features. It has recently been shown that committing to a categorical choice biases a subsequent report of the stimulus estimate by selectively increasing the weighting of choiceconsistent evidence (Talluri, Urai, Tsetsos, Usher, & Donner, 2018). This phenomenon is known as confirmation bias. The underlying neural mechanisms remain poorly understood. Here, we developed a computational network model that can integrate a continuous stimulus feature such as motion direction and can also account for a subsequent categorical choice. We then studied potential mechanisms underlying confirmation bias and found that including feature-based attention in the model can explain the experimentally observed bias in stimulus estimation.

Methods
Ring Model. The dynamics of the model are described in terms of the firing rate, r (θ,t), of a neural population arranged in a ring, θ ∈ [−π, π), obeying the following equation where τ = 20 ms is the neurons' time constant, and Φ is the current-to-rate transfer function. The synaptic input consists of a recurrent current due to the presynaptic activity at a location θ with a weight w (θ − θ ), and an external current I (θ,t). Both functions, I and w are written in terms of their Fourier coefficients, I k and w k (k = 0, 1, 2, . . . ), respectively. The homogeneous external input is I 0 = 2, and the connectivity function w is given by w 0 = −2, w 1 = 1 and w 2 = 0.5. We simulated Eq. (1) following the Euler scheme with a time step ∆t = 0.5 ms, and discretizing the space θ into N = 200 evenly distributed angular locations. Stimulus inputs I stim = I 1 · G (x, θ * ) + n (θ,t) are defined as the combination of a circular Gaussian function G with average direction θ * , and noisy fluctuations n (θ,t). The stimulus amplitude is I 1 = 0.1 and its width is σ stim = 10 • . Noisy inputs are modeled as independent Ornstein-Uhlenbeck processes for each neuron (τ OU = 1 ms, σ OU = 0.4). The urgency signal of Fig. 2B is modeled as an external input that combines two Gaussians of amplitude 0.25, mean ±90 • and standard deviation of 10 • . Finally, the attentional modulation of Fig.   2C is modeled by multiplying the second stimulus by a fac- Dynamics of the bump attractor. For stationary homogeneous inputs, Eq. (1) has either an homogenous or a localized solution depending on the spatial profile of the connectivity function w (Ben-Yishai, Bar-Or, & Sompolinsky, 1995).
Near the bifurcation, the dynamics of the bump reduce to the normal form of a supercritical Turing bifurcation, and can be expressed in terms of the amplitude, R, and phase, Ψ, of the bump as where c is a constant that depends on the particular form of the connectivity profile, θ * is the average direction of the stimulus, and ξ 1 (t) and ξ 2 (t) are independent Ornstein-Uhlenbeck processes.

Results
Stimulus integration with bump attractor dynamics. We studied the integration of noisy stimuli with a continuous fea-473 This work is licensed under the Creative Commons Attribution 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/3.0 ture (e.g. a random dot stimulus with different net motion directions), in a neural network model based on a ring attractor network (Fig. 1). Due to strong recurrent connectivity, a bump of activity emerges in this model at a position determined by the input, and this bump state persists when the input is removed (Fig. 1B). Similar ring models have previously been studied in the context of orientation selectivity in primary visual cortex (Ben-Yishai et al., 1995) and in the context of spatial working memory (Compte, Brunel, Goldman-Rakic, & Wang, 2000). However, here we take a different perspective and interpret the position (phase) of the activity bump as the estimate of the integrated stimulus direction.
We found that the transient population response to changing stimulus input effectively integrates the stimulus as is needed in typical decision making and estimation tasks that require accumulation of evidence over time (Fig. 1B). In the example trial (Fig. 1B), the activity bump starts at 0 • (the initial position may correspond to a reference in a given task or it may be random if no prior knowledge is available). To investigate whether the stimulus-dependent drift of the bump is actually yielding a continuous estimation of accumulated sensory input, we simulated many trials with noisy input stimuli of different average motion directions (Fig. 1C-E). We found that the probability of a CW choice depends on the evidence strength (Fig. 1C), similar to the observed performance ins psychophysical experiments (Talluri et al., 2018). This psychometric curve becomes steeper for longer stimulus duration (blue and orange lines in Fig. 1C), indicating that the model is able to integrate sensory evidence over long timescales. The precision of the estimation of the stimulus average also improves with stimulus duration (Fig. 1D).
In order to directly test how the stimulus is integrated over time, we computed the model's psychophysical kernel, obtained from the difference between the average stimuli yield- ing each choice (Fig. 1E). Stimulus fluctuations impacted the choice throughout the trial, with increasing weight towards the end of the stimulus (recency effect; e.g. Cheadle et al. (2014)). This integration dynamics can be understood with the help of a reduced two-dimensional equation for the amplitude and the phase of the bump (Eqs. (2a) and (2b) in Methods). The model can account for qualitatively distinct decision behaviors, depending on the relative strength of sensory stimuli, I 1 , compared to the amplitude of the bump, R. When sensory inputs dominate over the intrinsic network dynamics, later parts of the stimulus have an higher impact on the final phase and the categorical choice than earlier parts (recency regime; Fig. 1E). On the other hand, when the internal dynamics are stronger, the temporal weighting of stimulus information is uniform (not shown). In sum, the ring model can accumulate evidence at prolonged time scales and it can reproduce the experimentally observed dependence of discrimination and estimation accuracy on stimulus direction (Talluri et al., 2018).

Neural network mechanisms underlying confirmation bias.
In order to reveal the potential neural mechanisms underlying confirmation bias, we then used the model to mimic a recent psychophysical experiment that required both a categorical choice and a continuous estimation (Talluri et al., 2018). In the experiment, subjects viewed two successive random dot motion stimuli, and they had to make a categorical choice (CW vs CCW) after the first stimulus and a continuous estimation of the average direction across both stimuli after the second stimulus. Subjects successfully integrated evidence across both stimuli but their estimations were biased reflecting selectively enhanced sensitivity for the second stimulus if it was consistent with the intermittent choice.
We tested whether two plausible mechanisms in the model can account for the observed confirmation bias: an urgency signal applied after the first stimulus and an attentional gain modulation during the second stimulus (Fig. 2). For the ease of exposition we restricted our simulations to trials where the first stimulus has 0 • average direction, and the bias effect can simply be visualized by comparing the estimation as a function of the average stimulus direction for CW vs CCW choices (Fig.  2, bottom panel). Since the first stimulus is uninformative, the intermittent choice is determined only by stimulus fluctuations and internal noise ( Fig. 2A, middle), and approximately half of the trials lead to a CW and a CCW choice, respectively. This effect of the noise explains why in the model without any additional mechanism the estimation is slightly different for CW vs. CCW choices (Fig. 2A, bottom). This difference is however much smaller than the psychophysically observed biases.
A possible way of introducing a choice-dependent bias is to include an urgency signal in the model (Fig. 2B). This urgency signal consists of an unspecific input to neurons around −90 • and 90 • in the ring (see Methods), causing a movement of the bump towards either −90 • or +90 • , depending on which one is closer to the current bump position. Thus, the bump moves further away from the decision boundary, making it easier to read out the categorical choice. Moreover, the magnitude of the displacement is almost independent of the bump phase and its direction is always consistent with the intermittent choice. This leads to a choice-dependent shift modulation of the final stimulus estimation (Fig. 2B, bottom), an effect that contributed to the confirmation bias in the psychophysical experiment.
The second way of introducing a choice-dependent bias was through including an attentional modulation of the second stimulus (Fig. 2C). The attention effect is modeled as a feature-dependent gain modulation of visual neurons (Martinez-Trujillo & Treue, 2004) that provide the input to the ring model. Importantly, the "attentional spotlight" is chosen to be consistent with the intermittent choice (Fig. 2C, top). This gain modulation leads to a boost of the second stimulus if it is consistent with the choice, and to a decrease if it is inconsistent. The boost is larger for stimuli that are further away from the 0 • reference, yielding a choice-dependent gain-modulation of the final estimation (Fig. 2C, bottom). This gain-modulation resembles the stimulus-dependent confirmation bias that was the main signature of confirmation bias in human subjects.

Discussion
We showed that bump attractor dynamics emerging in a neural network model with ring-like connectivity structure provides a mechanistic model for stimulus integration of continuous features such as motion direction. A categorical choice (e.g. CW vs. CCW relative to a reference direction) can then be obtained by "reading out" the bump position. Using this model, we studied the potential neural mechanisms underlying the recently observed choice-dependent bias that an intermittent choice has on the estimation of average motion direction across two stimuli (Talluri et al., 2018). We found that a shift modulation of the estimation curve, observed in a minority of subjects, can be explained by an urgency signal applied after the first stimulus. Gain modulation of the estimation curve, as observed in most of the subjects, can be explained by a feature-based attention signal that boosts stimuli that are consistent with the intermittent choice. In future work we plan put to test the hypothesis that feature-based attention underlies the confirmation bias in human MEG experiments.
Our novel model has several unique features: Modulation of the excitability of the network results in either recency or uniform temporal integration, a temporal weighting that has for example been observed in a category-level averaging task (Cheadle et al., 2014;Wyart, de Gardelle, Scholl, & Summerfield, 2012). This uniform or recency temporal integration is in contrast to classical attractor models of decision making that typically give more weight to the early part of the stimulus (primacy effect; Wang (2002); Wimmer et al. (2015)). Furthermore, the architecture of our model is similar to the classical model of parametric working memory (Compte et al., 2000) but operating in a regime where it slowly integrates inputs. Both models show persistent stimulus-related activity in delay periods after stimulus presentation. Bump attractor dynamics might therefore be a unifying neural mechanism underlying both working memory and decision making, and it may provide a substrate for evidence integration at prolonged timescales (Waskom & Kiani, 2018).