Attentional orienting relies on Bayesian estimates of expected and unexpected uncertainty

Computational modelling work proposes that the attentional system relies on Bayesian estimates of two forms of uncertainty. Expected uncertainty tracks the unreliability of predictive relationships within a familiar context. Unexpected uncertainty signals sudden changes of the environmental context. In the current study, we empirically dissociated expected and unexpected uncertainty in a spatial orienting paradigm. Furthermore, we probed the postulated link between these two forms of uncertainty and neuromodulatory brainstem responses using two measures of phasic pupil dilation: pupil diameter and its temporal derivative. Expected and unexpected uncertainty levels in the task were estimated using an approximate Bayesian learning algorithm. Uncertainty influenced attentional orienting on the behavioural level. Attentional efficiency decreased with increasing levels of unexpected and expected uncertainty. Pupil diameter and its temporal derivative differently fluctuated with expected and uncertainty, thus supporting the links between computational estimates of uncertainty and neuromodulatory systems.


Introduction
Attentional orienting can be considered as probabilistic inference about which spatial locations are likely to be relevant in the near future. Bayesian principles can be applied to estimate uncertainty of such beliefs. Computational modelling work proposes that the attentional system relies on two forms of uncertainty estimates, which are linked to distinct neuromodulatory brainstem systems (Yu & Dayan, 2005). Expected uncertainty, associated with acetylcholine (ACh) levels, tracks the unreliability of predictive relationships within a familiar context. Unexpected uncertainty, likely triggered by noradrenaline (NE) release, signals sudden changes of the environmental context.
Several previous studies that investigated the role of uncertainty in attentional orienting provided evidence that the attentional efficiency depends on the probabilistic context provided by cues (i.e. the proportion of valid vs. invalid trials; see e.g. Doricchi, Macci, Silvetti, & Macaluso, 2010;Vossel et al., 2014). Yet, these studies only considered the role of expected uncertainty. In the current study, we addressed the empirical dissociation between expected and unexpected uncertainty in the attentional system and investigated behavioural responses under these two forms of uncertainty. We expected that the validity effect (VE; difference between invalidly and validly cue trials), considered as an index of attentional efficiency, will decrease under high expected and unexpected uncertainty levels.
Given the theoretical proposal that the two forms of uncertainty are related to distinct neuromodulatory systems (Yu & Dayan, 2005), we probed the link between uncertainty estimates and neuromodulatory brainstem responses. We capitalised on the fact that the latter are reflected by measures of phasic pupil dilation -the pupil diameter and its temporal derivative. Recent evidence showed links between pupil diameter and ACh on the one hand; and between pupil derivative and NE levels on the other (Reimer et al., 2016). Based on this, we hypothesised that pupil diameter following 689 This work is licensed under the Creative Commons Attribution 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/3.0 the target presentation would mostly be sensitive to expected uncertainty, while pupil derivative would fluctuate with levels of unexpected uncertainty. We also tested for interaction effects between two uncertainty estimates. The computational model predicts stronger effects of unexpected uncertainty under low expected uncertainty.

Experimental task
Participants (N=24) performed a spatial cueing task (see Fig. 1), in which they were asked to covertly orient their attention in response to centrally presented spatial cues (black and white arrows), and to discriminate a target (i.e., low-contrast Gabor grating) presented in the left or the right visual field (LVF/RVF). Each trial started with the presentation of two cues (white and black arrows). One of the cues was predictive of the spatial location (LVF/RVF) of the forthcoming grating with a certain probability (i.e., cue validity 70% or 85%). The other cue predicted the forthcoming stimulus location at chance level (50% validity). For efficient attentional orienting and a detection of the low-contrast grating, participants were asked to infer which of the two cues correctly predicted the target location. Cue identity remained unchanged during a series of trials. The number of trials within a series was determined by an exponential distribution with min. 40 and max. 120 trials (average of 80 trials); thus, the exact number of trials in the series was variable. After a variable series of trials, a cue switch was introduced, inducing unexpected uncertainty (UUn). The validity of the previously relevant cue (e.g., black cue) dropped to chance level, while the other cue was predictive (e.g., white cue). Participants were queried about the currently relevant cue 16 times during the experiment to ensure they were paying attention to the cues. Expected uncertainty (EUn) was manipulated by presenting different levels of cue validity in the 1st and 2nd half of the experiment (i.e., low expected uncertainty: 85% cue validity, high expected uncertainty: 70% cue validity, in an order counterbalanced between participants).

Computational model of attentional orienting
We simulated an approximate Bayesian learning algorithm (Yu & Dayan, 2005) which describes how EUn and UUn guide attention in a spatial cueing task with cue identity switches (see Fig. 1). An intractable belief distribution indicating the belief in the currently relevant cue is approximated by a set of variables; namely the most likely current cue ( " ) at trial t, having current cue validity ( " ), the confidence that this cue is correct ( " ), and the number of observations in the current context ( " ). EUn refers to the estimated validity of the cue, i.e. an observed agreement or disagreement between cue and target (i.e., EUn = 1-" ). UUn reports the uncertainty that the cue, which is currently assumed EUn and UUn to be relevant, is indeed predictive of cue-target relationships (i.e., UUn = 1-" ). On trials correctly

Data analysis
Model optimization. The computational model was optimized using Nelder-Mead simplex method to increase correlation coefficient between estimated trialby-trial PE values, which combine both EUn and UUn estimates, and trial-by-trial response times (RT) values.

Statististical analyses.
Linear mixed effects (LME) approach was used to analyse RT, pupil diameter, and pupil derivative measures. In each case, LME model included model-based trial-by-trial EUn estimates, model-based trial-by-trial UUn estimates, and validity (invalid versus valid cue) as fixed effects. A random intercept for participant was included. EUn and UUn predictors were mean-centered. We also performed LME separately for valid and invalid trials with trial-bytrial EUn and UUn estimates as fixed effects, and a random intercept for participant.

Behavioral results
Inference about the relevant cue. Participants reported the currently relevant cue with 80.21% accuracy (range: 56.25 -100%). The estimate of the currently relevant cue by the approximate Bayesian learning algorithm had 81.77% (range: 62.50-100%) accuracy.

Pupil size results
Pupil diameter data. LME model with the same regressors, but on the mean pupil diameter from the 500-1200 ms interval after the target presentation showed VE, 2 (1) = −0.038, p<0.001, with larger pupil diameter on invalidly cued than on validly cued trials. Pupil diameter increased with higher EUn estimates, 2 (1) = −0.057, p = 0.054. Specifically on valid trials, an interaction between EUn and UUn, 2 (1) = 0.43, p = 0.004, showed that the increase of pupil diameter with EUn is mostly under low levels of UUn.

Discussion
We empirically dissociated expected and unexpected uncertainty during attentional orienting. We found evidence that attentional orienting relies on Bayesian estimates of these two types of uncertainty. Behavioral responses were modulated by levels of uncertainty in the environment estimated by the computational model. Validity index -considered as an index of attentional efficiency -decreased with higher levels of unexpected uncertainty. Furthermore, response times were slower with increasing levels of both expected and unexpected uncertainty on trials validly cued by the currently relevant cue. We probed the link between uncertainty estimates and neuromodulatory brainstem responses by investigating two different measures of phasic pupil dilation -the pupil diameter and its temporal derivative. We found a reliable index of attentional orienting: both the pupil diameter and the derivative were larger on invalidly cued compared to validly cued trials. Both of these measures also fluctuated with levels of uncertainty in the environment. Specifically, the pupil diameter increased with expected uncertainty. This result is consistent with previous evidence showing that pupil diameter reflects surprise (Preuschoff, 't Hart, & Einhäuser, 2011). On the other hand, the temporal derivative seemed mostly influenced by the interaction of expected and unexpected uncertainty. It increased with increasing unexpected uncertainty estimates on validly cued trials with low expected uncertainty levels. This interactive pattern is predicted by the computational model -effects of unexpected uncertainty are larger under low expected uncertainty. Taken together, these findings support the postulated links between computational estimates of uncertainty and neuromodulatory brainstem systems.