State Anxiety Biases Precision Estimates in Volatile Environments

In an uncertain world, we must learn the statistical regularities to make accurate predictions about upcoming events. Real-world experiences are rarely certain, and residual uncertainty may motivate growing levels of anxiety: the excessive worry over future outcomes. How alterations in processing uncertainty affect learning in individuals with high levels of trait anxiety have been previously studied, yet still little is known about how states of anxiety shape learning processes. To test this, a state anxious and control group performed a reward-based learning task in a volatile environmental setting while we recorded electrophysiological (EEG) data. By using a hierarchical Bayesian model of performance, we quantified the effect of state anxiety on Bayesian belief updating and estimates of precision-weighted prediction errors in the brain. Our results reveal that state anxiety is characterised by a lower learning rate, lower precision estimates about volatility, and higher precision estimates about beliefs concerning understanding of task probability mappings. EEG analysis provided additional evidence linking the anomalous precision estimates in state anxiety to brain regions previously described to be involved in poorer learning in this population. These findings extend prior computational work on trait anxiety, suggesting states of anxiety in healthy human participants bias computational learning mechanisms.


Introduction
Present cognitive and computational neuroscience work suggests that humans, through action and perception, create a generative model of the world that the brain inverts and uses to predict hidden states. This is thought achieved in a hierarchical fashion by utilising prediction errors (PEs): a ubiquitous learning policy integrating the difference between predictions and observed outcomes (Friston & Kiebel, 2009;Iglesias et al., 2013). Importantly, as the brain learns from several sources of uncertainty, PEs are weighted by their estimated precision (inverse variance), with recent research detailing how numerous hierarchicallystructured precision-weighted PEs might be computed when learning from volatile environments (Mathys, Daunizeau, Friston, & Stephan, 2011).
Analysis of precision, volatility, and belief estimates from a Bayesian perspective is becoming increasingly influential to provide mechanisms underlying various neuropsychiatrical conditions. Anxiety is typified by an excessive worry concerning uncertain outcomes in the future. Accordingly, studies have focused on understanding how anxiety affects learning when in volatile environments to offer a mechanistic description of anxiety-related disorders (Bishop, 2007). The latest work on this uses highly trait anxious individuals to reveal a distinct disadvantage when adapting learning rates to volatile environments in aversive settings (Browning, Behrens, Jocham, O'Reilly, & Bishop, 2015). However, it remains unclear how states of anxiety in healthy individuals impacts learning from volatile environments and consequently precision estimates and belief updating in the brain. This is vital as we may be able to discover the process of how state anxiety biases beliefs, connecting to anxiety-related disorders.
Here we tested the effects of inducing a state of anxiety on reward-learning from a volatile context. We further estimated the changes to cortical dynamics by recording electroencephalography (EEG), associating anxiety-induced neural changes to potential computational adjustments. This was achieved by combining a Bayesian learning model with a general linear model (GLM), using precision-weighted PEs as regressors. Fitting a GLM to single-trial EEG data is a particularly useful technique to show the influence of hierarchical PEs and precision quantities on evoked brain responses (Diaconescu et al., 2017).

492
This work is licensed under the Creative Commons Attribution 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/3.0

Method
Participants Forty-two healthy individuals (age 18-35; 28 females) participated in this reward-based learning study. Participants were randomly assigned to one of two groups: state anxiety (StA) and control (Cont). Participants in each group were matched in their trait anxiety scores and all were below clinical levels (as in previous computational work in anxiety: Browning et al., 2015).

Task
The experiment was adapted from an aversive learning task concerning volatility and stress (de Berker et al., 2016). It was split into four blocks, resting state 1 (R1: baseline), reward-learning task block 1 (TB1), reward-learning task block 2 (TB2), and resting state 2 (R2). Both R1 and R2 were 5 minutes of sitting with eyes open relaxing. The reward learning task was a one-armed bandit binary choice where the probability of reward associated with each of two images (complementary: p, 1-p) changed over the course of 400 trials (200 trials per task block), representing environmental volatility. Each task block comprised five reciprocal probabilistic relationships that were randomly ordered and varied in length (between 26 and 38 trials) for each participant. The possible stimulus-outcome contingencies for each block ranked from strongly biased (90/10), moderately biased (70/30), to unbiased (50/50), and repeated in mirrored relationships (10/90; 30/70) so that a total of ten were represented across the two task blocks (de Berker et al., 2016). Participants were rewarded (+5p) for correctly predicting which out of the two images was rewarding (incorrect prediction = 0p) across the 400 trials. Participants were instructed to modify their predictions according to any inferred change in the probability of reward throughout the experiment.

State Anxiety A state of anxiety was induced in the
StA group just before beginning the reward learning task. Participants were told they had been randomly selected to conduct a secondary task where they will be required to perform public speaking. They were to present a piece of abstract art to a panel of three academic experts directly after completing the reward learning task. This threat of public speaking was then revoked after finishing TB2.

EEG
and Electrocardiogram (ECG) Recordings: Both heart rate (ECG) and brain responses (EEG: 64 channels) were recorded continuously throughout the experiment using the BioSemi ActiveTwo system. Heart rate variability (HRV) was calculated for each block baselined to R1. State anxiety scores were computed four times throughout the experiment.

Model Space
We used four computational models of learning, two hierarchical Bayesian models (HGF with three and two levels), a Rescorla Wagner (RW) model and a Sutton K1 (SK1) model. Models were then compared at the group level for fit using random effects Bayesian model selection (BMS). The log-model evidence (LME) from both Bayesian models was combined to get the log-family evidence (LFE) and was compared to the family of reinforcement learning models (RW & SK1) to assess which provided more evidence. BMS provided model frequencies and exceedance probabilities reflecting how optimal each model performed (Soch, Haynes, & Allefeld, 2016).
Statistics All statistical tests were computed using permutation testing and controlled for multiple comparisons using the false discovery rate (FDR) with adaptive linear step-up procedures (Benjamini, Krieger, & Yekutieli, 2006). We present the mean and standard error of the mean (SEM) alongside non-parametric effect sizes and bootstrapped confidence intervals.

Results
HRV Using a 2 x 3 (Group: StA, Cont; Block: TB1, TB2, R2) non-parametric permutation-based factorial statistical test on HRV showed no significant interaction effect but significantly lower HRV in the StA group compared to Cont in TB1 (PFDR < 0.05), and a within group significant drop from baseline to TB1, and significant increase to R2 in StA (PFDR < 0.05). These results confirmed physiological changes corresponding to an anxious state (Chalmers, Quintana, Abbott, & Kemp, 2014).

Model-free Behavioural Analysis
We found no between-group reaction time or error rate differences, but a trend level increase (p = 0.065) in the standard deviation of group error rates in StA compared to Cont, suggesting state anxiety influences the reliability of successful learning.

Computational Modelling BMS supported that
there was stronger evidence for the 3-level HGF above all other models. Trial-wise HGF parameters were averaged into 4 bins of ~100 trials. Between-group statistical analysis of relevant model parameters across the 4 bins demonstrated the StA group exhibited significantly lower learning rates across time in comparison to the Cont group (PFDR < 0.05, Figure 1). Also, the variance (inverse precision) or uncertainty of each level (2 & 3) was significantly different between groups. Uncertainty concerning beliefs about estimations of probabilistic contingencies (2) was significantly lower in StA across all 4 bins compared to Cont (PFDR < 0.05, Figure 2). This outcome suggested StA participants overestimate their ability to infer the probabilistic stimulus-outcomes relationships (higher precision). This pattern was then reversed regarding uncertainty about volatility estimates, 3, with significantly higher values in the StA group across all 4 bins compared to Cont (PFDR < 0.05, Figure 3). This demonstrates StA exhibit greater uncertainty about volatility in the task environment. The EEG analysis used precision-weighted PEs for belief and volatility estimates as regressors in a GLM revealing specific spatiotemporal patterns linked to state anxiety and its influence on the learning process under a Bayesian perspective.

Conclusion
The results of our study present evidence that state anxiety leads to more uncertain estimates about environmental volatility, yet more precise belief estimates about the probabilistic mapping between stimuli when compared to controls. These together also consistently inform a lower learning rate. Our results are important as they demonstrate how states of anxiety in healthy individuals can bias computational estimates involved in Bayesian belief updating. This may provide a mechanistic explanation of how states of anxiety bias beliefs over time to fit a profile of anxiety resembling high trait or clinical levels.