The role of intolerance of uncertainty in classical threat conditioning

Intolerance of uncertainty (IU), the tendency to find uncertainty aversive, is an important transdiagnostic dimension in mental health disorders. Over the last decade, there has been a surge of research on the role of IU in classical threat conditioning procedures, which serve as analogues to the development, treatment, and relapse of anxiety, obsessive-compulsive, and trauma-and stressor-related disorders. This review provides an overview of the existing literature on IU in classical threat conditioning procedures. The review integrates findings based on the shared or discrete parameters of uncertainty embedded within classical threat conditioning procedures. Under periods of unexpected uncertainty, where threat and safety contingencies change, high IU, over other self-reported measures of anxiety, is specifically associated with poorer threat extinction learning and retention, as well as overgeneralisation. Under periods of estimation and expected uncertainty, where the parameters of uncertainty are being learned or have been learned, such as threat acquisition training and avoidance learning, the findings are mixed for IU. These findings provide evidence that individual differences in IU play a significant role in maintaining learned fear and anxiety, particularly under volatile environments. Recommendations for future research are outlined, with discussion focusing on how parameters of uncertainty can be better defined to capture how IU is involved in the maintenance of learned fear and anxiety. Such work will be crucial for understanding the role of IU in neurobiological models of uncertainty-based maintenance of fear and anxiety and inform translational work aiming to improve the diagnosis and treatment of relevant psychopathology.


Introduction
The ability to learn and update, as well as select and adjust behaviours to threat and safety associations, is critical for wellbeing and protection against psychopathology (Carpenter et al., 2019;Pittig et al., 2018).Principles of classical threat conditioning have provided a theoretical framework for examining the development, treatment, and relapse of anxiety, obsessive-compulsive, and trauma-and stressorrelated disorders (Boschen et al., 2009;Hofmann, 2008;Jacoby and Abramowitz, 2016;McNally, 2007;Milad and Quirk, 2012;Zuj and Norrholm, 2019;Zuj et al., 2016).Based on animal and human evidence, it is well established that uncertainty (also known as ambiguity) is central to anxiety and stress (Brosschot et al., 2016;Grupe and Nitschke, 2013;Hirsch et al., 2016;Morriss et al., 2019a;Peters et al., 2017;Pulcu and Browning, 2019), and plays a fundamental role in the maintenance and recovery of threat learning (Bouton, 2002;Levy and Schiller, 2021).
However, only recently has research begun to synthesise and highlight the importance of individual differences in intolerance of uncertainty (i.e., the transdiagnostic personality and cognitive bias construct measuring the tendency to find uncertainty aversive) (Birrell et al., 2011;Carleton, 2016aCarleton, , 2016b) ) in classical threat conditioning mechanisms (Lonsdorf and Merz, 2017;San Martín et al., 2020;Tanovic et al., 2018).
Four years have passed since Lonsdorf and Merz's (2017) seminal review of individual differences in classical conditioning, which included a section on intolerance of uncertainty.Yet, in that time, the literature has grown substantially on individual differences in intolerance of uncertainty in both classical threat conditioning.However, as it currently stands, it is unclear how and in what circumstances (i.e., under different parameters of uncertainty) intolerance of uncertainty modulates classical threat conditioning mechanisms.Thus, an updated synthesis and review of the topic is warranted.Outlining how individual differences in intolerance of uncertainty is involved in threat and safety learning phenomena has many benefits.Firstly, it will allow us to further understand existing neurobiological models of uncertainty-based maintenance of fear and anxiety (Grupe and Nitschke, 2013;Peters et al., 2017).Secondly, it may inform us as to why some individuals are less responsive to current mental health treatments based on principles of threat conditioning (i.e., exposure therapy) (Lonsdorf and Merz, 2017) and provide the basis for future work aiming to develop or modify mental health treatments that are more appropriate for a particular individual or group (Fernandes et al., 2017;Insel, 2014).
The current review will integrate, outline, and discuss recent developments related to intolerance of uncertainty in classical threat conditioning and highlight avenues for future research.Firstly, for the purposes of the review, intolerance of uncertainty will be defined.Secondly, the existing literature on intolerance of uncertainty in classical threat conditioning will be separated based on different experimental procedures.Namely, we discuss intolerance of uncertainty regarding threat acquisition training, threat extinction training, threat extinction retention, reinstatement, reversal, generalisation and avoidance (see, Fig. 1).In each of the review sections, the procedures will be described, the parameters of uncertainty will be discussed (Levy and Schiller, 2021;Pulcu and Browning, 2019), and a summary of the intolerance of uncertainty findings from different read-out measures will be provided (i. e., skin conductance response, auditory-startle blink, corrugator supercilii ('frowning'), pupillometry, neural activation, anxiety or expectancy ratings, and avoidance responses).Thirdly, an integrated summary of the subsections will be presented with reference to the parameters of uncertainty that are most relevant for IU.Lastly, implications and directions for future research will be discussed.

Intolerance of uncertainty
Fear of the unknown is "an individual's propensity to experience fear caused by the perceived absence of information at any level of consciousness or point of processing" (Carleton, 2016a, p. 5).The behavioural inhibition system, which is thought to represent the neurobiological basis of fearful and anxious states (i.e.increased vigilance and arousal), is activated by an array of stimuli associated with pain, loss and unknowns (Gray and McNaughton, 2003).
A proximal measure of fear of the unknown is individual differences in self-reported Intolerance of Uncertainty (IU) (Carleton et al., 2007;Freeston et al., 1994).Based on Carleton's (2016b, p. 31) recent definition: "IU is a dispositional incapacity to endure the aversive response triggered by the perceived absence of salient, key, or sufficient information, and sustained by the associated perception of uncertainty".IU is considered a lower-order (fundamental) factor that underlies higher- order (emergent) factors related to negative affectivity such as trait anxiety and neuroticism (Carleton, 2016a(Carleton, , 2016b)).In particular, parallels have been drawn between IU and a sub component of the neuroticism construct (Carleton, 2016b), which accounts for the need for predictability or controllability (Barlow et al., 2014).
The most common Intolerance of Uncertainty Scales are the 27-item (Freeston et al., 1994) and 12-item variants (Carleton et al., 2007).Initially, the 27-item variant of the IUS was created to distinguish anxiety-related features in Generalised Anxiety Disorder (GAD) (Freeston et al., 1994).The reduced 12-item variant of the IUS is considered superior as the 12-item IUS removes high inter-item correlations and factor instability, and delineates from a unilateral scale into two subscales (Birrell et al., 2011;Carleton et al., 2007;Hong and Lee, 2015).The two subscales consist of prospective IU, which refers to the desire for predictability and active seeking of certainty, and inhibitory IU, which refers to paralysis of cognition and action in the face of uncertainty.
Self-reported IU is normally distributed across community samples and is notably higher in mental health disorders with an anxiety or negative affect component (i.e., anxiety, depression, and obsessivecompulsive disorders) (Carleton et al., 2012;Gentes and Ruscio, 2011;McEvoy et al., 2019).Given the transdiagnostic role of IU in mental health disorders and its potential promise as a target for mental health interventions (Boswell et al., 2013;Oglesby et al., 2017;Robichaud and Dugas, 2006;van der Heiden et al., 2012), understanding the neurobiological basis of IU has become critical (Shihata et al., 2016;Tanovic et al., 2018).

Threat acquisition training
Threat acquisition training is achieved by pairing a neutral stimulus (conditioned stimulus, CS+; i.e., a visual stimulus such as a coloured shape on a computer screen) with an aversive outcome (unconditioned stimulus, US; i.e., a mild electric shock or loud noise) (see, Fig. 1).With repeated pairings, a conditioned response develops to the CS+ (i.e., greater physiological responding or self-reported anxiety ratings).The reinforcement rate between the CS and US can be continuous (i.e., 100%), partial (i.e., commonly between 33 and 66%) (Lonsdorf et al., 2017), or a combination of continuous and partial (Grady et al., 2016).Typically, a control stimulus is also presented (CS-; i.e., another visual stimulus that is a different colour), which is never paired with an aversive outcome.The CS+ and CS-are directly compared to assess the extent of conditioning to the CS+, while controlling for non-associative processes such as habituation.The CS+ and CS-contingencies are commonly uninstructed (i.e., there is no explicit mention of the contingencies -'Pay attention to the shape stimuli presented'), although in some cases, the contingencies are instructed in a general (i.e., part of the contingency information is provided -'One of the shapes is paired with shock, the other is not, please pay attention to work it out') or precise way (i.e., all of the contingency information is provided -'Only the yellow shape will be paired with shock and the blue shape will not') (Atlas, 2019;Mertens et al., 2021;Mertens et al., 2018).The process of threat acquisition is thought to capture the development of a learned fear or anxiety (Mineka and Zinbarg, 2006).For example, a psychologically traumatic event can act as a significantly distressing US evoking a natural unconditioned response of intense fear and arousal, which can later result in distress triggered by cues associated with a psychologically traumatic event (e.g., loud noises; Zuj and Norrholm, 2019).
At the beginning of uninstructed threat acquisition training, the CS+ and CS-contingencies are unknown.Thus there is estimation uncertainty (also referred to as ambiguity) related to the probabilistic structure of the environment (Payzan-LeNestour and Bossaerts, 2011).However, with time and experience, the CS+ and CS-contingencies can be learned, thus decreasing estimation uncertainty, and increasing expected uncertainty or certainty (also referred to as irreducible uncertainty or risk) related to the probabilistic structure of the environment (Kobayashi and Hsu, 2017) (see, Fig. 2).During threat acquisition training, the extent to which a state of expected uncertainty or certainty is reached will vary based on the reinforcement rate (i.e., reaching a state of expected uncertainty when presentation of the US if partially reinforced; reaching a state of expected certainty when the presentation of the US if continuously reinforced).The time taken to learn the contingencies and therefore reduce uncertainty in the acquisition phase will depend on the complexity of the environment based on a number of factors such as the (1) the reinforcement rate and temporal connection between the CS+ and US (Dunsmoor et al., 2007;Grady et al., 2016;Knight et al., 2004), (2) the type of CS's and US's (Lipp, 2006;Lonsdorf et al., 2017), and (3) contingency instructions (Mertens et al., 2018).Particularly, higher reinforcement rates (e.g., 100% vs. 50%), easier to discriminate CSs (e.g., geometric shapes vs. subtle Gabor patches), and explicit instructions regarding CS-US contingencies promote faster contingency learning and higher rates of contingency awareness (e.g., Mertens et al., 2021), which in turn reduces estimation uncertainty.This is presumably so because conditioning supports organisms' learning to predict their environment and this crucially relies on attention towards the contingencies (Mackintosh, 1975;Mitchell et al., 2009;Rescorla, 1988).
A few studies have found IU to modulate skin conductance (Kanen et al., 2020) and auditory startle blink during acquisition training (Chin et al., 2016;Sjouwerman et al., 2020).Kanen et al. (2020) found that higher IU, over trait anxiety, was associated with poorer discrimination between the two CS+'s and CS-under 37.5% reinforcement.Chin et al. (2016) found that higher IU, over trait anxiety, was associated with greater auditory startle blink to the CS+ during 50% vs. 75% reinforcement.In addition, Sjouwerman et al. (2020) found that higher IU, over trait anxiety and neuroticism, was associated with greater auditory startle blink to the CS-under 100% reinforcement.However, Mertens and Morriss (2021) reported no relationship between IU and auditory startle blink under 75% reinforcement.The difference in findings across the three studies may be due to the predictability of the startle probe.Notably, the two studies that found relationships between IU and auditory-startle blink used unpredictable startle probes (i.e., a startle probe was only presented on some of trials and was presented at varying times in the trials).The majority of studies have found no effects of IU on measures of skin conductance, pupil dilation, neural activity or selfreport ratings of anxiety or expectancy during acquisition training with varying levels of reinforcement (50-100%) (Mertens and Morriss, 2021;Morriss, 2019;Morriss et al., 2015Morriss et al., , 2016a;;Morriss et al., 2019b;Morriss et al., 2019c;Morriss and van Reekum, 2019;Morriss et al., 2020b;Sjouwerman et al., 2020;Wake et al., 2021;Wake et al., 2020).Due to IU not being the central focus of previous work, many studies have recorded IU and different readout measures during threat acquisition training but have not reported the necessary statistics (Dunsmoor et al., 2015;Flores et al., 2018Flores et al., , 2020;;Lommen et al., 2010;Lucas et al., 2018;San Martín et al., 2020;Vervliet and Indekeu, 2015;Xia et al., 2017;Zuj et al., 2020).For instance, these studies have not included correlations between self-reported IU and differential responses to the CS+ and CS-or not included IU in factorial models such as ANCOVAs or MLMs.
Given the mixed findings it is difficult to conclude what the role of IU is in threat acquisition.It can be speculated that IU-related effects do not occur in threat acquisition because the level of uncertainty is relatively low, as the probabilistic structure of the environment is simple and learned quickly.IU-related effects may only occur during threat acquisition when the level of uncertainty is relatively high.For example, when the probabilistic structure of the environment includes more layers of uncertainty (i.e., partial reinforcement rate, more than two CS's, additional stimuli such as startle probes).Based on the lack of data examining IU and threat acquisition across time, it is difficult to assess the extent to which IU is related to periods of estimation (i.e., when contingencies are being learned) or expected uncertainty (i.e., when contingencies have been learned) during the acquisition phase.

Threat extinction training
Threat extinction training is achieved by presenting the CS+ without the US (see, Fig. 1).With repeated omission of the US, the conditioned response diminishes to the CS+ (i.e., less physiological responding or self-reported anxiety ratings).Eventually, responding to the CS+ is comparable to the CS-, indicative of successful threat extinction.Thus, during threat extinction training two associations compete for expression (i.e., the old CS+ threat association and the new CS+ safety association).In the laboratory, threat extinction training usually consists of one session and follows threat acquisition training immediately or after a period of time (Lonsdorf et al., 2017).During threat extinction training, the CS+ and CS-contingencies are typically uninstructed, although in some cases, the CS+ and CS-contingencies are instructed (Luck and Lipp, 2016).The process of threat extinction is thought to reflect the attenuation of learned fear or anxiety either naturally or via prolonged exposure-based therapies.Impaired threat extinction is considered to be a core transdiagnostic feature of pathological fear or anxiety contributing to ongoing symptom expression (Craske et al., 2008;Milad and Quirk, 2012;Pitman et al., 2012).
At the start of uninstructed threat extinction training, the change in contingency for the CS+ is unknown.Thus there is estimation and unexpected uncertainty related to the probabilistic structure of the environment (Angela and Dayan, 2005;Behrens et al., 2007;Browning et al., 2015;Gagne et al., 2020).Again, with time and experience, the new CS+ contingency can be learned, thus decreasing estimation/unexpected uncertainty, and increasing expected certainty (i.e., the CS+ no longer predicts the US) related to the probabilistic structure of the environment (Kobayashi and Hsu, 2017;Payzan-LeNestour and Bossaerts, 2011) (see, Fig. 2).The time taken to realise the change in CS+ contingency and therefore reduce uncertainty in the threat extinction training phase, will vary based on the reinforcement rate previously used in the acquisition training phase (i.e., slower rates of threat extinction are observed after partial, compared to continuous reinforcement (Chan and Harris, 2019;Grady et al., 2016;Leonard, 1975)), the type and number of CS's and US's (Lonsdorf et al., 2017) and contingency instructions (i.e., threat extinction is observed rapidly with instruction) (Luck and Lipp, 2016).
The findings above suggest that IU plays a critical role in modulating threat extinction training, particularly across psychophysiological and neural readout measures.At the start of threat extinction training, the level of uncertainty is high, as the probabilistic structure of the environment includes both estimation and unexpected uncertainty.The extent to which estimation, expected and unexpected uncertainty operate in the threat extinction training phase may vary depending on IU.For example, in individuals with high IU, relative to low IU, the absence of information regarding the change in contingency of the CS+ and omission of the US may heighten and prolong the perception that the environment is uncertain and volatile, and therefore lead to longer states of unexpected uncertainty.Indeed, individuals with high IU show successful threat extinction, indexed via skin conductance response when information about the CS+ and CS-contingencies are provided before the start of threat extinction training (Morriss and van Reekum, 2019).Such findings suggest that individuals with high IU, relative to low IU require more information during threat extinction training to alleviate perceptions of uncertainty and volatility and to enter a state of expected uncertainty.Fig. 2. Shared and discrete parameters of uncertainty embedded within classical threat conditioning procedures.Estimation uncertainty, also referred to as ambiguity, occurs when attempting to learn the probabilistic structure of the environment.Expected uncertainty/certainty, also referred to as irreducible uncertainty, arises when the probabilistic structure of the environment has been learned and is stable (i.e., no new evidence is available to decrease uncertainty or increase certainty).Unexpected uncertainty arises when the probabilistic structure of the environment unknowingly changes.In classical threat conditioning procedures with initial learning, the parameters of estimation and expected uncertainty/certainty are present (A).In classical threat conditioning procedures where the contingencies change, the parameters of estimation, expected uncertainty/certainty, and unexpected uncertainty are present (B).

Threat extinction retention
Threat extinction retention is a return of threat manipulation (Lonsdorf and Merz, 2017).In terms of procedure, threat extinction retention is near identical to threat extinction training (see Section 4), except that a threat extinction retention session usually follows threat acquisition and extinction training after a temporal delay (i.e.hours, days or weeks) (Lonsdorf et al., 2017) (see, Fig. 1).At the beginning of threat extinction retention, conditioned responses to the CS+ return, relative to the CS-(i.e., also referred to as spontaneous recovery), and tend to dissipate rapidly (Bouton, 2002).Similar to threat extinction training, in threat extinction retention, two associations compete for expression (i.e., the old CS+ threat association and the new CS+ safety association).Poorer threat extinction retention (or return of fear or anxiety) is generally considered to be an indicator of low response to prolonged exposure-based treatments via impaired extinction memory consolidation mechanisms (Bouton, 2002;Carpenter et al., 2019;Zuj et al., 2016).
At the start of threat extinction retention, the exact contingencies are not known.Although from prior experience (i.e., previous threat acquisition and extinction training) the contingencies may be estimated more easily.Therefore, the threat extinction retention phase includes elements of estimation, unexpected and expected uncertainty related to the probabilistic structure of the environment (Angela and Dayan, 2005;Behrens et al., 2007;Browning et al., 2015;Gagne et al., 2020).Because of prior experience, the CS+ and CS-contingencies can be identified relatively quickly, thus decreasing estimation/unexpected uncertainty, and increasing expected certainty related to the probabilistic structure of the environment (Kobayashi and Hsu, 2017; Payzan-LeNestour and Bossaerts, 2011) (see Fig. 2).
A handful of studies have reported the relationship between IU and skin conductance during threat extinction retention (Dunsmoor et al., 2015;Kanen et al., 2020;Morriss et al., 2020aMorriss et al., , 2020b;;Wake et al., 2021).Dunsmoor et al. (2015) and Wake et al. (2021) found that higher IU is associated with greater skin conductance responding to the CS+ versus CS-at the start of the threat extinction retention phase indicating a stronger return of learned threat.Morriss et al. (2020aMorriss et al. ( , 2020b) ) found a similar pattern, whereby higher inhibitory IU (i.e., a subscale related to paralysis under uncertainty) over trait anxiety specifically predicted greater skin conductance responding, but not pupil dilation, to the CS+ versus CS-at the beginning of the threat extinction training phase.Kanen et al. (2020), reported no such relationship between IU and skin conductance during threat extinction retention.Interestingly, threat extinction retention can be promoted in individuals with high IU or inhibitory IU by: (1) pairing a novel stimulus with the CS+ the day before (Dunsmoor et al., 2015), (2) presenting an extended extinction training session the day before (Morriss et al., 2020a(Morriss et al., , 2020b)), and (3) providing an acute dietary tryptophan depletion to lower serotonin temporarily on the same day (Kanen et al., 2020).
While there are only a few studies that have examined IU and threat extinction retention, the results appear to be relatively consistent for skin conductance and demonstrate that IU likely plays a key role in the modulation of threat extinction retention.As in threat extinction training, the level of uncertainty is high at the beginning of the threat extinction retention phase because the probabilistic structure of the environment includes estimation, unexpected and expected uncertainty.Again, the extent to which estimation, expected and unexpected uncertainty operate in the threat extinction retention phase may vary depending on IU.Tentatively, the results above suggest that even with prior experience of the contingencies, individuals with high IU, relative to low IU may find the beginning of the threat extinction retention phase distressing as there is still an absence of information regarding the contingency of the CS+ and omission of the US.

Reinstatement
Reinstatement is a return of threat manipulation (Haaker et al., 2014).The reinstatement procedure generally occurs immediately following threat extinction training and involves one or several unsignaled US deliveries without the CS+ (see, Fig. 1).As a result, the conditioned response to the CS+ can be 'reinstated' (Norrholm et al., 2006), and sometimes also generalised to the CS- (Kull et al., 2012;Zuj et al., 2018).Reinstated threat is thought to partly explain the relapse of fear or anxiety following successful treatment (Bouton, 2002).For example, in clinical situations, reinstatement of severe anxiety can occur due to an unsignaled panic attack or other episode that produces similar physiological sensations and emotional experiences previously associated with a distressing trigger.
By the reinstatement phase, participants have typically experienced two types of CS-US contingency, as outlined above.Specifically, participants have learned that (1) the CS+ reliably predicts the aversive US during threat acquisition training while the CS-does not, and (2) the CS+ no longer predicts the aversive US during extinction training, with the CS-continuing to signal the absence of threat.During reinstatement, however, associations between the CS+, CS-and the US are thrown into question as the US is delivered in the absence of the CS+ (or CS-), creating significant uncertainty regarding the CS-US relationship (Haaker et al., 2014).Like the threat extinction retention phase, the reinstatement phase includes elements of estimation, unexpected uncertainty and expected certainty related to the probabilistic structure of the environment (Angela and Dayan, 2005;Behrens et al., 2007;Browning et al., 2015;Gagne et al., 2020) (see, Fig. 2).However, due to prior experience, the CS+ and CS-contingencies can be identified relatively quickly, thus decreasing estimation/unexpected uncertainty, and increasing expected uncertainty related to the probabilistic structure of the environment (Kobayashi and Hsu, 2017;Payzan-LeNestour and Bossaerts, 2011).
Previous research has found that higher IU predicts greater reinstatement of the conditioned response to the CS+ vs. the CS-, indexed via skin conductance (Dunsmoor et al., 2015;Lucas et al., 2018).Notably, however, the reinstatement phase used by Dunsmoor et al. (2015) occurred after an extinction retention phase rather than after an extinction training phase, potentially confounding reinstatement effects with extinction retention effects.Further investigations with rigorous reinstatement testing are needed to understand the relationship between IU and reinstatement.Therefore, evidence on the role of IU in threat reinstatement is currently lacking but holds promise due to the uncertainty in the CS-US relationship that is stimulated by unsignaled US exposure (i.e., unexpected uncertainty).

Reversal
Reversal immediately follows threat acquisition training and involves changing the pairings between the CS's and the US.During the reversal phase, the former CS-is paired with the US, whereas the former CS+ is no longer paired with the US (Morris and Dolan, 2004;Schiller and Delgado, 2010) (see, Fig. 1).Across the reversal phase, the conditioned responses to the CS+ and CS-typically reverse.Thus, during reversal, multiple associations compete for expression (i.e., the old CS+ threat association, the new CS+ safety association, the old CS-safe association and the new CS-threat association).As in other conditioning procedures, the contingencies during the reversal procedure can be uninstructed and instructed (Atlas, 2019;Atlas and Phelps, 2018;Costa et al., 2015;Mertens et al., 2021;Mertens and Morriss, 2021).Reversal is a relatively understudied conditioning procedure.Because reversal includes both threat acquisition and extinction processes, the reversal procedure likely captures the general flexibility of learned fear or anxiety (Schiller and Delgado, 2010).Further, in trauma-exposed populations there is evidence of an impairment in the ability to reverse contingencies in instances of both threat-safety and safety-threat (Levy-Gigi et al., 2014).
Within the reversal procedure, there is uncertainty about the relationship between the CS's and US.During reversal, the contingencies are unclear (i.e., do old contingencies still apply or are there new contingencies?).Thus, the reversal procedure includes estimation, unexpected uncertainty and expected uncertainty/certainty related to the probabilistic structure of the environment (Angela and Dayan, 2005;Behrens et al., 2007;Browning et al., 2015;Gagne et al., 2020).Given prior experience and time, the newly reversed contingencies can be learned, thus decreasing estimation/unexpected uncertainty, and increasing expected uncertainty and certainty related to the probabilistic structure of the environment (Kobayashi and Hsu, 2017; Payzan-LeNestour and Bossaerts, 2011) (see, Fig. 2).As in threat acquisition training (see Section 3), the time taken to learn the contingencies and therefore reduce uncertainty in the reversal phase will depend on the complexity of the environment, particularly with regards to the reinforcement rate, number of stimuli, and contingency instructions (Mertens et al., 2018).
Only a few studies have examined whether threat and safety reversal is related to IU (Mertens and Morriss, 2021;Morriss et al., 2019b).Morriss et al. (2019aMorriss et al. ( , 2019bMorriss et al. ( , 2019c) ) found that higher IU, over trait anxiety, is associated with more pronounced updating of threat and safety contingencies as measured via skin conductance response (i.e., greater discrimination between the new threat and safe cue after reversal for high IU individuals).However, a later conceptual replication of the Morriss et al. (2019aMorriss et al. ( , 2019bMorriss et al. ( , 2019c) study found the opposite pattern.Particularly, Mertens and Morriss (2021) found that lower IU, over trait anxiety, is specifically associated with greater discrimination of threat and safety cues after reversal, indexed by skin conductance response (but not auditory startle-blink).Nonetheless, as this was not an exact replication, there were procedural differences between the two studies outlined above that may account for the divergent findings.Importantly, Morriss et al. (2019aMorriss et al. ( , 2019bMorriss et al. ( , 2019c) ) used uninstructed threat conditioning and uninstructed reversal, whereas in Mertens and Morriss (2021) contingency instructions prior to threat conditioning were manipulated and instructed reversal was used.Mertens and Morriss (2021) only found the negative relationship between IU and reversal learning when participants were instructed about the contingencies prior to conditioning.
Taken together, the reversal and IU findings tentatively suggest that under conditions with no contingency instructions, individuals with high IU more quickly learn the updated contingencies during reversal.In contrast, individuals with low IU may be more flexible under conditions with precise contingency instructions during reversal.IU likely modulates threat reversal because the level of uncertainty is high at the beginning the reversal procedure, particularly when uninstructed, because the probabilistic structure of the environment includes estimation, unexpected and expected uncertainty.

Generalisation
The generalisation procedure commonly follows threat acquisition training (but can be merged with threat acquisition training, see Bauer et al., 2020;Morriss et al., 2016aMorriss et al., , 2016b) ) and involves presenting generalisation stimuli (GS's) that resemble the CS+ (Dymond et al., 2015) (see, Fig. 1).The resemblance of the GS's to the CS+ can be situated on a perceptual scale (i.e., geometric circles that are more or less similar in diameter to a circle that was used as the CS+) (Lissek et al., 2008), a symbolic scale (i.e., pictures of stimuli within a category such as 'tools' or 'animals') (Bennett et al., 2015;Boyle et al., 2015), and many other types of scales (Dunsmoor and Murphy, 2015).For the generalisation phase, the size of the conditioned response matches the perceptual or symbolic gradient, with the largest response to the CS+, followed by the stimulus most similar (i.e., GS1) and so on (i.e., GS2, GS3, etc.).Overgeneralisation is commonly identified in clinical populations where fear or anxiety can rapidly spread to previously benign environmental cues, serving to increase and sustain fear or anxiety (Dunsmoor and Paz, 2015;Dymond et al., 2015;Lissek, 2012).
In the generalisation procedure, there is uncertainty about the relationship between the CS's, and US, as well as the GS's and US.Similarly, to all conditioning procedures, the generalisation procedure includes elements of estimation and expected uncertainty or certainty related to the probabilistic structure of the environment (Angela and Dayan, 2005).Furthermore, the generalisation procedure may also include unexpected uncertainty to some extent (Browning et al., 2015;Gagne et al., 2020), particularly if there is a shift in the type of stimuli presented from the threat acquisition training phase (CS's) to the generalisation phase (GS's) (see, Fig. 2).With experience and time, the contingencies of the CS's and GS's can be learned, thus decreasing estimation/unexpected uncertainty, and increasing expected uncertainty or certainty (i.e., depending on the reinforcement rate of the US) related to the probabilistic structure of the environment (Kobayashi and Hsu, 2017;Payzan-LeNestour and Bossaerts, 2011).As in threat acquisition training (see Section 3), the time taken to learn the contingencies and therefore reduce uncertainty in the generalisation phase will depend on a number of different factors (e.g., reinforcement rate, type of generalisation stimuli, and contingency instructions).Particularly, in order for threat to generalize, threat should first be successfully conditioned, which depends on the reinforcement rate, CS and US discriminability, and contingency instructions (see Section 3).Thereafter, threat generalisation can depend on verbal instructions (e.g., regarding the crucial dimension on which CSs differ; Vervliet et al., 2010a), the dimensional properties of the GS's (e.g., threat generalisation is easier towards a GS higher in fear intensity such as a terrified face; Dunsmoor et al., 2009), and the reinforcement rate (e.g., threat generalisation can be attenuated by prior safe pre-exposure to GSs; Vervliet et al., 2010b).
While many studies have examined the relationship between threat generalisation and anxious personality traits (Sep et al., 2019), only a few studies have investigated IU specifically (Bauer et al., 2020;Hunt et al., 2019;Morriss et al., 2016b;Nelson et al., 2015).In an uninstructed conditioning and generalisation procedure Morriss et al. (2016aMorriss et al. ( , 2016b) ) found that higher IU, over trait anxiety and worry, was related to greater generalisation to GSs as measured with skin conductance responses.In a recent series of replication studies of the study by Morriss et al. (2016aMorriss et al. ( , 2016b)), Bauer et al. (2020) investigated the relationship between IU and generalisation further.The replication studies consisted of one direct replication of Morriss et al. (2016aMorriss et al. ( , 2016b) ) and two conceptual replications in which the timing parameters and trial order were slightly adjusted, and Event Related Potentials (ERPs) were also measured.Surprisingly, the direct replication failed to confirm the original findings of Morriss et al. (2016aMorriss et al. ( , 2016b)).Still, the two conceptual replication studies found results that were more in line with the findings of Morriss et al. (2016aMorriss et al. ( , 2016b)), with high IU individuals showing more generalised responding in skin conductance (but not ERPs).The authors suggest that this may be due to delayed learning of threat cues in high IU individuals.Initially, this results in less discrimination between threat and safety cues and therefore more generalisation.However, towards the end of the acquisition phase, high IU individuals can learn the contingencies (i.e., reduce estimation and unexpected uncertainty) and therefore show less generalisation.In comparison, in an instructed generalisation procedure, higher prospective IU was related to attenuated late positive ERPs (assumed to reflect arousal and attention) towards the GS's (Nelson et al., 2015).Importantly though, participants in the Nelson et al. (2015) study were instructed about which stimulus was the CS+ and that none of the GSs would be followed by the US.Similarly, to the research on threat extinction training and reversal (see Sections 4 and 7), instructions prior to the generalisation procedure may have quickly resolved uncertainty and therefore reduced uncertainty related distress in individuals with high IU.
In sum, the exact relationship between IU and generalisation remains somewhat unclear.From the results above it can be speculated that individuals with high IU are prone to threat generalisation.However, overgeneralisation in individuals with high IU may depend on the extent of uncertainty within the environment.As shown above, verbal instructions about contingencies or prolonged experience with threat acquisition training during generalisation procedures may reduce unexpected uncertainty and thereby uncertainty-related distress in individuals with high IU.Lastly, it is worth highlighting that no studies so far have examined the relationship between IU and different types of symbolic generalisation (e.g., semantic, intensity, etc.).

Avoidance
Experimentally, avoidance paradigms begin with a threat acquisition training phase where participants learn the CS-US contingency before beginning an avoidance learning phase (a type of instrumental conditioning) (Pittig et al., 2018) (see, Fig. 1).Here, participants are generally instructed that delivery of the US can be prevented if a particular action is performed (e.g., a spacebar press) after CS onset.These methods have been validated in producing reliable avoidance responses in the presence of the CS+ compared to the CS- (Vervliet and Indekeu, 2015;Xia et al., 2017;Zuj et al., 2020).The goal of an avoidance extinction task is in the reduction of the avoidance behaviour independently of the conditioned threat association (Dymond, 2019).That is, if participants learn that avoidance is no longer useful in preventing the threatening outcome then avoidance should, theoretically, be used less, opening the conditioned threat association for extinction.Reducing the reliability (reinforcement) of avoidance (Xia et al., 2017), increasing the effort required to avoid (Meulders et al., 2016), or increasing the cost (e.g., monetary) of an avoidance response (Pittig, 2019;Rattel et al., 2017;Vervliet and Indekeu, 2015) can be useful in extinguishing avoidance behaviours.
Avoidance behaviours are a natural response to threat, however maladaptive avoidance is considered a significant barrier to effective treatment by preserving conditioned threat relationships before, during, and after prolonged exposure-based therapies (Craske et al., 2014;Dymond, 2019;Pittig et al., 2018).For example, an individual that undergoes cognitive-behavioural therapy may engage in low-cost avoidance behaviours such as carrying anti-anxiety medications during public outings just in case of an anxious episode (Vervliet and Indekeu, 2015).Such behaviours prevent the extinction of the original threat associations.
At the beginning of the avoidance learning procedure, the relationship between the CS+ and US has been established, and the behaviour needed to avoid the CS+ is typically known through instruction (i.e.pressing a button) (Vervliet and Indekeu, 2015;Xia et al., 2017;Zuj et al., 2020).There may be some uncertainty with regards to when avoidance behaviours should be enacted (i.e. when to press the button) (Flores et al., 2018(Flores et al., , 2020)).The avoidance learning phase therefore has a period of estimation uncertainty but the avoidance learning phase mainly consists of a period of expected uncertainty or certainty related to the probabilistic structure of the environment (Angela and Dayan, 2005) (see, Fig. 2).However, the extent to which avoidance is enacted during different phases of conditioning following avoidance learning (e. g., threat extinction training, generalisation) will alter perceptions of the probabilistic structure of the environment.While, engaging in avoidance to the CS+ reduces uncertainty related to threatening outcomes, the act of avoidance also prevents the learning of potential new contingencies (Pittig et al., 2018).Thus, pervasive avoidance to the CS+ (or CS-, GSs) may not allow for an accurate assessment of the probabilistic structure of the environment.
The literature on IU and avoidance learning is mixed (Flores et al., 2018(Flores et al., , 2020;;Hunt et al., 2019;Lommen et al., 2010;Morriss et al., 2018;San Martín et al., 2020;Vervliet and Indekeu, 2015;Xia et al., 2017;Zuj et al., 2020).The majority of studies report no significant relationships between IU and avoidance of the CS+ or CS-during avoidance learning (Lommen et al., 2010;Morriss et al., 2018;San Martín et al., 2020;Vervliet and Indekeu, 2015;Xia et al., 2017;Zuj et al., 2020).One study has found that higher prospective IU (i.e., engagement in seeking behaviours to reduce uncertainty), over inhibitory IU and trait anxiety, is associated with greater frequency of avoidance behaviour to both the CS+ and CS-during avoidance learning (Flores et al., 2018(Flores et al., , 2020)).Notably, the experiment by Flores et al. (2018Flores et al. ( , 2020) ) used a probabilistic structure with more layers of uncertainty, as the experiment included multiple CS's, a 50% reinforcement schedule for the US and a temporally uncertain US (i.e., the onset of the US varied).
Beyond initial avoidance learning, there is some evidence that IU is involved in avoidance behaviour under conditions with greater unexpected uncertainty such as threat extinction training and generalisation.For instance, high prospective IU is associated with greater frequency of avoidance behaviour to the CS+ during threat extinction training (Flores et al., 2018(Flores et al., , 2020)).Furthermore, high IU is associated with greater frequency of avoidance behaviour to both the CS+ and CS-after periods of extinction training with response prevention (Zuj et al., 2020).Note, however, a few studies have not reported significant relationships between IU and avoidance behaviour during threat extinction training (Xia et al., 2017;Lemmens et al., 2021) or after threat extinction training (Morriss et al., 2018).Lastly, high IU is associated with greater generalisation of avoidance behaviour and self-reported relief following successful avoidance (San Martín et al., 2020).
Overall, such findings suggest that IU, particularly prospective IU, may be more important for initial learning of avoidance behaviours in environments with more layers of uncertainty (i.e., estimation uncertainty), and in the extinction and generalisation of avoidance behaviours (i.e., phases with unexpected uncertainty).The findings above suggest that investigating the subscales of IU may be beneficial for understanding the role of IU in avoidance learning or, indeed, associative learning models more generally.

Summary
Taken together, the findings suggest that IU is involved in modulating psychophysiological, self-report and avoidance responses during classical threat conditioning procedures with periods of unexpected uncertainty, where contingencies change or appear volatile (i.e., threat extinction training, extinction retention, reinstatement, reversal and generalisation).More specifically, the majority of IU-related effects upon psychophysiological measures, particularly skin conductance, were observed under conditions of unexpected uncertainty where information is absent regarding threat and safety contingencies and the occurrence of the US, such as threat extinction training and retention.There is also some evidence for IU-related effects upon psychophysiological measures under conditions of unexpected uncertainty where there is an absence of information regarding threat and safety contingencies and where the US is still present, such as reversal and generalisation.Furthermore, while the evidence is mixed, the literature tentatively suggests that IU is involved in modulating psychophysiological responses during classical threat conditioning procedures with periods of estimation and expected uncertainty/certainty (i.e., threat acquisition training, avoidance learning), but only when more layers of uncertainty related to the contingencies are embedded (e.g., uninstructed, more stimuli such as auditory probes, partial reinforcement).Importantly, the majority of empirical research on IU and classical threat conditioning also indicated the specificity of IU over other measures of self-reported anxiety (i.e., trait anxiety, worry, neuroticism).
Overall, the findings suggest that IU plays a critical role in classical threat conditioning mechanisms when the probabilistic structure of the environment includes a greater quantity of unknowns (i.e., the absence of information about threat and safety contingencies and the omission of the US).The results of the literature review provide direct evidence for modern IU theory outlined by Carleton (2016aCarleton ( , 2016b) ) and through methods recommend by Shihata et al. (2016) that individual differences in IU, and by proxy fear of the unknown activates the behavioural inhibition system (i.e.increased vigilance and arousal) (Gray and McNaughton, 2003), and is in part responsible for the maintenance of learned fear and anxiety (Brosschot et al., 2016;Brosschot et al., 2017;Grupe and Nitschke, 2013;Tanovic et al., 2018).

Future directions
There are a number of opportunities and potential avenues for future work on the role of IU in classical threat conditioning.What was clear from the literature review was that many empirical studies conflate different parameters of uncertainty (for discussion see, Bennett et al., 2018;Davies and Craske, 2015;Morriss et al., 2021;Morriss et al., 2020a), making it difficult to assess what parameters of uncertainty drive IU-related maintenance of learned fear and anxiety.To avoid conflating different parameters of uncertainty, future work should further address the role of IU in classical threat conditioning mechanisms by varying the level of uncertainty in a hierarchical or linear way (see, Fig. 3).For example, by layering specific aspects of conditioning procedures that involve outcome, spatial or temporal uncertainty such as contingency instruction, reinforcement rate, number of CS's, and predictability of additional stimuli (i.e., startle probes).The quantity of unknowns should be reported clearly in the methods section of reports for clarity and replication purposes.In doing so, the level of uncertainty in the probabilistic structure of the environment can be quantified and be related to broader aspects of conditioning procedures that involve periods of estimation, unexpected and expected uncertainty/certainty.
Further research is needed to examine the extent to which the absence of information about threat and safety drives IU-related effects (i.e., continued responding or avoidance) during phases with unexpected uncertainty (i.e., threat extinction training, retention, reinstatement, and reversal).For example, by manipulating: (1) contingency instruction (Mertens et al., 2018), (2) the level of US omission via the reinforcement rate in prior threat acquisition training (partial vs. continuous) or the reinforcement rate in threat extinction training itself (Knowles and Olatunji, 2018;Lipp et al., 2020;Thompson et al., 2018) (i.e., complete omission of the US, compared to partial omission or unpaired presentation of the US), (3) replacing the US with a novel stimulus (Dunsmoor et al., 2015) or positive stimulus (Keller et al., 2020) and ( 4) the presentation of new CS's or GS's (Lipp et al., 2020).Moreover, in phases with unexpected uncertainty, particularly return of threat procedures, the stability of IU-related effects across time (e.g., multiple sessions) and in relation to interventions (e.g., more exposure experience, pharmacological) should be examined to determine the pervasiveness of IU-related biases in maintaining learned fear and anxiety.
Earlier in this review we highlighted two distinct forms of IU: prospective-and inhibitory-IU.Prospective-IU refers to the desire for predictability and actively seeking certainty, whereas inhibitory-IU refers to the paralysis of cognition and action in the face of uncertainty (Birrell et al., 2011;Hong and Lee, 2015).Investigating the prospective and inhibitory subscales of IU separately may provide unique explanations of how different aspects of IU modulate classical threat conditioning phenomena (Flores et al., 2018(Flores et al., , 2020;;Morriss et al., 2020aMorriss et al., , 2020b) ) and are related to different internalising psychopathology.
Replication work is required to address the reliability and robustness of IU-related effects, over other self-reported measures of anxiety, in classical threat conditioning procedures, particularly for threat extinction retention, reinstatement, reversal, and generalisation phases.Replication efforts should be preregistered (Krypotos et al., 2019), well powered (Ney et al., 2018) and include diverse samples, in order to assess the generalisability of IU-related effects in non-WEIRD samples (Henrich et al., 2010).Similarly, replication is warranted in clinical samples to examine whether IU-related profiles of responding during classical threat conditioning procedures are transdiagnostic across internalising psychopathology or vary based on different disorders (Shihata et al., 2016).

Conclusion
In sum, individual differences in IU play an important role in modulating classical threat conditioning mechanisms, particularly when there is unexpected uncertainty related to threat and safety contingencies.Under periods of unexpected uncertainty, high IU, over other self-reported measures of anxiety, is specifically associated with poorer threat extinction learning and retention, as well as overgeneralisation.Under periods of estimation and expected uncertainty, the role of IU is less clear.More research is needed to examine the reliability, robustness and stability of IU-related effects in classical threat conditioning procedures.Specifying and manipulating parameters of uncertainty hierarchically or linearly within classical threat conditioning procedures will be beneficial for identifying which parameters of uncertainty are most crucial for uncertainty-based maintenance of learned fear and anxiety.Such work will be critical in understanding the relevance of IU in neurobiological models of uncertainty-based maintenance of fear and anxiety and inform translational work aiming to improve the diagnosis and treatment of mental health disorders with an anxiety component.

Fig. 1 .
Fig. 1.Image depicting the different classical conditioning procedures outlined in the review: threat acquisition training, threat extinction training, threat extinction retention, reinstatement, reversal, generalisation and avoidance.Yellow and Blue tiles represent conditioned or generalisation stimuli.Shocks represent an aversive outcome paired with the conditioned stimulus or an unsignaled aversive outcome in the case of reinstatement.The button press represents an action required to avoid an aversive outcome.(For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 3 .
Fig. 3. Image demonstrating how the level of uncertainty in classical threat conditioning procedures can be organised in a hierarchical or linear way.Parameters in classical threat conditioning procedures that have uncertainty embedded (A) can be layered or stacked to linearly increase the level of uncertainty (B).