From safety to frustration: The neural substrates of inhibitory learning in aversive and appetitive conditioning procedures

Inhibitory associative learning counters the effects of excitatory learning, whether appetitively or aversively motivated. Moreover, the affective responses accompanying the inhibitory associations are of opponent valence to the excitatory conditioned responses. Inhibitors for negative aversive outcomes (e.g


Introduction
The capacity to inhibit thoughts and behaviours is a fundamental component of impulsivity and, as such, important to our understanding of a wide range of psychological processes and disorders, from selfcontrol to obesity, drug and behavioural addictions in the appetitive domain. However, the inhibition of learned associations has been most extensively investigated in the aversive domain, in the context of safety learning and anxiety disorders, and fear (or rather lack of fear) is also a key determinant of impulsivity. Hence inhibition underpins selectivity in a variety of cognitive processes and behaviours, with disinhibition providing some explanation of wide-ranging dysfunctions, and scope for a better understanding of its translational implications in the appetitive domain.
Whilst readily understood as a construct, inhibition spans diverse cognitive and behavioural phenomena and underlying mechanisms (Nigg, 2000;Sosa & dos Santos, 2019;Sosa, 2022). In studies of experimental psychology, measures of inhibitory (dys)function have typically used volitional response measures of the kind Nigg (2000) classified as effortful, involving conscious control, e.g. Stroop, 'stop signal' and Go-NoGo. In contrast, the topic of the present review -inhibition of associative learning -falls within Nigg's classification of automatic inhibition (Nigg, 2000;Kantini et al., 2011). Even if automatic, without requiring any executive control, inhibition of associative learning can also be seen to relate to impulsivity (inhibitory control), and has been a relatively neglected area of study (Sosa & dos Santos, 2019;Sosa, 2022).
Associative learning is typically excitatory, in the sense of learning what goes with what, when an excitatory conditioned stimulus (CS) is paired with the unconditioned stimulus (UCS) outcome. Such excitatory conditioning is demonstrated by the frequency and/or strength of the conditioned response (CR). Motivationally such excitatory learning is classified as appetitive or aversive. Appetitive conditioning procedures can take many forms, including sexual conditioning using access to a mate, or conditioning to cues associated with the administration of an addictive substance as the UCS. However, appetitive conditioning using food reward is the most commonly used laboratory procedure. Nosepoking activity in the part of the apparatus where food is delivered (seen as the unconditioned response) is also a Pavlovian conditioned response and provides a convenient measure. Aversive conditioning procedures also take many forms, from eyeblink conditioning using air puffs delivered to the cornea to conditioned taste aversion following experimentally-induced nausea. Fear conditioning using foot shock is most commonly used, and is typically measured by quantifying freezing responses, or as conditioned suppression of an ongoing appetitive response. Relatedly, theories of active avoidance (Mowrer, 1947(Mowrer, , 1956(Mowrer, , 1960, which specifically requires the learning of a new response to avoid experiencing any further shock, propose that anticipatory anxiety is initially learned through Pavlovian conditioning of the signal for foot shock.

Pavlovian inhibitory learning procedures
Humans and other animals also show inhibitory associative learning, which can counter the effects of excitatory learning, whether appetitively or aversively motivated. Specifically, conditioned inhibition is said to occur when a particular stimulus (the conditioned inhibitor) signals the absence of an otherwise expected outcome (Pavlov, 1927). Experimentally, conditioned inhibition may be established using a socalled feature negative discrimination procedure in the classic Pavlovian paradigm followed by confirmatory tests ( Fig. 1; see below).
For example, following on from an excitatory conditioning phase, the CS (A+) presented in isolation continues to be followed by the UCS, but on trials when the CS is compounded with the inhibitory cue (AX− ) the otherwise expected UCS does not occur (Fig. 1a). As a result of the feature negative discrimination learning (A+/AX− ), the potential conditioned inhibitor comes to signal that the UCS outcome, which would normally occur following the CS, will not now occur (Pavlov, 1927). Related feature negative designs may include additional cues (AB+/AX− ; Fig. 2a). Differential inhibition involves presentations of the CS (A+) and the inhibitory cue (X− ), with only the excitor followed by the reinforcer (A+/X− ; Fig. 2b).
Other methods have also been proposed to produce inhibitory learning. For example, an explicitly unpaired procedure (+/X− ) can be used to generate inhibition akin to that seen in a feature negative designs (A+/AX− ): occurrences of the UCS are specifically unpaired in time with the inhibitory cue (X− ), such that the context is the more reliable excitatory cue (+). The UCS is presented only on occasions which are temporally removed from X, hence no excitation accrues to X and moreover X develops inhibitory properties. Similarly, in an inhibition of delay procedure, the UCS is presented at the end of an extended CS. Due to the length of time the CS is presented, the early part of the CS Fig. 1. Conditioned inhibition established by (a) feature negative discrimination learning (A+/AX− ) and confirmed by (b) summation test of transfer of inhibition to an alternative previously reinforced cue C (comparing C versus CX, or CX versus CY) and/or (c) retardation tests of new learning to X versus a novel stimulus (Y). effectively signals a period of no UCS, thus a delay-conditioned CS can also develop inhibitory properties (Rescorla, 1967). Likewise, in backward conditioning procedures, when the notional CS occurs only after the UCS, inhibitory learning may be demonstrated.
Fundamental to all of the associative learning procedures which result in inhibition is the idea that the inhibitor can be seen to signal the omission of an expected outcome. As described above, the different methods to produce inhibitory learning have in common that the UCS is removed (or reduced, see below). Whilst inhibition is readily demonstrated under controlled experimental conditions, the underlying psychological mechanisms are not firmly established. According to the Rescorla-Wagner model of associative learning, the change in associative strength on any particular learning trial depends on the discrepancy between what is expected based on the available cues and what actually occurs, known as the prediction error (Rescorla & Wagner, 1972). Thus, expectations are an important factor in determining learning; in other words, new learning typically requires that an association is not already at full strength (so that there is still mismatch between what is expected and what actually occurs). Positive prediction error generates excitatory conditioning; negative prediction error generates inhibitory conditioning. According to the Rescorla-Wagner model (1972), inhibition is a negative form of learning that occurs when the trained CS now 'overpredicts' the UCS, in that this does not occur or is in some way attenuated. For example, in a feature-negative procedure, the putative inhibitory component of the compound (comprised of the experimental stimuli presented at the point of UCS omission or reduction) starts with zero associative strength (as nothing is known about this new cue), its associative strength becomes negative over trials because the CS with which it is presented predicts the UCS.
In order to demonstrate that conditioned inhibition has occurred and that the designated inhibitory cue is a true inhibitor, its presentation should measurably counteract the excitatory response, and any responding to the conditioned inhibitor as such must be clearly distinguishable from that elicited by the excitatory CS. In fact, inhibitory stimuli clearly do acquire special properties and these have been demonstrated using approach-withdrawal as well as transfer tests. Based on the associated conditioned emotional response (CER), approachwithdrawal methods make use of the fact that successfully conditioned subjects will approach an excitatory cue (A+) and withdraw from or avoid an inhibitor (X− ) for a positive UCS outcome (Hearst et al., 1980;Hearst & Franklin, 1977;Wasserman et al., 1974). Although not widely used as a definitive test of conditioned inhibition, approach-withdrawal does provide a good behavioural measure. However, summation (Fig. 1b) and retardation tests (Fig. 1c) have been suggested to be definitive (Hearst, 1972;Rescorla, 1969). The summation test involves taking an additional excitatory cue (C+), pre-trained but not previously presented in conjunction with any inhibitor, and examining the reduction in responding seen when C+ is paired together with an established conditioned inhibitor (X− ), pre-trained in conjunction with a different CS (A+). According to the Rescorla-Wagner model (1972), because it has negative associative strength, a true inhibitor should reduce the responding which would otherwise be elicited by any conditioned excitor (CS+) (for the same UCS) with which it is subsequently paired. The retardation test similarly relies on the notion that a conditioned inhibitor develops negative associative strength. Retardation is assessed when the previously trained conditioned inhibitor is now paired with a UCS. Compared with the rate of acquisition of excitatory learning seen with a neutral cue, conversion to a CS+ is slowed for a conditioned inhibitor. It has been suggested that, ideally, a true conditioned inhibitor should pass both summation and retardation tests in order to rule out alternative explanations of apparent inhibition, for example based on changes in attention to the stimuli. In a summation test, the presumed conditioned inhibitor might rather distract from the accompanying CS+ with which it has not previously been paired, hence the apparently reduced excitation occurs because the conditioned inhibitor attracts attention. Conversely, in the retardation test procedure too little attention may be paid to the conditioned inhibitor because the prior training history necessarily involves non reinforced exposures; non-reinforced exposures also reduce learning but through a different mechanism (latent inhibition; Lubow & Moore, 1959;Rescorla, 1969). Thus, between them, the two tests should rule out explanations of apparent inhibition based on too much or too little attention to the inhibitor.
The two-test method to confirm conditioned inhibition (by both summation and retardation tests) has been widely adopted in animal learning studies, but has been harder to implement in human studies (Papini & Bitterman, 1993;Cole et al., 1997;Sosa & dos Santos, 2019;Sosa & Ramίrez, 2019). Moreover, behavioural neuroscience studies may introduce practical limitations on how behaviour is trained and tested, for example relating to the number of days on which treatments can be given, or the half-life of a drug constraining session length. They also introduce the need for further controls in order to evaluate the effects of the selective brain manipulations, which further increase the overall study sample size (Sosa & dos Santos, 2019). Thus the two-test method remains an ideal which is not much put into practice.

The role of motivation
In animal studies, the differences between appetitively and aversively motivated procedures typically conflate differences in the number of trials necessary to achieve the underpinning excitatory conditioning. Appetitive procedures inevitably require more conditioning trials than aversive procedures because food reward is less salient than aversive UCSs such as foot shock (Pezze et al, 2016). A relatively high number of learning trials and extended duration of training may promote consolidation (McGaugh et al., 1996;Squire et al., 2015). Inter-trial-intervals and CS durations are also parameters showing systematic differences in appetitive versus aversive procedures (Domjan, 1980;Pezze et al., 2016;Thrailkill et al., 2020).
Nonetheless, there are clear differences in the emotional reactions to inhibitors established in appetitive and aversive procedures. So the effects of motivation as such (generated by the use of different primary reinforcers rather than other procedural differences) seem strong.
An inhibitor trained in conjunction with an excitatory cue for an aversive outcome becomes a positively valenced stimulus; for example, inhibitors for shock signal safety. This line of thinking has had important implications for our understanding of avoidance learning (Miller, 1948;Mowrer, 1947Mowrer, , 1956Mowrer, , 1960. Since the avoidance response results in the omission of the expected UCS, it has the potential to generate inhibitory cues. The Miller-Mowrer theory was later developed to include a key role for the safety signals generated by the successful execution of an avoidance response, through consideration of the likely role of inhibitory learning (Gray, 1987;Lovibond & Shanks, 2002). Animals successfully avoid in a situation where an aversive UCS is predicted (by a warning signal), but when the avoidance response is made, the UCS is omitted, so the warning signal should eventually extinguish, and the avoidance response should subsequently decline. However, in practice this decline in the avoidance response is not seen -avoidance learning is very persistent. The Miller-Mowrer theory can explain the persistence of avoidance learning: the avoidance response generates inhibitory feedback, counteracting the effect of the warning signal, so the UCS is not expected. Hence there is no prediction error to drive extinction of the warning signal. Indeed, there is experimental evidence to show how a conditioned inhibitor acting as a safety signal can protect the classically conditioned fear response from extinction and thus account for the (otherwise unexplained) persistence of the avoidance behaviour (Soltysik et al., 1983). Moreover, safety signals have been shown to be 'relieving' in a free-operant lever press avoidance procedure, in the sense that rats preferentially responded on the lever that produced an explicit safety signal (in addition to the omission of the foot shock) as feedback (Fernando et al., 2014a).
Animal experimental studies confirm that safety signals moderate stress reactivity, buffering the deleterious effects of uncontrollable stressors (Christianson et al., 2011). Such studies have employed discrimination learning procedures, for example comparing behavioural and physiological reactions to a stimulus reliably associated with shock termination with reactions shown to a stimulus presented in a random relationship to shock as the control (Christianson et al., 2011). Of the various procedures in use to model safety signal processing, the AX+/ BX− 1 paradigm is closest to the design typically used in studies of Pavlovian inhibitory learning (Kazama et al., 2013). The rationale for the AX+/BX− (over a simpler A+/B-) design is that the requirement to discriminate the A versus B elements of the compound cues should minimise the contribution of external inhibition to the observed differences in responding (Myers & Davis, 2004). Although many such fear conditioning studies fall short of the two-test requirement for conditioned inhibition (Rescorla, 1967(Rescorla, , 1969, the safety signal literature has greatly informed our understanding of anxiety and post traumatic stress disorders (Jovanovic et al., 2009;Sosa & dos Santos, 2019;Sangha, et al., 2020).
In contrast to safety learning, an inhibitor trained in conjunction with an excitatory cue for a positive appetitive outcome becomes a negatively valenced stimulus, eliciting frustration and/or disappointment. This distinction resonates with seminal work on the emotional effects of frustration (Amsel, 1962), identifying safety cues as reinforcers (Miller, 1948) and on 'cross-tolerance', providing evidence for the similarity of punishment and non-reward (Brown & Wagner, 1964). However, the theoretical implications of the different emotional responses generated by different kinds of inhibitor have received little attention to date. Studies of conditioned inhibition in human participants have typically used positively motivated tasks, in the sense that the goal is to successfully predict outcomes and participants are motivated to complete the task successfully. Motivation may be less of a consideration in human studies because the UCS outcomes to be predicted are rarely primary reinforcers like those used in studies of conditioning in animals, so the presence or absence of the UCS outcome is unlikely to elicit strong emotional responses directly. The use of emotionally salient cues such as those provided by the International Affective Picture System potentially brings us closer to the traditional Pavlovian paradigm, but it is challenging to match the salience of the UCSs for direct comparison between appetitive and aversive procedures (Thurston & Cassaday, 2015). There have been studies using biologically important UCSs (such as food) in human studies (e.g. Colagiuri & Lovibond 2015;Quail et al., 2017), but these are the exceptions. In general, animal studies are not directly comparable with studies of human participants, which are usually motivated indirectly (by the reward of successfully completing the task). Moreover, we cannot exclude the possibility that humans use reasoning to solve discrimination problems of the kind posed in such tasks (Lovibond & Shanks, 2002;Thurston & Cassaday, 2015, Williams, 1995. The profile of similarities or differences in the neural substrates identified with inhibitory learning in humans and animals will shed light on the issue of whether different strategies and/or processing mechanisms are involved in formally equivalent tasks implemented in humans and animals (using inevitably very different procedures).
Logically, there could be an independent neural substrate for coding event omission. However, in animal studies, based on the primary reinforcers in use, task motivation and the corresponding emotional 1 The standard denotation AX+/BX− is well-established in the fear discrimination literature and B represents the inhibitor, whereas in the denotations A+/X− and A+/AX− (also widely used) X is the inhibitor. We follow the conventions used in the cited literature and note this inconsistency in the representations in use. responses to conditioning cues will certainly engage different neural substrates. A signal for food engages an appetitive state, and an inhibitor for food engages an opponent, aversive one. But that inhibitor also requires the appetitive statefor it to form, and for its effects to be manifest.

Mechanisms of inhibitory learning
Inhibitory and excitatory learning are inevitably inter-dependent, because a conditioned inhibitor signals the absence of an outcome otherwise predicted by the CS. In other words, inhibitory learning requires some existing expectation, based on acquired excitatory learning. Procedurally the excitatory expectation may be established before the inhibitory contingency is introduced. Inhibitory learning during feature negative discrimination also requires additional processing, to assess net prediction error to a combination of cues, in this case the compound (AX− ) signalling the absence of the expected outcome (Pavlov, 1927;Rescorla, 1967Rescorla, , 1969Rescorla & Wagner, 1972). Excitatory learning can of course be studied independently of inhibitory learning, but it may be challenging to identify the additional mechanisms necessary for inhibitory learning (over and above the excitatory learning mechanisms on which it depends).
The motivations driving the learning and the emotions generated in different learning procedures add yet another layer of complexity (Thurston & Cassaday, 2015). Appetitive conditioned stimuli have opposing valence ('motivationally opposite') to those provided in aversively motivated conditioning, as do the respective conditioned inhibitors for these different types of associative learning. Specifically, in an appetitive context, the conditioned inhibitor signifies frustration rather than safety, mirroring the motivational valence of the excitatory cues. We have seen how the appetitive or aversive motivation of the learning procedure in use influences the psychological mechanisms which come into play to make associative learning possible. It follows that the distinctive emotional states of relief and disappointment, respectively associated with safety and frustration, will similarly depend on dissociable mechanisms (Soltysik & Jelen, 2005;Christianson et al., 2011;2012;Thurston & Cassaday, 2015;Sangha et al., 2020).
It therefore also follows that the neural substrates of inhibitory learning should (1) extend beyond those identified with the corresponding excitatory learning process and (2) be subject to differences arising from the opposing emotional responses inherent in anticipating the presence versus absence of different kinds of motivationally significant outcome (within the general categories of aversive or appetitive). For the same kinds of UCS outcomes, the CS+ versus its conditioned inhibitor would be predicted to be reacted to differently based on the opposing emotional valences attaching to these cues, and indeed to elicit opponent responses (Konorski, 1967). Presentation of a food-associated CS results in approach and presentation of a shock-associated CS results in freezing, withdrawal, or escape. Conversely, an inhibitor for shock becomes a safety signal and an inhibitor for food becomes a frustration cue. Behaviourally, presentation of an inhibitor for an appetitive UCS elicits withdrawal or even avoidance responses (Hearst et al., 1980;Hearst & Franklin, 1977;Wasserman et al., 1974). Conversely, safety signals (potentially including also afferent feedback stimuli generated by the animals' actions in avoidance procedures) have been shown to provide secondary reinforcement, in this case acting as positive reinforcers in that animals will work to earn their presentation (Cándido et al., 1991;Cicala & Owen, 1976;Dinsmoor, 2001;Galvany & Twitty, 1978;Morris 1975).
In line with the importance of valence, electrophysiological studies have provided evidence that safety and reward learning use overlapping mechanisms in basolateral amygdala (BLA): different populations of neurons responded to fear plus safety, safety, and safety plus reward cues (Sangha et al., 2013). There may be unique substrates specific to inhibitory versus excitatory learning, but it is also to be expected that the emotional and motivational consequences of signals for the presence or absence of different kinds of outcome should also recruit additional brain systems.

Neural substrates of excitatory learning
The neural substrates of excitatory learning have been extensively reviewed elsewhere (Berridge, & Robinson, 2003;Fanselow & LeDoux, 1999;Fanselow & Wassum, 2015;Iordanova et al., 2021;LeDoux, 2000). Our focus here is on inhibitory learning and performance, but the nature of the interdependence between excitatory and inhibitory learning requires consideration of the wider network of neural substrates for associative learning. As discussed above, the relationship between excitatory and inhibitory learning is inherently asymmetric: there can be no inhibition in the absence of excitatory expectation. Second the motivational qualities of excitatory and inhibitory cues show overlap: fear and frustration (in the absence of expected food reward) are aversive; like food CSs, safety signals for shock omission are positively reinforcing.
The dopaminergic projection from the ventral tegmental area (VTA) to the nucleus accumbens (NAc) plays a key role in appetitive reward motivated tasks (Ikemoto, 2007;Koob, 1992;Schultz, 1997;Wise, 2006;Wise & Rompre, 1989). The available evidence suggests that natural rewards such as food, water, access to mate, as well as other rewards such as brain stimulation and drugs of abuse, all activate this same reward pathway (Kelley & Berridge, 2002;Milner, 1991). As might be expected, the same reward processing systems also mediate secondary reinforcement acquired through appetitive associative learning. For example, NAc plays a role in Pavlovian approach behaviours (Parkinson et al., 2002) and at the associative level VTA dopamine (DA) is strongly implicated in the mechanisms of reward prediction (Iordanova et al., 2021). However, it is now recognised that the identification of DA with the reward signal is too simplistic (Schultz, 2019). Work on opioids suggests there are also non-dopaminergic mechanisms in VTA for mediating reward whose effect can depend on the animal's behavioural state (Fields & Margolis, 2015). Moreover, gamma aminobutyric acid (GABA) neurons in VTA have been found to respond to predicted reward (Cohen et al., 2012).
The amygdala is most commonly implicated in aversive learning, fear conditioning based on the use of foot shock UCS in particular (Iordanova et al., 2021;LeDoux, 2000). However, depending on the nature of the aversive stimulus, other brain areas have also been identified as important, for example the neural substrates of conditioned taste aversion include the parabrachial nucleus and medial thalamus in addition to the amygdala (Yamamoto et al., 1994). And, although air puffs to the eye are mildly aversive, the cerebellum plays the predominant role in eyeblink conditioning because the blink reflex is a form of motor learning (Bracha et al., 2009;Gerwig et al., 2007).
Nonetheless commonalities in the neural substrates mediating appetitive and aversive conditioning have been suggested. For example, the central nucleus of the amygdala (CeA) is a key component of the circuitry required to encode general motivational value, though the empirical evidence for this claim derives largely from appetitive studies (Fanselow & Wassum, 2015). Similarly, there is evidence to suggest that the encoding of outcome representations depends on a network which includes orbitofrontal cortex, BLA, NAc (core and shell) and mediodorsal thalamus. These substrates might also be expected to be shared across different motivational variants, but the majority of studies of the neural substrates of outcome devaluation have also been conducted in appetitive procedures (Fanselow & Wassum, 2015).
In summary, there may be some commonality in neural substrates responsible for the encoding of general UCS properties, but a large body of evidence points to dissociation in the substrates underlying appetitive and aversive conditioning. The amygdala (BLA in particular) is the fundamental component of the mammalian fear circuit (Fanselow & LeDoux, 1999); whereas the mesolimbic DA system mediates reward processing and appetitive conditioning (Berridge & Robinson, 2003;Kelley & Berridge, 2002;Milner, 1991). However, there is also good evidence that such dissociations are not absolute. Despite the wellknown role of the BLA in aversive learning, the CeA has also been shown to be involved in appetitive learning (Parkinson et al., 2000;Everitt et al., 2003;Knapska et al., 2006).
Similarly, the mesolimbic DA system is involved in aversive as well as appetitive learning (Young et al., 1993;Saul'skaya and Marsden, 1995), with evidence for hemispheric asymmetry and intraaccumbens regionalization in shell and core sub-regions (Besson & Louilot, 1995), as well as in forming associations between neutral cues in sensory preconditioning (Young et al., 1998). More recently, Tang et al. (2020) confirmed the role of DA neurons in the VTA projecting to BLA in fear conditioning, using both electrophysiological recording and optogenetic methods. These neurons were excited by foot shock and acquired a response to the CS used in conditioning, whilst optogenetically silencing these neurons reduced the strength of the fear memory measured 24 hr later. Possibly previous distinctions the pathways of reward and aversion have been too crude and insufficiently fine-grained given that that the VTA projection areas include the amygdala. In NAc, there is regional variation in the shell and core subterritories, with some differences by laterality in the DA response seen after conditioning, to appetitive in the right NAc and to aversive stimuli in left NAc (Besson & Louilot, 1995).

Neural substrates of inhibitory learning
In the case of conditioned inhibition as confirmed by summation and/or retardation test, a role for the DA system has also been suggested, for example by the correlation of inhibitory learning with measures of reward sensitivity in humans (Migo et al., 2006), as well as by a variety of animal studies summarised in Table 1. For example, 7 days of systemic pre-treatment with the indirect DA agonist amphetamine (prior to any conditioning) was found to enhance conditioned inhibition in an appetitive procedure (Harmer & Phillips, 1999). The fact that amphetamine can be effective when administered as a pre-treatment is consistent with actions mediated by sensitisation and/or secondary neuroadaptations in the DA system (Robinson & Berridge, 1993). However, the role of DA has been confirmed in electrophysiological (Tobler et al., 2003) and optogenetic studies (Chang et al., 2016(Chang et al., , 2018 of appetitive conditioned inhibition (see also below).
Using the same free-operant lever press avoidance procedure used to demonstrate that safety signals are relieving (Fernando et al., 2014a), these authors also report evidence that DA in NAc shell plays a key role in the mediation of the conditioned reinforcing properties of the safety signal (Fernando et al., 2014b). Indeed, the role of NAc in inhibitory learning may depend on such conditioned reinforcement: in an earlier study of conditioned inhibition of fear, which did not assess whether the safety signal produced an appetitive state, no such role for NAc could be demonstrated (Josselyn et al., 2005).
One possibility is that there should be roles for the neural substrates identified with excitatory conditioning (appetitive and aversive) also in their inhibitory learning counterparts, with opponency of the behavioural response reflected in opponency of the neural response. As we have seen above, behavioural studies suggest that inhibitory and excitatory learning show some asymmetric dissociation, and electrophysiological studies have shown that positive and negative prediction error are coded opponently at the neuronal level (Tobler et al., 2003). This was demonstrated with electrodes implanted in the VTA for extracellular recordings (in A8, A9 and A10 of the primate). Presentation of A+ produced activation. Around half the neurons tested with an AX− inhibitory compound showed a biphasic response (activation followed by depression). X− alone produced pure depression in the majority of cases but also some biphasic responses (Tobler et al., 2003). Such electrophysiological studies, while valuable, are inevitably correlational. Bidirectional optogenetic modulation of DA neurons in VTA has since been used to show cause and effect in relation to the putative neural substrates. Specifically, optogenetic activation of VTA DA at the time of reward omission interfered with the development of conditioned inhibition (Chang et al., 2016). This was demonstrated as the failure to pass the summation and retardation test in the optogenetic intervention group (VTA DA neuronal activation) as compared to the controls (Chang et al., 2016). Conversely, brief (but not prolonged) inhibition of VTA DA neurons at the time of reward delivery resulted in the additional target cue becoming a conditioned inhibitor, as confirmed by summation and retardation tests (Chang et al., 2018).
The demonstration of opponent responses in response to conditioned inhibitors versus excitors was initially restricted to the VTA DA neurons sampled by Tobler et al. (2003) and to appetitively motivated tasks (Chang et al., 2016(Chang et al., , 2018Tobler et al., 2003). Electrophysiological and optogenetic studies of this kind are very time consuming and expensive to conduct. Conditioned inhibition is not always confirmed by summation and/or retardation tests and studies of inhibitory learning more broadly defined (i.e. the learning produced by negative prediction error) are also in principle informative. There have been fewer studies of VTA activity when an aversive outcome is omitted in studies of negative prediction error (Iordanova et al., 2021). However, there is some evidence of opponent responses in aversively motivated tasks. Recording theta oscillations (and haemodynamic responses) in BLA, McHugh et al. (2014) found evidence for opponent responses to the omission of expected foot shock. Such evidence for opponency is consistent with the hypothesis that the same neural substrates identified with excitatory appetitive and excitatory aversive conditioning code for inhibition, albeit opponently. Similarly, there is evidence for bidirectional regulation of dorsomedial prefrontal cortex (PFC) activity by VTA DA neurons in safety signalling and fear discrimination: safety signalling increased activation of VTA DA neurons, whereas impaired safety signalling was associated with a lack of VTA DA neuron activation (Yan et al., 2019).
The focus on DA may be too narrow and developments in our understanding of its pathways suggest that a more fine-grained approach should be adopted. VTA DA neurons show cellular heterogeneity which has not so far been addressed in studies of inhibitory learning (some corelease glutamate, GABA or peptides with DA), with these distinct neuronal populations linked to different aspects of motivated behaviour (Morales & Margolis, 2017). Differences in the afferent and efferent connectivity in medial and lateral subregions suggest that cellular heterogeneity relates anatomically to the organisation of these neuronal populations. Different midbrain systems, projecting to mPFC, NAc core and shell, and BLA, participate in distinct circuits for reward and aversion (Lammel et al., 2008(Lammel et al., , 2011(Lammel et al., ,2012Verharen et al., 2020). For example, VTA neurons projecting to the dorsomedial striatum show reduced activity in response to aversive stimulation, whereas those projecting dorsolateral striatum show the opposite response to the same aversive stimulus (Lerner et al., 2015). Similarly, distinctive neuronal subpopulations showing molecular and anatomical heterogeneity have been identified in NAc (Verharen et al., 2020). Thus increases and decreases in DA activity will also depend on the neuronal subpopulations sampled.
With a wider focus, we can identify the scope of neural substrates identified with inhibitory learning processes across a number of behavioural procedures. Successful feature negative discrimination is fundamental to conditioned inhibition but we separate out the studies showing confirmation of conditioned inhibition by summation, and/or retardation tests in Table 1. The Table 1 summary of conditioned inhibition studies also includes backward conditioning, explicitly unpaired (+/X− ) and differential inhibition (A+/X− ) studies which confirmed conditioned inhibition by summation, and/or retardation tests. We provide some summary overview of the findings of studies run without confirmatory summation, and/or retardation tests, for feature negative discrimination in Table 2, and for differential inhibition (also known as differential conditioning) in Table 3. Tables 1 and 2 also includes related designs using compound cues (AX+/BX− and AB+/AX− ), more elaborate than the simple A+/X− designs used in the Table 3 studies of differential inhibition. AX+/BX− resembles differential inhibition with a Table 1 Conditioned inhibition studies.

Species
Procedure Target/treatment/measure Key findings Desrochers & Nautiyal (2022) Mice Appetitive 5-HT 1B receptor KO 5-HT 1B receptor KO mice showed normal responding to excitatory stimuli (A+/B + ), but increased responding on inhibitory (AX− ) trials. This difference in responding was not seen at summation (BX− ). Nelson et al. (2018) Rats Appetitive Excitotoxic lesions in retrosplenial cortex Retrosplenial cortex lesions had no effect on the acquisition of A+/AX− discrimination or on retardation test performance, however these lesions impaired performance in the summation test. Chang et al. (2018) Rats Appetitive Optogenetics to inactivate DA neurons in the VTA Sudden brief (but not prolonged) inactivation of VTA DA neurons (mimicking the profile produced by negative prediction error) produced a cue meeting the classic criteria for a conditioned inhibitor. Chang et al. (2016) Rats Appetitive Optogenetics to inactivate DA neurons in the VTA Optogenetic activation of VTA DA at the time of reward omission interfered with the development of conditioned inhibition; demonstrated as the failure to pass the summation and retardation test in the optogenetic intervention as compared to the control group. Rhodes & Killcross (2007) Rats Appetitive Excitotoxic lesions in mPFC (IL region) IL lesions had no significant influence on A+/AX− acquisition or summation test performance, however these lesions impaired performance in the retardation test. Tobler et al. (2003) Non-human primates Appetitive DA neurons in A8, A9 and A10 cell bodies Extracellular recordings demonstrated differential responses: the conditioned excitor (A+) produced activation, the conditioned inhibitor (X− ) largely produced depression and the compound cue (AX− ) produced depression or a biphasic response (activation followed by depression). Chan et al. (2003) Rats Appetitive Excitotoxic lesion in hippocampus Hippocampal lesions did not impair the performance of the A+/ AX− discrimination task nor the retardation test, however these lesions marginally impaired performance in the summation test. Harmer & Phillips (1999) Rats Appetitive i.p. d-amphetamine (indirect DA and NA agonist); 2 mg/kg d-amphetamine pre-treatment facilitated both the conditioned response to the conditioned excitor (A+) and the inhibition of this response to the compound cue (AX− ). Retardation was initially increased by amphetamine sensitisation. Lister et al. (1996) Rats Appetitive Neurotoxic depletion in 5-HT pathways Ablation of the 5-HT pathways impaired A+/AX− discrimination, as well as performance in both summation and retardation tests.

Rats
Aversive Fiber photometry to measure DA activity in VTA AX+/BX− discrimination procedure to establish B as a safety signal that passed both summation and retardation tests. Medial VTA calcium transients on BX− trial predicted the level of safety learning and expression at summation test. Sengupta et al.
Rats Aversive Optogenetic photoinhibition of BLA AX+/BX− discrimination procedure to establish B as a safety signal that passed both summation and retardation tests. Photoinhibition in BLA following presentation of the BX− compound selectively slowed AX+/BX− discrimination learning (simple A+, B− discrimination was unaffected). Summation test performance was selectively reduced following AX+/BX− training (but not by latent inhibition of B− ).  Rats Aversive Micro-infusion of AP5 into anterior, medial or posterior insular cortex NMDAR antagonism in the posterior (but not the anterior or medial) insular cortex prevented acquisition of conditioned inhibition in a fear conditioning procedure. Ostroff et al. (2014) Rats Aversive Lateral amygdala Explicitly unpaired training (+/X− ) was compared to paired training (fear conditioning) and no training (naïve) groups. Fear conditioning transiently increased the density of synapses with no astrocytic contacts relative to explicitly unpaired and control groups. Changes in astrocytic contacts may relate to changing size of synapse (enlarging or shrinking). Summation and retardation tests had been previously used with the same protocol (Osteroff et al., 2010). Synapses with astrocytic contacts were smaller after conditioned inhibition (+/X− ). Genaud-Gabi et al.
Non-human primates Aversive BLA A+/X− discrimination but with retardation test. BLA responses to X− cues of differing modalities were mostly comparable in proportion, direction of firing rate change (increase or decrease), onset and magnitude, with some differences by modality of cue (within-modality generalisation was seen when X− and A+ were of the same modality). Ostroff et al. (2012) Rats Aversive Lateral amygdala Explicitly unpaired training (+/X− ) was compared to paired training (fear conditioning) and no training (naïve) groups. Fear conditioning altered synaptic morphology relative to explicitly unpaired and control groups. Same dataset reported previously with summation and retardation tests passed after +/X− training (Osteroff et al., 2010). Conditioned inhibition prevented the increase in synaptic connectivity otherwise seen in lateral amygdala following fear conditioning.
(continued on next page) H.J. Cassaday et al. redundant cue, but is closer to feature negative than differential conditioning procedures because part of the excitatory cue is present on nonreinforced trials. As might be expected, Table 1 is shorter because a smaller proportion of studies potentially meet the criteria for conditioned inhibition in terms of passing summation and/or retardation tests. Table 4 provides an overview comparison of the neural substrates targeted, by task motivation and inhibitory learning variant in use. Inhibition resulting from other procedures (e.g. by explicitly unpaired cues in excitatory conditioning ) was beyond the scope of this review, and there were relatively few behavioural neuroscience studies of these phenomena. Further studies of the neural substrates of (broadly defined) negative prediction error have been reviewed elsewhere (Iordanova et al., 2021;Sosa & Ramίrez, 2019;Sosa, 2022).
The aversive studies summarised in Table 1 also included the use of lesions and electrophysiological methods, as well as microinfusions, measures of synaptic morphology and measures of DA activity. These show modulatory effects mediated in VTA DA neurons (Yau and McNally, 2022), and at N-methyl-D-aspartate (NMDA) receptors in the posterior insular cortex  and identify BLA (Genaud-Gabi et al., 2013;Ostroff et al., 2010Ostroff et al., , 2012Ostroff et al., ,2014Sengupta et al., 2018) and perirhinal cortex (Campolattaro & Freeman, 2006) as additional brain regions involved in regulating conditioned inhibition. Blockade of GABA A receptors in the anterior interpositus of the cerebellum was without effect on aversively motivated conditioned inhibition (Nolan et al., 2002), but a number of other studies confirm the role of cerebellum (Freeman & Nicholson, 1999;Nolan & Freeman, 2005;Freeman et al., 2005).
Thus, there is incomplete overlap in the brain substrates implicated across the different motivational variants of conditioned inhibition. Some differences can be seen to relate to the use of specific task variants. For example, the role of cerebellum in aversive procedures likely relates to the use of conditioned eyeblink procedures (Campolattaro & Freeman, 2006;Freeman & Nicholson, 1999;Freeman et al., 2005;  (2006) Rats Aversive Electrolytic lesion in perirhinal cortex Lesion impaired A+/AX discrimination and summation but not retardation test expression in an eyeblink conditioning procedure; no effect on differential inhibition (A+/X− ). Freeman et al.
Rats Aversive Micro-infusions in cerebellum (anterior interpositus nucleus): muscimol (GABA A receptor agonist) Muscimol inactivation of the cerebellum during excitatory training (ipsilateral to the conditioned eye) blocked excitatory (A+) eyeblink conditioning. Muscimol inactivation of the inhibitory training suppressed A+/AX− discriminative responding but not the subsequent drug-free A+/AX− discrimination, or retardation test expression of conditioned inhibition. Nolan & Freeman (2005) Rats Aversive OX7-saporin micro-infusion in cerebellar cortex (Purkinje cells) Purkinje cell destruction impaired excitatory re-acquisition and conditioned inhibition in an eyeblink conditioning procedure.  Rats Aversive Anterior interpositus nucleus Neuronal activity increased in response to the conditioned excitor (A+) and reduced in response to the conditioned inhibitor (X− ) or safety compound (AX− ) in an eyeblink conditioning procedure. Nolan et al. (2002) Rats Aversive Picrotoxin micro-infusion in anterior interpositus nucleus Blockade of GABA A receptors reduced the conditioned response during excitatory trials but had no effect on the conditioned response during inhibitory trials in an eyeblink conditioning procedure. Freeman & Nicholson (1999) Rats Aversive Interpositus nucleus and lateral pontine nuclei Neuronal activity increased in response to the conditioned excitor (A+) and reduced in response to the safety compound (AX− ) in an eyeblink conditioning procedure. Watkins et al.

Rats
Aversive Electrolytic lesions to amygdala, dorsal raphe and ventral medullary; spinal cord laminectomy and cut Backwards conditioning design previously confirmed to produce conditioned inhibition (Wiertelak et al., 1992a). Conditioned inhibition of analgesia was abolished by lesions to the raphe dorsalis, raphe magnus and spinal dorsolateral funiculus. Amygdala lesions were without effect on conditioned inhibition of analgesia. Wiertelak et al. (1992a,b).

Rats
Aversive Morphine analgesia (morphine sulphate 2 mg/kg s.c.); intrathecal injection 5µg morphine sulphate and/or 1. Nicholson Nolan & Freeman, 2005;Nolan et al., 2002). Studies of conditioned inhibition have also extended beyond the brain. In a backwards conditioning design (Maier et al., 1976), safety signals were confirmed as conditioned inhibitors by summation and retardation tests (Wiertelak et al., 1992a). Inhibition was subsequently demonstrated at the neuronal level in the spinal cord following the release of cholecystokinin, which inhibits conditioned fear-induced hypoalgesia (Wiertelak et al., 1992b). Moreover, lesions to the spinal cord and the descending projections from the brainstem and midbrain, but not amygdala lesions, prevent the conditioned fear-induced hypoalgesia, inducing 'anti-analgesia' by interfering with opiate and nonopiate analgesia signalling mechanisms (Watkins et al., 1997(Watkins et al., , 1998. The anti-analgesia component of the safety response is thus partially independent of the fear circuitry and can be behaviourally silent. This body of work also provides evidence for opponency outside of the DA pathways.

Neural substrates of feature negative discrimination
The appetitive studies summarised in Table 2 include mostly lesion methods but also systemic drug studies, fMRI and a dietary intervention. These confirm the modulatory role of acetylcholine (Macleod et al., 2006, and identify mPFC Meyer & Bucci, 2014), hippocampus (Holland et al., 1999), posterior parietal cortex (Robinson & Bucci, 2012) and retrosplenial cortex (Keene & Bucci, 2008;Robinson et al., 2011) as brain regions involved in regulating the learning of feature negative discriminations. Non-specific neuronal disturbance caused by the interference with the blood-brain barrier produced by eating a high energy diet (Kanoski et al., 2010) may also be sufficient to impair feature negative discrimination learning. Lesions to the CeA were without effect on learning appetitive feature negative discriminations (Holland, 2012). Further negative findings from studies investigating structures of a priori interest are also included in Table 2.
As shown in Table 4, additional substrates (including amygdala, posterior parietal cortex, thalamus, inferior colliculus, periaqueductal grey,) have been identified for feature negative discrimination, compared to conditioned inhibition as confirmed by summation and/or retardation test. Some of these differences likely relate to the relatively greater use of feature negative discrimination procedures with shock stimuli of sufficient intensity to engage the periaqueductal grey (Fendt, 1998(Fendt, , 2000Waddell et al., 2003). Some differences are likely attributable to the use of acoustic cues, which require auditory processing in the thalamus and inferior colliculus , 2006McIntosh & Gonzalez-Lima, 1995).

Neural substrates of differential inhibition
The four appetitive studies summarised in Table 3 used mostly drug treatments and also include an electrophysiology study. These evidence the modulatory role of DA (Brom et al., 2016;Morutto & Phillips, 1999;Phillips & Morutto, 1998), and identify the reticular nucleus of the thalamus (Moldavan, 1999) and the lateral hypothalamus (Morutto & Phillips, 1999;Phillips & Morutto, 1998) as brain regions mediating (effects on) the learning of differential inhibition.
The aversive studies summarised in Table 3 are more numerous and used diverse methodological approaches: in addition to electrophysiology, drug treatments, fMRI and lesions, for example also using fluorodeoxyglucose uptake of brain regions, extracellular citrulline to measure activation of the nitrergic system, genetic approaches (comparing polymorphisms) and comparisons between other human participant groups. Together these studies confirm the modulatory role of DA (Yan et al, 2019) and/or NA (Gruss et al., 2016), acetylcholine (Palmisano et al., 2022;Thiel et al., 2002;Whalen et al., 1994) and glutamate at NMDA receptors (Jami et al., 2007;Murphy & Glanzman, 1999). Aversive studies also identify VTA (Yan et (Lindner et al., 2015) as brain regions involved in regulating the learning of differential inhibition. Aversive differential inhibition is also modulated by more general effects on arousal caused by activity in the reticular formation (Pascoe & Kapp, 1993) or differences in catecholamine levels (Gruss et al., 2016). As was the case for the feature negative discriminations, substrates in addition to those identified for conditioned inhibition as confirmed by summation and/or retardation test have been identified as important for differential inhibition. As above, some such differences are likely attributable to the modality of the cues in use (Moratti et al., 2017).

Conclusions and implications
The test procedures required to confirm conditioned inhibition have been identified as barriers to its wider use in experimental studies (Papini & Bitterman, 1993), to the extent relaxation of these stringent requirements has been suggested (Sosa & dos Santos, 2019;Sosa & Ramίrez, 2019). Certainly studies of feature negative (A+/AX− ) discrimination and differential inhibition (A+/X− ) procedures have been useful to help us to understand the differences in sensitivity to fear and safety contingencies seen under experimental conditions and we have included review of the neural substrates of differential inhibition Both nicotine and RJR-2403 improved A+/AX− discrimination performance. Nicotine increased the conditioned response to A+ and reduced responding to AX− . Since RJR-2403 only increased conditioned responding to A+, it is unlikely that nicotine's effect on learning the A+/AX− discrimination was mediated by α4β2 nAChRs alone.
MacLeod & Bucci (2010) Rats Appetitive Neurotoxic lesion in mPFC (PL or IL region) PL lesions made prior to, but not following, training slowed the rate of serial feature negative (A+, X → A− ) discrimination learning. Thus, PL lesions disrupted the acquisition but not the performance of serial feature negative discrimination. IL lesions were without any effect (before or after training). Kanoski et al.
Rats Appetitive High energy diet (high in glucose and saturated fat) to compromise blood-brain barrier integrity and hippocampus Rats on the high energy diet showed impaired acquisition of A+/AX− discrimination and responded more to AX− compared to those on the control diet. Impaired performance in this hippocampal-dependent task was associated with reduced blood-brain barrier integrity. Keene & Bucci (2008) Rats Appetitive Electrolytic lesions in retrosplenial cortex Retrosplenial cortex lesions impaired the acquisition of the feature negative discrimination. Conditioning to a lone excitatory stimulus was normal, consistent with a role in the processing of simultaneously presented stimuli. Macleod et al.
Rats Appetitive s.c. nicotine (nAChR agonist); 0.35 mg/kg Nicotine improved the magnitude and speed of acquisition of serial A+, X → A− discrimination learning in rats. The enhanced discrimination persisted even after seven days without further drug injection. Holland et al., Rats Appetitive Excitotoxic lesion in hippocampus Hippocampal-lesions impaired serial feature negative (A+, X → A− ) discrimination as measured by food-cup activity and food-cup directed behaviours. Serial feature positive (X → A+, A− ) discrimination was little affected and nonconditional discriminations were unaffected by the hippocampal lesions. Laing et al.
Humans Aversive fMRI whole-brain corrected Activation to feature negative discrimination (A+/AX− ) compared with a standard safety signal (BC− Humans Aversive fMRI of ventral mPFC in children with and without anxiety disorders Children with anxiety disorders displayed more fMRI activity in the right ventral mPFC in response to the safety (AX− ) versus novel (AB+) compound in an aversive noise prediction task. Children without anxiety disorders showed more fMRI activity in the right ventral mPFC to the novel (AB+) versus the safety (AX− ) compound.
(continued on next page) H.J. Cassaday et al. Systemic and BLA infusion of either DA D1 receptor agonist or antagonist impaired fear suppression during AX− in a fear conditioning procedure. Systemic administration of D1 receptor agonist reduced reward seeking response to the reward cue in animals also trained on threat and safety cues, but not in animals trained on only reward or nonreward cues. BLA infusion of either DA D1 receptor agonist or antagonist had no effect on reward seeking behaviour. Lack of effects in comparison task suggests impairments not mediated by cue discrimination. Inactivation of the IL region impaired A+/AX− discrimination between fear and safety cues in a fear conditioning procedure, but was without effect on cued reward-seeking. Inactivation of the PL region impaired discriminative reward seeking but not the A+/AX− discrimination. Sangha et al. (2013) Rats Aversive (with appetitive comparison) Basal amygdala: in vivo single unit recordings Neurons altered their firing rate to AX− but not A+ in a fear conditioning procedure. Response profiles showed two subpopulations (1) 'safety' responding to the AX− compound alone, and (2) 'safety + reward' responding to AX− and the separately trained reward cue. Glover et al. (2013) Humans Aversive Oestrogen receptors: compared follicular (lower oestrogen) and luteal (higher oestrogen) phases of menstrual cycle Lower oestrogen in women was associated with impaired AB+/AX− discrimination in a fear potentiated startle procedure. Kazama et al. (2012) Non-human primates Aversive Neonatal excitotoxic lesions to amygdala Neonatal lesion to the amygdala did not significantly impair adults' overall AB+/AX− discrimination performance in a fear potentiated startle procedure. However, the two neonatally-lesioned animals with the most extensive damage never learnt the safety discrimination. Jovanovic et al. (2010a, b) Humans Aversive HPA axis in PTSD patient and matched control groups PTSD subjects demonstrated behaviourally impaired AX+/ BX− discrimination in a fear potentiated startle procedure, in the absence of any impairment in contingency awareness. Plasma adrenocorticotropic hormone levels were positively correlated with startle responses in the PTSD group but not in the control group. Cortisol levels were not correlated with startle responses. Zyablitseva et al.  an auditory X− . Fear potentiated startle with an auditory A+ was also impaired. Waddell et al. (2003) Rats Aversive Excitotoxic lesions to superior colliculus or the dorsal PAG Post-training lesions of the superior colliculus, but not the dorsal PAG, impaired serial A+/X → A− discrimination in a fear potentiated startle procedure. Heldt et al. (2002) Rats Aversive Aspiration lesions to hippocampus Post-training, but not pre-training, hippocampal lesions impaired serial A+/X → A− discrimination in a fear potentiated startle procedure. After additional training, hippocampally lesioned rats learned to discriminate A+ and X → A-. Fendt (2000) Rats Aversive Micro-infusions in dorsal PAG: kainic acid (AMPA/kainate receptor agonist); NBQX (AMPA/kainate receptor antagonist); picrotoxin (GABA A receptor antagonist) and piperidine (GABA A receptor agonist) Modulation of AMPA/kainate receptor activity in PAG influenced the expression of fear potentiated startle. However, neither modulation of AMPA/kainate receptors nor of GABA A receptor activity had any effect on A+/AX− discrimination. Vouimba et al. (2000) Mice Aversive Electrolytic lesions to dorsomedial PFC Lesions to dorsomedial PFC overall increased freezing in a fear conditioning procedure but did not affect acquisition of the A+/AX− discrimination. Shul'gina (1998) Rabbits Aversive s.c. sodium nitrite (11 or 5.5 mg/kg; nitric oxide producing compound) Sodium nitrite impaired feature negative discrimination in a fear conditioning procedure (cutaneous shock to hind limb UCS), by increasing conditioned responding to the safety compound (AX− ). Heldt & Falls (1998) Rats Aversive Electrolytic lesions to auditory thalamus Lesions of the auditory thalamus impaired acquisition of the conditioned response to an auditory conditioned excitor (A+) but did not impair A+/AX− discrimination with an auditory inhibitor (X− ) in a fear potentiated startle procedure. Fendt (1998) Rats Aversive Micro-infusions in dorsal, lateral or ventrolateral PAG: picrotoxin (GABA A receptor antagonist) Infusion into the dorsal PAG impaired A+/AX− discrimination in a fear potentiated startle procedure, but was without effect on potentiated startle per se. Picrotoxin infusion into the lateral PAG increased the expression of potentiated startle, while infusion in ventrolateral PAG blocked it, but these infusions were not shown to affect A+/ AX− discrimination. Gewirtz et al. (1997) Rats Aversive Electrolytic lesions to ventral mPFC mPFC lesions did not disrupt A+/AX− discrimination in a fear potentiated startle procedure. Falls et al. (1997) Rats Aversive Electrolytic lesions to perirhinal cortex Lesions of the perirhinal cortex did not impair expression of a previously acquired A+/AX− discrimination in a fear potentiated startle procedure. Some initial impairment in the expression of fear potentiated startle to the A+ but X− retained its ability to inhibit fear-potentiated startle to the retrained A+.

Rats
Aversive; one group underwent A+/ AX− training Auditory system: autoradiography; structural equation modelling A+/AX− discrimination training with tone A and light X; control group trained with tone excitor. Medial geniculate nucleus showed reduced activity in response to AX− . Structural equation modelling suggested differences in activity patterns in the auditory system depending on whether the same auditory stimulus (A) was accompanied by the inhibitor (X). Falls & Davis (1995) Rats Aversive Electrolytic lesions to amygdala Amygdala lesions did not impair expression of a previously acquired A+/AX− discrimination in a fear potentiated startle procedure.

Rats
Aversive Auditory system: autoradiography; structural equation modelling A+/AX− discrimination training with tone/light identities of A and X counter-balanced. Rats trained with the tone as X demonstrated reduced activity in the dorsal cochlear nucleus and external nucleus of the inferior colliculus, compared to those in which the tone was trained as A. The opposite associative significance of the tone was also reflected at the network level. Schmajuk & DiCarlo (1992) N/A; computer model Aversive Computer-simulated lesion to hippocampus tested in simulated A+/AX− discrimination procedure; modelling based on multiple animal studies of classical conditioning In contrast to previous experimental findings (prior to 1992), the model simulations predicted hippocampal lesions should impair A+/AX− discriminations. Kirkpatrick-Steger et al.

Rabbits
Aversive Electrolytic lesions to hypothalamus or mesencephalon Lesions of the lateral and posterior hypothalamus had no effect on the expression a previously acquired A+/AX− discrimination in a nictitating membrane response preparation. Some transient impairment following lesions of the mesencephalon. ( Table 3) by way of comparison and context for consideration of the neural substrates of feature negative discrimination (Table 2) and conditioned inhibition as confirmed by summation and/or retardation test (Table 1). The feature negative discrimination protocol has been argued to provide a more convincing proxy for conditioned inhibition than differential inhibition because of the behavioural conflict generated by the need to withhold responding when the excitatory (A) and the inhibitory (X) cues are simultaneously presented in the compound (AX− ) trials (Sosa & dos Santos, 2019;Sosa & Ramίrez, 2019). This follows because, according to the Rescorla-Wagner model, conditioned inhibition requires that X acquires negative associative strength (Rescorla & Wagner, 1972). However, it must be acknowledged that the AX+/BX− design which is widely used in studies of fear conditioning also has advantages (Myers & Davis, 2004;Jovanovic et al., 2009). When summation and/or retardation tests are passed, safety signals can be identified as conditioned inhibitors, but -although fear conditioning studies have dominated the behavioural neuroscience literatureconditioned inhibitors do not necessarily indicate safety. Inhibitory learning is a more general phenomenon demonstrated also in positively motivated conditioning situations and hence spanning motivationally opposed systems. Evidence for the role of NAc and other parts of the striatum, which receive the densest DA projections, is notable by its absence. This lack of evidence may relate to the largely overlooked complexities introduced by the heterogeneity in DA neurons in VTA and its projection areas (Lammel et al., 2008(Lammel et al., , 2011(Lammel et al., ,2012Morales & Margolis, 2017;Verharen et al., 2020). Moreover, where there is evidence to implicate NAc, this seems to relate to the conditioned reinforcement component of inhibitory learning (Fernando et al., 2014b;Josselyn et al., 2005). The role of NAc in appetitively-motivated conditioned inhibition remains to be determined.
Nonetheless, systematic examination of its neural substrates of inhibitory learning (broadly defined) reveals relatively consistent evidence implicating DA and its mesocorticolimbic projection areas, including the amygdala and mPFC (Fig. 3), as measured by a variety of procedures and across different motivational states in the case of mPFC (Table 4). In the case of amygdala, the available evidence suggests that the role of amygdala may be restricted to aversively motivated procedures, in both animal studies (Collins & Paré, 2000;Ostroff et al., 2010Ostroff et al., , 2012Ostroff et al., ,2014Genaud-Gabi et al., 2013;Sangha et al., 2013;Harris et al., 2015;Ng et al., 2018), as well as in the available human studies which have used fear conditioning (Straube et al., 2014;Lindner et al., 2015;Weber et al., 2016;Muench et al., 2021). Negative or inconclusive evidence in aversively motivated procedures has also been reported following neonatal amygdala  and CeA lesions (Falls & Davis, 1995). An appetitively motivated study of feature negative discrimination following CeA lesions reported negative findings (Holland, 2012). CeA is the most likely candidate subregion for appetively motivated inhibitory learning because of its role in appetitive excitatory learning (Parkinson et al., 2000;Everitt et al., 2003;Knapska et al., 2006). The role of BLA in appetitively motivated procedures remains to be determined. As above, heterogeneity in the DA projections points to the need for more precise molecular tools, to target distinct neuronal populations and their projection areas in future in vivo studies of inhibitory learning (Lammel et al., 2008(Lammel et al., , 2011(Lammel et al., , 2012. Even taking such complexity into account, the role of BLA may turn out to be restricted to the aversive domain. In an optogenetic study of conditioned inhibition (AX+, BX− ), photoinhibition in BLA following presentation of the BX− compound slowed discrimination learning and reduced summation test performance (Sengupta et al., 2018). In the same study, BLA photoinhibition was also found to impair fear conditioning, to increase fear loss during extinction, to impair the relearning of fear but not appetitive conditioning (when the extinguished fear CS was paired with sucrose pellets), whilst having no effect on simple fear discrimination or latent inhibition. Taken together, these findings have been argued to suggest that BLA maintains aversive emotional salience (Sengupta et al., 2018).
There have been fewer studies, but retrosplenial cortex is consistently implicated in inhibitory learning (Keene & Bucci, 2008;Nelson et al., 2018;Robinson et al., 2011). Retrosplenial cortex and other projection areas implicated in inhibitory learning have in common that they are identified with the broader construct of relational learning, of the relationships between individual stimuli as measured in other kinds of task, for example spatial navigation processing cues separated in space or episodic memory tasks processing the order in which events occur (Eichenbaum & Cohen, 2014). Relational learning is required to form more complex conditional associations between stimuli and/or over a time interval (Cassaday et al., 2014;Holland et al., 1999;Kochli et al., 2015;Raybuck & Lattal, 2014;Robinson et al., 2011). The amygdala role in relational memory is more affective, mediating the emotional state of relief (and possibly also frustration and/or disappointment) engendered in the inhibitory learning network (Cahill et al., 1995;Hermans et al., 2014;McGaugh et al, 1996). Retrosplenial cortex, part of the circuitry necessary for episodic memory, is compromised in disorders which impair memory (Vann et al., 2009) and is increasingly of interest in relation to Alzheimer's because of the early changes seen here (Poirier et al., 2011). To date, however, relevant research has primarily focused on memory as measured by spatial and object recognition procedures rather than associative learning tasks (de Landeta et al., 2020;Walsh et al., 2022). Hippocampus, which shows a fairly consistent pattern of involvement in inhibitory learning across various task variants, has been identified with relational learning (Bergmann et al., 2016;Jarrard, 1993;Konkel & Cohen, 2009;Lavenex et al., 2006;Monti et al., 2015;Schmajuk & DiCarlo, 1992).
If this is the correct interpretation -that relational learning is an underpinning mechanism -then it should follow that other structures involved in relational processing should similarly impair inhibitory learning across the range of task variants. For example, damage to posterior parietal cortex which has so far been found to result in impairment in an appetitive feature negative discrimination procedure (Robinson & Bucci, 2012) should similarly impair performance in conditioned inhibition and differential inhibition tasks.
As well as the consistencies emerging across task and motivational variants, we also find differences in the substrates identified in appetitive and aversively motivated procedures. Some heterogeneity in terms of substrates is to be expected because of the motivational system engaged, and in some cases the modality of the CSs in use , 2006. Moreover the selection of target substrates for study in this context will be based on the established knowledge base. For example, the role of the periaqueductal grey in pain modulation and behavioural responses to aversive stimuli is fully consistent with its role in aversively motivated feature negative discrimination and differential inhibition procedures (Arico et al., 2017;Fendt, 1998Fendt, , 2000Lindner et al., 2015;Waddell et al., 2003;Walker et al., 2019). Sex differences in behavioural profile have been identified in fear motivated procedures: apparent lack of safety discrimination in freezing relating to sex differences in behavioural expression, females darting and males freezing in anticipation of fear (Greiner et al., 2019;Krueger & Sangha, 2021), though with some reported differences by the modality and timing of the stressor (Adkins et al., 2022). Sex differences are consistent with impairments in aversive feature negative discriminations found to be mediated by oestrogen receptors (Glover et al., 2013;Toufexis et al. 2007). Sex differences have received less attention in appetitive procedures but there is evidence for increased reward responsivity in females (Greiner et al., 2019). nAChR = nicotinic acetylcholine receptor; PAG = periaqueductal gray; PFC = prefrontal cortex; PL = prelimbic; PPT = propyl pyrazole triol; PTSD = post-traumatic stress disorder; s.c. = sub-cutanous; UCS = unconditioned stimulus.

Table 3
Differential inhibition studies.

Species
Procedure Target/treatment/measure Key findings Brom et al. (2016) Humans Appetitive Haloperidol (oral 3 mg; DA D2R antagonist) DA D2R antagonism reduced A+/X− discrimination, measured as participants' subjective ratings. Moldavan (1999) Cats Appetitive Neuronal activity: reticular nucleus (thalamus) Differential A+/X− neuronal response; background activity dampened in response to X− . In two thirds of neurons sampled, the introduction of X− also reduced the neuronal response to the A+ presentation. Morutto & Phillips (1999) Rats Appetitive Quinpirole (selective DA D2R & D3R agonist); micro-infusions in perifornical region (of lateral hypothalamus) Quinpirole dose-dependently impaired the acquisition of the CR to A+ but had no impact on the response to X− . Phillips & Morutto (1998) Rats Appetitive Sulpiride (DA D2R & D3R and 5-HT1a antagonist); micro-infusions in lateral hypothalamus Sulpiride dose-dependently enhanced the acquisition of a conditioned response to A+ but had no impact on the response to X− . Palmisano et al. (2022) Humans Aversive Nicotine (oral 2 mg) Nicotine-treated participants with high trait anxiety showed reduced differential inhibition (exploratory analysis). Labrenz et al.
Humans Aversive CRH receptor polymorphisms (HPA axis) CRHR1 risk allele rs17689918 was associated with reduced BOLD response to differential conditioning in the frontal regions. In females, this risk allele was associated with an increased amygdala activation in response to X− specifically. Gruss et al. (2016) Humans Aversive COMT val158met polymorphisms Polymorphisms in COMT are associated with A+/X− discrimination ability. Met allele carriers demonstrated an impaired A+/X− discrimination compared to non-Met carriers.  Rats Aversive Nitrergic system in the mPFC; N ω -propyl-L-arginine (neuronal NO synthase inhibitor; microdialysis) Citrulline levels (a by-product of NO synthesis) were significantly higher in response to A+ compared to X− . Inhibition of NO production impaired A+/X− discrimination, increasing the conditioned response to X− .

Rats Aversive
Nitrergic system in the mPFC Animals that performed well on the A+/X− discrimination task had higher citrulline levels compared to those that performed poorly. Harris et al. (2015) Mice Aversive; +/X− (explicitly unpaired training procedure) Fear circuity; fMRI Paired fear conditioning led to a higher BOLD response to the CS in the left amygdala compared to unpaired fear conditioning. Unpaired conditioning had a marginally higher BOLD response in the NAc core, though not significantly. Lindner et al. (2015) Humans Aversive Amygdala, anterior insula, ACC, PAG, and vmPFC; fMRI BOLD responses were higher for A+ compared to X− in the anterior insula, the ACC and PAG. Conversely, BOLD responses were higher for X− (compared to X− ) in the vmPFC. Straube et al.
Humans Aversive HTR1A polymorphisms; fMRI During A+/X− discrimination acquisition, HTR1A rs6295 risk allele carriers (with panic disorder and agoraphobias) typically exhibited higher activity in the amygdala and hippocampus, as well as in the parietal, cerebellar and temporal regions.
(continued on next page) H.J. Cassaday et al. Work in behavioural neuroscience has been more focused on fear conditioning procedures because of the implications for our understanding of anxiety (Gadenzam et al., 2013). It is also a simpler model to use than appetitive learning: conditioning and retrieval test stages can be completed in just two days in animal studies. In human studies, safety learning has been reported to be enhanced or impaired in connection with different kinds of anxiety-related and post-traumatic stress disorder symptoms (Orr et al., 2000;Jovanovic et al., 2010bJovanovic et al., , 2012Jovanovic et al., , 2013Duits et al., 2021). In addition, a recent meta-analysis confirmed reduced acquisition of differential inhibition of fear responses in individuals with schizophrenia (Tuominen et al., 2022). Hence, in clinical settings, individual inhibitory learning profiles could in principle be used as prognostic markers for individualised precision interventions (Duits et al., 2021). It has been suggested that safety learning approaches to understanding health anxiety will have particular implications for our understanding of the mental health impact of the Covid-19 pandemic (Thurston & Cassaday, 2022). Safety signals produced by hand washing and sanitisation (e.g. the smell of soap and alcohol) will develop secondarily reinforcing properties. Mask wearing and social distancing will also become established as safety behaviours which may be difficult to reverse even when the objective risk is reduced. In line with this account, the mental health impact of the Covid-19 pandemic has been particularly severe for those with pre-existing obsessive compulsive disorder (OCD) (Jassi et al., 2020) or health anxiety (Cannito et al., 2020).
The neural substrates of appetitive inhibitory learning have been under-investigated in comparison to the substrates of safety signal learning in aversive inhibitory procedures, this difference being most stark for differential inhibition procedures (Table 4). However, learning about signals of frustration and disappointment will have implications for human health and wellbeing: for our understanding of impulsive decision making in general (Hertel, 2007), as well as of diagnosed disorders and individual differences. Conditioned inhibition measured in an appetitively motivated procedure was impaired in schizophrenia and Lesions of the telencephalon did not impair the acquisition of differential inhibition.
Chen et al.

Rats
Aversive; A+/X− plus standard fear conditioning group (A+) Anterior piriform cortex; electrophysiology Both standard and differential conditioning reduced spontaneous single-unit activity in the anterior piriform cortex. The size of the receptive field reflected the extent to which fear was generalised, and was increased by standard conditioning and decreased by differential conditioning. Jami et al. Animals that underwent differential inhibition training had reduced fluorodeoxyglucose uptake in numerous regions including within the basal forebrain, the hippocampus, the retrosplenial cortex and the inferior colliculus. Collins & Paré (2000) Cats Aversive Lateral amygdala; electrophysiology Auditory-evoked increases in firing responses in the lateral amygdala did not differ between A+ and X− before differential conditioning. Following differential conditioning, the firing response increased to A+ and decreased to X− . Murphy & Glanzman (1999) Sea slug (Aplysia) Aversive; cellular analogue of A+/X− discrimination procedure.
Glutamatergic system; APV (NMDA antagonist; 100 µM; in artificial seawater preparation) Differential conditioning in normal artificial seawater (control) lead to a differential enhancement of EPSPs (higher for A+). This differential response was prevented in the presence of APV. Whalen et al. (1994) Rabbits Aversive Nucleus basalis of Meynert (basal forebrain); EEG Out of the neurons that demonstrated a differential activation, the majority responded to A+ with an increase in activity and to X− with a decrease; however a few neurons demonstrated the opposite profile of activity. Pascoe & Kapp (1993) Rabbits Aversive Dorsolateral mesopontine reticular formation; extracellular electrophysiology Some neurons (23/55) demonstrated differential activity in response to A+ versus X− , though also with some variability.

Gonzalez-Lima
& Agudo personality disorder (He et al., 2011(He et al., , 2012 and reduced in individuals with higher neuroticism and behavioural inhibition (He et al., 2013). Some divergence in terms of the underlying neural pathways is to be expected based on the motivational valences of the excitatory versus inhibitory cues, but more focused approaches should improve translational relevance. For example, Pavlovian appetitive procedures using food UCS have direct relevance to our understanding of human food intake. In its widest sense, inhibition (or rather lack thereof) is thought Legend: Overview of neural substrates targeted, by task motivation and inhibitory learning variant in use. See Tables 1-3 for fuller study details. Negative findings for targets of a priori interest are included (-ve). Abbreviations: AMPA = α-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid; GABA = gamma aminobutyric acid; 5-HT = 5-hydroxytryptamine; mPFC = medial prefrontal cortex; NMDA = N-methyl-D-aspartate. Fig. 3. The neural substrates most consistently identified with inhibitory learning (as measured in conditioned inhibition, feature negative discrimination and/or differential inhibition tasks) by task motivation: both aversively and appetitively motivated (dark green shaded) or aversively motivated only (light orange shaded). To date no areas have been identified with appetitively motivated inhibitory learning only. Projections are shown as dark green (directly implicated in inhibitory learning) or light grey lines (established projections consistent with the role of the interconnected structures implicated). Abbreviations: HPC = hippocampal formation; mPFC = medial prefrontal cortex; RSC = retrosplenial cortex; VTA = ventral tegmental area. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.) to play a role in the obesity epidemic. For example, based on the experimental findings from a Go-NoGo task, it has been suggested that deficits in the ability to restrain responses to food could lead to overeating, resulting in obesity (Chen et al., 2018). An fMRI imaging study investigating the effects of body mass index (BMI) on the neural processes involved in rating a pleasant food odour CS+ (in a hungry versus satiated state), found an inverse correlation between BMI and levels of activation in reward (and olfactory and memory) areas. For example, high BMI was associated with decreased activation of the right caudate nucleus, supporting the argument that the conditioned reward response is blunted in obesity (Jacobson et al., 2019). Such procedures could be adapted to measure responses to inhibitory cues. Moreover, if impairments to inhibitory learning lead to overeating, and eating a high-energy diet leads to impairments to inhibitory learning (Kanoski et al., 2010) then we have a positive feedback loop. With respect to the translational relevance to addiction, the sensitization to amphetamine produced by repeated intermittent drug exposure is context-dependent and such sensitization is subject to conditioned inhibition in that it can be blocked by contextual cues that predict the absence of drug. Consistent with other evidence, this inhibition was modulated by transient inhibition in mPFC, BLA or ventral subiculum (Guillory et al., 2022).
As in the case of fear conditioning and safety learning, many potentially relevant studies of appetitive inhibitory learning have not employed summation and/or retardation tests. As in the case of fear conditioning and safety learning, the same pragmatic approach can be applied to appetitive procedures. In contrast to the reassurance provided by safety signals in the context of feared outcomes, the emotional reaction to an inhibitor in the context of a positive outcome is one of frustration or disappointment (Gray & McNaughton, 2000). Thus, delineating the mechanisms of inhibitory discriminations will also help us to understand differences in sensitivity to reward and frustration. Reward sensitivity and conditioned inhibition have been argued to relate to impulsivity (Sosa & dos Santos, 2019;Sosa 2022). Impulsivity in turn contributes to a constellation of other symptoms and disorders, from those of obesity and drug addiction, to schizophrenia and personality disorder. Thus the implications of inhibitory learning studies extend well beyond the fear conditioning and safety learning studies which have been the primary focus of translational studies to date.
At the neural level, the breadth of inhibitory learning phenomena is underpinned by different motivational systems, and other fine-grained delineations of neural substrates relate to the conditioning procedure in use and in some cases the stimulus modality of the conditioning cues. In a general sense, the time for in depth understanding of neuronal correlates of behaviour has come (Cooper and Shallice, 2010;Lisman, 2015), and the translational implications of our understanding of the neural substrates of inhibitory learning go hand in hand with technological developments in the cognitive neurosciences. There will be challenges extrapolating from animal studies, not least the fact that individual variation in networks has already been identified in studies of patients, but these are exciting times (Denison and Morrell, 2022). Precision techniques for neuromodulation are already in use, primarily for neurological disorders at present, but with rapidly developing technological advances and new applications (Denison and Morrell, 2022;Lozano et al., 2019). Transcranial magnetic stimulation was approved by the UK National Institute of Health Care Excellence in 2018 with some UK National Health Service provision for depression and anxiety, plus for a range of other disorders, including drug addiction, borderline personality disorder and OCD, through private healthcare providers. Multi-session transcranial direct current stimulation over the PFC has been reported to improve cortical connectivity in disorders of consciousness (Zhang et al., 2021) and is starting to be trialled for a wider range of conditions. The use of deep brain stimulation (DBS) enables adjustable stimulation of deeper structures and is also an option to treat non-neurological conditions affecting limbic as well as motor systems. Mesolimbic and mesocortical DA systems have been targeted in preclinical trials for schizophrenia and studies of the effectiveness of DBS for OCD are ongoing (Lozano et al., 2019). At the same time, genome editing, silencing and regulation therapies are progressing to clinical trials to enable precision medicine for a number of neurodevelopmental and neurodegenerative disorders (Lubroth et al., 2021).
As we have argued, the emotional and motivational consequences of signals for the presence or absence of different kinds of outcome should also recruit additional brain systems, above and beyond the core substrates for inhibitory learning. This emotional aspect to inhibitory learning is important given focus of research effort to investigate fear and safety learning, with translational implications of our understanding of anxiety. Studies of appetitively motivated learning have provided a useful counterpoint and differences in motivation (safety versus frustration) can be an aspect of the translational fit (to anxiety versus drug addiction for example).
Examining inhibitory learning in both appetitive and aversive studies (as we have done here) has allowed us to identify brain areas and pathways common to both. Nonetheless, a further corollary would be that to minimise the emotional aspect of the behavioural task should help to directly identify whether there are any specific neural substrates required for establishing knowledge about the omission of an event per se, regardless of its motivational value. Indeed associations between neutral cues (measured in a sensory preconditioning procedure) have been successfully used to identify the substrates of dopaminergic excitatory learning (Young et al. 1998;Roughley et al., 2021). Inhibitory learning can also be established between neutral cues, also using sensory preconditioning designs (Artigas et al., 2001;Dwyer & Mackintosh, 2002;Espinet et al., 2004Espinet et al., , 2008, and confirmed in both summation and retardation tests (Artigas et al., 2001;Dwyer & Mackintosh, 2002;Espinet et al., 2004). In order to generate observable behaviour all these experiments required subsequent valuation of some of the cuesinhibition cannot be tested behaviourally without the induction of a motivational state. However, it would be possible to test the effects of brain manipulations during the acquisition stage, to see if there are neural substrates specific to the inhibitory learning process. Such procedures to examine associative learning between neutral cues would be the procedures of choice to identify any pure substrates of inhibition.

CRediT authorship contribution statement
H.J.C, C.B. and C.W.S. acquired the funding for the research on which this review is based. H.J.C., C.M. and C.W.S. developed the paper conceptualization. H.J.C., C.M., C.W.S., C.B., R.H. and L.W. contributed to the writing of the original drafts of the manuscript. All authors provided reviewed and edited the manuscript, and approved of the final version for submission.

Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Data availability
No data was used for the research described in the article.