Research ReportWhy trace and delay conditioning are sometimes (but not always) hippocampal dependent: A computational model
Highlights
► We present a hippocampal model of delay and trace conditioning. ► We additionally simulate the effects of hippocampal lesion on conditioning. ► Our model simulates hippocampal role in short and long delay/trace conditioning
Introduction
Classical conditioning, in which a cue (the conditioned stimulus or CS) is paired with a reflex-evoking unconditioned stimulus (US) until the CS comes to produce an anticipatory response (the conditioned response or CR) has proven a useful testbed for examining the psychological principles and neurobiological substrates of learning. Under many circumstances, delay conditioning, in which the CS and US overlap and co-terminate, is spared or even mildly facilitated following hippocampal damage (e.g., Berger et al., 1983, Gabrieli et al., 1995, Ito et al., 2005, Ito et al., 2006, Schmaltz and Theios, 1972); conversely, under many conditions, hippocampal lesion disrupts trace conditioning, in which CS offset occurs before US onset, producing a temporal gap known as the trace interval (e.g., Beylin et al., 1999, McGlinchey-Berroth et al., 1997, Solomon et al., 1986, Weisz et al., 1980). This has led some researchers to assume that the hippocampus plays a more important role in trace than in delay conditioning (Beylin et al., 1999, McGlinchey-Berroth et al., 1997, Solomon et al., 1986)—as we discuss below, this assumption ignores empirical findings on the role of the hippocampus in delay conditioning. Along the same lines, some studies of humans and non-human animals use trace conditioning as a canonical task to demonstrate evidence of hippocampal dysfunction in transgenic models, healthy aging, and pharmacological models (Brown et al., 2010, Disterhoft et al., 1999).
However, there are strong reasons to challenge this delay/trace dichotomy. First, it has long been known that the hippocampus shows learning-related changes during acquisition of the delay CR in intact animals and humans. These changes include the development of responses by some hippocampal pyramidal neurons that precede the behavioral eyeblink CR and mirror its form (Berger et al., 1976, Berger et al., 1983, Berger and Thompson, 1978, Green and Arenos, 2007, Thompson et al., 1980). Initially, these responses occur in the US period, but increases in the CS period occur at about the time that behavioral CRs appear, and decline with continued training (for review, see Christian and Thompson, 2003). Similar hippocampal activity occurs in rabbits given trace conditioning (Weiss et al., 1996) but not in rabbits given unpaired CS/US trials (Solomon et al., 1986).
Functional imaging studies in humans show a similar pattern of learning-related activity in the hippocampus during delay eyeblink conditioning to those observed in animals (Blaxton et al., 1996, Cheng et al., 2008, Knight et al., 2004 Stein and Helmstetter, 2004; Logan and Grafton, 1995, Schreurs and Alkon, 2001). Thus, these data suggest that – even if the hippocampus is not necessary for acquisition of a delay CR – it nevertheless normally plays a role. This challenges the simple view of delay conditioning as hippocampal-independent, and begs a more nuanced view of the difference between brain substrates that are sufficient to mediate a learned response, versus those that are normally involved.
Second, under many conditions, delay conditioning is spared or slightly enhanced following hippocampal lesion. Specifically, the ability of hippocampal lesioned animals to acquire a delay eyeblink CR depends on the length of the CS interval. Specifically, while hippocampal-lesioned rats can acquire an eyeblink CR when the delay between CS onset and US onset is short, they are impaired when the delay is lengthened (Beylin et al., 2001). Thus, short-delay conditioning is spared by hippocampal lesion, but long-delay conditioning is not. Further, disruption of the hippocampus, via electrical stimuli or pharmacological intervention, can retard acquisition of even a short-delay CR (Kaneko and Thompson, 1997, Sakamoto et al., 2005, Salafia et al., 1979, Salafia et al., 1977, Solomon and Gottfried, 1981, Solomon et al., 1983). Together, these results document that delay conditioning is not always spared following hippocampal lesion or disruption.
Third, although trace conditioning is often disrupted by hippocampal lesion, this is not always the case. For example, Thompson et al. (1980) speculated that the hippocampus might be involved in trace conditioning, to bridge the temporal gap between CS and US. Solomon et al. (1986) presented an early study showing that dorsal hippocampal lesions disrupted trace eyeblink conditioning in rabbits, by decreasing the number of CRs. However, other studies followed that reported no trace conditioning impairment in hippocampal-lesioned animals (James et al., 1987, Port et al., 1986).
Another factor affecting the hippocampal-dependence of trace conditioning may be differences in the trace interval used. In studies where the trace interval has been explicitly varied, a deficit in trace conditioning appears only for long trace intervals. Thus, for example, hippocampal-lesioned rabbits are impaired on eyeblink CR acquisition with a long (500 ms) but not a short (100 ms CS or 300 ms) trace interval (Moyer et al., 1990). There may also be interactions between CS duration, and trace interval: Steinmetz and colleagues (Walker and Steinmetz, 2008) found that hippocampal-lesioned rats were impaired relative to controls on acquisition of an eyeblink CR when the CS duration was 50 ms and the trace interval was 500 ms, but not when the CS duration was 500 ms and the trace interval was 50 ms. In addition, Shors and colleagues (Beylin et al., 2001) showed that – although hippocampal-lesioned rats were impaired at both trace and long-delay eyeblink conditioning – once the lesioned animals had acquired a long-delay CR, they could then learn and perform the trace CR. Together, these results document that, at least under some circumstances, subjects with hippocampal lesion can acquire a trace CR as well as matched controls.
In summary, while the idea that trace conditioning is hippocampal-dependent whereas delay conditioning is hippocampal-independent provides a useful rule of thumb, it is not sufficient to adequately address the full range of existing data. An additional complication involves the inter-stimulus interval (ISI). When the ISI is short, response systems (such as the brainstem and cerebellum for eyeblink conditioning) can successfully learn CS–US associations and produce a well-timed CR. When the ISI is longer, the hippocampus helps to bridge the temporal gap between CS and US, facilitating production of a well-timed CR. Thus, in the case of eyeblink conditioning, both short-delay and short-trace paradigms can be acquired without impairment by hippocampal-lesioned animals; however, both long-delay and long-trace paradigms are disrupted following hippocampal lesion.
Consistent with this view, although infant rats (with immature hippocampus) can acquire short-delay conditioning, they are impaired at both long-delay and trace conditioning, which emerge in parallel during later development (Barnet and Hunt, 2005, Ivkovich et al., 2000). Thus, these data all suggest that in addition to the presence of a trace interval, the duration between CS onset and US arrival determines whether hippocampal mediation is required, as is the case in short- and long-delay conditioning.
Interestingly, when Hoehler and Thompson (1980) first speculated that trace conditioning might be hippocampal dependent, they did so based on their studies of ISI manipulations in eyeblink conditioning, and on their findings that the hippocampus appeared to be involved in forming a temporal map of the learned behavioral response to be made, allowing for the CR to be accurately timed even when the ISI is beyond the timing parameters that are “optimal” for the basic associative substrate. Thus, for example, the “optimal” parameters for eyeblink conditioning (operationalized in terms of acquisition speed) may be a few hundred milliseconds and may reflect temporal processing mechanisms in other structures such as the cerebellum.
As discussed by Christian and Thompson (2003), optimal temporal parameters for learning in the cerebellum are 50–200 ms between CS and US presentation. Most trace conditioning studies on animals use a trace interval of 500 ms to induce hippocampal involvement in task learning. Optimal parameters for fear conditioning tend to be an order of magnitude longer (usually>1 s, see for example, Bevins and Ayres, 1995), and may reflect temporal processing mechanisms in the amygdala. But in fear conditioning, just as in eyeblink conditioning, hippocampal lesions affect trace conditioning as a function of the trace interval, so that hippocampal lesions impair expression of contextual fear conditioning with long but not short trace intervals (Chowdhury et al., 2005, Pang et al., 2010). In other words, we argue that the hippocampus plays a similar role in both eyeblink and fear conditioning, but temporal differences in optimal ISI length for eyeblink and fear conditioning acquisitions are, respectively related to processing in the cerebellum and amygdala.
Extending this principle beyond classical conditioning, other brain systems mediate other behavioral responses, and each may have an operating window of temporal delays that can be spanned; in each case, these brain systems alone may be capable of mediating learning with sufficiently short delays, but as the delay is lengthened, hippocampal mediation becomes critical. This basic idea is consistent with a large number of theories of hippocampal-region function (e.g., Hoehler and Thompson, 1980, Rawlins, 1985, Wallenstein et al., 1998) and finds broad support from a range of preparations. For example, in delayed non-matching-to-place in an eight-arm radial maze, rats with hippocampal lesion can learn under short delays, but are impaired when the delay period is extended (Lee and Kesner, 2003). Similarly, in delayed non-match to sample (DNMS), primates with lesions limited to the hippocampus (sparing nearby medial temporal areas) can learn the non-matching task as well as controls, but show increasing impairments as the delay between sample and response is lengthened (Zola-Morgan and Squire, 1986). In each case, the important factor determining hippocampal dependence is not presence or absence of a stimulus-free gap, but rather the length of the interval across which information must be maintained before responding.
Again, similar findings have also been reported in human studies. For example, humans with bilateral hippocampal damage are impaired relative to healthy controls on temporal and spatial estimation at long, but not short, delays (Kesner and Hopkins, 2001). Patients with medial temporal lobe and hippocampal lesions also perform much better on relational memory tasks when the interval between learning and memory test is short than long. Specifically, Squire and colleagues (Jeneson et al., 2010) found that patients with medial temporal lobe damage do not show impairment in performing relational learning tasks when the delay between learning and test is short (1 s). Similarly, Ryan and Cohen (2004) have tested amnesic patients on a relational memory task using both short and long delay periods. They have found that patients are impaired only for the long-interval condition.
Thus, a conceptualization of eyeblink conditioning in which the length of the ISI, in addition to the presence of a trace interval, determines hippocampal dependence would appear to help integrate our understanding of the eyeblink conditioning literature with that of other preparations.
In this paper, we present an extension of our prior trial-level computational models of hippocampal function and stimulus representation. Our models assumed that the hippocampal region interacted with other brain systems, such as cortex and cerebellum, during associative learning, specifically by forming new stimulus representations that provided information about stimulus–stimulus and contextual regularities (Gluck and Myers, 1993, Gluck and Myers, 2001, Myers and Gluck, 1994, Moustafa et al., 2009). These models were correctly able to account for the effects of hippocampal lesion and disruption on various trial-level phenomena such as acquisition, discrimination, latent inhibition, and contextual shift effects. A later extension which modeled the effects of cholinergic manipulations by altering the hippocampal region learning rate was correctly able to address the effects of cholinergic agonists and antagonists on classical conditioning (Myers et al., 1996, 1998; Moustafa et al., 2010). However, these earlier models simulated trial-level information only, meaning that they could simulate whether a CR is given on a particular trial, but could not address within-trial events, such as the relative timing of CS and US onset. As such, these earlier trial-level models could not address the differences between delay and trace conditioning, nor the effects of manipulating the length of the ISI or trace interval. The need to address these aspects of the empirical data partially motivates the current work.
Our new model simulates performance in various delay and trace eyeblink conditioning data within a unified framework. Specifically, the current model includes adaptive recurrent collateral connections that aid in the representation of intra-trial temporal information. With this model, as in our prior models, we argue that the hippocampus is a general-purpose system that learns to predict the next state of all stimuli given the current state of variables encoded by activity in recurrent collaterals. As such, the model correctly predicts that hippocampal involvement in associative learning, including classical conditioning, should be most critical not only when there is an intervening trace interval, but also when there is a long delay between CS onset and US onset, as in short-delay vs. long-delay conditioning.
Fig. 1 shows a schematic diagram of the current model, which builds off our prior models of hippocampal-region processes in classical conditioning. Like our earlier models (Gluck and Myers, 1993, Gluck and Myers, 2001, Moustafa et al., 2009), the present model conceives of the hippocampal region (Fig. 1, green) as a predictive autoencoder, which learns to predict the next state of the world given current inputs. In the process, the hippocampal-region network forms new stimulus representations in its internal layer that compress (or make more similar) the representations of co-occurring inputs while differentiating (making less similar) the representations of inputs that make different predictions about future events such as US arrival. In other words, the essential function of the hippocampus in our model is monitoring environmental regularities and using prior experience and current inputs to predict what (out of all possible events) is likely to happen next. Classical conditioning is a good example of prediction processes because the most salient event – the US arrival – can be predicted with high accuracy by learning the CS and the ISI. In our model, the hippocampal network learns not only whether a particular CS will be followed by a US, but when this will occur.
Also as in our prior models, the hippocampal-region network communicates with a second motor network (Fig. 1, blue), which is assumed to represent some of aspects cortical and cerebellar substrates of motor learning. The motor output network is modeled as a single adaptive node that learns to map from weighted inputs specifying the presence of CSs, as well as contextual or background stimuli and an efferent copy of the CR. The activities of the hippocampal-region network hidden layer units are also provided as inputs to the motor output network, allowing the motor output network to incorporate the adaptive representations formed in the hippocampal-region network into its own ongoing learning. The output from the motor response network represents the behavioral CR. The difference between this output (CR) and the US constitutes an error signal that can be used to train the connection weights in the motor response network, using an error correction rule such as the least-mean squares or LMS rule (Widrow and Hoff, 1960); full details of the learning rule and other model details are provided in Section 4.
The major differences between this model and the prior models are (1) the consideration of each trial not as a discrete event, but as a series of timepoints, (2) the addition of recurrent pathways within the hippocampal-region and motor output networks. We discuss each of these points below; full simulation details are provided in Section 4.
First, to simulate within-trial events, each trial is divided into a number of timesteps, which represent small time intervals within a conditioning trial (e.g., 50 ms). Typically, contextual inputs are present during the entire trial, and are present alone during the first several timesteps of a trial; then one or more CSs may be presented for a specified number of timesteps; the US, when present, appears for a single timestep. The US may overlap with the CS (as in delay conditioning) or may occur after CS cessation (as in trace conditioning). A further series of context-alone presentations ends the trial (simulating the intertrial interval or ITI). Fig. 2 provides schematic illustrations of some example paradigms that we simulate in our model.
A second difference between the prior and current models is that Fig. 1 includes recurrent connections within the hippocampal and motor output models. Specifically, the hippocampal region network includes recurrent connections within the internal layer, while the motor output network contains a feedback CR pathway, carrying information regarding the current state of the CR. Provision of feedback within the hippocampal network allowed the activation of internal layer nodes at any time step to be a function of external (CS and contextual) input and also of the adaptive representation of input from a previous timestep. Because the weights on these recurrent connections are adaptive, it is possible for a sequence of activation patterns to be stored in the network that “buffers” input information over several timesteps. Importantly, this buffering function is not pre-wired into the network, but emerges dynamically as a result of training and learning. Anatomical studies support the existence of such recurrent loops in the hippocampal region, particularly hippocampal subfield CA3 (Amaral et al., 1990, Amaral and Witter, 1989) as well as the dentate gyrus (Amaral et al., 2007). Prior models of the hippocampus have also included recurrent connections to simulate conditioning tasks (see for example, Rodriguez and Levy, 2001).
In this model as in our prior models, hippocampal lesion is simulated by disabling learning in the hippocampal-region network, in which case the motor output network can still learn new responses by modifying weights from the CS and contextual inputs, but no new adaptive stimulus representations are formed in the hippocampal region (Gluck and Myers, 1993, Myers and Gluck, 1994). In addition, and also as in our prior models, the effects of cholinergic agonists and antagonists are simulated by raising (agonist) or lowering (antagonist) learning rates in the hippocampal-region network (Myers et al., 1996, 1998; Moustafa et al., 2010).
Section snippets
Results
The recurrent model of Fig. 1 successfully simulates the basic findings usually interpreted as evidence that trace conditioning is hippocampal-dependent but delay conditioning is hippocampal-independent. Fig. 3A shows that, for a short ISI (ISI=4), delay conditioning (Fig. 2A) is acquired by the intact system more quickly than trace conditioning; this is consistent with empirical data (Beylin et al., 2001). As mentioned above, the role of the hippocampus in our model is predicting next state of
Discussion
Here, we have presented a computational model of the hippocampal region and its role in stimulus representation, that includes the ability to simulate within-trial events, and thus to address not only trial-level data regarding whether a behavioral response is emitted, but within-trial data regarding the timing of that response. Applied to classical eyeblink conditioning, the model is correctly able to account for the findings that, in intact animals, learning is slower as the ISI increases
Experimental procedures
The recurrent hippocampal model was implemented in objective-C++ using the Xcode 3.0 applications development suite for Macintosh OS 10.5.
Acknowledgments
This work was partially supported by the NSF/NIH Collaborative Research in Computational Neuroscience (CRCNS) program and by NIAAA (RO1 AA018737-01).
References (149)
- et al.
Neurons, numbers and the hippocampal network
Prog. Brain Res.
(1990) - et al.
The three-dimensional organization of the hippocampal formation: a review of anatomical data
Neuroscience
(1989) - et al.
Neuronal plasticity in the limbic system during classical conditioning of the rabbit nictitating membrane response. I. The hippocampus
Brain Res.
(1978) - et al.
The role of the hippocampus in trace conditioning: temporal discontinuity or task difficulty?
Neurobiol. Learn. Mem.
(2001) - et al.
Hippocampectomy disrupts the topography of the rat eyeblink response during acquisition and extinction of classical conditioning
Brain Res.
(1992) - et al.
Classical conditioning, awareness, and brain systems
Trends Cogn. Sci.
(2002) - et al.
Cholinergic facilitation of trace eyeblink conditioning in aging rabbits
Life Sci.
(1999) - et al.
Temporal encoding of place sequences by hippocampal cell assemblies
Neuron
(2006) - et al.
Interfering with theories of sleep and memory: sleep, declarative memory, and associative interference
Curr. Biol.
(2006) - et al.
Computational models of the hippocampal region: linking incremental learning and episodic memory
Trends Cogn. Sci.
(2003)
Different brain networks underlying the acquisition and expression of contextual fear conditioning: a metabolic mapping study
Neuroscience
Hippocampal and cerebellar single-unit activity during delay and trace eyeblink conditioning in the rat
Neurobiol. Learn. Mem.
Banishing the homunculus: making working memory work
Neuroscience
Hippocampal lesions and trace conditioning in the rabbit
Behav. Brain Res.
On the role of the hippocampus in learning and memory in the rat
Behav. Neural. Biol.
Hippocampal “time cells” bridge the gap in memory for discontiguous events
Neuron
The contribution of sleep to hippocampus-dependent memory consolidation
Trends Cogn. Sci.
Recall of fear extinction in humans activates the ventromedial prefrontal cortex and hippocampus in concert
Biol. Psychiatry
Classical conditioning of the rabbit’s nictitating membrane response under fixed and mixed CS–US intervals
Learn. Motiv.
A neural model of hippocampal-striatal interactions in associative learning and transfer generalization in various neurological and psychiatric patients
Brain Cogn.
A neurocomputational model of classical conditioning phenomena: a putative role for the hippocampal region in associative learning
Brain Res.
A computational model of cholinergic disruption of septohippocampal activity in classical eyeblink conditioning
Neurobiol. Learn Mem.
Methods for reducing interference in the complementary learning systems model: oscillating inhibition and autonomous memory rehearsal
Neural. Networks
Hippocampal and neocortical contributions to memory: advances in the complementary learning systems framework
Trends Cogn. Sci.
A comparison of latent inhibition and learned irrelevance pre-exposure effects in rabbit and human eyeblink conditioning
Integr. Physiol. Behav. Sci.
The dentate gyrus: fundamental neuroanatomical organization (dentate gyrus for dummies)
Prog. Brain Res.
Trace and long-delay fear conditioning in the developing rat
Learn. Behav.
Double dissociation of conditioning and declarative knowledge relative to the amygdala and hippocampus in humans
Science
Neuronal substrate of classical conditioning in the hippocampus
Science
Single-unit analysis of different hippocampal cell types during classical conditioning of rabbit nictitating membrane response
J. Neurophysiol.
One-trial context fear conditioning as a function of the interstimulus interval
Anim. Learn. Behav.
Functional mapping of human learning: a positron emission tomography activation study of eyeblink conditioning
J. Neurosci.
Application of a model to paired-associate learning
Psychometrika
Trace eyeblink conditioning is impaired in alpha7 but not in beta2 nicotinic acetylcholine receptor knockout mice
Front. Behav. Neurosci.
Attention, configuration, and hippocampal function
Hippocampus
Dissociable effects of hippocampus lesions on expression of fear and trace fear conditioning memories in rats
Hippocampus
Neural substrates underlying human delay and trace eyeblink conditioning
Proc. Nat. Acad. Sci. USA
Dorsal hippocampus involvement in trace fear conditioning with long, but not short, trace intervals in mice
Behav. Neurosci.
Neural substrates of eyeblink conditioning: acquisition and retention
Learn. Mem.
Classical conditioning and brain systems: the role of awareness
Science
Acquired equivalence and distinctiveness of cues: II. Neural manipulations and their implications
J. Exp. Psychol. Anim. Behav. Process
Dorsal versus ventral hippocampal contributions to trace and contextual conditioning: differential effects of regionally selective nmda receptor antagonism on acquisition and expression
Hippocampus
Dissociating space and trace in dorsal and ventral hippocampus
Hippocampus
Classical conditioning after temporal lobe lesions in man: impairment in conditional discrimination
Behav. Neurosci.
Altered brain response to verbal learning following sleep deprivation
Nature
Enhancement of passive avoidance learning through small doses of intra-amygdaloid physostigmine in the young rat. Its relation to the development of acetylcholinesterase
Dev. Psychobiol.
Effects of physostigmine and scopolamine on rats’ performances in object-recognition and radial-maze tests
Psychopharmacology (Berl)
A cholinergic-dependent role for the entorhinal cortex in trace fear conditioning
J. Neurosci.
Differential acetylcholine release in the prefrontal cortex and hippocampus during pavlovian trace and delay conditioning
Neurobiol. Learn. Mem.
Cited by (25)
Impact of fornix lesions on tone-off delay vs tone-on trace eyeblink conditioning in rats
2023, Physiology and BehaviorA neurocomputational model of creative processes
2022, Neuroscience and Biobehavioral ReviewsCitation Excerpt :Intrinsically, novelty per se can integrate previous knowledge in a novel manner, such as motor plans in the basal ganglia (BG) (Chakravarthy and Moustafa, 2018; Mandali et al., 2015, 2018; Moustafa et al., 2014, 2018; Moustafa and Srinivasa Chakravarthy, 2018; Redgrave, 2007; Srinivasa Chakravarthy and Moustafa, 2018; Houk, 2007), which is essential in the motor sequencing process. It is widely known that the cerebellum is essential in motor learning and control, particularly for the precise control of movements (Desmurget and Grafton, 2000; Garrido et al., 2013; Kalveram et al., 2005; Kawato and Gomi, 1992; Moustafa et al., 2013; Sergio et al., 2005; Stroeve, 1997). The cerebellum uses signals from the surrounding brain regions to learn and process sensorimotor information (from the associative sensory cortex and the PFC).
Antidepressant-relevant behavioral and synaptic molecular effects of long-term fasudil treatment in chronically stressed male rats
2020, Neurobiology of StressCitation Excerpt :Interestingly, similar observations were described by Bondi et al. using chronic desipramine treatment, which could prevent stress-increased anxiety-like behavior (Bondi et al., 2008). Both associative memory and anxiety-like behavior may comprise hippocampal circuitries, especially those involving the ventral hippocampus (Bannerman et al., 2003; Moustafa et al., 2013; Wang et al., 2015). However, the dorsal hippocampus is also sensitive to chronic stress (Luine, 2002) and antidepressants (Liu et al., 2012).
Case-based decision neuroscience: Economic judgment by similarity
2018, Goal-Directed Decision Making: Computations and Neural CircuitsHippocampal contributions to memory for time: evidence from neuropsychological studies
2017, Current Opinion in Behavioral SciencesCitation Excerpt :Such findings may be better understood with reference to the notion that the hippocampus ‘jumps back in time’ (discussed above). Specifically, a jump back is thought to provide information about how long ago the CS occurred so that an appropriately timed conditioned response can ensue ([21••,22], also see [59] for a different view). In other words, the hippocampus does not just serve as a bridge over a delay, but provides a contextual representation of when the CS was first presented.