Long time-scales in primate amygdala neurons support aversive learning

Taub, Aryeh H.; Shohat, Yosef; Paz, Rony

doi:10.1038/s41467-018-07020-4

Download PDF

Article
Open access
Published: 26 October 2018

Long time-scales in primate amygdala neurons support aversive learning

Nature Communications volume 9, Article number: 4460 (2018) Cite this article

2171 Accesses
7 Citations
3 Altmetric
Metrics details

Subjects

Abstract

Associative learning forms when there is temporal relationship between a stimulus and a reinforcer, yet the inter-trial-interval (ITI), which is usually much longer than the stimulus-reinforcer-interval, contributes to learning-rate and memory strength. The neural mechanisms that enable maintenance of time between trials remain unknown, and it is unclear if the amygdala can support time scales at the order of dozens of seconds. We show that the ITI indeed modulates rate and strength of aversive-learning, and that single-units in the primate amygdala and dorsal-anterior-cingulate-cortex signal confined periods within the ITI, strengthen this coding during acquisition of aversive-associations, and diminish during extinction. Additionally, pairs of amygdala-cingulate neurons synchronize during specific periods suggesting a shared circuit that maintains the long temporal gap. The results extend the known roles of this circuit and suggest a mechanism that maintains trial-structure and temporal-contingencies for learning.

Affective memory rehearsal with temporal sequences in amygdala neurons

Article 25 November 2019

Tamar Reitich-Stolero & Rony Paz

Neuromodulation in circuits of aversive emotional learning

Article 24 September 2019

Ekaterina Likhtik & Joshua P. Johansen

Neural dynamics underlying associative learning in the dorsal and ventral hippocampus

Article 03 April 2023

Jeremy S. Biane, Max A. Ladow, … Mazen A. Kheirbek

Introduction

In associative learning, the passage of time between trials—the inter-trial interval (ITI), can potentially serve as a cue of trial expectation. The time that passes from the offset of one trial carries information about the onset of the next trial, and if this time can be internally represented and kept, it can aid to form higher-order representations of the environment. Evidence that the ITI is indeed used for timing and auto-shaping of responses was first described in a series of classical work^1,2. Importantly, the ITI length affects acquisition rate and memory strength^3,4,5, and the ratio between the ITI and the inter-stimulus interval (ISI) predicts acquisition characteristics^6,7. This suggests that acquisition of associative memory involves assessment and integration of multiple temporal contingencies, and specifically, that temporal information about the ITI is internally represented even in the absence of behavioral changes during it.

Because the ITI usually lasts from few seconds to dozens of seconds, it raises the question where is this temporal lag being maintained? Neurons in the basolateral complex of the amygdala (BLA) play a role in acquisition and expression of affective associations^8,9,10, and show tonic responses and baseline changes that last after a specific trial or stimulus ends^11,12,13. The amygdala also plays a role in the acquisition of trace-conditioning, where a temporal gap between the conditioned stimulus (CS) and the unconditioned stimulus (US) must be bridged^14,15,16,17, and even in longer lags that were traditionally thought to be dependent on the hippocampus^18,19. In addition, the amygdala plays a role in second-order conditioning^20,21,22, which can be formed between the ITI itself and the CS²³ and even hold abstract representations of the trial structure beyond the CS-US relationship^24,25. Further, the amygdala can use its bidirectional connectivity with the anterior cingulate cortex (ACC)²⁶. The ACC is required for trace-conditioning with long gaps²⁷ likely due to its role in attention^27,28,29. It forms and integrates representations of task structure^29,30, and exchange information with the amygdala for flexible updating of contingencies^10,16,17,31.

We therefore hypothesized that the BLA together with the ACC hold a representation of long timescales during the ITI of an aversive conditioning task, even in the absence of an explicit external cue. To address this, we recorded the activity of single neurons in the amygdala and the dorsal ACC (dACC) in nonhuman primates during a tone-odor conditioning paradigm with a relatively long (dozens of seconds) ITI that varies on a trial-by-trial basis. Our results show that longer ITI duration increases rate and strength of aversive learning, and that single units in the primate amygdala and dACC develop and hold a temporal representation of this long temporal gap, both in single-cell activity and at the population level. As a result, this circuitry maintains timescales at the order of dozens of seconds and can contribute to the formation of trial-structure and modulate affective learning and memory.

Results

We recorded neuronal activity during the ITI in a tone (CS)-odor (US) conditioning task (n_days = 82), where an aversive odor was paired with a pure-frequency tone (250 ms, randomly chosen each session from 900 to 2400 Hz). Tones were triggered by onsets of respiration cycle detected in real-time, and odors were presented at the onset of the following respiration cycle (mean ISI duration of 1.6 ± 0.24 s). As shown in our previous studies^16,32 and here (Fig. 1a, b), pairing resulted in a CS-evoked augmented inhale that preceded the presentation of the odor, and reflects a learned preparatory conditioned response (CR).

ITI length contributes to learning rate

As might be expected from the long ITI duration and its trial-by-trial variability (mean ITI 38.7 ± 4.4 s, Fig. 1c), and from the fact that the following CS completely predicted the US, we did not observe any preparatory behavioral response during the ITI (Fig. 1d), neither in the depth of inhale modulation (Fig. 1e) nor in its frequency (Fig. 1f).

However, when sorting the CR based on the duration of the previous trial ITI, we found that the length of the ITI induced a higher CR, both in absolute value compared to habituation (Fig. 2a, b), and in increased learning rate (Fig. 2c). These results show that longer ITIs contribute to and enhance the learned behavioral response.

Neurons modulate their activity during the ITI

We recorded single-unit activity in the amygdala and the dACC (Fig. 3a, amygdala: n = 291; dACC: n = 263), and first validated previous results by comparing pre-CS to post-CS activity (paired t-tests, p < 0.05). We found that 24% (n = 70) of amygdala and 29% (n = 76) of dACC neurons had CS-evoked activity, confirming learning-related changes in these regions (p < 0.001, binomial test)^16,33.

We next turned to examine the activity during the ITI. On average, no specific epoch of the ITI showed modulation of activity (Fig. 3b). However, close inspection of the data suggested that many of the neurons do modulate their firing rates during the ITI, but over specific temporal scales (Fig. 3c). To test this, we analyzed the neuronal activity between the offset of each trial and the expected initiation of the next trial, namely the average ITI length (mean ITI), computed separately for each session. We omitted the bins where the CS occurred before the mean ITI to avoid CS-evoked activity in the analyses. Neurons’ discharge during the ITI was binned into segments, and the strength of the ITI modulation was assessed by comparing the original ITI firing rate (ITI-FR) in each bin to shuffled data (Fig. 3d). We found that the activity of roughly 50% of amygdala and dACC cells differed from baseline activity for 20% or more of the total duration of the ITI (Fig. 3e, p < 0.01, binomial test). For robustness, we tested and observed identical results under different segmentations (6, 12, 20, and 40 bins, Fig. 3d, e). For comparison, a null distribution derived from shuffled data showed that almost all cells have significant activity in <5% of ITI duration, as expected from chance-level modulation (Fig. 3e).

We conclude that in both structures, large and similar proportions (p > 0.1, binomial tests, Supplementary Fig. 1) of cells had significant modulation during the ITI.

ITI modulation strengthens during conditioning

If indeed ITI modulation contributes to the acquisition, then we can expect it to strengthen during learning. To evaluate the strength of the ITI-FR modulation we computed the excess in spikes after normalization to the mean and standard deviation of the shuffled data (Fig. 4a, b). The root mean square (RMS) of 21% (n = 61) of amygdala neurons and 26% (n = 68) of dACC neurons was increased compared to habituation in one or more stages of the acquisition (early, mid, or late, 10 trials each; one-way analysis of variance (ANOVA), amygdala: df = 60, f = 2.94, p = 0.04 and ACC: df = 67, f = 2.81, p = 0.046). These acquisition responsive neurons (23%, 129/554) were further inspected during same-day extinction that started 15–30 min after the acquisition ended. Their ITI modulation diminished significantly yet remained higher than habituation (one-way ANOVA, amygdala: df = 60, f = 3.66, p = 0.0098 and ACC: df = 67, f = 3.71, p = 0.008, Fig. 4b). This result suggests that during acquisition of emotional memory, the activity of neurons in the amygdala and the dACC becomes locked to cycles of CS-US and the long ITI (dozens of seconds) between them.

Although odor exposure can remain in the receptors for few seconds and even induce prolonged behavioral effect, it cannot account for the ITI modulation we observe. First, the neural modulation often occurred during the middle or late segments of the ITI (see also next sections), yet to completely exclude this possibility we introduced “catch” trials with un-reinforced CS during the acquisition (<1/3 of the trials, during half of the sessions). We found that 11% (29/247; p < 0.05, binomial test) of the cells had significant modulation in the ITI following un-reinforced trials (the lower number of cells can stem from the lower number of trials used for analysis, hence lower statistical power). Importantly, the ITI response of these cells was highly similar in reinforced and un-reinforced trials (Fig. 4c, d, t-test, p > 0.1). Here again, their response diminished during extinction (Fig. 4d; t-test, p < 0.05). In sessions that included both aversive and appetitive odors conditioned to two different tones (discrimination learning), we found that the proportion of ITI-FR modulation was significant after both types of valence, but higher following aversive trials (Supplementary Fig. 1).

Overall, the findings are in line with the hypothesis that the FR modulations during the ITI occur during learning.

Trial-by-trial modulation of ITI signaling

If indeed expectation is formed based on the mean of the ITI, and ITI length contributes to learning rate (Fig. 2), one can hypothesize that slight variations in length can induce transient updates in representation. Specifically, the duration of the ITI in the previous trial is expected to correlate with the time of modulation during the following trial. To explore this possibility, correlations were calculated between the center of mass for the ITI-FR in the current trial and the duration of the ITI of the previous trial, for all available neurons. Indeed, 15% of the neurons significantly correlated with the duration of the previous ITI (Fig. 5a, b). Correlations were significantly more common in dACC neurons than in amygdala neurons (17.4% and 10.6%, respectively, p < 0.01, χ² test for independence), yet in both regions their prevalence exceeded chance level (p < 0.05, binomial test). Earlier ITIs (trials n − 2, n − 3, and n − 4) had only limited effect on the center of mass (7–9% of cells, p > 0.05), and a multiple regression model including n − 1/2/3/4 suggested that the combined duration affect only few cells (4%).

This result demonstrates that neural activity during the ITI is constantly modified by the most recent trial.

To search for a more direct link between representation of time in neural activity and strength of conditioning, we correlated the behavioral change in CR (as in Fig. 2c) to the center of mass for the ITI-FR, for all trials and all neurons. This revealed a significant positive relationship (Fig. 5c, r = 0.08, p < 0.0001) that was slightly higher in the amygdala than in the dACC (Supplementary Fig. 2).

These results demonstrate that neurons adapt their temporal characteristics during the ITI based on the duration of the previously encountered period, and that this modulation is likely related to the behavioral change.

Amygdala and dACC show peak modulations in different periods

Our results so far suggest that amygdala and dACC neurons modulate their activity during the ITI, but is it a neural representation that spans the complete ITI duration? To address this we inspected single-cell peak modulation and found that 35% of amygdala cells (101/291) and 33% of dACC cells (89/263) exhibited a significant peak modulation (p < 0.01 for both, binomial tests).

Although the proportion of amygdala and dACC cells with significant modulation was highly similar (p > 0.1, binomial test), the distributions of peak locations were different (Fig. 6a, b, p < 0.05, permutation test comparing the regional distributions). Both regions had early peaks, likely US-related, but whereas amygdala neurons had an additional density of peaks around the middle of the ITI, additional dACC peaks occurred later in the ITI (Fig. 6a, b). Despite this, the mean width of individual tuning was similar in the amygdala and dACC (Fig. 6c left, p > 0.01, t-test), with no change in tuning width across conditioning trials (Fig. 6c, d). These results suggest that the amygdala and the dACC span the duration of the ITI, potentially enabling maintenance of long time periods in this shared network.

Synchronized activity communicates temporal information

If amygdala and dACC represent different types of information and time during the ITI, it is possible that they transfer and share such information to aid in the maintenance of long timescales. To examine this, we computed inter-regional correlations between pairs of amygdala and dACC neurons that were recorded simultaneously. The cross-correlations were computed in 3-s window that advanced in 1-s steps from the end of each trial.

Evaluation of significance in such cross-correlations requires two sets of shuffled data to address two different null hypotheses (Supplementary Fig. 3). First, data were shuffled within-ITI, so that spikes from a specific ITI were shuffled within itself, thus maintaining the overall firing rate in each ITI. This shuffle targets pseudo-correlations that may arise when the overall ITI-FR rate of the two units covaries. Second, data were shuffled across-ITI, so that spikes are shuffled between different trials’ ITI—but from the same period within the ITIs. This addresses pseudo-correlations that may originate when the ITI-FR is repeatedly modulated along the ITI (Supplementary Fig. 3). Only cross-correlations that differed from both were deemed significant. A total of 41% (344/839) pairs of amygdala-dACC neurons were found to have a significant cross-correlation. Most correlations had a zero-lag with no clear physiological direction, suggesting reciprocal interaction between the amygdala and the dACC (Fig. 7a). However, the distribution of significant cross-correlations peaked mainly at early and late periods of the ITI (Fig. 7a).

We next examined if this double-peaked pattern reflects an average correlation map of prototypic pairs of neurons, or a mixture of single-peaked correlations. A principle component analysis (PCA) on the normalized distribution of correlation density along the ITI identified three separable clusters (Fig. 7b, c; k-means clustering). The distribution of cross-correlations of each cluster strengthens the previous finding and shows that it is comprised of different pairs that synchronize during a single and confined segment of time within the ITI. These segments tile the complete duration on one hand (Fig. 7c), but are much more dominant early and late during the ITI.

The results suggest that BLA-dACC neurons signal and potentially transfer temporal information in a pairwise-specific manner.

Estimating time during the ITI with neural activity

If neurons have temporal-tuning with peaks in different times during the ITI, and these are relatively homogenously distributed when combined across the amygdala and the dACC, then one should be able to decode time during the ITI (and as a result, its total length). We used an optimal linear estimator (OLE) with cross-validation and found that time could be estimated based on the population activity within a reasonable error (Fig. 8a), that dropped with the size of the population being used, yet remained within few (<4) seconds accuracy (Fig. 8b). Notice the true error is likely larger because we assume independence between neurons (similar results were obtained with a population vector approach and a Naïve Bayes classifier). Averaging the estimates showed that it is unbiased (Fig. 8c), and therefore a downstream network can use this information reliably to assess ITI duration. Supporting this, we used a hazard-rate approach and found that a proportion of the neurons had gradual elevated activity toward the end of the ITI, one that correlates with the increasing probability for CS appearance as the time in the ITI progresses (Fig. 9).

Discussion

We show here that neurons in the basolateral complex (BLA) of the primate amygdala together with neurons in the dACC acquire temporal-tuning modulations during the ITI. As a result, the population faithfully represents the time that passes between trials, and can potentially represent anticipation of the aversive event that will be cued by the CS. In addition, because the length of the ITI contributes to learning rate and the strength of the memory, as shown here for aversive conditioning, our results suggest that this is mediated by amygdala-prefrontal networks. This is the first demonstration of representation of long timescales (dozens of seconds) in primate amygdala networks, an order of magnitude longer than the typical duration used for the ISIs. Therefore, this network can support formation and maintenance of temporal contingencies not only when they are cued explicitly by external stimuli, but also when embedded in the temporal structure of the task. Below we discuss the implications of this finding to valence-based learning.

Our finding of neural representation of time in BLA-dACC network could support several mechanisms that contribute to the effect of ITI duration on acquisition (learning) and CR (performance), as demonstrated across species and learning tasks^34,35,36. Accordingly, we show here that the learning rate and size of the CR were positively correlated with ITI duration^3,4,5,6,7. The underlying mechanisms for this effect can consist of multiple parallel processes.

One appealing explanation of ITI-related enhancement stems from the ratio between the ITI and the ISI (CS-US duration). Larger ratio is correlated with accelerated learning^5,7, in line with the larger informative power of the CS on the occurrence of the US³⁷. Here we demonstrate one major component that of time within the ITI. This is a more challenging aspect for a neural network and hence the main novelty in our study, because it requires representation of long timescales beyond that of persistent activity in single cells (as widely demonstrated in CS-US trace-conditioning, CS-delayed-response memory tasks, and interval-reproduction tasks). Nevertheless, full examination of this specific hypothesis would require longer ISI to allow the CS response to decay, and it would be intriguing to explore neural implementation of ratio-dependent acquisition rate. Here the ISI was short and predictable, and hence our finding of unbiased decoding error along the ITI can enable a representation of duration and ratio. Moreover, this interpretation is further supported by the finding that even single neurons shift the temporal characteristics of their activity based on the duration of the previous trial, and this shift was correlated with the instantaneous learning rate, hence providing a direct link between single-cell representation and behavior. Interestingly, we found a relationship only with the previous trial, suggesting that neurons use short-term history to update their expectation about ITI duration. This could be a result of the statistics used in the current study because the ITIs were normally distributed, and the mean duration can be computed additively on a trial-by-trial basis.

Despite the elegance of this temporal account of ITI duration and/or ITI-to-ISI ratio, several studies suggest that it might be insufficient. This is based on the observation that the proximity of an ITI-interfering event to the preceding³⁸ or consecutive³⁹ trial increased the effect on learning performance. As alternative, the opponent process theory postulates that after an aversive US, an opponent relief-like process is initiated that interferes with the learning and longer ITI allows for this process to terminate and for learning to be enhanced^34,40. Our finding that a larger proportion of BLA neurons are tuned to the middle of the ITI can support this, as the relief process evolves after the termination of the US yet decays toward the following trial, in line with safety signals reported in BLA neurons^41,42. Further, the opponent process can also be activated by CS presentation alone as learning progresses, and reduced following extinction⁴⁰. Accordingly, we found higher response magnitude during un-reinforced trials that diminish with extinction.

Another option is the formation of second-order associations mediated by the amygdala^20,21,22, first between the CS and the US, and then between the ITI itself and the CS, or between the US offset and the next CS (via the ITI “filler”²³). However, we did not observe any behavioral changes during the ITI in expectation of the next trial, and we therefore think it is a less probable interpretation. Even if such second-order association indeed takes place, it would require temporal activity as we report here that tiles and bridges the duration of the interval. Albeit, the ITI here was of varying random duration and hence the next CS cannot be faithfully predicted from US offset, yet the coming CS is a complete predictor for the US. Together with the lack of preparatory (CR-like) behavior during the ITI, this supports the interpretation that it is not a mediated association, but rather a representation of trial structure and/or statistics of the environment that is represented in amygdala networks²⁴.

In addition, behavioral³⁸ and neural^43,44 studies suggest that rehearsal of memory can occur during the ITI, where additional processing contributes and consolidates learning⁴⁵. Although it is not clear at this point how representation of time and rehearsal are related and integrated, and evidence for neural rehearsal processes in the amygdala is still scarce, further characterization of BLA and dACC contribution to ITI-related processing could shed light on these mechanisms and their putative complementary role.

The representation of long timescales on the order of dozens of seconds, and without an external cue as an anchor, is a new finding in the primate amygdala. One can consider several brain regions as candidates to supply this information to the amygdala. The striatum is one, shown to underlie and pace short timescales at hundreds of milliseconds⁴⁶, but also to signal longer durations^47,48, yet mostly in specific dedicated time tasks^35,49. However, the striatum does not project directly to the BLA, yet the BLA does project to parts of the striatum^50,51. In contrast, temporal information can reach the ACC via cortico and cortico-striatal loops^52,53,54, and from there to the amygdala²⁶. Paralleling this, we observed a higher proportion of dACC neurons that scale their peak ITI modulation with the length of the recent ITI, on a trial-by-trial basis. Similarly, the representation of time can reach the amygdala via time-cells and event-integration in the hippocampus^55,56,57. Whereas the exact functional role of such loops remains to be examined, it supports the idea that time perception is mediated by multiple overlapping neural systems, which are flexibly engaged depending on the task requirements⁵⁸. In the current case, they can play a role in valence/emotional-based learning and support the demonstrated contribution of the ITI to memory strength.

An additional interpretation for the differential peaks of modulation observed in the two regions can come from functional considerations. The amygdala was shown to signal safety^41,42, in accordance with its view as an absolute valence decoder^59,60. ACC neurons signal anxiety, threat, and risk^61,62, and importantly, the ACC plays a major role in error, context, and attention^28,29,63. These functions are higher either right after or right before a trial. Therefore, the subtle distribution of roles might underlie the differential preference of ACC neurons to mediate information early and late in the interval, whereas amygdala neurons signal its middle.

Here we focused on the reciprocal functional loop between the BLA and the dACC and found that together they span the complete ITI duration. Moreover, we observed potential transfer of temporal information in confined periods, early and late, by zero-lag synchronization between simultaneously recorded pairs of neurons. We identified three classes of pairwise interactions that synchronize early in the ITI, late, and more diffused population that spans the middle of it. This is in line with other types of information that transfer between the amygdala and the dACC at the single-cell level^{16,17,31,64,65,66,67} and orchestrated by field-oscillations^31,32.

These two seemingly different interpretations for the difference in peak distributions are not mutually exclusive and can serve both purposes. Namely, utilizing the tendency of one region to report a specific state aids to maintain the temporal marker for that state. This enable the network to use the underlying state of the animal to maintain additional information about the temporal structure across the network. Even if the immediate goal of the network is to represent the overall length of the ITI, it cannot do so by single-cell sustained activity, because of the high energetic demands for maintaining firing rates. The temporal-tuning-like approach allows this, and especially when distributed across regions.

To conclude, we find here that the activity of amygdala and dACC neurons hold information about long timescales, and together support learning rate and memory strength during aversive learning. The results provide a new role for this network in maintaining timescales to build the temporal and/or statistical structure of the environment, and suggests a mechanism to how longer intervals can promote learning and memory. In turn, it can also explain why under circumstances of multiple subsequent experiences, aversive memories are formed faster and exhibit strong-to-abnormal responses. Hence, in addition to existing learning models that implicate this specific network in anxiety and fear-disorders^9,10,68,69, our findings suggest a new model for how deviations in representation and computation in this circuitry can lead to maladaptive and exaggerated behaviors.

Methods

Animals

Two male macaca fascicularis (5–7 kg) were implanted with a recording chamber (27 × 27 mm) above the amygdala and cingulate cortex under deep anesthesia and aseptic conditions. All surgical and experimental procedures were approved and conducted in accordance with the regulations of the Weizmann Institute Animal Care and Use Committee, following National Institutes of Health regulations and with Association for Assessment and Accreditation of Laboratory Animal Care (AAALAC) accreditation. Food, water, and enrichments (e.g. fruits and play instruments) were available ad libitum during the whole period, except before medical procedures that require deep anesthesia.

Recordings

The monkeys were seated in a dark room and each day 3–6 microelectrodes (0.6–1.2 MΩ glass/narylene-coated tungsten, Alpha Omega, Israel or We-sense, Israel) were lowered inside a metal guide (Gauge 25xxtw, OD: 0.51 mm, ID: 0.41 mm, Cadence Inc., USA) into the brain using a head-tower and electrode-positioning system (Alpha Omega, Israel). The guide was lowered to penetrate the dura and stopped at 2–6 mm in the cortex. The electrodes were then moved independently further into the amygdala or the dACC. Electrode signals were pre-amplified, 0.3 Hz–6 kHz band-pass filtered and sampled at 25 kHz; and online spike sorting was performed using a template-based algorithm (Alpha Lab Pro, Alpha Omega, Israel). Anatomical magnetic resonance imaging scans were acquired before, during, and after the recording period and these scans were used to guide the positioning of the chamber on the skull at the surgery and to calibrate the positioning of the electrodes in the amygdala and the dACC.

Behavior paradigm and stimuli

Each daily session consisted of tone presentations (900–2400 Hz) that were triggered by real-time detection of respiration onset and followed by odor release at the onset of the following respiration cycle (but not earlier than 1 s from tone presentation). Odors (propionic acid or banana-melon organic extracts diluted in mineral oil) were actively evacuated from the nasal mask by a vacuum hose and respirations were constantly monitored with two parallel connected pressure sensors (1/4″ and 1″ H2O pressure range, AllSensors).

During the habituation phase the tones (novel tones on each day) were presented 10 times, without any odor to follow. During the acquisition stage, the tones were presented 30 times each, and paired with aversive odors in all occasions. In some sessions, aversive and pleasant trials were intermingled in a pseudorandom order and paired with different tones (discrimination learning). In addition, in half of the sessions, the acquisition stage also included 10 presentations of unpaired (catch) trials, i.e. tones that were not followed by an odor.

Behavioral analysis

Learning-dependent changes in the response to the CS were computed as area under the curve during 350 ms following CS onset compared with mean inhale volume in response to the same tone during habituation, as in previous work¹⁶.

Behavioral ITI data were aligned to the end of the ITI, namely to the beginning of the next trial (CS occurrence), because our goal was to examine if any anticipatory/preparatory behavior develops toward the next trial. To evaluate changes in inhale power during the ITI across trials we averaged the area under the curve of breathing data during 30 s of the ITI with a running window of 7 s and overlap of 2 s. To evaluate changes in inhale frequency (for 0.2–2 Hz, steps of 0.01 Hz) we quantified spectrograms by obtaining the multitaper power spectral density estimation (Thomson multitaper method) for breathing data during 30 s of the ITI (7 s windows with and 2 s overlap).

Neural data analysis

Neural activity during the ITI was aligned to the end of each trial, and taken until the end of the mean ITI in each specific session, with exclusion of bins from trials with shorter ITI (to avoid sampling bias and CS response contaminating the dataset). This is because our hypothesis was that neural activity signals the time during the ITI that passes from the previous trial (as the reliable way to estimate the duration of the ITI).

ITI-FR modulation: Spikes discharge that occurred between the US offset and the mean ITI were binned into 6, 12, 20, or 40 bins, and counted to produce an estimation of the ITI-FR. To assess the strength of modulation, spikes were shuffled 100 times and the mean ITI-FR was compared to the mean and standard deviation of the shuffled ITI-FR distribution. Data were taken only from bins that occurred before the ITI-mean, to avoid sampling bias. Up and down modulations were determined when the original spike count at a specific bin exceed ±2 standard deviation of the shuffled spike count at the corresponding bin.

Acquisition of ITI response: ITI-FR was computed separately for habituation, acquisition, and extinction ITIs. The ITI-FR of the acquisition and extinction was further separated into three and two equally sized (10 trials) sub-stages, respectively. Each ITI-FR was then normalized according to its own shuffled dataset (as above). This normalization assures that the ITI-FR at all stages will be on the same scale even in cases that the neurons increase or decrease their firing rate along the session (i.e. non-stationary). Next, the RMS of the ITI-FR was derived and compared between the different stages. ANOVA test was employed to examine whether the RMS scores changed between the habituation and the acquisition. When the ANOVA test indicated that the RMS was modulated during the session, we further performed post hoc t-test comparisons between the early, middle, and late acquisition phases and the habituation. Neurons whose ANOVA test was significant and with one or more post hoc comparisons that indicate that their RMS is increased during the acquisition stages were classified as acquisition responsive neurons.

Peak modulation: Spikes were binned into 40 bins during the ITI, and the bin with the highest firing rate (averaged over trials) was marked as the peak. This bin was then gradually joined by surrounding bins if they had a higher-than-baseline firing rate (with confidence interval > 95%). The number of bins were taken as the tuning-width (in seconds; units with tuning of 1 bin only were excluded).

ITI correlations: Pearson correlations between the center of mass of the ITI-FR and the length of the previous ITI (duration) were computed. Correlation at p < 0.05 were considered significant, and binomial test was employed to determine whether the percentage of units with significant correlation is higher than chance level (5%). Similarly, Pearson correlations were also computed between the RMS of individual ITI-FR during the acquisition and the length of the ITI at the previous trial.

ITI cross-correlation: Cross-correlations were computed between all pairs of amygdala-dACC that were recorded simultaneously and which firing rate exceed a mean of 1 Hz during the ITI (to allow sufficient reliable statistical power). Cross-correlations were computed in a 3-s window that was advanced in 1-s steps from the offset of the US until 5 s before the mean ITI. In all cases, amygdala spikes were used as the reference for the occurrence of dACC spikes, and spike that occur up to 750 ms before or after an amygdala’s spike were included in our analysis. Next, spike times were binned into 20 ms bins and counted. To evaluate whether the observed cross-correlations were significant we repeated the same procedure but this time, the dACC spikes were shuffled in two different manners: (1) within-ITI shuffling, in which we shuffled spikes in each individual ITI, destroying their temporal resolution, but maintaining the overall firing rate in each individual ITI; and (2) across-ITI shuffling, in which spikes were shuffled between the different ITIs, but maintaining the same epoch within the ITI. This procedure changed the overall firing rate of individual ITIs, but maintains the temporal structure of the ITI-FR. Cross-correlations were deemed significant only if they exceeded both criteria, with a further imposed requirement that they maintain significance for at least 15% of the total ITI duration.

Decoding time from neural activity: We used cross-validation and an OLE. In each iteration, we pseudorandomly chose 25 trials from the acquisition and derived the optimal weights given the normalized firing rate in each bin of the actual time during the ITI (40 bins). These weights were then used to estimate/decode the time in each of the remaining 5 trials separately by using the real firing rates. We then averaged across neurons. The process was repeated 100 times, and also as a function of N neurons pseudorandomly selected from the different sessions (Fig. 8). A population vector approach gave highly identical results.

Center of mass: Center of mass was expressed in seconds and computed as

$${\rm Center\;of\;mass} = \frac{{\mathop {\sum }\nolimits_{1:n} {\rm Firing\;Rate}\left( i \right) \ast {\rm Time}(i)}}{{\mathop {\sum }\nolimits_{1:n} {\rm Fring\;Rate}(i)}}$$

Hazard rate: Hazard rate was computed empirically for each session as the cumulative probability for CS occurrence in each time point, and normalized (warped) to the maximal ITI in each session for averaging across sessions. To identify matching neural activity, we correlated the activity of each cell firing rate with a portion of the hazard function, from the first point that rises from 0 to 0.75 (we used 0.75 because there only few trials that can be used for neural activity beyond it, inducing unreliable noise in the results). Firing rate was calculated in spikes/s in windows of 1 s advancing in steps of 0.5 s, normalized to the actual number of trials per ITI duration, and z-scored for purposes of averaging across units. Peak location was taken as in Fig. 6.

We repeated the main analyses separately for each monkey to validate the main results (Supplementary Fig. 5).

Code availability

Custom code for behavioral and electrophysiological tests is available from the corresponding author upon reasonable request.

Data availability

All data supporting the findings of this study are available from the corresponding author upon reasonable request.

References

Skinner, B. F. Superstition in the pigeon. J. Exp. Psychol. 38, 168–172 (1948).
Article CAS Google Scholar
Staddon, J. E. R. & Simmelhag, V. L. Superstition experiment—a reexamination of its implications for principles of adaptive behavior. Psychol. Rev. 78, 3–43 (1971).
Article Google Scholar
Spence, K. W. & Norris, E. B. Eyelid conditioning as a function of the inter-trial interval. J. Exp. Psychol. 40, 716 (1950).
Article CAS Google Scholar
Fanselow, M. S. & Tighe, T. J. Contextual conditioning with massed versus distributed unconditional stimuli in the absence of explicit conditional stimuli. J. Exp. Psychol. Anim. Behav. Process. 14, 187–199 (1988).
Article CAS Google Scholar
Lattal, K. M. Trial and intertrial durations in Pavlovian conditioning: issues of learning and performance. J. Exp. Psychol. Anim. Behav. Process. 25, 433–450 (1999).
Article CAS Google Scholar
Holland, P. C. Trial and intertrial durations in appetitive conditioning in rats. Learn. Behav. 28, 121–135 (2000).
Article Google Scholar
Gibbon, J., Baldock, M., Locurto, C., Gold, L. & Terrace, H. Trial and intertrial durations in autoshaping. J. Exp. Psychol. Anim. Behav. Process. 3, 264 (1977).
Article Google Scholar
Pape, H. C. & Pare, D. Plastic synaptic networks of the amygdala for the acquisition, expression, and extinction of conditioned fear. Physiol. Rev. 90, 419–463 (2010).
Article CAS Google Scholar
Salzman, C. D. & Fusi, S. Emotion, cognition, and mental state representation in amygdala and prefrontal cortex. Annu. Rev. Neurosci. 33, 173–202 (2010).
Article CAS Google Scholar
Likhtik, E. & Paz, R. Amygdala-prefrontal interactions in (mal)adaptive learning. Trends Neurosci. 38, 158–166 (2015).
Article CAS Google Scholar
Lee, S. C., Amir, A., Haufler, D. & Pare, D. Differential recruitment of competing valence-related amygdala networks during anxiety. Neuron 96, 81–88 (2017).
Article CAS Google Scholar
Herry, C. et al. Switching on and off fear by distinct neuronal circuits. Nature 454, 600–606 (2008).
Article ADS CAS Google Scholar
Belova, M. A., Paton, J. J. & Salzman, C. D. Moment-to-moment tracking of state value in the amygdala. J. Neurosci. 28, 10023–10030 (2008).
Article CAS Google Scholar
Paz, R., Pelletier, J. G., Bauer, E. P. & Pare, D. Emotional enhancement of memory via amygdala-driven facilitation of rhinal interactions. Nat. Neurosci. 9, 1321–1329 (2006).
Article CAS Google Scholar
Paton, J. J., Belova, M. A., Morrison, S. E. & Salzman, C. D. The primate amygdala represents the positive and negative value of visual stimuli during learning. Nature 439, 865–870 (2006).
Article ADS CAS Google Scholar
Livneh, U. & Paz, R. Amygdala-prefrontal synchronization underlies resistance to extinction of aversive memories. Neuron 75, 133–142 (2012).
Article CAS Google Scholar
Klavir, O., Genud-Gabai, R. & Paz, R. Functional connectivity between amygdala and cingulate cortex for adaptive aversive learning. Neuron 80, 1290–1300 (2013).
Article CAS Google Scholar
Chowdhury, N., Quinn, J. J. & Fanselow, M. S. Dorsal hippocampus involvement in trace fear conditioning with long, but not short, trace intervals in mice. Behav. Neurosci. 119, 1396–1402 (2005).
Article Google Scholar
Quinn, J. J., Oommen, S. S., Morrison, G. E. & Fanselow, M. S. Post-training excitotoxic lesions of the dorsal hippocampus attenuate forward trace, backward trace, and delay fear conditioning in a temporally specific manner. Hippocampus 12, 495–504 (2002).
Article Google Scholar
Setlow, B., Gallagher, M. & Holland, P. C. The basolateral complex of the amygdala is necessary for acquisition but not expression of CS motivational value in appetitive Pavlovian second-order conditioning. Eur. J. Neurosci. 15, 1841–1853 (2002).
Article Google Scholar
Gewirtz, J. C. & Davis, M. Second-order fear conditioning prevented by blocking NMDA receptors in amygdala. Nature 388, 471–474 (1997).
Article ADS CAS Google Scholar
Gaffan, D. & Harrison, S. Amygdalectomy and disconnection in visual learning for auditory secondary reinforcement by monkeys. J. Neurosci. 7, 2285–2292 (1987).
CAS PubMed Google Scholar
Bolles, R. C., Collier, A. C., Bouton, M. E. & Marlin, N. A. Some tricks for ameliorating trace-conditioning deficit. B. Psychon. Soc. 11, 403–406 (1978).
Article Google Scholar
Madarasz, T. J. et al. Evaluation of ambiguous associations in the amygdala by learning the structure of the environment. Nat. Neurosci. 19, 965–972 (2016).
Article CAS Google Scholar
Saez, A., Rigotti, M., Ostojic, S., Fusi, S. & Salzman, C. D. Abstract context representations in primate amygdala and prefrontal cortex. Neuron 87, 869–881 (2015).
Article CAS Google Scholar
Ghashghaei, H. T., Hilgetag, C. C. & Barbas, H. Sequence of information processing for emotions based on the anatomic dialogue between prefrontal cortex and amygdala. Neuroimage 34, 905–923 (2007).
Article CAS Google Scholar
Han, C. J. et al. Trace but not delay fear conditioning requires attention and the anterior cingulate cortex. Proc. Natl Acad. Sci. USA 100, 13087–13092 (2003).
Article ADS CAS Google Scholar
Bryden, D. W., Johnson, E. E., Tobia, S. C., Kashtelyan, V. & Roesch, M. R. Attention for learning signals in anterior cingulate cortex. J. Neurosci. 31, 18266–18274 (2011).
Article CAS Google Scholar
Heilbronner, S. R. & Hayden, B. Y. Dorsal anterior cingulate cortex: a bottom-up view. Annu. Rev. Neurosci. 39, 149–170 (2016).
Article CAS Google Scholar
Lee, D., Rushworth, M. F., Walton, M. E., Watanabe, M. & Sakagami, M. Functional specialization of the primate frontal cortex during decision making. J. Neurosci. 27, 8170–8173 (2007).
Article CAS Google Scholar
Likhtik, E., Stujenske, J. M., M, A. T., Harris, A. Z. & Gordon, J. A. Prefrontal entrainment of amygdala activity signals safety in learned fear and innate anxiety. Nat. Neurosci. 17, 106–113 (2014).
Article CAS Google Scholar
Taub, A. H., Perets, R., Kahana, E. & Paz, R. Oscillations synchronize amygdala-to-prefrontal primate circuits during aversive learning. Neuron 97, 291–298 (2018).
Article CAS Google Scholar
Livneh, U. & Paz, R. Aversive-bias and stage-selectivity in neurons of the primate amygdala during acquisition, extinction, and overnight retention. J. Neurosci. 32, 8598–8610 (2012).
Article CAS Google Scholar
Barela, P. B. Theoretical mechanisms underlying the trial-spacing effect in Pavlovian fear conditioning. J. Exp. Psychol. Anim. Behav. Process. 25, 177–193 (1999).
Article CAS Google Scholar
Gallistel, C. R. & Gibbon, J. Time, rate, and conditioning. Psychol. Rev. 107, 289–344 (2000).
Article CAS Google Scholar
Sunsay, C. & Bouton, M. E. Analysis of a trial-spacing effect with relatively long intertrial intervals. Learn. Behav. 36, 104–115 (2008).
Article Google Scholar
Balsam, P. D. & Gallistel, C. R. Temporal maps and informativeness in associative learning. Trends Neurosci. 32, 73–78 (2009).
Article CAS Google Scholar
Wagner, A. R., Rudy, J. W. & Whitlow, J. W. Rehearsal in animal conditioning. J. Exp. Psychol. 97, 407–426 (1973).
Article CAS Google Scholar
Sunsay, C., Stetson, L. & Bouton, M. E. Memory priming and trial spacing effects in Pavlovian learning. Learn. Behav. 32, 220–229 (2004).
Article Google Scholar
Solomon, R. L. The opponent-process theory of acquired motivation: the costs of pleasure and the benefits of pain. Am. Psychol. 35, 691–712 (1980).
Article CAS Google Scholar
Genud-Gabai, R., Klavir, O. & Paz, R. Safety signals in the primate amygdala. J. Neurosci. 33, 17986–17994 (2013).
Article CAS Google Scholar
Sangha, S., Chadick, J. Z. & Janak, P. H. Safety encoding in the basal amygdala. J. Neurosci. 33, 3744–3751 (2013).
Article CAS Google Scholar
Diba, K. & Buzsaki, G. Forward and reverse hippocampal place-cell sequences during ripples. Nat. Neurosci. 10, 1241–1242 (2007).
Article CAS Google Scholar
Carr, M. F., Jadhav, S. P. & Frank, L. M. Hippocampal replay in the awake state: a potential substrate for memory consolidation and retrieval. Nat. Neurosci. 14, 147–153 (2011).
Article CAS Google Scholar
Girardeau, G., Inema, I. & Buzsaki, G. Reactivations of emotional memory in the hippocampus-amygdala system during sleep. Nat. Neurosci. 20, 1634–1642 (2017).
Article CAS Google Scholar
Soares, S., Atallah, B. V. & Paton, J. J. Midbrain dopamine neurons control judgment of time. Science 354, 1273–1277 (2016).
Article ADS CAS Google Scholar
Mello, G. B., Soares, S. & Paton, J. J. A scalable population code for time in the striatum. Curr. Biol. 25, 1113–1122 (2015).
Article CAS Google Scholar
Matell, M. S., Meck, W. H. & Nicolelis, M. A. L. Interval timing and the encoding of signal duration by ensembles of cortical and striatal neurons. Behav. Neurosci. 117, 760 (2003).
Article Google Scholar
Allman, M. J., Teki, S., Griffiths, T. D. & Meck, W. H. Properties of the internal clock: first- and second-order principles of subjective time. Annu. Rev. Psychol. 65, 743–771 (2014).
Article Google Scholar
Fudge, J. L., Kunishio, K., Walsh, P., Richard, C. & Haber, S. N. Amygdaloid projections to ventromedial striatal subterritories in the primate. Neuroscience 110, 257–275 (2002).
Article CAS Google Scholar
Averbeck, B. B. & Costa, V. D. Motivational neural circuits underlying reinforcement learning. Nat. Neurosci. 20, 505–512 (2017).
Article CAS Google Scholar
Buhusi, C. V. & Meck, W. H. What makes us tick? Functional and neural mechanisms of interval timing. Nat. Rev. Neurosci. 6, 755–765 (2005).
Article CAS Google Scholar
Murray, J. D. et al. A hierarchy of intrinsic timescales across primate cortex. Nat. Neurosci. 17, 1661–1663 (2014).
Article CAS Google Scholar
Finnerty, G. T., Shadlen, M. N., Jazayeri, M., Nobre, A. C. & Buonomano, D. V. Time in cortical circuits. J. Neurosci. 35, 13912–13916 (2015).
Article CAS Google Scholar
Eichenbaum, H. Time cells in the hippocampus: a new dimension for mapping memories. Nat. Rev. Neurosci. 15, 732–744 (2014).
Article CAS Google Scholar
Paz, R. et al. A neural substrate in the human hippocampus for linking successive events. Proc. Natl Acad. Sci. USA 107, 6046–6051 (2010).
Article ADS CAS Google Scholar
Buzsaki, G. & Llinas, R. Space and time in the brain. Science 358, 482–485 (2017).
Article ADS CAS Google Scholar
Wiener, M., Matell, M. S. & Coslett, H. B. Multiple mechanisms for temporal processing. Front. Integr. Neurosci. 5, 31 (2011).
Article Google Scholar
Morrison, S. E. & Salzman, C. D. Re-valuing the amygdala. Curr. Opin. Neurobiol. 20, 221–230 (2010).
Article CAS Google Scholar
Li, J., Schiller, D., Schoenbaum, G., Phelps, E. A. & Daw, N. D. Differential roles of human striatum and amygdala in associative learning. Nat. Neurosci. 14, 1250–1252 (2011).
Article CAS Google Scholar
McCoy, A. N. & Platt, M. L. Risk-sensitive neurons in macaque posterior cingulate cortex. Nat. Neurosci. 8, 1220–1227 (2005).
Article CAS Google Scholar
Dejean, C. et al. Prefrontal neuronal assemblies temporally control fear behaviour. Nature 535, 420–424 (2016).
Article ADS CAS Google Scholar
Wallis, J. D. & Kennerley, S. W. Contrasting reward signals in the orbitofrontal cortex and anterior cingulate cortex. Ann. N. Y. Acad. Sci. 1239, 33–42 (2011).
Article ADS Google Scholar
Klavir, O., Prigge, M., Sarel, A., Paz, R. & Yizhar, O. Manipulating fear associations via optogenetic modulation of amygdala inputs to prefrontal cortex. Nat. Neurosci. 20, 836 (2017).
Article CAS Google Scholar
Burgos-Robles, A. et al. Amygdala inputs to prefrontal cortex guide behavior amid conflicting cues of reward and punishment. Nat. Neurosci. 20, 836–844 (2017).
Article Google Scholar
Sotres-Bayon, F., Sierra-Mercado, D., Pardilla-Delgado, E. & Quirk, G. J. Gating of fear in prelimbic cortex by hippocampal and amygdala inputs. Neuron 76, 804–812 (2012).
Article CAS Google Scholar
Senn, V. et al. Long-range connectivity defines behavioral specificity of amygdala neurons. Neuron 81, 428–437 (2014).
Article CAS Google Scholar
Averbeck, B. B. & Chafee, M. V. Using model systems to understand errant plasticity mechanisms in psychiatric disorders. Nat. Neurosci. 19, 1418–1425 (2016).
Article CAS Google Scholar
Etkin, A., Buchel, C. & Gross, J. J. The neural bases of emotion regulation. Nat. Rev. Neurosci. 16, 693–700 (2015).
Article CAS Google Scholar
Martin, R. F., Bowden, D. M., Wu, J., Dubach, M. & Robertson, J. Primate Brain Maps (Elsevier, Amsterdam, 2000).

Download references

Acknowledgements

We thank Drs. Rita Perets and Yoav Kfir for scientific advice; Dr. Uri Livneh for providing experimental data and analyses; Dr. Eilat Kahana for help with medical and surgical procedures; and Dr. Edna Furman-Haran, Nachum Stern, and Fanny Attar for MRI procedures. This work was supported by ISF #26613 and ERC-2016-CoG #724910 grants to R.P.

Author information

Authors and Affiliations

Department of Neurobiology, Weizmann Institute of Science, Rehovot, 76100, Israel
Aryeh H. Taub, Yosef Shohat & Rony Paz

Authors

Aryeh H. Taub
View author publications
You can also search for this author in PubMed Google Scholar
Yosef Shohat
View author publications
You can also search for this author in PubMed Google Scholar
Rony Paz
View author publications
You can also search for this author in PubMed Google Scholar

Contributions

R.P. and A.H.T. designed the study; Y.S. performed the experiments; A.H.T. and R.P. analyzed the data; R.P. and A.H.T. wrote the manuscript.

Corresponding author

Correspondence to Rony Paz.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Supplementary Information

Peer Review File

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Cite this article

Taub, A.H., Shohat, Y. & Paz, R. Long time-scales in primate amygdala neurons support aversive learning. Nat Commun 9, 4460 (2018). https://doi.org/10.1038/s41467-018-07020-4

Download citation

Received: 24 February 2018
Accepted: 08 October 2018
Published: 26 October 2018
DOI: https://doi.org/10.1038/s41467-018-07020-4

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.