Model Framework

To study the processes described above computationally, we use an adaptation of the classical attractor network model [23], [24]. These highly connected neural networks, which can store memories as neuronal activation patterns based on Hebbian modifications of synaptic weights, have been proposed to be simple correlates of autoassociative networks such as the one believed to exist in region CA3 of the hippocampus [25], [26]. Attractor-like functioning has been shown to be compatible with both firing-rate and spike-time dependent plasticity in spiking neuronal networks [27], [28]. For the sake of simplicity, however, and for better correlation with previous models dealing with the effect of mismatch and memory representations (e.g. [29]), we use the classical firing rate implementation, which remains a useful tool for studying emergent network properties related to learning and memory.

Neuronal activities in the attractor network (meant to represent a hippocampal auto-associative storage network in our model) are determined by equation (1):(1)where τ is the neural time constant and represents the level of activation of neuron i in a network comprised by N neuronal units, varying continuously from 0 to 1 for each neuron, and not from −1 to 1 as in classical formulations (see Methods). This can reflect the firing rate and connectivity of neurons in a more realistic way, as it solves a series of biologically unfeasible features of the original formulation, including (a) the requirement of symmetric connections between neurons, (b) the strengthening of connections between neurons with low activity and (c) the occasional retrieval of mirror patterns diametrically opposite to those originally learned. The term causes the activation level to decay towards 0, while the term represents the influence of presynaptic neurons within the attractor network, weighed by the strength of the synaptic connections . Finally, the term represents synaptic influences from cue inputs. These cue inputs are thought to represent cortical afferents providing the hippocampus with the animal's current representation of its environment, based both on external (i.e. sensory input) and internal information (i.e. retrieved memories) (Figure 1A). The interplay between sensory information and hippocampal feedback is not modeled explicitly; instead, the presented cues will be modeled as relying more on external or internal input depending on behavioral parameters (see below).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 1. Model description.

(A) General scheme of the model architecture. (B) Patterns of 100 neurons used to represent memory 1 (unrelated memory), memory 2 (shock memory), memory 3 (non-shock memory) and retrieval cue. Red denotes activation, while blue denotes inhibition. Shock and non-shock memories share the activation of context neurons (red square), which are used to test retrieval. (C) Transformation of cue pattern according to reexposure duration. For short durations (low t values), the pattern resembles the shock memory (left), while cues for long durations (high t values) resemble the non-shock memory (right) and intermediate durations (center) yield mixed patterns. (D) Retrieval-induced synthesis in fear learning. Activation of shock (circle) and context (square) neurons is driven by excitatory cues (red curved arrows), while inhibition of a non-shock neuron (triangle) is caused by an inhibitory cue (blue curved arrow). This leads to the establishment of excitatory/inhibitory synaptic weights between neurons (red/blue arrows), which allow reinstatement of the pattern by presentation of the context (right panel). (E) Mismatch-induced degradation in nonreinforced reexposure. A cue exciting non-shock and context neurons and inhibiting shock neurons is presented (left panel), but retrieval of the shock pattern occurs due to previously established synaptic weights, causing mismatch between activation patterns in the cue and attractor networks. This leads to degradation of synaptic weights responsible for the mismatch, causing reinstatement of the shock pattern in response to context to be weakened in a subsequent test session (right panel).

https://doi.org/10.1371/journal.pone.0023113.g001

Learning in the model occurs through presentation of an activation pattern by the cue inputs, which leads to changes in the synaptic weight matrix , as determined by equation (2):(2)where is a time-dependent synaptic decay factor [30], [31], and and stand for Hebbian Learning Plasticity and Mismatch-Induced Degradation, respectively, expressed in array form. Both of these matrices are dependent on the steady state pattern of neuronal activation that is reached by the network upon cue presentation (Eq. (1)). The precise meaning of the term and its equation will be explained below; for now, we will mention that all entries in the matrix are related to mismatch between the cue and a retrieved attractor and, as such, equal zero during initial learning. The term represents a modified Hebbian learning factor (see Methods), and it is given by(3)where the vector is the steady state of the network and corresponds to a factor representing a sum of the biochemical requirements for Hebbian synaptic plasticity, such as receptor activation, intracellular signaling and protein synthesis. Thus, if two neurons are maximally active (u_i = u_j = 1), the u_i→u_j connection gets reinforced by S; if the presynaptic neuron u_i is active and the postsynaptic neuron u_j is silent (u_j = 0), then the connection u_i→u_j changes by −S. If u_i is silent, nothing happens to the connection u_i→u_j. Intermediate values of u_i and u_j lead to intermediate effects of these factors. The value of is what is modified in simulations studying the influence exerted by pharmacological agents on initial memory consolidation, reconsolidation and extinction. The effect of protein synthesis inhibition by anisomycin, for instance, is modeled by setting S to 0, thereby blocking Hebbian plasticity.

Training, reexposure and testing in a simple one-trial learning task, such as contextual fear conditioning, are modeled by setting up appropriate cue patterns. Training sessions consist of presentation of one of three complete patterns (Figure 1B): pattern 1, representing a memory which is unrelated to fear conditioning; pattern 2, representing fear conditioning training, in which a set of neurons representing the context is activated along with another set of neurons representing the presence of danger or an aversive stimulus (i.e. an electric shock); and pattern 3, representing fear conditioning extinction, in which the same context neurons are activated along with a different set of neurons representing absence of danger. The use of a specific pattern to represent extinction is motivated by experimental data suggesting that the extinction process represents the active learning of a new memory trace [5], [6], as well as by studies suggesting that it may be encoded by neuronal populations which are at least partially distinct from those involved in the original learning [32], [33].

Memory retrieval is tested by presenting the cue pattern that represents the context (Figure 1B), and observing the attractor to which the network evolves. We model the animal's behavioral response by assuming that retrieval of pattern 2 leads to a far greater degree of conditioned behavior in response to danger than when the network reaches another attractor (see Methods and Figure S1). In analogy to the experimental literature, we refer to the fear conditioned response as “freezing”, and use the percentage of time spent freezing during the test as a measure of memory in the task.

Nonreinforced reexposure to the context is modeled similarly to training, except that the cue pattern in this case is a mix of patterns 2 and 3. This is based on the assumption that, upon reexposure to the context in which fear learning occurred, the memory network will initially retrieve the aversive memory, with feedback from the hippocampus signaling the activation of neurons representing danger in the animal's contextual representation. Later within the trial, however, the absence of shock will lead the animal to start perceiving the context as non-threatening, with sensory information prevailing over the stored attractor and causing activation of neurons encoding the absence of danger. Thus, we assume that the animal's contextual representation changes gradually from pattern 2 to pattern 3 as the reexposure session becomes longer, leading the learning which occurs based on this session to become progressively more biased toward the new context rather than toward internal cues, as has been suggested to occur experimentally [34]. Therefore, the final cue pattern can vary from a pattern close to pattern 2 (for short reexposure times, in which the animal perceives the context as aversive throughout the session) to one close to pattern 3 (for long reexposure times, in which the animal perceives the context as non-threatening for most of the session), with reexposure times in between these two extremes yielding intermediate activation patterns (Figure 1C). In other words, we assume that the degree of mismatch between the final context representation during reexposure and the original representation formed upon initial learning is proportional to reexposure duration. We refer to this duration as t, and use a transformation from pattern 2 to pattern 3 which is a function of t to create the cue patterns representing different durations of reexposure (see Methods).

As in the initial training session, synaptic weights are updated after the reexposure session following equation (2), and the Hebbian learning rule acts by means of the HLP term (Figure 1D). However, the existence of a previously stored attractor for the context in the reexposure session can lead the memory network to retrieve an attractor which is different from the cue pattern employed, leading to mismatch between the two patterns. We thus introduce a memory updating system which degrades synaptic weights between the different sets of neurons responsible for this mismatch, reducing the strength of connections which cause disagreement with the new cue pattern (Figure 1E). This effect is modeled by the term MID in (2), which follows the equation:(4)where the degradation factor D represents biochemical requirements for mismatch-induced updating of synaptic connections – which are thought to involve, among other things, protein degradation [35], [36] – and is the mismatch vector (where is a normalized cue vector varying between 0 and 1). Note that when the retrieved attractor is equal to the cue input (as during initial learning) there is no mismatch, since in these cases, leading all entries in vector m to equal zero.

Although the biochemical elements in the model are an obvious simplification (i.e. synaptic plasticity is certainly more complex than a synthesis/degradation balance, and involves many other mechanisms), there is much evidence to suggest that protein synthesis is a defining factor in long-term memory consolidation [37], as well as some evidence [35], [36] to suggest that protein degradation through the ubiquitin-proteasome system is involved in trace labilization during reconsolidation. Therefore, we focus on these two parameters in our simulations of pharmacological experiments. The synaptic weight changes induced by these processes are modeled as occurring during the post-reexposure period, based upon the activation state reached during the reexposure session (which presumably sets in motion the biochemical cascades and transcriptional information which will drive the protein changes occurring later). Pharmacological interventions after reexposure are thus modeled as changing either S or D during the synaptic weight updating process caused by the reexposure session (Eq. (2)), and the effects of these interventions are measured by evaluating subsequent retrieval in response to the cue representing the context.

Learning and extinction in the model

Figure 2 shows normal learning in the model. We first present the network with two orthogonal patterns with no overlapping active neurons, one at a time: pattern 1 (an unrelated memory) and pattern 2 (the shock memory). Presentation of these patterns leads to the formation of local energy minima corresponding to the two memories (Figure 2A). Retrieval of either one can occur upon random network initialization, while presentation of a partial cue for either of the two patterns biases retrieval towards the corresponding attractor (Figure 2B). Although we perform our simulations using only 3 patterns in a small network of 100 neurons, our network framework is capable of storing larger numbers of memories, with the absolute capacity depending on parameters such as network size and on the number of active neurons in each memory pattern, as has been shown to be the case for other attractor-based models [38], [39]. Estimations of storage capacities for different network sizes and sparseness values are shown in Figure S2, demonstrating that the model can store a reasonable number of memories, provided the number of neurons is large enough and memory patterns are reasonably sparse.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 2. Normal learning and extinction in the model.

(A) Energy landscape showing the relative basins of attraction after learning of memories 1 and 2 (see Methods). The learned patterns are seen as energy minima in blue, while no basin of attraction is observed for the point corresponding to memory 3. (B) Attractor retrieval after learning of memories 1 and 2. Upon random activation of the network (no cue), retrieval of both patterns occurs with similar probabilities, while presentation of a cue involving weak (I = 0.1) activation of 4 neurons pertaining to either pattern leads to preferential retrieval of this pattern. Bars represent mean ± S.E.M. of percentages of retrieved attractors over 10 sets of 100 simulations. (C) Fear conditioning and extinction. Memories 1, 2 and 3 are learned sequentially, with bars showing freezing percentages (mean ± S.E.M. of 100 simulations) in retrieval tests. After learning of memory 1 (green bar), little freezing occurs. Freezing increases after learning of memory 2 (red bar), but decreases again (blue bar) after extinction learning (corresponding to a single reexposure session with t = 10). (D) Extinction over multiple sessions. Learning of memories 1 and 2 occurs as in (C) (green and red bars). Extinction learning occurs through 6 reexposure sessions of intermediate duration (t = 6), leading to a decrease in freezing behavior in retrieval tests performed after each session (blue bars). Time-related decay (γ) occurs only before the first extinction session to allow comparison with the single-session protocol. (E) General protocol used to model nonreinforced contextual reexposure. Learning of memories 1 and 2 is followed by a nonreinforced reexposure session, represented by a cue pattern which varies according to reexposure duration. Memory retrieval is then tested by presentation of the context.

https://doi.org/10.1371/journal.pone.0023113.g002

Similarly to what occurs behaviorally, extinction in the model (represented as learning of pattern 3) can occur either in a single retrieval session with a cue similar to pattern 3 (i.e. a high t value, representing a long retrieval session) (Figure 2C) or in multiple retrieval sessions with intermediate cues (representing multiple short sessions in which pattern 2 and 3 are both reflected in the cue) (Figure 2D). Extinction over multiple sessions occurs due to gradual weakening of the shock attractor, which is repeatedly retrieved in the presence of mismatch and thus undergoes degradation, allowing learning of a new attractor (the extinction memory) to occur eventually. This is in contrast with single session extinction, in which prompt learning of the extinction memory prevents retrieval of the original attractor and weakening of the shock representation (see Figure S3).

The sequence of patterns used to model learning followed by nonreinforced reexposure to the context, which will be used throughout the simulations concerning the effects of anisomycin, is shown in Figure 2E. Learning of patterns 1 and 2 is followed by a nonreinforced reexposure session of variable duration (modeled by changing the value of t), and retrieval is later measured through presentation of the context cue.

Effects of anisomycin on different reexposure protocols

Figure 3 shows the effects of anisomycin administration (i.e. setting S to 0) in different learning and reexposure protocols. During initial learning, blockade of protein synthesis inhibits Hebbian modifications and prevents formation of the shock memory (Figure 3A), a finding which is consistent with the effect of anisomycin in various behavioral paradigms of learning, including fear conditioning [40].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 3. Effects of anisomycin administration in initial learning and reexposure.

(A) Effect of anisomycin in initial learning. Memories 1 and 2 are learned sequentially, and bars represent freezing percentages (mean ± S.E.M. of 100 simulations) before (left set of bars) and after (right set of bars) learning of memory 2. Administration of vehicle (S = 0.8, dark blue) or anisomycin (S = 0, light blue) occurs in the learning session, as indicated by the syringe. Freezing increases in the vehicle group, but remains at baseline values in the anisomycin group, as learning of fear conditioning is blocked. (B) Effect of anisomycin in a short reexposure session. Memories 1 and 2 are learned sequentially in normal conditions (S = 0.8), and the left set of bars shows retrieval after learning of memory 2. During a short reexposure session (t = 1), vehicle (S = 0.8, dark blue) or anisomycin (S = 0, light blue) is administered, and a retrieval test after this session (right set of bars) shows that freezing remains high in both groups. (C) Effect of anisomycin in a reexposure session of intermediate duration. Learning sessions are performed as in (B), but with t = 6. Freezing remains high after the reexposure session in the vehicle group, but decreases markedly in the anisomycin group, demonstrating reconsolidation blockade. (D) Effect of anisomycin in a long reexposure session. Learning sessions are performed as in (B) and (C), but with t = 10. Freezing decreases in the vehicle group due to extinction learning, but remains high in the anisomycin group due to blockade of extinction. (E) Effect of anisomycin in the absence of contextual reexposure. Learning of memories 1 and 2 occurs as in (B)–(D), but the reexposure session is replaced by the learning of an unrelated pattern. Freezing in both groups remains high, demonstrating the lack of effect of anisomycin in the absence of contextual reexposure. (F) Summary of the effect of anisomycin in reexposure sessions of various durations. The x axis represents reexposure duration, while lines and blue contours show freezing percentages (mean ± S.E.M, represented in the y axis) of anisomycin and vehicle groups in retrieval tests performed after reexposure.

https://doi.org/10.1371/journal.pone.0023113.g003

In Figures 3B to 3E, learning of the shock memory occurs normally (S = 0.8), and anisomycin administration is modeled in various nonreinforced reexposure protocols with different contextual cues (see Figures 1C and 2E). In very short reexposure trials, in which the shock memory is retrieved over the full course of the retrieval session and dominates the contextual representation (i.e. low t values), anisomycin will have little effect on subsequent retrieval of that memory, as the degree of mismatch-induced degradation will be small even in the absence of protein synthesis (Figure 3B). This is compatible with the “simple retrieval” condition observed with short reexposure durations in experimental studies [8], [16], [41].

In reexposure trials with intermediate durations (i.e. “reconsolidation” conditions), inhibition of protein synthesis starts to exert a significant amnestic effect on subsequent retrieval trials (Figure 3C), as Hebbian learning is blocked and cannot compensate for mismatch-induced degradation of the shock memory. This effect is analogous to the reconsolidation blockade effect described in various experimental studies [3], [4]. Finally, in long reexposure trials, in which the cue pattern will be distinct enough from pattern 2 to prevent its retrieval, extinction (i.e. formation of a new attractor representing pattern 3) will occur after the reexposure session in control conditions. The burning of a new attractor in the network will also prevent mismatch degradation of the shock representation; in this case, therefore, anisomycin will block formation of the extinction memory, but will not affect the existing shock attractor, leading to preservation of the shock memory in treated animals (Figure 3D). Such results closely match the effects of reexposure time on reconsolidation and extinction found in experimental studies [8], [16], [41], [42].

In agreement with all experimental studies of reconsolidation, anisomycin administered in the absence of the original learning context for the shock memory (i.e. upon reexposure to an unrelated cue) will have no effect on its subsequent retrieval in our model, demonstrating the context-specificity of the reconsolidation blockade effect [43] (Figure 3E). The effect of reexposure duration in control conditions and in anisomycin-treated animals upon subsequent memory retrieval is summarized in Figure 3F. One can observe that the amnestic effect of anisomycin increases along with reexposure duration until the minimum duration required for extinction to occur in controls is reached (around t = 8). In longer reexposure conditions, on the other hand, freezing decreases in controls with increasing reexposure duration due to extinction, while anisomycin preserves the original memory by preventing extinction learning.

Reconsolidation and extinction after different strengths of training

As observed experimentally [7], [8], [44], the protocols necessary to induce reconsolidation and extinction in our model vary according to the strength of the original learning. In some reexposure conditions which normally induce reconsolidation in controls (and amnesia in anisomycin treated animals) (Figure 4A), anisomycin will have no effect if the initial learning of the shock memory is made stronger by increasing S during the training session (Figure 4B), as the stronger memory will not be as affected by the degradation caused by reexposure. Such results are in accordance with the behavioral data indicating that longer reexposure trials are needed to induce reconsolidation of stronger or more consolidated memories [8]. Another consequence of strengthening the shock memory is that longer durations of reexposure, which normally yield extinction (Figure 4C), will lead to reconsolidation instead (Figure 4D). In this case, anisomycin will not lead to memory preservation (as occurred after normal training conditions) but to reconsolidation blockade and amnesia, similarly to what has been described experimentally [7]. The effect of reexposure duration on retrieval of the shock memory for various strengths of initial learning is summarized in Figures 4E (control conditions) and 4F (anisomycin treatment during reexposure).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 4. Reconsolidation and extinction after different strengths of training.

(A) Effect of anisomycin in a reexposure session of intermediate duration after regular training. Learning sessions are performed as in Figures 3B–3E, with S = 0.8. Bars indicate freezing in vehicle (dark blue) and anisomycin (light blue) groups before (left set) and after (right set) reexposure with t = 4 (mean ± S.E.M. of 100 simulations). As occurs in Figure 3C, freezing is decreased by anisomycin administration, indicating reconsolidation blockade. (B) Effect of anisomycin in a reexposure session of intermediate duration after strong training. Learning and reexposure sessions are performed as in (A) except that memory 2 is strengthened by setting S to 0.95 during the learning session. The same reexposure protocol used in (A) causes a much lower decrease of freezing in the anisomycin group, indicating that stronger learning can protect the shock memory from reconsolidation blockade. (C) Effect of anisomycin in a long reexposure session after regular training. Learning sessions are performed as in (A), with S = 0.8. As in Figure 3D, freezing decreases in the vehicle group after reexposure with t = 10 due to extinction, but remains high in the anisomycin group due to extinction blockade. (D) Effect of anisomycin in a long reexposure session after strong training. Learning and reexposure sessions are performed as in (C), except that memory 2 is strengthened by setting S to 0.95 during the learning session. The same reexposure protocol used in (C) is now insufficient to cause extinction in controls, while a decrease in freezing indicating reconsolidation blockade is observed in the anisomycin group. (E) Color plot summarizing the effect of reexposure on freezing behavior in the vehicle group after various strengths of initial learning. Freezing percentages after a reexposure session with S = 0.8 (color scale) are indicated for various reexposure durations (x axis) with different values of S used in initial learning (y axis). Extinction (blue regions indicating low freezing) occurs at long reexposure durations, but becomes increasingly hard to induce with stronger initial learning. (F) Color plot summarizing the effect of reexposure on freezing behavior in the anisomycin group after various strengths of initial learning. Freezing percentages after a reexposure session with S = 0 (color scale) are indicated for various reexposure durations (x axis) with different values of S used in initial learning (y axis). Increased strengths of initial training favor maintenance of freezing (red regions) instead of reconsolidation blockade (blue regions) in short reexposure sessions, but can favor reconsolidation blockade in long reexposure sessions by preventing extinction.

https://doi.org/10.1371/journal.pone.0023113.g004

Effect of memory-enhancing drugs on different reexposure protocols

Experimental data suggests that administration of memory-enhancing drugs such as D-cycloserine (a partial agonist of the coactivator site at the NR1 subunit of the NMDA receptor) during contextual reexposure can improve either reconsolidation or extinction, leading to an effect which is the opposite of that of anisomycin [42]. We have simulated that by increasing the value of S during the reexposure session, based on the enhancing effect of such drugs upon synaptic plasticity [45]. As found experimentally with D-cycloserine [42] and protein kinase A (PKA) activation [46], stimulating Hebbian plasticity during reexposure in reconsolidation conditions (t = 4) slightly improves subsequent retrieval of the shock memory (Figure 5A, third set of bars). This improvement was small in our simulations due to a ceiling effect, as memory in controls already approached saturation values after normal reconsolidation. On the other hand, increasing S during extinction conditions (t = 8) improves extinction and lowers subsequent fear memory retrieval (Figure 5A, fourth set of bars). These trends hold true for a range of parameters, as shown in Figure 5B, which summarizes the effects of increasing or decreasing S during reexposure sessions of different durations.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 5. Effects of enhanced plasticity during the reexposure session on reconsolidation and extinction.

(A) Effects of memory-enhancing drugs on reconsolidation and extinction. Learning of memories 1 and 2 occurs normally with S = 0.8, followed by a reexposure session of variable duration. Bars (mean ± S.E.M. of 100 simulations) indicate freezing after learning of memories 1 and 2 (first set) and after reexposure sessions of various durations in animals treated with vehicle (S = 0.8, blue) or a memory-enhancing drug (S = 0.95, yellow) in the reexposure session. Freezing is slightly increased by memory enhancement in short and intermediate reexposure sessions due to reinforcement of the shock memory. In long reexposure sessions, meanwhile, memory enhancement improves extinction and decreases freezing. (B) Color plot summarizing the effect of reexposure with various durations and S values on freezing behavior. Freezing percentages (color scale) after reexposure sessions with different durations (x axis) and S values (y axis) are indicated. For t values of up to 7, decreasing S below the usual value of 0.8 decreases freezing due to reconsolidation blockade, while increasing S values cause either no effect or slight enhancement of freezing. For t values of 8 or more, decreasing S causes preservation of freezing by blocking extinction, while increasing S decreases freezing by enhancing extinction.

https://doi.org/10.1371/journal.pone.0023113.g005

Effects of blocking mismatch-induced degradation

Experimental evidence for the effects of blocking protein degradation on memory (usually achieved through the use of inhibitors of the ubiquitin-proteasome cascade) is somewhat controversial, with different effects described on initial learning [36], [47], [48], [49] and reconsolidation [35], [36], [47]. It has recently been suggested, however, that protein degradation is necessary for the amnestic effect of anisomycin on reconsolidation to occur [35], [36]. This indeed occurs by blocking mismatch-induced degradation (i.e. setting D to 0) in our model, which does not affect memory reconsolidation by itself, but prevents the effect of anisomycin on subsequent retrieval (Figure 6A). Blocking mismatch-induced degradation will also prevent multiple session extinction (Figure 6B), as shown experimentally in one of these studies [36]. This result demonstrates that the mismatch-induced degradation system has a physiologic role in our model, as it allows nonreinforced trials of intermediate duration to lead to extinction when performed repeatedly, as opposed to the reinforcement of the original memory which occurs in the absence of degradation. When compared to experimental findings, it also suggests that protein degradation through the ubiquitin-proteasome system could be one of the mechanisms involved in mismatch-induced degradation of synaptic changes.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 6. Effects of blocking degradation on reconsolidation and extinction.

(A) Effects of blocking degradation on reconsolidation and reconsolidation blockade. Bars (mean ± S.E.M. of 100 simulations) indicate freezing percentages after initial learning with S = 0.8 (red) and after an intermediate reexposure session (t = 6) with vehicle (S = 0.8, D = 1.25, dark blue), anisomycin (S = 0, D = 1.25, light blue), degradation blockade (S = 0.8, D = 0, light gray) or anisomycin+degradation blockade (S = 0, D = 0, dark gray). Freezing decreases after anisomycin administration in reexposure as previously shown; degradation blockade has no effect on freezing on its own, but reverses the effect of anisomycin when both treatments are administered together. (B) Effects of blocking degradation on multiple session extinction. After learning of memories 1 and 2, multiple session extinction is induced in the vehicle group (blue bars) by performing 6 reexposure sessions with t = 6 as in Figure 2D. Blocking degradation by setting D to 0 (light gray bars) blocks the decrease in freezing, demonstrating that multiple session extinction depends on mismatch-induced degradation.

https://doi.org/10.1371/journal.pone.0023113.g006

General model framework

In line with previous research, we model the attractor network responsible for storing the memory patterns as a fully connected neural network [24], [29], [30], [31]. Neuronal activities in this network are determined by Eq. (1), which fully defines its dynamics and constrains neural activation (u) to values between 0 and 1 through the term ½(1+tanh(Σw_iju_j + I_j). This represents a change from the original Hopfield formulation, in which u is unbounded and can achieve negative values as well. In that formulation, however, u is typically regarded as the membrane potential, while V(u) = (tanh(u)+1)/2 would represent the firing rate or activity level of a neuron. In this sense, in our model u can be thought of as a direct measure of the firing rate, without the intermediate step of calculating the membrane potential.

As mentioned in the results session, the 0/1 implementation can reflect the firing rate and connectivity of neurons in a more realistic way, as it does not assume unrealistic features such as symmetric connectivity and reinforcement between silent neurons; this kind of change from the original Hopfield formulation has also been implemented by other authors in different ways [89], [90]. Furthermore, the use of a 0/1 activation scheme also prevents the retrieval of “mirror attractors” (i.e. retrieval of a pattern diametrically opposite to the one which was originally learned) and diminishes the retrieval of spurious patterns when sequences of correlated patterns are learned, as in the case of our simulations, since it prevents the strengthening of connection between inactive neurons, which can lead to the development of abnormal connectivity between neuronal populations when the patterns used are not completely arbitrary.

Modeling of synaptic weight strengthening and degradation

The final steady state pattern achieved by Eq. (1) will in turn induce changes in the synaptic weight matrix as determined by Eq. (2), with the HLP and MID terms defined in equations (3) and (4), respectively. In these equations, the notation B*A denotes connections from A to B. Therefore, note that the first term in Eq. (3) represents the strengthening of positive synaptic connections among coactive neurons, while the second term models a reduction in the strength of synaptic connectivity from active to inactive neurons. Moreover, as the entries of can achieve negative values, the second term in (3) can bring about inhibition from active to inactive neurons, as in classical formulations (see Figure 1D). To prevent a memory or a set of memories from completely dominating and suppressing the other memories, we require that the magnitude of synaptic entries in the matrix saturates at a maximum value s₀. We implement this by truncating the entries that become too large back to s₀, and by using a similar procedure for synaptic values that decrease below −s₀.

After reaching the steady state upon a cue presentation, all units belong to one of four categories: AA, SA, AS and SS, where A stands for Active and S stands for Suppressed, with the first letter indicating the nature of the cue currents and the second letter denoting the final unit activity upon reaching the steady state. For the computation of the mismatch vector, we normalize the cue currents , making them achieve values between 0 and 1 through a transformation defined by . Therefore, it follows from Eq. (4) that when the retrieved pattern is in complete agreement with the input cue, all entries in the matrix MID have zero values, since all units belong either to category AA or to category SS.

When mismatch occurs between the attractor network pattern and the cue currents, this means that there are units pertaining to either AS or SA categories – that is, there are neurons that were suppressed in the retrieved pattern despite activation by the cue current and, conversely, neurons that were active despite cue suppression. The synaptic changes induced by mismatch occur only at the connections linking: (a) active units (AA or SA) to AS, and (b) active units to SA. As a result, in the first case, mismatch-induced degradation acts to decrease the inhibition from active units towards units that are rendered inactive despite the existence of excitatory cue currents arriving at these neurons (AS units; see Figure 1E). Thus, upon subsequent presentation of the same cue pattern, the overall drive (i.e., cue currents plus within network influences) to the AS units is increased, making these units more likely to switch to the AA category. Similarly, in the second case, the strength of connections from active units to SA units decays to lower values as a result of the mismatch-induced degradation (Figure 1E). Consequently, SA units become more likely to switch to the SS category upon subsequent presentation of the same pattern. Note that the effects mentioned here are a consequence of the definition of the mismatch vector (), upon which MID is dependent (Eq. (4)). For instance, an AS unit j has its correspondent m entry value equal to m_j = +1−0 = +1, whereas a SA unit k has m_k = 0−1 = −1. Therefore, by employing , the connection strength from an active unit i (i.e, u_i = +1) to an AS unit j will be changed by MID_ji = +D, whereas the connection to a SA unit k will change by MID_ki = −D. Taken together, these two types of mismatch-induced degradation facilitate the learning of new memory patterns that introduce conflicting information.

Modeling of retrieval tests and nonreinforced reexposure

Memory retrieval is tested by presenting the cue pattern which represents the context, with of its strength at training for context neurons j and 0 for other neurons (Figure 1B), and observing the attractor to which the network evolves. In order to have a closer correlation between attractor retrieval in our computational model and the behavioral measures of memory used in experimental studies of fear conditioning, we model the retrieval of a particular memory pattern as leading to a certain amount of freezing during the test session. Therefore, we assume that upon retrieval of the shock pattern the animal exhibits a high amount of freezing (90% of the test duration), while other memory patterns induce a low, baseline freezing time (10% of the test duration) (see Figure S1).

Cue inputs during the reexposure sessions are modeled by:(5)where vectors and represent the cue inputs for patterns 2 and 3 (Figure 1B), respectively, and t represents the amount of reexposure time (which varies between minimum and maximum values t_min = 0 and t_max = 10 in the simulations). The function monotonically varies between 0 and 1 and determines the ratio between pattern 3 and pattern 2 present in the cue input , which is thus assumed to depend on the reexposure time t. We use a sigmoid defined by , but any monotonically increasing function onto [0,1] provides qualitatively similar results.

Visualization of attractor basins

In agreement with previous research, the strength of the stored memories could be estimated from statistics of full pattern retrieval induced by either partial cue presentation or random initialization of the neural units. In addition, we also developed a new method (used in Figure 2 and Figures S3 and S6) to estimate the basins of attraction for these patterns, defined as follows. Although each pattern constitutes a point in a large N-dimensional space (N = 100), the number of patterns P presented to the network is low (P = 3). This allowed us to use Multiple Discriminant Analysis (MDA) [91] to project these patterns into a low-dimensional encoding subspace of dimension P−1 (in the examples used in this work the resulting dimension is D = 2, a plane). This projection can be obtained by performing and eigenvalue/eigenvector decomposition of the total covariance matrix S_b given by the formula:(6)Here, is the corresponding pattern for each class and is the global mean vector. This method allows the projection of continuous N-dimensional neural states into this subspace, using the matrix comprised by the first P−1 eigenvectors. We then compute their corresponding energy (Lyapunov) function [24] in the original space, using the formula:(7)

Finally, the average energy corresponding to a region in the low-dimensional space is determined as the local mean energy over a set of nearest neighbors and displayed as a 3D color map. While we do not prove that network dynamics converge to a local minimum for all possible initial states, numerical simulations indicate that this is indeed true for all cases analyzed with the 0/1 network used in our work. As a result, this method is useful for providing enhanced intuition about the relative strength of the stored patterns, allowing direct visualization of their corresponding basins of attraction.

Model parameters

The model parameter values employed in simulations were τ = 1 and N = 100 and γ = 0.15; s₀ = 1; S = 0.8; D = 1.25 (unless where otherwise noted). varied from −5 to +5 for each neuron according to the learning pattern presented (see Figures 1C and 2E). Although parameters such as S and D cannot be directly estimated from experimental evidence, results qualitatively similar to ours can be obtained for a reasonably wide range of parameters, albeit with reconsolidation and extinction occurring at different values of t, as observed in Figure S8.

For each protocol studied, we ran 100 simulations with different initial conditions of activation randomly chosen from a uniform distribution on [0,0.1], and the reported result constitutes the mean ± S.E.M. of these simulations, unless where otherwise specified.