Modulation of hippocampal plasticity in learning and memory

Synaptic plasticity plays a central role in the study of neural mechanisms of learning and memory. Plasticity rules are not invariant over time but are under neuromodulatory control, enabling behavioral states to influence memory formation. Neuromodulation controls synaptic plasticity at network level by directing information flow, at circuit level through changes in excitation/inhibition balance, and at synaptic level through modulation of intracellular signaling cascades. Although most research has focused on modulation of principal neurons, recent progress has uncovered important roles for interneurons in not only routing information, but also setting conditions for synaptic plasticity. Moreover, astrocytes have been shown to both gate and mediate plasticity. These addi-tional mechanisms must be considered for a comprehensive mechanistic understanding of learning and memory. This highly influential paper suggests a two-stage process of hippo- campal memory formation, whereby a weak and transient memory trace is initially laid down in CA3 during theta activity, which later during sharp wave-ripples drives the persistent synaptic potentiation of CA3-CA1 synapses. This study showed that firing of medial septal cholinergic neurons is highly correlated with hippocampal theta states during exploration and rapid eye movement sleep. Optogenetic activation of these neurons during slow-wave sleep, a behavioural state not related to theta activity, suppressed ripple oscillations in CA1, blocked by muscarinic M2 and M4 receptor antagonists. Optogenetic and pharmacogenetic tools were used to investigate the contribution of the medial entorhinal cortex (mEC) and hippocampal CA3 to network oscillations and place cell activity in CA1. The authors found that silencing inputs from the mEC abolished theta and gamma oscillations in CA1, while not affecting firing rates. Conversely, CA3 silencing Monosynaptic retrograde tracing single-cell optogenetic stimula- tion were used to investigate the microcircuit dynamics of CA1 place cells during navigation. The authors found less activity in connected presynaptic interneurons when the animal traverses a stable place field, but not a newly formed one, and further that optogenetic activa- tion of a place cell recruits other functionally coupled pyramidal neurons. The authors suggest that plastic reorganization in local circuits drives spatial selectivity of CA1 neurons. used optogenetics to study the role of parvalbumin (PV) and somatostatin (SST) interneurons onto CA1 pyramidal neurons in long-term inhibitory synaptic plasticity in hippocampal slices. Inhibitory PV synapses show iLTD, while SST interneurons undergo iLTP using a spike-timing dependent plasticity protocol. Using this study shows that astrocytes are in direct contact with individual dendritic spines forming a functional component of tripartite synapses. Ca 2+ imaging revealed that the astrocytic nodes form biochemical compartments and Ca 2+ microdomains. This study demonstrates a high expression of the GluN2C NMDAR subunit in hippocampal astrocytes. The authors suggest that synaptic weight distribution of Schaffer collateral inputs is regulated by a feed- back signaling mechanism involving astrocytic NMDARs, which may affect long-term synaptic plasticity. This study describes a developmental switch in spike-timing dependent plasticity rules from t-LTD to a form of t-LTP that is mediated by astrocytes and expressed presynaptically. The induction of this form of t- LTP is independent of NMDARs but requires postsynaptic and astrocytic Ca 2+ signaling, and vesicular gliotransmitter release. This study used advanced imaging methods and an optical glutamate sensor combined with patch clamp recordings to study the effect of LTP on perisynaptic astroglial processes (PAPs). They found PAP with- drawal from potentiated synapses, leading to retreat of astroglial glutamate transporters and an increase in extrasynaptic glutamate enhancing NMDAR-mediated cross talk among synapses. This study reveals an important role for astrocytic phagocytosis in the activity-dependent elimination of excitatory synapses. Interference with phagocytosis led to an accumulation of functionally impaired synapses. This caused deficits in LTP and led to impairment of hippocampus- dependent memory formation.


Introduction
Learning and memory enable animals to continuously adapt to their environments. The cellular basis of these adaptations is thought to be synaptic plasticity leading to changes in the functional connectivity of neuronal networks in the brain. Following the seminal discovery of long-term potentiation (LTP) [1], spike timingdependent plasticity (STDP) [2] and behavioral timescale synaptic plasticity (BTSP) [3,4**] have emerged as physiologically relevant forms of synaptic plasticity that are likely to be important for learning and memory processes. However, synaptic plasticity is dynamic and varies with behavioral state. Different behavioral states, such as sleep or awake state, and different behaviors, such as exploring an environment or consuming a reward, are associated with release of different neuromodulators. Long-range neuromodulators, such as acetylcholine and monoamines, bind to and activate G-protein-coupled receptors, which directly control the gating and polarity of synaptic plasticity through intracellular signaling cascades [5,6]. Neuromodulators further exert control (1) at the neural network level by directing information flow, (2) at the circuit level through excitation/inhibition balance determining spike-timing patterns, and (3) at the synaptic level through modulation of astroglia.
This review focuses on modulation of synaptic plasticity in the hippocampus, a structure important for spatial learning and memory [7]. Considering hippocampal processing during spatial navigation and memory tasks, acetylcholine and dopamine are particularly relevant. We review evidence that these neuromodulators guide information flow in neuronal circuits and coordinate spike-timing patterns to efficiently store relevant memory traces. Direct effects of neuromodulators on intracellular signaling pathways in neurons have been extensively reviewed elsewhere [8], and here we will focus on the modulation of plasticity at the network and circuit levels and discuss the involvement of astroglia as part of the "tripartite synapse" [9] (Figure 1).

External neuromodulators control network states supporting different types of plasticity
Whenever an animal switches from one behavioral state to another (e.g. from sleep to wakefulness, or from immobility to exploration) or receives an external salient stimulus (e.g. finding a reward), this shift is correlated with changes in neuronal network activity, typically captured by local field potential (LFP) or electroencephalographic (EEG) recordings. A classic example is the transition from slow-wave sleep to wakefulness, when the cortical activity shifts from synchronous oscillations to an asynchronous state [10]. Shifts in brain states are associated with changes in cholinergic and monoaminergic (histaminergic, noradrenergic, dopaminergic, and serotonergic) neuromodulators. Acetylcholine has diverse effects on brain states depending on the site of release and the receptor subtypes in the target neuronal population, and is generally associated with behaviors that are adaptive to environmental stimuli, such as exploration [11]. Dopamine neurons are well established to encode reward, while more recently they have additionally been suggested to transmit nonreward signals related to novelty and other salient events [12].
The link between the release of neuromodulators and memory formation is not well understood. In an influential model, it was suggested that hippocampal memory formation occurs in two stages. Memories are initially encoded as a weak and transient synaptic potentiation during exploration of an environment when hippocampal theta activity dominates. Subsequently, during awake immobility or slow wave sleep, sharp wave ripple (SWR)-associated reactivation of neuronal activity induces LTP at hippocampal CA3-CA1 synapses [13]. These two network states in the CA1 area of the hippocampus rely on neuronal input from distinct regions, occur temporally separated, and are mutually exclusive. Theta-gamma oscillations, which occur during exploration, are associated with cholinergic activity. Acetylcholine binds to two types of receptors, ionotropic nicotinic receptors and metabotropic muscarinic receptors coupled to either Gq proteins (M1, M3, and M5 subtypes), which activate phospholipase C, or Gi/o proteins (M2 and M4 subtypes), which couple negatively to adenylyl cyclases [11]. In the hippocampus M1 muscarinic receptors are present at high levels, followed by M3, which are found at moderate levels, whereas M2 are found at much lower densities. M1 muscarinic receptors are predominantly located on somato-dendritic membranes of neurons. Their activation leads to membrane depolarization and increases in cellular excitability through several different mechanisms, and they are required for induction of gamma oscillations (25e80 Hz [14]. M2 muscarinic receptors are often located on presynaptic axon terminals (for a review see [15]). Optogenetic activation of cholinergic neurons in the medial septum in vivo suppresses CA1 ripple oscillations through M2 and M4 muscarinic receptors [16*]. Theta-gamma oscillations require input from the medial entorhinal cortex (MEC), an area providing sensory information. MEC silencing abolishes extracellular theta-gamma currents in CA1, while firing rates remain largely unaffected [17**]. Conversely, silencing inputs from CA3, thought to mediate internal representations, decreases firing of CA1 neurons but leaves theta currents intact [17**]. By way of enhancing gamma oscillations [18], acetylcholine may set appropriate time windows for STDP, a mechanism suitable for the initial encoding of memories when an animal explores an environment. SWRs, which are associated with burst activity in hippocampal neurons [13,19], might then consolidate those initial memory traces through induction of LTP, although during sleep, SWRs were reported to downregulate synaptic weights overall [20]. SWRs occur during low-cholinergic states. Interestingly, rewarded outcomes, which are associated with activity in dopamine neurons, enhance SWR-associated reactivation events [21,22] and activation of dopamine neurons promotes hippocampal reactivation and spatial memory persistence [23], enabling actions leading to rewarding outcomes to be stored in memory, although it is unknown whether the action of dopamine is in the hippocampus itself. Dopamine acts on two types of receptors. The D1-like receptors, including D1 and D5, are coupled to Gs-proteins, which activate adenylyl cyclases, and the D2-like receptors, which include D2, D3, and D4, are coupled to Gi/o-proteins, which inhibit adenylyl cyclases.
Another recently discovered form of plasticity, BTSP [3], which may be bidirectional [4**], is induced when synaptic inputs are active within 1e2 s of calcium plateau potentials, which are long-duration dendritic calcium spikes of large amplitude. Plateau potentialinduced plasticity, which has been associated with an increased activity of neurons during subsequent SWRs [24], and strong single-cell stimulation, possibly by triggering plateau potentials, can induce new place fields [24e26]. Generation of place fields may not require synaptic plasticity, however, as subthreshold place fields in initially silent cells can be unmasked by depolarizing current [27] or optogenetic stimulation [28**]. Moreover, hippocampal activity represents many behaviorally important variables [29] and also stores non-spatial choice-related information [30]. A recent study that investigated the effect of blocking LTP by interfering with AMPAR insertion in vivo suggested that rather than being required for spatial memory per se, LTP in CA1 stores reward information, suggesting a role of dopamine modulation on learning and memory in vivo [31**].
These recent findings are consistent with a two-stage model of memory formation [13]. Neuromodulation may add a further dimension to this model by informing the network about the behavioral state of the animal and thereby prioritize salient information for memory storage ( Figure 2).

Neuromodulators control plasticity by guiding information flow in the network
Neuromodulators have pathway-specific effects on synaptic transmission and thereby guide the information flow at the circuit level. In the hippocampus, the major output region CA1 receives Schaffer collateral (SC) input from hippocampal CA3 and temporoammonic (TA) input from the entorhinal cortex (EC). These two CA1 inputs are crucial for memory formation and the balance between them depends on acetylcholine and dopamine. Cholinergic agonists inhibit excitatory SC input more than TA input [32]. A recent study using optogenetic stimulation of cholinergic fibers confirmed this finding; excitatory inputs from both the entorhinal cortex and CA3 were depressed but stronger suppression of feedforward inhibition in the TA pathway compared to the SC pathway resulted in a selective increase of E/I balance in TA inputs. These effects were mediated by different interneuron subpopulations and muscarinic receptor subtypes in the two pathways, M3 for TA inputs and M4 for CA3 inputs [33**]. Conversely, application of dopamine, which is associated with reward and novelty in behavioral contexts, induces strong suppression of TA inputs to CA1 in hippocampal slices, with little or no effect on SC input [34]. This led to the suggestion that dopamine favors information flow through the trisynaptic circuit via CA3, while inhibiting the direct input from the EC, thus enhancing memory storage by increasing the relative influence of CA3 [35] ( Figure 2). More recent studies using optogenetic release of dopamine partly support but also challenge this view. Optogenetic release of dopamine from ventral tegmental area (VTA) inputs in the hippocampus had no Two-stage model of memory formation. The release of neuromodulators is associated with different behavioral states. Acetylcholine (ACh) is released during theta activity when an animal explores the environment. If the animal finds a reward (orange circle) dopamine (DA) is released, enhancing the reactivation of neuronal activity associated with sharp wave ripple (SWR) activity. Theta-gamma and SWR traces reproduced from [18]. effect on TA input, but induced an activity-dependent modulation of SC input [36]. Low levels of dopamine release, simulating tonic VTA neuron firing, depressed SC responses through enhancement of parvalbumin (PV) interneuron-mediated feedforward inhibition. Higher levels of dopamine release, simulating phasic firing of VTA neurons, increased SC responses through a D1 receptoredependent enhancement of excitatory synaptic transmission [36]. However, fibers from the VTA mainly target stratum oriens and pyramidal layers and less the stratum lacunosum-moleculare, where the TA inputs terminate, explaining the lack of a direct effect on those inputs. Nevertheless, the projections from the VTA are not the only source of dopamine in the hippocampus. Dopamine is also co-released with noradrenaline from locus coeruleus (LC) fibers, which are more spread out in the hippocampus than fibers from the VTA. Environmental novelty activates LC neurons and leads to memory enhancement mediated by D1/D5 receptors independent of VTA activation [37,38]. Interestingly, spatial reward learning and overrepresentation of reward locations by place cells also require LC fibers [39**]. In summary, release of acetylcholine in the hippocampus favors information flow of inputs from TA over SC inputs by modulation of feedforward inhibition. This is associated with explorative behaviors, where mediation of sensory input may be prioritized. In contrast, dopamine is generally thought to favor information flow from SC compared to TA inputs. However, while tonic release of dopamine from VTA inputs decreases SC inputs, phasic release enhances those inputs. Similarly, phasic dopamine release from LC fibers increases information flow over CA3-CA1 connections. The effect of dopamine release from LC fibers on TA inputs has yet to be investigated.

Neuromodulators affect inhibition coordinating spike-timing patterns
A balance of excitatory and inhibitory synaptic inputs is important for functional stability of neuronal circuits. Inhibitory interneurons are important both for balancing excitation and controlling spike timing. For example, SWRs in CA1 occur when increased excitation from CA3 (the sharp wave) results in activation of CA1 pyramidal cells and feedback inhibition from reciprocallyconnected PV basket cells create time windows of alternating pyramidal cell and interneuron firing, synchronized at ripple frequency [40]. External neuromodulators often influence both excitatory and inhibitory synaptic transmission and thereby reconfigure the E/I balance, which can result in spike-patterns conducive to the induction of synaptic plasticity (Figure 3). Acetylcholine differentially affects different interneuron types (for a review see [41]). Cholinergic input directly activates oriens lacunosum-moleculare (O-LM) interneurons, and thereby regulates CA1 neuronal firing [42]. Some types of interneuron also express dopamine receptors [43], which can modulate their intrinsic properties and synaptic transmission [44,45]. Deletion of D2 receptors from PV interneurons causes schizophrenia-like phenotypes [46], suggesting important functional roles of dopaminergic modulation of interneurons.
On longer timescales, the physical architecture of the network is sustained through several homeostatic cellular mechanisms, including long-term plasticity of GABAergic inhibitory synapses. While the cellular mechanisms of plasticity at excitatory synapses have been well-studied, less is known about the plasticity rules of inhibitory GABAergic synapses. Moreover, due to the extensive diversity of interneuron types targeting specific membrane subdomains of both pyramidal neurons and other inhibitory interneurons [47], it has been more challenging to establish a unified view of the role of inhibitory plasticity (for a recent review see [48]). A recent study using single-cell retrograde tracing and 2photon calcium imaging in vivo revealed that activity in interneurons presynaptic to a place cell is lower when the animal traverses a stable place field but not newly formed ones, suggesting that local circuit plasticity amplifies spatial selectivity of CA1 neurons [49**]. The cellular basis of the circuit level changes observed in vivo is synaptic plasticity, which varies between different interneuron types. In the hippocampus, many PV interneurons target CA1 pyramidal neurons in the perisomatic region, whereas somatostatin (SST)-expressing interneurons preferentially target dendrites. Both interneuron types require coincident inhibitory input and pyramidal neuron depolarization for plasticity induction at inhibitory synapses onto pyramidal cells. This plasticity requires T-type voltage-gated calcium channels [50*]. However, PV and SST interneurons display distinct plasticity rules. A timing-dependent plasticity protocol induced depression in PV interneuron synapses (PV-iLTD), while SST interneuron synapses were potentiated (SST-iLTP). Interestingly, PV-iLTD increased the average spike probability of CA1 pyramidal neurons when CA3-CA1 inputs were activated, with no effect on TA inputs. In contrast, SST-iLTP led to a substantial reduction in CA1 spiking when the TA input was stimulated, with no effect on SC stimulation [50*].
Interneurons upstream of CA1 also show differential effects. Suppressing DG and CA3 PV interneurons increased coupling from CA3 to CA1 but decreased coupling from entorhinal cortex to CA1; suppressing SST interneurons had the opposite effect [51]. Longterm plasticity in PV interneurons in CA3 depends on local D1/5 dopamine receptor activation, resulting in enhanced sharp-wave ripples and long-term memory consolidation [52]. Taken together, these findings highlight a central role of neuromodulation of GABAergic neurons for spike patterns and plasticity.

Astrocytes as synaptic modulators of plasticity and their control by external neuromodulators
The core mechanisms that drive activity-dependent plasticity have long been thought to be exclusively neuronal. However, recent studies have highlighted important roles of glial cells, and in particular of astroglia, in the modulation of synaptic plasticity, neural circuits, and networks (for other recent reviews of their roles in synaptic, computational, and behavioral functions, see [53e55]). Beyond their well-documented metabolic functions, as part of the "tripartite synapse" [9], astrocytes increase their cytoplasmic calcium levels in response to synaptic signals, including endocannabinoids acting on CB1 receptors. This induces the release of neuroactive substances ("gliotransmitters") which in turn modulate synaptic activity and plasticity. A recent super-resolution microscopy study identified a structural basis for astrocytic calcium microdomains associated with individual synapses [56*]. There is strong evidence that astroglia regulate D-serine and/or glycine availability as coagonists at NMDA receptors (NMDARs), gating the induction of both LTP [57] and LTD [58]. Furthermore, hippocampal astrocytic Dserine release appears to be important for cognitive flexibility [59]. Whether astroglia themselves release Dserine or they produce L-serine, which is converted to D-serine by the enzyme serine racemase in neuronal compartments, is debated [60,61]. Complicating factors in interpreting astrocytic modulation of NMDARdependent plasticity is that astrocytes themselves express NMDARs [62*] and that their involvement changes during development [63*]. The role of astrocytes in plasticity is not limited to the induction but may also be involved in expression of plasticity. It was recently shown that induction of LTP prompts withdrawal of perisynaptic astroglia, boosting extrasynaptic glutamate actions [64*]. Conversely, astrocytes have a major role in the neuronal activity-dependent elimination of excitatory synapses through astrocytic phagocytosis [65*].
The involvement of astroglia in modulation of plasticity appears to be important for learning and memory, as astrocyte-specific genetic deletion of CB1 receptors impaired LTP and recognition memory, both of which were completely rescued by elevation of D-serine levels [66]. Moreover, a recent study showed that block of an astrocytic pathway required for LTD at CA3-CA1 synapses affects contextual fear conditioning [67], whilst chemogenetic activation of astrocytes in the mouse CA1 area of the hippocampus enhances fear learning [68]. These examples indicate that astroglial modulation of plasticity is of behavioral relevance. Interestingly, astrocytes are themselves controlled by long-range neuromodulators. Acetylcholine can activate astrocytic a7 nicotinic receptors causing an increase in extracellular D-serine and enhanced synaptic NMDAR function influencing contextual fear conditioning [69]. Astrocytes also respond to dopamine (for a review see [70]). In the hippocampus, dopamine triggered a bidirectional calcium response in stratum radiatum astroglia, a D1/D2 receptor-dependent elevation followed by a D2 receptor-dependent below-baseline decrease [71]. Dopamine elicits cAMP increase in hippocampal primary astroglial culture [72]. Directly increasing astrocytic cAMP is sufficient to facilitate memory formation in an objectelocation memory task [73*]. These findings show that glia, which have been referred to as "the glue holding memories together" [74], are now considered important modulators of plasticity and learning. Whether and how astrocyte stimulation by dopamine affects synaptic plasticity in the hippocampus in vivo is an interesting question for future research.
Neuromodulators directly affect the outcome of plasticity and memory Acetylcholine has been reported to facilitate both LTP and LTD [75] and is often thought to enhance the contrast between these two forms of synaptic plasticity. Dopamine enhances LTP and has specifically been related to protein-synthesis dependent late-phase LTP [76]. In hippocampal slices, while application of acetylcholine biases STDP in SC inputs towards depression, subsequent dopamine application induces potentiation [77]. Whilst the source of dopamine for reward learning in the hippocampus is unknown, it has been shown that optogenetic dopamine release from LC fibers leads to LTP [37]. A computational model showed that sequential neuromodulation of plasticity enables effective navigation in an environment with changing reward locations, similar to natural behavioral settings [77]. Moreover, in a behavioral learning task, a phasespecific effect of acetylcholine release was observed. While optogenetic release of acetylcholine during navigation had no effect, its release at a goal location impaired sharp wave ripple activity and spatial longterm and short-term memory [18,78]. Since acetylcholine release depresses CA3 inputs [33**] and leads to decreased SWR activity [16*], this offers a possible explanation for memory impairment upon cholinergic activation at inappropriate times. Whilst more studies will be needed to fully understand the effect of acetylcholine and dopamine on hippocampal-dependent learning, the findings summarized above support a two-stage model of memory formation.

Future directions
While studies investigating effects of individual aspects of neuromodulation on plasticity are essential to our understanding of learning and memory, these factors should not be viewed in isolation. Recent technical developments, such as fluorescent reporters for acetylcholine and dopamine and 2-photon imaging in freely behaving animals, allow investigations of network effects of neuromodulators in the context of behavioral learning and memory. Furthermore, optogenetic tools enable studies of endogenous release of neuromodulators and their circuit-specific effects on neuronal activity and plasticity. Computational models should take into account how plasticity rules are affected by internal and external modulators [79]. Moreover, behavioral studies of learning and memory need to take into account that plasticity rules show sex differences that may vary over development. A recent study reported opposite changes in LTP and spatial learning in female and male mice during the transition from prepubertal to postpubertal age [80]. With new techniques rapidly emerging the prospects look good for substantial progress in our understanding of the neural mechanisms of learning and memory.

Conflict of interest statement
Nothing declared.  Knockdown of a postsynaptic SNARE complex eliminated hippocampal LTP by interfering with AMPAR insertion but did not affect accurate representations of space and produced only limited effects on learning and memory. However, the authors found that LTP in CA1 in vivo is required for incorporation of salient information in memory.

* *
. Palacios-Filardo J, Udakis M, Brown GA, Tehan BG, Congreve MS, Nathan PJ, Brown AJH, Mellor JR: Acetylcholine prioritises direct synaptic inputs from entorhinal cortex to CA1 by differential modulation of feedforward inhibitory circuits. Nat Commun 2021, 12:5475, https://doi.org/10.1038/ s41467-021-25280-5. The authors used optogenetic tools to investigate the effect of endogenous acetylcholine release on synaptic transmission in the temporoammonic (TA) and Schaffer collateral (SC) inputs toCA1. They found that acetylcholine release enhances the relative weight of TA inputs over SC inputs mediated through changes in feedforward inhibitory synaptic transmission.