A non-canonical on-demand dopaminergic transmission underlying olfactory aversive learning

Dopamine (DA) is involved in various brain functions including associative learning. However, it is unclear how a small number of DA neurons appropriately regulates various brain functions. DA neurons have a large number of release sites and release DA non-specifically to a large number of target neurons in the projection area in response to the activity of DA neurons. In contrast to this “ broad transmission ” , recent studies in Drosophila ex vivo functional imaging studies have identified “ on-demand transmission ” that occurs independent on activity of DA neurons and releases DA specifically onto the target neurons that have produced carbon monoxide (CO) as a retrograde signal for DA release. Whereas broad transmission modulates the global function of the target area, on-demand transmission is suitable for modulating the function of specific circuits, neurons, or synapses. In Drosophila olfactory aversive conditioning, odor and shock information are associated in the brain region called mushroom body (MB) to form olfactory aversive memory. It has been suggested that DA neurons projecting to the MB mediate the transmission of shock information and reinforcement simultaneously. However, the circuit model based on on-demand transmission proposes that transmission of shock information and reinforcement are mediated by distinct neural mechanisms; while shock transmission is glutamatergic, DA neurons mediates reinforcement. On-demand transmission provides mechanical insights into how DA neurons regulate various brain functions.


Introduction
Dopaminergic (DA) neurons play an important role in various brain functions, including sleep-wake regulation, attention, reward, motor control, and learning and memory. The anatomical features of DA neurons are widely projecting axons and a large number of release sites. In canonical synaptic transmission, action potentials generated in presynaptic neurons are propagated to each release site, where Ca 2+ influx via voltage-gated Ca 2+ channels leads to fusion of synaptic vesicles with the plasma membrane, resulting in neurotransmitter exocytosis (Katz and Miledi, 1967). Using this canonical mechanism, DA neurons carry out "broad transmission" by which DA is released from many release sites to numbers of target neurons nonspecifically (a presynaptic basis of volume transmission in mammal). While broad transmission can globally modulate the function of the target regions, it is not appropriate for regulating certain networks, target neurons, and synapses. In the rat striatum, there are a large fraction of release sites that do not release DA upon propagation of action potentials and Ca 2+ influx (Liu et al., 2018;Pereira et al., 2016). Using Drosophila functional ex vivo imaging, we have identified a localized and atypical DA release mechanism (on-demand release) that occurs upon simultaneous stimulation of odor input and somatosensory input pathways to the mushroom body (MB), a brain structure for olfactory memory formation, storage, and retrieval (Busto et al., 2010;Dubnau et al., 2001;Heisenberg, 2003;Hirano et al., 2016;McGuire et al., 2001). MB neurons that have received coincident inputs from two different sensory pathways induce DA release from their presynaptic release sites by generating carbon monoxide (CO) (Ueno et al., , 2017. In this review, we outline the mechanisms of on-demand release of DA and discuss their possible contributions to associative learning.

Olfactory aversive conditioning and long-term enhancement of odor responses
In Drosophila aversive olfactory conditioning using a teaching machine (Fig. 1A), a conditioned stimulus (CS) odor of CS+, which is paired with unconditioned stimulus (US) electric shocks (namely CS+), is presented for one minute. Next, another odor, which is unpaired with US (namely CS-), is presented for one minute (presentation of CS-alone). In the memory test, flies placed in the center of a T-maze are given a choice between the CS + and CS-. It has been suggested that CS + odor information and US shock information become associated in the MBs to form aversive olfactory memory. The MBs are bilateral structures, and each unilateral component is composed of approximately 2000 neurons, named Kenyon cells (KCs). In the calyx, KCs receive olfactory input from projection neurons in the antennal lobe (AL). KCs project their axons through the peduncle (pedc) and then branch out their axons to form either αβ, α'β', or γ lobes. Therefore, KCs are classified as αβKCs, α'β' KCs, or γ KCs ( Fig. 2A). The MB lobes are both output and input sites of the MBs. Thus, while the lobes receive inputs from DA neurons and other external neurons, they output their information to MB output neurons (MBONs). The MB lobes are further divided into 16 compartments (pedc, (Aso et al., 2014;Tanaka et al., 2008), and axon terminals from each DA neuron and dendrites from each MBONs show specific projection patterns to defined compartments (Fig. 2B and 2C).
In vivo imaging in living flies mounted under a microscope shows that MBs exhibit Ca 2+ responses to odor and shock stimuli (Akalal et al., 2010;Cohn et al., 2015;Dylla et al., 2017;Hattori et al., 2017;Wang et al., 2008). Olfactory aversive conditioning increases CS + odor but not CS-odor induced Ca 2+ responses in α'3 compartment of the MB vertical lobes (Cervantes-Sandoval et al., 2013;Tan et al., 2010;Wang et al., 2008;Yu et al., 2006). In addition, repetitive conditioning with rest intervals (spaced training), which produces long-term memory, increases CS + odor induced Ca 2+ responses in α3 compartment (Akalal et al., 2011;Yu et al., 2006). Notably, synaptic outputs from MBON-α'3, which receives MB output in α'3 compartment, and MBON-α3, which receives MB output in α3 compartment, is required for expression of short-term memory and long-term memory, respectively (Aso et al., 2014;Zhang et al., 2019). To address how sensory input pathways modulate MB responsiveness, ex vivo Ca 2+ imaging of MB vertical lobes was performed in isolated Drosophila brains (Ueno et al., 2013) in which the AL and the ascending fibers of the ventral nerve cord (AFV), which transmits sensory information including electric shock from the ventral nerve cord to the brain, were stimulated by microelectrodes (Fig. 4A). Through ex vivo imaging analyses, we found a novel circuitry model that explains the plastic change in MB neurons (Fig. 3B). Similar to in vivo imaging, Ca 2+ responses are observed in the α3α'3 compartments upon AL and AFV stimulation. Furthermore, after the AL and AFV are simultaneously stimulated, AL-evoked Ca 2+ responses in these compartments are enhanced for more than 2 h, a phenomenon called long-term enhancement (AL-MB LTE) (Fig. 4B). Similar to olfactory aversive conditioning, LTE requires nAChR, Dop1R1, and Rut-AC. In addition, LTE shows several phenotypes that are reminiscent olfactory aversive memory. First, trained fly display increased avoidance to the CS + odor but not to US electrical shocks. Similarly, LTE occurs for AL to MB transmission but not for AFV to MB transition. Second, olfactory aversive conditioning requires coincident odor and shock stimulation. Similarly, LTE occurs upon coincident AL and AFV stimulation but not upon AL or AFV stimulation alone. Third, LTE is extinguished by repeated stimulation of the ALs, similar to how olfactory aversive memory is extinguished by repeated CS + odor presentation (Ueno et al., 2013).
When synaptic vesicular exocytosis is monitored by synapto-pHluorin (Miesenbock et al., 1998), cholinergic projection neurons show exocytosis in response to AL stimulation. However, significant exocytosis in the DA neuron terminals is not observed in response to AFV stimulation, and the Ca 2+ response of MBs evoked by AFV stimulation is not suppressed by either Dop1R1 inhibitors nor Dop1R1 mutations. Instead, the Ca 2+ responses from AFV stimulation are impaired by NMDA receptor blockers, a result that is consistent with the robust expression of NMDA receptors in the MB (Miyashita et al., 2012). AFV-evoked Ca 2+ responses in the MB are not completely abolished by an NMDA receptor inhibitor, suggesting that information from the AFVs is transmitted to the MBs through both NMDA receptors and other receptors. Nevertheless, NMDA application can replace AFV stimulation during AL-MB LTE induction, suggesting that the major input from the Fig. 1. Olfactory aversive conditioning and memory testing using a teaching machine.
AFVs for AL-MB LTE is through glutamatergic (Glu) neurons and NMDA receptors rather than DA neurons and Dop1Rs (Ueno et al., 2017). To date, Glu neurons that transmit AFV signal to the MB have not been identified. Although some glutamatergic MBON (MBON-β1 > α and MBON-γ4 > γ1γ2) project their axon terminals to the MB, these MBONs are unlikely to receive AFV information (Aso et al., 2014). While these MBONs express Drosophila homolog of vesicular Glu transporter (DVGLUT), there is another candidate of DVGLUT in the Drosophila genome (Daniels et al., 2004), implicating that another unidentified Glu neurons may transmit AFV information to the MBs.

On-demand DA release
While LTE requires Dop1R1s, odor information is conveyed to the MBs through cholinergic inputs and shock information seems to be transmitted through Glu inputs. Why and when is DA required for LTE formation? Consistent with the early circuitry model, AL-MB LTE is induced by simultaneous AL stimulation and DA application. However, LTE formation is also induced by DA alone. LTE formed by AL + NMDA requires Dop1R1 in the MB, whereas LTE induced by DA alone does not require NMDA receptors, indicating that DA release is downstream of simultaneous AL and NMDA (or AFV) stimulation (AL + NMDA). In addition, DA release occurs specifically onto the MB lobes that have received coincident AL + NMDA (or AFV) stimulation, although PPL1-DA neurons project their terminals to the MBs bilaterally (Fig. 4C). Thus, while projection neurons in the AL project their axons to the ipsilateral MB, AFV stimulation excites the MB on both sides, and only MB neurons ipsilateral to the stimulated AL receive simultaneous AL and NMDA receptor (or AFV) inputs and subsequent DA stimulation (Ueno et al., 2017). Interestingly, the simultaneous inputs to the MB induce DA release in a Rut-AC-dependent manner. In addition, Rut-AC is required again for DA-induced plastic changes in the MB neurons. These results indicate that Rut-AC functions twice during LTE induction. However, it is unclear whether the identical Rut-AC or different Rut-ACs are responsible for DA release and DA-induced plasticity.
How is DA locally released to coincidentally activated MB neurons? During typical synaptic transmission, the neural activity of DA neurons should cause DA exocytosis bilaterally. However, the fact that local DA release was observed only in coincidentally activated MB neurons suggests that coincidentally activated MB neurons may provide signaling MBON-α3, and MBON-γ1ped>α/β project dendrites into the MBα'3, MBα3, and γ1ped compartments, respectively. C) Terminal projection patterns of DA neurons labelled using a THpromoter. All DA neurons in the PPL1 cluster and a portion of DA neurons in the PAM cluster can be labeled using a TH promoter. Each DA neuron projects to both ipsilateral and contralateral MB lobe compartments.

Fig. 3. Schematic model of sensory input pathways to the MBs and LTE formation.
A) An LTE induction model based on the classical network model. Stimulation of PN in AL activates nicotinic AChR (nAChR). AFV stimulation releases DA from DA neurons (DANs) and activates Dop1R1s (D1Rs). B) LTE induction model based on the on-demand release; AFV stimulation releases Glu from glutamatergic neuron (GluN) and activates NMDA receptors (NMDA receptors). Ca 2+ signaling derived from nAChRs and NMDA receptors activates Rut-AC that stimulates HO. CO generated from HO activates RyRs in DA release sites thereby promotes Ca 2+ efflux from Ca 2+ storage, resulting in local DA release.
for exocytosis to their presynaptic DA release sites. It has been reported that DA neurons and MB neurons form reciprocal synapses (Cervantes-Sandoval et al., 2017;Eichler et al., 2017;Takemura et al., 2017). However, local DA release to the lobes of coincidentally activated MB neurons is not suppressed by inhibiting synaptic outputs from MB neurons. Instead, it is suppressed by inhibiting activity of heme oxygenase (HO), which mediates generation of carbon monoxide (CO), in the MBs. DA release is also suppressed by application of a CO chelator to the extracellular medium . Furthermore, application of CO-saturated saline or a CO donor to the MB induces DA release. Similar to DA release, CO is also specifically generated in the lobes of MB that have received simultaneous AL + NMDA stimulation (Michel et al., 2012;Morstein et al., 2020;Ueno et al., 2020). These results suggest a novel "on-demand DA release" mechanism in which coincidently activated MB neurons generate CO as a retrograde signal for local DA release that gives rise to plastic changes (Fig. 3B).
Neurotransmitter release requires an increase in intracellular Ca 2+ (Katz and Miledi, 1967), and an increase in Ca 2+ in DA neuron terminals is observed after CO administration. During typical neurotransmitter release, an action potentials propagate to release sites and activate voltage-gated Ca 2+ channels, which cause the Ca 2+ influx required for exocytosis. However, on-demand DA release occurs even in the presence of TTX, a sodium channel blocker. Instead, it requires activity of ryanodine receptors (RyRs) in the DA neuron, suggesting that an atypical release mechanism that depends on Ca 2+ efflux from internal Ca 2+ stores via RyRs rather than on Ca 2+ influx through voltage-gated Ca 2+ channels may be involved in DA release. In the mammalian striatum, local suppression of DA release by Glu neurons via metabotropic GluRs or local enhancement/suppression of DA release by cholinergic internal neurons via nAChRs or muscarinic AChRs have been reported. In addition, H 2 O 2 produced by medium spiny neurons upon Glu input via ion-permeable GluRs (iGluRs) acts as a retrograde signal that locally suppresses DA release, suggesting a mechanism that locally regulates DA release by target neurons (Avshalumov et al., 2008;Cachope and Cheer, 2014). However, these mechanisms all involve local control of DA release in response to neural activity of DA neurons, and the physiological principle is different from that of on-demand transmission.
The mechanism by which CO causes Ca 2+ efflux through RyRs during LTE is still unknown. Ca 2+ influx through voltage-gated Ca 2+ channels is known to cause Ca 2+ efflux through RyRs in a process known as Ca 2+induced Ca 2+ release, but a different mechanism is likely involved for LTE since DA release by CO occurs in the absence of extracellular Ca 2+ or neural activity of DA neurons. In addition to Ca 2+ -induced Ca 2+ release, RyR activity is increased by calmodulin, ATP, PKA, cADP-ribose, PKG, and NO (Endo, 2009;Kakizawa, 2013;Lanner et al., 2010;Takasago et al., 1991;Verkhratsky, 2005;Xu et al., 1998;Zalk et al., 2007). CO can activate guanylyl cyclase (GC), which produces cGMP. In Drosophila, there are eight membrane-bound GCs (mGCs), two cytosolic GCs (cGCs), and three atypical GCs. However, the expression of PKG in DA neurons has not yet been confirmed. lobes ipsilateral to the stimulated AL (ipsi), and not on the contralateral lobe (cont).

Is on-demand release the basis for associative learning? (Is DA involved in reinforcement but not US transmission?)
What is the physiological role of on-demand release of DA? Ondemand DA release is induced by coincident AL and AFV inputs, suggesting that it may be involved, at least in part, in aversive olfactory learning. In support of this idea, knock-down of HO in the MBs or knockdown of RyRs in DA neurons impairs aversive conditioning . However, on-demand release during aversive conditioning has not yet been observed in vivo, and to detect it in living fly by functional imaging is a future challenge.
It is assumed that DA acts as a reinforcer in the classical conditioning of flies as well as in other mammalian learning models (Horiuchi, 2019;Waddell, 2013). In behavioral psychology, a US presented with or after the presentation of CS + is synonymous with reinforcement of associative learning (Rescorla, 1967). Therefore, in many learning models, including Heisenberg's, DA neurons are assumed to be responsible for both the transmission of USs and reinforcement of learning at the same time, and the failure of conditioning in Dop1R1 mutants and transgenic flies in which output from DA neurons is suppressed could be due to lack of transmission of US information to the MBs. On the other hand, results from ex vivo functional imaging suggest that US transmission and learning reinforcement are separate neural mechanisms that are mediated by Glu and DA, respectively (Horiuchi, 2019;Ueno et al., 2017). Coincident cholinergic CS and glutamatergic US inputs determine which MB neurons receive DA to undergo plastic changes. Therefore, the failure of olfactory aversive conditioning in Dop1R1 mutants and in transgenic flies in which output from DA neurons is blocked is not due to lack of transmission of US information to the MBs, but rather due to lack of the reinforcement of CS and US association by DA.
Although on-demand release restricts DA release to specific MB neurons, DA may activate neighboring neurons that have not produced CO through diffusion. There may be an undefined mechanism that allows specific enhancement of Dop1R1 signaling in MB neurons that have received coincident CS and US inputs, inducing plastic changes. If US shock information is transmitted to the MBs by NMDA receptor/Glu, due to Mg 2+ block of NMDA receptors, Ca 2+ influx through NMDA receptors would occur selectively in MB neurons that have been depolarized by odor information. Considering that D2R/Gi signaling is inhibited by Ca 2+ /CaM (Bofill-Cardona et al., 2000;Park et al., 2005), it is possible that Dop2R signaling is inhibited by Ca 2+ /CaM in MB neurons that have received coincident CS and US inputs, making Dop1R1 signaling dominant to increase cAMP production. In mammals, it has been suggested that Gi coupled D2 dopamine receptors (D2Rs) have a higher affinity for DA than Gs coupled D1 dopamine receptors (D1Rs) (Richfield et al., 1989). Given these affinity profiles, D1R signaling may be dominant in the vicinity of DA release sites, while D2R signaling may be dominant distal to DA release sites, making it possible to induce plastic change specifically in the postsynaptic MB neurons that have generated CO. However, in Drosophila, affinities of Gs coupled Dop1R1s and Gi coupled Drosophila D2Rs (Dop2R) to DA has been suggested to be comparable (Han et al., 1996;Hearn et al., 2002;Sugamori et al., 1995). Even though, enhancing effect of cAMP production by Dop1R1 may be greater than the inhibitory effect by Dop2R, as application of DA alone induces LTE in ex vivo.
Each projection neuron in the AL projects terminals to multiple MB neurons and each MB neurons receives inputs from multiple projection neurons (Caron et al., 2013). Thus, information about individual odors is delivered to multiple MB neurons, and each MB neuron receives multiple odor inputs (Honegger et al., 2011). Based on the molecular mechanism of on-demand release and physiological data from ex vivo imaging on α3α'3 compartments, we propose following cellular model of memory-associated plasticity and memory encoding in particular MB neurons (Fig. 5). i) During olfactory aversive conditioning, specific CS odor depolarizes the subpopulation of MB neurons thereby activating voltage-gated Ca 2+ channels and removing Mg 2+ block from NMDA receptors, while neither activation of voltage-gated Ca 2+ channels nor removal of Mg 2+ block occur in the other MB neurons. ii) In response to in the antennal lobe transmit specific CS odor information to subpopulation of MB neurons, while US shock is transmitted to all MB neurons. CS odor inputs depolarize the MB neurons, thereby activating voltage gated Ca 2+ channels (VGCCs) and removing Mg 2+ block from NMDA receptors (NRs) to gate US shock inputs. Combined Ca 2+ signaling from NRs and VGCCs activates Rut-AC to generate CO. CO activates RyRs in the presynaptic DA release sites and induce local DA release onto coincidentally activated MB neurons. Released DA activates Rut-AC to encode memory and to produce associated plasticity, AL-MB LTE.
US shock presentation, combined Ca 2+ influx through NMDA receptors and voltage-gated Ca 2+ channels generate CO (Horiuchi, 2019;Ueno et al., 2017) in a Rut-AC-dependent manner. iii) DA is locally released to CO-producing MB neurons to encode memory and strengthen the synaptic connections between MB neurons and olfactory input pathways in a Rut-AC-dependent manner.
Our model assumes that removing Mg 2+ block gates association of CS odor information with US shock information which is reinforced by ondemand DA release. While flies expressing NMDA receptors lacking Mg 2+ block fail to form long-term memory, they still can form shortterm memory (Miyashita et al., 2012). In addition, NMDA application alone does not induce DA release in ex vivo preparations. These results imply that NMDA receptor-mediated Ca 2+ influx alone is not sufficient to activate Rut-AC for generating CO. We suspect that the increase in Rut-AC activity sufficient to produce CO may require the convergence of Ca 2+ influx through voltage-gated Ca 2+ channels and Ca 2+ influx through NMDA receptors. Rut-AC is thought to be synergistically activated by Ca 2+ /CaM signaling from cholinergic input and Gs signaling from DA input (Tomchik and Davis, 2009). However, our cellular model hypothesizes that sum of Ca 2+ /CaM signaling mediated by NMDA receptor and voltage-gated Ca 2+ channel is sufficient to activate Rut-AC for CO generation.
While ex vivo imaging studies have suggested that on-demand DA release occurs during or immediately after coincident CS and US inputs to reinforce their association, the importance of DA release after olfactory aversive conditioning is also suggested in forgetting of short-term memory (Berry et al., 2012) and consolidation of long-term memory . Interestingly, both forgetting and consolidation are mediated by DA neurons project onto α'1γ2 and γ1pedc compartments of the MB. Since forgetting is promoted by increasing neural activity of these DA neurons (Berry et al., 2012), we assume that DA release for forgetting may not be on-demand release but broad release. On the other hand, consolidation of long-term memory is suppressed by blocking DA release during the rest intervals of spaced training . Given that this DA release consolidates specific memory information, we speculate it could be an on-demand DA release rather than broad release. While our circuit model hypothesizes learning-induced plasticity at the cellular level, Bilz et al. demonstrate heterogeneous modulation in axonal boutons within the same compartment on the same axon to learned odor (i.e. a mixture of increased or decreased bouton response to CS+) (Bilz et al., 2020). If these bouton-specific modulation requires DA signaling, it is interesting to investigate possible involvement of on-demand DA transmission.
Although this cellular model of on-demand DA release provides provocative insights into mechanisms of learning and memory, there are some conflicts with results from previous behavioral studies and in vivo imaging. First, in vivo imaging of flies expressing genetically encoded DA probes in the MB has shown that, unlike ex vivo studies, DA is released onto the MBs even when shocks or odors are presented alone (Sun et al., 2018), indicating that AFV stimulation does not exactly mimic shock presentation. While these in vivo imaging study detect DA release in the compartments of γ lobes, ex vivo imaging has not done on the horizontal lobes. Hence, it is still unclear whether AFV stimulation evokes DA release on the γ lobe.
In ex vivo imaging, the intensity of AFV stimulation is determined by the appearance of Ca 2+ responses in the MBs. Therefore, if the response threshold of DA neurons to AFV stimulation is higher than that of the MBs, DA neurons may not have been activated sufficiently to release DA even when the MBs show Ca 2+ responses. The pathway from the AFV to the MBs is unknown, and AFV stimulation activates not only the nociceptive pathway but also inhibitory pathways to DA neurons. Another difference between in vivo and ex vivo functional imaging is that while sensory inputs are processed in intact aroused brains receiving multiple sensory inputs in in vivo, sensory inputs are highly restricted in ex vivo. Fly DA neurons release DA upon novel sensory stimulation (Hattori et al., 2017). Given that an aroused state is required for novelty detection, it makes sense that DA release in response to odor or shock alone is absent in ex vivo imaging. In addition, if coordinated activity of various neurons is required for primary responses to novel sensory stimuli and DA is required for such coordinated activity, broad transmission rather than on-demand transmission may be occurring in these situations.
A second conflict is that olfactory aversive conditioning can also be established without US electrical shocks when a CS-odor is presented at the same time as artificial stimulation of PPL1-DA neurons using thermogenetic and optogenetic techniques (Aso et al., 2010;Claridge-Chang et al., 2009). This result is consistent with an early model of memory circuitry where US aversive information is transmitted by PPL1-DA neurons. However, artificial release of DA from PPL1-DA neurons may increase the sensitivity of the MBs to sensory information, similar to symptoms of schizophrenia or sensory hypersensitivity induced by methamphetamine where increases in extra cellular DA level increase sensitivity to sensory stimuli. Therefore, heat or photo stimuli, which do not normally function as noxious or aversive stimuli, may do so in the presence of excess DA release. Consistent with this, it has been reported that the NMDA receptor currents in rat neostriatal neurons are also increased by DA (Cepeda et al., 1998;Flores-Hernandez et al., 2002;Hallett et al., 2006). Furthermore, recent study demonstrated the expression of vesicular Glu transporter, DVGLUT in DA neurons, suggesting co-release of Glu from DA neurons (Aguilar et al., 2017) Artificially stimulated PPL1-DANs may release a larger amount of DA and Glu than in actual aversive conditioning. Also, heat or photo stimuli may induce the release of Glu, which is normally insufficient to provide aversive information. If NMDA receptor conductance is elevated by artificially released DA, these released Glu may induce a sufficient amount of NR-mediated Ca2 + influx in the MB neurons depolarized by odor information even in the absence of shock presentation.
A third conflict is that, in in vivo imaging, CS + odor-evoked Ca 2+ responses in the MBON-γ1pedc>αβ, which receives inputs from the MB γ1pedc compartments and project their axon terminals back onto αβ lobes, are depressed rather than enhanced in trained flies (Perisse et al., 2016). DA release in the γ1pedc compartment is critical for establishing olfactory aversive conditioning  and it has been suggested that training-dependent decreases in CS + odor-evoked Ca 2+ responses are dependent on DA release (Hige et al., 2015). It remains to be examined whether the Ca 2+ responses in the γ1ped compartment and MBON-γ1pedc>αβ are also reduced by AL + AFV stimulation in ex vivo.
Similarly, CS + evoked Ca 2+ response in α3 compartment and in MBON-α'3 is also decreased during first 15 min after conditioning (Zhang et al., 2019). This short-term depression in MBON-α'3 is suggested to be due to decreased ACh release from cholinergic MB neurons. Interestingly, it has been suggested that D1R activity modifies voltage-gated Ca 2+ channel in certain Glu input pathways in the mouse prefrontal cortex, reducing Ca 2+ influx in response to action potentials, thereby decreasing Glu release (Burke et al., 2018). Notably, however, responsiveness to CS + odor in α'3 MB compartment turns to be enhanced later than 30 min after conditioning (Tan et al., 2010;Wang et al., 2008). Furthermore, synaptic output from MBON-α'3 is required for memory recall (Zhang et al., 2019). Similarly, Owald et al. show enhancement in responsiveness to CS + odor in MBON-γ5β'2a (Owald et al., 2015), and DA neurons project on β'2 is required for aversive conditioning (Aso et al., 2010). These previous studies suggest that olfactory aversive learning expresses both enhancement and depression, depending on compartment and time course, and that function of DA is not homogeneous across MBs and varies from compartment to compartment (Cohn et al., 2015). In ex vivo imaging analyses, we have looked at AL-MB LTE in α3 and α'3 MB compartments. Hence, we assume that AL-MB LTE recapitulates an increase in responsiveness to CS + in α3 and α'3 MB compartments that emerges at the later timepoint.
Notably, it was recently reported that the extracellular Mg 2+ concentration in the Drosophila brain is approximately 18 mM (Raccuglia et al., 2019). Indeed, NMDA receptors in Drosophila requires Mg 2+ concentrations higher than 10 mM Mg 2+ to induce Mg 2+ block (Miyashita et al., 2012). While our ex vivo imaging studies use 15− 20 mM extracellular Mg 2+ , most of other imaging studies use much lower Mg 2+ concentrations, around 2− 4 mM. Since Ca 2+ influx required for synaptic vesicle exocytosis is inhibited by extracellular Mg 2+ (Dodge and Rahamimoff, 1967;Douglas, 1968), release of neurotransmitter, including DA, might be facilitated by lower concentrations of extracellular Mg 2+ . Such a difference in physiological conditions may also have led to the discrepancy in results between ex vivo and in vivo imaging.
Taking account of the results obtained from ex vivo and in vivo imaging, we speculate that there are two distinct modes of DA release during olfactory aversive conditioning: shock-associated (DA neuron activity-dependent) broad DA release during conditioning, and subsequent CO-dependent on-demand release. Interestingly, Cohn et al. demonstrate that paring DA release with stimulation of KC results in depression of MBON-γ4 responsiveness to KC stimulation, while DA release alone results in enhancement (Cohn et al., 2015). We suspect that broad DA release upon shock presentation is phasic and terminated at the end of conditioning, but on-demand DA release is rather tonic and lasts even after conditioning. Based on the results by Cohn et al., broadly released DA may cause short-term depression upon association with CS + odor, while on-demand DA release after conditioning may cause long-term enhancement that becomes dominant later in α'3 and α3 compartments. Recently, Adel and Griffith combine ex vivo and in vivo imaging data and propose an integrative model (Adel and Griffith, 2021). They propose that DA released by shock presentation is insufficient for aversive conditioning. However, in the MB neurons that have been coincidentally activated by odor and shock inputs, DA is additionally released by CO thereby becoming a sufficient level to establish conditioning.

Does on-demand DA release exist in mammals?
In mammals, DA neurons have highly branched axons with a huge number of release sites. For example, DA neurons project their axons from the substantia nigra to broad areas of the striatum. There is a report of a single DA neuron whose axon terminals occupy nearly 6% of the striatum (Matsuda et al., 2009). In mammal, most of the DA receptors are expressed at extrasynaptic sites. Therefore, DA signaling is thought to be carried out by volume transmission, in which diffusely released DA acts also on receptors distant from the release sites. It is still not clear whether Drosophila DA receptors are expressed at extrasynaptic sites, but like broad transmission in Drosophila, volume transmission is evoked by activity of DA neurons that releases DA from many release sites simultaneously. However, more than half of the DA release sites in the striatum are silent release sites that do not release DA in response to activity of DA neuron (Liu et al., 2018;Pereira et al., 2016). While the active release sites that undertake volume transmission express active zone proteins such as RIM and Bassoon, silent release sites do not (Liu et al., 2018).
It is unclear whether silent release sites differentiate into active release sites under certain physiological conditions or whether they release DA under specific conditions. It will be interesting to see whether atypical and localized on-demand release of DA, which does not require neural activity or extracellular Ca 2+ influx, is responsible for DA release from these sites. On-demand release also requires an elevation of intracellular [Ca 2+ ]i at the release site. However, the silent release sites of striatal DA axons, unlike active release sites, do not release DA even when Ca 2+ is elevated by activity of DA neuron. This suggests that the concentration of Ca 2+ required for DA release from silent release sites may be higher than that of active release sites. The Ca 2+ concentration in the smooth endoplasmic reticulum [Ca 2+ ] er , an intracellular Ca 2+ storage site, is comparable to the extracellular Ca 2+ concentration [Ca 2+ ] out, in the mM range (Alonso et al., 1998). However, while neuronal voltage-gated Ca 2+ channels are self-limiting and inactivated by increased intracellular Ca 2+ (Eckert and Tillotson, 1981), RyRs do not seem to have this limit and release Ca 2+ without inactivation during CO exposure . It would be interesting to examine whether mammalian silent release sites express RyRs. Considering that RyRs are activated by NO as well as CO, other animals, including mammals, may employ NO or both NO and CO as retrograde signals for on-demand release. In Drosophila, the anatomical relation of the release sites responsible for classical DA release and on-demand release site are not clear. The integration of the physiological and anatomical findings of silent release sites in mammalian systems and on-demand release in Drosophila may help us understand the regulation of various brain functions by DA.

Conclusions
In early studies, DA was thought to act as a positive reinforcer for reward learning in mammals, whereas in flies, DA was thought to act as a negative reinforcer for aversive learning. However, it has now been shown that DA also acts as a positive reinforcer in flies (Burke et al., 2012;Kim et al., 2007;Krashes et al., 2009;Lin et al., 2014;Liu et al., 2012;Perisse et al., 2016;Selcho et al., 2009;Yamagata et al., 2016), and as a negative reinforcer in mammals (Ilango et al., 2012;Matsumoto and Hikosaka, 2009). In flies, as in mammals, DA has been found to be involved in sleep-wake regulation, motor control, and attention (Akiba et al., 2020;Kume et al., 2005;Pimentel et al., 2016;Ueno et al., 2012). This suggests that the diversity of physiological roles and functions of DA is conserved in flies and mammals.
In this review, we proposed a learning model for aversive olfactory conditioning in which the function of DA as a negative learning reinforcer is segregated from that of US aversive signaling: US aversive information is transmitted to the MBs by Glu, while DA is specifically released to MB neurons that receive coincident CS and US inputs to reinforce CS-US associations (Horiuchi, 2019). As a neural basis for this learning model, we introduced on-demand release, in which post-synaptic MB neurons simultaneously stimulated with CS and US inputs from the ALs and AFV, respectively, induce local pre-synaptic DA release using CO as a retrograde signal . We proposed that DA release is important for the establishment of conditioning. It remains to be elucidated which neurotransmitter conveys positive US signals during reward conditioning and whether the on-demand DA release is also involved in the establishing of the reward associations. In addition, it remains unclear whether broad transmission is responsible for DA released evoked by odor or shock presentation alone, and what role this released DA plays in conditioning. Resolving these issues will allow us to understand how DA regulates learning and memory and how a small number of DA neurons regulate various brain functions.

Funding
This work was supported by a Grant-in-Aid for Scientific Research in Innovative Areas "Memory dynamism" (MEXT KAKENHI, JP25115006) and for Scientific Research (A) (JSPS KAKENHI, 19H02013), and Takeda Science Foundation to M.S.