INTRODUCTION

For more than a century following William James’ original thesis on emotion (James, 1884), psychologists have attempted to determine whether the diverse range of human affect can be understood using few independent factors. On the basis of self-reported emotional states, early theorists charted emotions in two or more dimensions (Nowlis and Nowlis, 1956; Russell, 1980; Schlosberg, 1954). Popular among these models is the two-dimensional circumplex model of emotion (Russell, 1980), wherein emotions arise from the interaction between two neuropsychological systems—one representing the degree of pleasantness, ranging from aversive to appetitive (valence), and the other representing alertness (arousal; Posner et al, 2005). Identifying and understanding the neurobiological substrates underlying these features of emotion is an active area of neuroscience research.

The idea that anatomically localized regions in the brain drive emotion and emotional behaviors was initially suggested by the finding that lesions to the temporal lobe and amygdala cause affective deficits (Klüver and Bucy, 1939). Following this early work, animal models for studying affect have been instrumental in advancing our understanding of the neurobiological basis of emotion. Although the subjective aspect of emotions cannot be directly tested in animal models, the behavioral and physiological responses elicited by emotionally relevant stimuli can be objectively assessed.

Arousal is commonly studied in relation to consciousness, sleep, attention, sex, and emotion. Emotional arousal is an important aspect of emotion that is known to enhance emotional memory, either positive or negative. For a detailed review of the neural representation of arousal, refer to (Adolphs et al, 1999; Harris and Aston-Jones, 2006; Lang et al, 1998; McGaugh, 2000, 2004; McIntyre and Roozendaal, 2007).

Monitoring neural activity evoked by emotionally salient stimuli in model organisms, such as non-human primates and rodents, has proved to be an invaluable method to investigate the neurobiological basis of valence. A stimulus that is inherently appetitive or pleasant is said to carry positive valence, whereas a stimulus that is inherently aversive is said to carry negative valence. These stimuli are sufficient to evoke appetitive or aversive responses, and are therefore designated positive or negative unconditioned stimuli (US), respectively. When a previously neutral stimulus (known as a conditioned stimulus or CS), such as a tone, odor, or image, predicts a positive or negative US, it acquires valence. Pavlovian conditioning (Pavlov, 1927; Rescorla, 1988), in which the CS and US are repeatedly paired, is a common behavioral paradigm for teaching an animal a CS–US association. After the acquisition of a successful CS–US pairing, a positive CS is sufficient to evoke appetitive behaviors such as approach toward a food dispenser, and a negative CS is sufficient to evoke fear- or avoidance-related behaviors such as freezing in response to a CS predicting a foot shock.

In several regions of the brain, neural responses to the CS change after the acquisition of CS–US pairing. Moreover, these changes are different depending on whether the CS is positive or negative. A neuron recorded during a behavioral paradigm such as Pavlovian conditioning or retrieval is said to represent valence if its output is differentially modulated by the positive and negative natures of the stimuli, independent of all other features. Some brain regions, such as the basolateral amygdala (BLA), contain a greater proportion of neurons signaling valence (~40%) compared with others, such as the hippocampus (~25%; Fuster and Uyeda, 1971). Valence-encoding neurons represent only a subset of neurons within each of these brain regions. Even among the valence-representing subset in a given region, there are some neurons signaling positive valence and some neurons signaling negative valence. Much effort is being devoted to isolate a common property within neurons signaling one valence vs neurons signaling the opposite valence. Projection target or genetic markers are promising candidates for properties that could distinguish neurons selectively signaling positive or negative valence.

In the next few years, we predict the characterization of several new valence-signaling populations identified by their projection target and/or genetic markers, and distributed in a wide network throughout the brain. Comparing and contrasting the extent of valence representation within and between these populations will be essential in directing the field’s efforts to understand the neurobiological basis of valence. Here we propose a model that will facilitate comparison and contrast between the multitudes of candidate populations signaling valence.

STRATEGIES TO ASSAY REWARD AND AVERSION IN RODENT MODELS

The field is now utilizing modern approaches to dissect the basis of valence in mammalian neural circuits, and, as a result, there have been a variety of classical and novel behavioral models employed to capture valence responding in rodents. Historically, many of these behavioral paradigms have relied on Pavlovian conditioning approaches. These models date back to early work in primates with both drugs of abuse and natural rewards (Spragg, 1940). In all examples, a CS is typically paired with a US, and the time animals spend in a given CS–US paired context is measured. Place preference models in rodents were then adopted from these early methods and are now widely used to assess reward and aversion, ranging from natural rewards (ie, sucrose) and drugs to aversive stimuli (Rossi and Reid, 1976). When a US is repeatedly paired with a neutral environmental stimulus, the motivational properties of the neutral stimulus change, and, over the course of the conditioning period, the neutral stimulus becomes a CS. Subsequent to training, the CS can elicit either approach or withdrawal from the environment, depending on whether the US is rewarding or aversive (Tzschentke, 2007). Alongside these efforts, Skinner (1938) developed the term ‘operant conditioning’ whereby behavioral manipulations can be achieved through the use of reinforcement, which is given after the desired behavioral response. Skinner organized potential outcomes into three basic categories: neutral, reinforcing, or punishing. Operant training has been expanded to include an intracranial self-stimulation (ICSS) paradigm (Carlezon and Chartoff, 2007). Rodents learn to deliver brief electrical pulses into the medial forebrain bundle (Carlezon and Chartoff, 2007), and more recently into other specific brain regions hypothesized to mediate both natural and ICSS rewards. These basic models have evolved over the last 75 years to include elaborate measures of valence responding during pharmacological, electrical, and genetic modifications. They are now commonly used for dissecting the contributions of individual brain regions and circuits in reward and avoidance.

In the last decade, the rise of optogenetic and chemogenetic approaches has greatly influenced these traditional behavioral paradigms in order to take advantage of the unique spatiotemporal features of these tools that facilitate discrete control over neural circuits. Most commonly, cre-recombinase/loxP technology is utilized to gain cell-type-selective expression and thus precise excitation, inhibition, or modulation of specific circuits in vivo (Atasoy et al, 2008; Tsai et al, 2009). In a seminal paper by Tsai et al (2009), it was demonstrated that closed-loop control of behavioral responding was possible with in vivo optogenetics, such that the animal’s presence in a certain context triggered phasic photostimulation of dopamine (DA) neurons to elicit place preference. Since then numerous adaptations of closed loop, ‘real-time’ in vivo optogenetic engagement of neural circuits have been used with behavioral models in Pavlovian, operant, and acute measures of valence. The increased spatiotemporal control over neural circuits afforded by these modern approaches has rapidly advanced our understanding of the role of the limbic system in reward and aversion (Al-Hasani et al, 2015; Nieh et al, 2013; Tye and Deisseroth, 2012). We have compiled several of these studies utilizing adapted behavioral models into Table 1 to summarize the most recent findings, the behavioral models employed, and the brain regions examined (for a comprehensive review of anxiety, see Calhoon and Tye, 2015). This provides a background reference for the remainder of the review, which will break down these related findings into a conceptual framework of valence in the limbic system. It is anticipated that further advances in optogenetic tools, hardware, in vivo imaging, and mouse genetic models over the next decade will further expand the types of behavioral measures that assess valence in the limbic system.

Table 1 A Summary of Studies Investigating the Causal Relationship Between Neuronal Populations and Valence within the VTA, NAc and BLA

NEURAL REPRESENTATION OF REWARD AND AVERSION

Where is Valence Represented in the Brain?

Certain anatomically localized populations of neurons show differential responses to reward or aversion-associated cues. These regions have been considered to represent valence, and include the BLA (Fuster and Uyeda, 1971; Paton et al, 2006; Shabel and Janak, 2009), nucleus accumbens (NAc; Roitman et al, 2005), ventral tegmental area (VTA; Bromberg-Martin et al, 2010; Cohen et al, 2012; Matsumoto and Hikosaka, 2009), orbitofrontal cortex (Schoenbaum et al, 1999), lateral hypothalamus (LH; Fukuda et al, 1990; Li et al, 2013; Nakamura et al, 1987; Ono et al, 1986; Yamamoto et al, 1989), dorsal raphe nucleus (DRN; Cohen et al, 2015), locus coeruleus (Bouret and Richmond, 2015; Hofmeister and Sterpenich, 2015; McCall et al, 2015), subthalamic nucleus (Sieger et al, 2015), and hippocampus (Fuster and Uyeda, 1971). Within many of these regions, there is marked heterogeneity among neural populations, such that neurons representing positive valence reside side-by-side with those representing negative valence.

Techniques that reveal two populations of neurons in the same brain—one active in response to a rewarding stimulus and the other active in response to an aversive stimulus—will be immensely useful for gaining a comprehensive understanding of brain regions containing topographically intermingled populations of neurons representing valence. These emergent strategies are already proving their utility; a recent study (Xiu et al, 2014) used a technique known as tyramide-amplified-immunohistochemistry–fluorescence in situ hybridization (TAI–FISH) to simultaneously label populations of neurons that respond to a rewarding stimulus such as cocaine, and an aversive stimulus, such as foot shock. TAI-FISH exploits the difference between the time courses of c-fos mRNA and protein expression to simultaneously label two populations of neurons, each active in response to a different stimulus. The authors found several brain regions containing intermingled populations of neurons activated by morphine and foot shock, including the ventral division of the lateral septum, dorsomedial shell of the NAc, and the fusiform nucleus of the bed nucleus of stria terminalis.

Therefore, valence is represented across several brain regions in distributed networks, and, more importantly, all of the brain regions discussed above represent both positive and negative valences. Hence, we can posit that several brain regions contain either largely distinct or overlapping populations of neurons representing positive and negative valences. An important challenge is to identify a unique feature shared by neurons representing only positive or only negative valence. Projection target (downstream target of axons) and genetic markers are two major classes of possible features shared by neurons selectively representing a single valence, and recent findings in valence representation within VTA neurons clearly illustrate this point.

Distinct Neural Populations Representing Reward and Aversion: the VTA as an Example

Recent evidence has identified separable populations of VTA neurons that differentially encode positive and negative valence. These populations can be segregated by genetic markers; for instance, phasic activation of VTA DA neurons causes place preference (Tsai et al, 2009), whereas activation of VTA GABA neurons causes place aversion (Kim et al, 2013; Tan et al, 2012). VTA DA and GABA neurons also show differences in their response properties during fear and reward conditioning (Cohen et al, 2012).

Projection target can also be used to disentangle VTA neurons encoding positive and negative valences. VTA DA neurons are a heterogeneous population in the context of valence, and dissecting VTA DA neurons by their projection target has elucidated their differential role in reward and aversion. Synaptic transmission onto largely nonoverlapping VTA DA neurons projecting to the medial shell of NAc and the medial prefrontal cortex (mPFC) can be modulated by a rewarding and an aversive experience, respectively (Lammel et al, 2011). VTA DA neurons projecting to NAc lateral shell are enriched in the lateral VTA and receive a relatively larger input from the laterodorsal tegmentum (LDT), and VTA DA neurons projecting to mPFC are enriched in the medial VTA and receive stronger input from the lateral habenula (LHb; Lammel et al, 2012). Activating LDT inputs to VTA is sufficient to evoke place preference, whereas activating LHb inputs to VTA is sufficient to evoke place aversion (Lammel et al, 2012; Stamatakis and Stuber, 2012). Therefore, the medial–lateral axis of the VTA has parallel pathways running alongside each other that process reward and aversion (Lammel et al, 2014a). The VTA provides but one example highlighting the importance of studying genetically defined (DA vs GABA VTA neurons) and projection-target-defined (mPFC vs NAc projecting VTA DA neurons) populations. For more in-depth reviews on these circuits, see (Lammel et al, 2014a, b).

In the following two sections, we will highlight valence representation in projection-target-, genetically and functionally defined populations of neurons (Figure 1), as well as the relationships between these populations of neurons in the BLA and the NAc. Although initial studies on the BLA were focused on its role in processing both appetitive and aversive stimuli (Fuster and Uyeda, 1971; Machne and Segundo, 1956), more than three decades of BLA research has focused primarily on the role of BLA in fear conditioning (Davis, 1992; LeDoux, 2000; LeDoux et al, 1990; Maren, 2001). Conversely, the NAc has been studied primarily in the context of reward learning (Carlezon and Thomas, 2009; Kelley, 2004; Robbins and Everitt, 1996; Salamone et al, 2005; Wise, 2004). Recent research has started to focus on the role of both the BLA and the NAc in representing valence.

Figure 1
figure 1

Three common ways to define subpopulations of neurons, based on functional (activity-tagged), genetic, and anatomical (eg, projection target) criteria. (a) Recent attempts to establish the identity of positive (represented in the figure by a +) and negative (represented in the figure by a −) valence-encoding populations of topographically overlapping neurons can be broadly classified into forward and reverse approaches. In a forward approach, properties of functionally defined populations of neurons are examined, whereas in the reverse approach the functional role of either anatomically- or genetically-defined populations of neurons is examined. An example of a forward approach in the basolateral amygdala (BLA) is activity-dependent labeling of BLA neurons activated by a positive and/or a negative stimulus (Gore et al, 2015; Nonaka et al, 2014; Redondo et al, 2014; Han et al, 2009), followed by the study of the properties of each subpopulation to understand their necessity, sufficiency, experience-dependent plasticity, and molecular identity. An example of a reverse approach in the BLA is examining the functional role of projection-target-defined BLA neurons (Namburi et al, 2015; Senn et al, 2014). With the explosion of tools recently made available for circuit dissection, including the ability to tag and manipulate neural populations with a common genetic marker, or a common projection target, we can now move at an accelerated pace toward understanding the relationship between functionally defined, genetically-defined and projection-target-defined neural populations. (b) The BLA is an example where stimulation of distinct projection-target-defined populations of neurons, those projecting to the NAc and those projecting to the CeM evoke positive or negative behaviors (Namburi et al, 2015). (c) The VTA is an example where genetically defined and projection-target (anatomical)-defined populations of neurons are known to play differential roles in positive and negative behaviors (Cohen et al, 2012; Lammel et al, 2011). CeM, medial division of the central amygdala; mPFC, medial prefrontal cortex; NAc, nucleus accumbens; VTA, ventral tegmental area.

PowerPoint slide

REPRESENTATION OF REWARD AND AVERSION IN THE BLA

Neural populations in the BLA have been dissected functionally, anatomically, and genetically in the context of fear and reward. In this section, we discuss these neural populations within the BLA, and the relationships between them.

Diversity of BLA Neural Populations

The BLA complex, consisting of the lateral (LA) and the basal (BA) subdivisions, contains topographically intermingled populations of neurons related to a variety of behaviors (Zhang et al, 2013) including fear conditioning, reward conditioning, anxiety, feeding, and social interaction (Janak and Tye, 2015). Here we will focus on the functional populations of BLA neurons related to fear and reward behaviors. Advances in activity-dependent regulation of transgene expression is fueling dissection of such functionally defined BLA neural populations (Denny et al, 2014; Gore et al, 2015; Liu et al, 2012; Redondo et al, 2014; Reijmers et al, 2007). Optogenetic advances have allowed us to dissect BLA neurons based on criteria other than a functionally defined population, including projection target (Felix-Ortiz and Tye 2014; Felix-Ortiz et al, 2013, 2015; Namburi et al, 2015; Senn et al, 2014; Stuber et al, 2011; Tye et al, 2011) and genetic markers (Wolff et al, 2014). The field is currently employing a substantial amount of effort toward understanding the relationship between populations of BLA neurons defined by the following three criteria: functional, anatomical (projection target), and genetic identity.

The idea that multimodal information converges onto single neurons in the BLA was established at the dawn of single-unit recordings in the amygdala (Machne and Segundo, 1956), and increased firing of BLA neurons to behaviorally relevant stimuli was observed shortly thereafter (Sawa and Dalgado, 1963). Almost a decade later, 37% of neurons in the amygdala were reported to be selectively responsive to motivationally significant stimuli—a higher proportion compared with neurons in the hippocampus and piriform cortex (Fuster and Uyeda, 1971). Some BLA neurons modulated their firing rate when the affective value of the outcome was reversed, establishing that BLA neurons were tracking the valence of the stimulus (Nishijo et al, 1988), and this reversal, or valence-tracking property, was present in about half of the BLA neurons selectively responsive to affective stimuli (Schoenbaum et al, 1999). Furthermore, valence-tracking BLA neurons reversed their response properties before behavioral reversal in the animal (Belova et al, 2007; Schoenbaum et al, 1999). Taken together, we should expect about a fifth of all randomly sampled BLA neurons to track valence. Consistent with single-unit recording data (Zhang et al, 2013), the complementary approach of activity-dependent labeling of BLA neurons using nicotine or a conspecific of the opposite sex as a positive US and foot shock as a negative US in the same animal reveal two largely nonoverlapping, but topographically intermingled, populations in the BLA (Gore et al, 2015; Redondo et al, 2014).

Populations of BLA neurons can also be classified into nonoverlapping sets based on the primary neurotransmitter they carry—glutamate or GABA (Sah et al, 2003). There are multiple partially overlapping subpopulations among the GABAergic population that can be distinguished based on their immunoreactivity to various proteins, such as parvalbumin (PV), somatostatin (SOM), and calbindin and calretnin (Capogna, 2014; Kemppainen and Pitkänen, 2000; McDonald and Mascagni, 2002).

Although the functional role of PV and SOM interneurons in the BLA during fear conditioning has been identified recently (Wolff et al, 2014), the functional role of BLA interneurons in reward learning has not yet been explored. PV cells primarily contact the soma of principal neurons (Muller et al, 2006) and inhibit SOM neurons. SOM neurons primarily contact the distal dendrites of principal neurons (Muller et al, 2007). PV neurons are active during an auditory CS, and inhibit SOM neurons, thereby disinhibiting distal principal neuron dendrites. Both PV and SOM neurons are inhibited by the US, thereby disinhibiting the principal neuron (Wolff et al, 2014). The authors also find a population of PV neurons that is inhibited by the CS. Perhaps distinct subpopulations of PV interneurons might disinhibit BLA principal neurons in the BLA differentially responsive to fear or reward cues (Janak and Tye, 2015).

BLA projection neurons have diverse targets in the brain, notably including the NAc (McDonald, 1991a), the lateral division of the central nucleus of the amygdala, medial division of the central nucleus of the amygdala (CeM; Pitkänen et al, 1997), ventral hippocampus (Pikkarainen et al, 1999), and the pre-limbic (PL) and infralimbic (IL) subdivisions of the mPFC (McDonald, 1991b). Although the exact extent of overlap between various projection-target-defined subpopulations of neurons remains to be elucidated, retrograde tracing using two tracers suggests that some of the projection-target-defined populations of BLA neurons are largely nonoverlapping, such as vHPC vs mPFC-projecting BLA neurons (Senn et al, 2014), whereas other projection-target-defined BLA neurons are largely overlapping, especially striatal and prefrontal cortex-projecting BLA neurons (McDonald, 1991b; Shinonaga et al, 1994).

The broad relationship between projection-target- and neurotransmitter-defined populations of BLA neurons is relatively straight forward. About 70–90% of the projection neurons in the BLA are glutamatergic (McDonald and Augustine, 1993; Millhouse and DeOlmos, 1983; Washburn and Moises, 1992), with the exception of some SOM+ GABAergic neurons that project to the basal forebrain (McDonald et al, 2012), entorhinal cortex (McDonald and Zaric, 2015), and preoptic–hypothalamic region (McDonald, 1987). However, by exploiting modern transcriptomic techniques such as RNA-seq, we are able to appreciate more subtle gene expression profile differences between projection-target-defined subpopulations of glutamatergic BLA projection neurons. The first of such attempts in the BLA shows several differentially expressed genes between NAc- and CeM-projecting BLA principal neurons, including some membrane-bound receptors (Namburi et al, 2015).

Recent studies are starting to shed light on the relationship between functionally defined and projection-target-defined BLA neurons. Activation of either nicotine-US (positive)-labeled subpopulation of BLA neurons (Gore et al, 2015) or NAc-projecting BLA neurons (Namburi et al, 2015) is sufficient to induce ICSS. Moreover, activation of either foot shock-US-labeled subpopulation of BLA neurons (Redondo et al, 2014) or CeM-projecting BLA neurons (Namburi et al, 2015) is sufficient to support place avoidance. Although these data suggest a general relationship between functionally defined and projection-target-defined populations of BLA neurons, the extent of overlap between functionally-defined and projection-target-defined populations of BLA neurons remains to be quantified.

It is also important to consider the interplay among different subpopulations of BLA neurons. Different populations of GABAergic interneurons target different cell compartments of glutamatergic BLA principal neurons (Capogna, 2014). Since projection-target-defined BLA neurons are being shown to have opposing functional roles (Namburi et al, 2015; Senn et al, 2014), there may be interpopulation inhibition within the BLA (Janak and Tye, 2015). The first evidence for functional opposition among projection-target-defined BLA populations came from the anxiolytic properties of BLA projections to the lateral subdivision of the CeA (Tye et al, 2011) and the anxiogenic properties of BLA projections to the ventral hippocampus (Felix-Ortiz et al, 2013). In the context of conditioned associations, there is also evidence for projection-target-defined functional opposition as seen by an inversely correlated pattern of FOS expression between PL- and IL-projecting BLA neurons (Senn et al, 2014). Further work is required to establish an understanding of the interplay between BLA neurons that process positive and negative valences.

Learning-Induced Plasticity in the BLA

Acquisition of an association between a CS and either an aversive or appetitive outcome leads to an increase in AMPAR/NDMAR ratio (a proxy for synaptic strength) of internal capsule inputs to BLA neurons (Clem and Huganir, 2010; Rumpel et al, 2005; Tye et al, 2008).

Inputs onto BLA neurons undergo plastic changes upon fear and reward learning (McKernan and Shinnick-Gallagher, 1997; Rogan et al, 1997; Tye et al, 2008). This long-term plasticity is mediated via AMPA receptor trafficking to the postsynaptic membrane (Clem and Huganir, 2010; Rumpel et al, 2005; Tye et al, 2008). Emerging evidence suggests that these plastic changes can vary by cell type and projection-target-defined populations of BLA neurons.

As BLA neural populations mediate a diverse set of behaviors, some of which are opposing, experience-dependent plasticity in the BLA is perhaps a function of the neural population sampled. A recent study examined experience-dependent changes in AMPAR/NMDAR ratios in NAc- and CeM-projecting BLA neurons, wherein photoactivation drives positive reinforcement or punishment, respectively (Namburi et al, 2015). AMPAR/NMDAR ratios in the internal capsule inputs to NAc- and CeM-projecting BLA neurons underwent opposing changes after fear and reward conditioning—AMPAR/NMDAR ratios onto NAc-projecting BLA neurons decreased after fear conditioning and increased after reward conditioning. Conversely, AMPAR/NMDAR ratios onto CeM-projecting BLA neurons increased after fear conditioning and decreased after reward conditioning.

Another study (Nonaka et al, 2014) expressed a fluorophore dVenus under the control of an Arc promoter (Eguchi and Yamaguchi, 2009) during a fear-conditioning paradigm, thus labeling the functional population of neurons involved in fear conditioning. The authors contrasted input synaptic transmission between dVenus-expressing and -non-expressing LA neurons after a cued fear-conditioning paradigm (Nonaka et al, 2014). The authors demonstrated an increase in synaptic transmission selectively onto dVenus-positive cells (ie, labeled by fear conditioning). This increase was at least in part due to an increase in probability of release from the presynaptic terminals of the cortical inputs. It is interesting to note that the authors did not see a difference in synaptic transmission from internal capsule inputs onto LA neurons with (dVenus-expressing) and without (dVenus-negative) activity-dependent labeling. Taken together with studies showing increased post-synaptic transmission from internal capsule inputs onto nonspecific LA neurons after fear conditioning (Clem and Huganir, 2010; Rumpel et al, 2005), perhaps changes in internal capsule synaptic transmission represent a more generalized sum of aversive experiences, and changes in cortical pathway synaptic transmission signal changes specific to one aversive experience. A reward-conditioning analog of this study has yet to be conducted.

Flexibility of Valence Representation in the BLA

Does a BLA neuron have the flexibility to encode either positive or negative valence, or is the valence encoded by a BLA neuron indelible? Might there be distinct populations that are flexible or indelible? If so, might the indelible population(s) be anatomically hard-wired?

The existence of valence-tracking BLA neurons (satisfying the reversal criterion) in the BLA (Paton et al, 2006; Schoenbaum et al, 1999) poses a strong argument in favor of indelible valence encoding on the timescale of multiple hours within the BLA. It is prudent to remember that only a fraction (about a fifth) of BLA neurons exhibit this property. Further work, such as monitoring valence-tracking neurons over multiple days would be required to determine whether these neurons represent the same valence over longer timescales. Using optogenetic advances to manipulate functionally defined or projection-target-defined populations of BLA neurons, recent studies have advanced our understanding regarding the flexibility of valence representation in the BLA.

Activity-dependent tagging in the BLA using foot shock and nicotine label two largely nonoverlapping BLA neural populations, thus supporting the idea of valence-coding neurons in the BLA (Gore et al, 2015). A more direct test of a BLA neuron’s flexibility in encoding valence was performed by attempting to reverse the valence encoded by a functionally defined population of BLA neurons, tagged under the control of a c-fos promoter (Redondo et al, 2014). The authors showed that activating foot shock-labeled BLA neurons after stimulating these neurons during a rewarding stimulus (female mouse) did not elicit place preference. Conversely, activating reward-labeled BLA neurons after associating these neurons with foot shocks did not elicit place avoidance. Whereas the authors could not reverse the valence-coding properties of activity-labeled BLA neurons, they were able to reverse the association of activity-labeled cells in the dentate gyrus.

A complementary approach to determine the flexibility of valence representation in the BLA is offered by efforts to determine the criteria for recruiting a cell into a memory trace. Neurons with elevated CREB expression are recruited into a fear memory trace (Han et al, 2009). Elevating CREB expression (Zhou et al, 2009) or neural excitability (Yiu et al, 2014) increases the probability of recruiting that neuron into the memory trace. It remains to be seen whether elevating CREB/neural excitability increases the probability of recruiting the neuron into a memory trace of any valence or one specific valence. The former outcome would suggest that the memory trace is flexible, whereas the latter would suggest that it is indelible. However, the outcome may depend on the subpopulation of BLA neurons under investigation, which would imply BLA neurons to contain both flexible and indelible subpopulations.

Finally, activating projection-target-defined populations of BLA neurons was sufficient to evoke either positive or negative behaviors (Namburi et al, 2015). Stimulating NAc-projecting BLA neurons is sufficient to support ICSS, whereas stimulating CeM-projecting BLA neurons is sufficient to cause place aversion, suggesting that BLA neurons with innate valence representations are anatomically hard-wired. Therefore, some BLA neurons have indelible valence representations, and some valence representing BLA neurons are anatomically hard-wired.

REPRESENTATION OF REWARD AND AVERSION IN THE NAc

As is the case in the BLA, specific ensembles of neurons within the NAc have been identified based on their functional, anatomical, and genetic characteristics, and these populations of cells have been found to differentially impact appetitive and aversive behaviors. Here we discuss these types of populations in the NAc, including cells functionally defined as responsive to rewarding or aversive events, genetically defined groups of cells such as D1 receptor (D1R) and D2 receptor (D2R) populations, and groups of cells anatomically divided into NAc core vs shell, dorsal vs ventral, and rostral vs caudal placements. We go on to discuss how these populations may interact, and each of their contributions to valence-coding in the NAc.

Diversity of NAc Neural Populations

The NAc has been widely identified as a key mediator of reward behaviors. However, as activity within this structure has been linked to processing both rewarding and aversive events (Reynolds and Berridge, 2002; Roitman et al, 2005; Salamone et al, 2005), it is also appropriately described as a valence-encoding system. Located in the ventromedial aspect of the striatum, the NAc can be roughly divided into core and shell subregions (although these two subregions have been further divided into anatomically defined sections; see below). Both the core and shell regions receive and integrate inputs from numerous afferent structures, including the mPFC, hippocampus, BLA, thalamus, and midbrain DA neurons in the VTA. They subsequently project downstream to basal ganglia nuclei, as well as interact directly with each other; however, there are known afferents from the core to the shell but very few from the shell to the core (van Dongen et al, 2005, 2008; Saddoris et al, 2013; Wenzel et al, 2015).

Single-unit recordings in awake, behaving animals have revealed that rewarding and aversive events are encoded by largely distinct populations of NAc medial shell neurons. Moreover, primary rewards such as sucrose preferentially result in inhibition among reward-responsive NAc neurons, whereas aversive stimuli such as quinine primarily drive excitatory responses (Roitman et al, 2005). Among the relatively small populations of cells responsive to both quinine and sucrose, responses to the two stimuli tend to be opposing (Roitman et al, 2005). Compellingly, direct electrical stimulation of NAc neurons observed to be inhibited during sucrose consumption results in the interruption of licking behavior (Krause et al, 2010). This finding indicates the necessity of inhibition in the NAc for the execution of appetitive behaviors.

These functionally defined populations of reward- and aversion-selective neurons have been identified in both the core and the shell; however, these regions appear to separately modulate reward learning and valence encoding, respectively. For example, although presentation of rewarding stimuli produces transient increases in extracellular DA in the core and shell (Roitman et al, 2004, 2008), the roles of DA release in these two regions in motivated behavior are distinct. DA release in the core of the NAc is important for acquisition of reinforced behavior, insomuch as it is necessary for animals to learn instrumental behaviors such as reward-seeking following presentation of a cue (Abercrombie et al, 1989; Bassareo et al, 2002; Young, 2004). By contrast, DA release into the NAc shell is necessary for hedonic and aversive responses to natural rewards and punishments (Aragona et al, 2008; Bassareo et al, 2002; Di Chiara and Bassareo, 2007; Goto et al, 2007; Stuber et al, 2005).

Consistent with the distinct impacts of DA in the core and shell, a large body of evidence indicates that the hedonic value of stimuli is preferentially encoded within the medial shell (Berridge and Kringelbach, 2015). Distinct populations within this region have been functionally defined based upon their sensitivity to opioids, which have been shown to evoke positive hedonic responses independent of DA signaling (Bardo, 1998; Berridge and Robinson, 1998; Cannon and Palmiter, 2003; Hyman et al, 2006; Pettit et al, 1984; Robinson et al, 2005). A specialized opioid ‘hotspot’ in the rostrodorsal quadrant of the medial NAc shell is composed of a population of neurons mediating hedonic responses following mu opioid receptor (MOR) activation. Conversely, another population of neurons in the caudal half of the shell—a so-called ‘coldspot’—reduces hedonic responses to sucrose when stimulated with MOR, delta opioid receptor (DOR), or kappa opioid receptor (KOR) agonists (Castro and Berridge, 2014), suggesting an opposing role for these populations in valence encoding.

The discrete anatomical placement of these populations has led to further mapping of an affective gradient within the NAc medial shell. Microinjections of the GABAA agonist, muscimol, at different rostro/caudal sites along the medial shell of the NAc result in distinct responses depending upon their placement; rostral injections evoke increased feeding behavior along with enhanced hedonic orofacial ‘liking’ responses to sucrose, whereas caudal injections instead promote fearful and defensive behaviors, and elicit aversive ‘disliking’ reactions to sucrose and quinine (Faure et al, 2010; Reynolds and Berridge, 2002). AMPA receptor antagonism within equivalent sites has an impact on motivated behaviors, increasing appetitive behaviors in rostral sites and increasing fear-related behaviors in caudal sites, without correspondingly affecting hedonic responses (Faure et al, 2010). By contrast, antagonism of metabotropic glutamate signaling in the medial shell of NAc shifts both motivated behaviors and affect from positive to negative valence homogenously throughout the entire shell (Richard and Berridge, 2011). These data underscore the interplay between anatomically defined populations of cells and the array of neurochemical signals in the NAc in the calculation of the valence of environmental stimuli.

Genetically defined populations of NAc neurons have also been shown to differentially contribute to motivated behaviors. The NAc primarily comprises GABAergic medium spiny neurons (MSNs), which make up >95% of the region’s neurons (Gerfen, 1992; Kita and Kitai, 1988); the remaining proportion is largely represented by GABAergic and cholinergic interneurons (Tepper and Bolam, 2004). Among MSNs, there are two partially overlapping subpopulations defined by their projection targets and DA receptor expression. Direct pathway MSNs, which express D1Rs (Gs-coupled), project to the midbrain, whereas indirect pathway MSNs express D2Rs (Gi-coupled) and project to the ventral pallidum (Spanagel and Weiss, 1999; Swanson, 1982). These populations of MSNs are not entirely segregated, however; recent evidence suggests that NAc projections to the ventral pallidum do not conform to the traditionally accepted model of D1-direct and D2-indirect neuronal circuitry (Kupchik et al, 2015).

Although activation of both D1Rs and D2Rs is involved in motivated behavior, the specific roles of the two populations of MSNs are not yet well understood. However, modern tools in neuroscience such as optogenetics are allowing for dissection of the discrete roles of the direct and indirect pathways in valence processing. For example, targeted activation of D2R MSNs was demonstrated to attenuate the behavioral response to cocaine, whereas activation of D1R MSNs enhanced cocaine’s reinforcing effects (Lobo et al, 2010). Similarly, direct optogenetic activation of D1R-expressing MSNs has been shown to induce persistent reinforcement, whereas activation of D2R-expressing MSNs is transiently punishing (Kravitz et al, 2012). Distinct roles for D1R- and D2R-expressing MSNs in appetitive and aversive behaviors have also been evaluated using pathway-specific blockade of NAc transmission; pharmacological activation of D1Rs in the direct pathway has demonstrated the necessity of this pathway for reward-based learning, whereas inactivation of D2Rs in the indirect pathway likewise demonstrated their necessity for aversive learning (Hikida et al, 2013). Taken together, these results illustrate the divergent roles of genetically and anatomically defined NAc MSNs, and emphasize the importance of investigating these populations individually (Kupchik et al, 2015).

Recent work is beginning to examine the interplay between anatomically-, genetically-, and functionally defined populations of NAc neurons in valence encoding. For example, a recent report demonstrated that photostimulation within discrete subregions of the NAc shell of cells expressing both dynorphin (the endogenous peptide ligand for KOR) and D1Rs drives opposing motivational behavioral states (Al-Hasani et al, 2015). In this study, it was found that photostimulation of these D1R/dynorphin-positive cells in the ventral shell drives aversion, whereas photostimulation in the dorsal shell drives preference (Al-Hasani et al, 2015). D1R expression patterns with dynorphin-containing cells did not differ in either region that produced the opposing behavior, suggesting that these cells are genetically similar; however, additional work to further define their genetic markers now that these subregions have been identified is warranted. Together, these findings suggest that anatomical specificity gates the valence of endogenous opioid signaling in genetically defined populations of NAc MSNs. This effect is likely modulated by functionally distinct inputs to the dorsal and ventral shell, as has been demonstrated to be the case in VTA neurons encoding opposite valence (Lammel et al, 2011, 2012), or potentially through divergent outputs from these two regions back to the VTA or other basal ganglia loci.

Together, this body of evidence suggests that valence encoding is dependent on multiple parallel circuits within discrete subregions of the NAc. Further work is required to clarify how different populations of valence-sensitive neurons within the NAc may interact, or, in some cases, overlap. For instance, the relationship between D1R- and D2R-expressing MSNs and the reward- and aversion-selective cells identified in the NAc via single-unit recording remains to be conclusively determined, particularly in light of recent evidence that these genetically identified populations do not perfectly map onto the anatomically distinct direct and indirect pathways (Kupchik et al, 2015). Determining how the diverse populations of NAc neurons interact to shape valence encoding will be instrumental in clarifying reward and aversion learning in the brain.

Experience-Dependent Plasticity in the NAc

As in the BLA, experience of aversive or appetitive events leads to lasting changes in the NAc. Because of the central role of NAc in driving motivated behavior, plastic changes in NAc structure and activity following exposure to drugs of abuse have received a great deal of attention, and are reviewed in depth elsewhere (Britt and Bonci, 2013; Gipson et al, 2014; Grueter et al, 2012; van Huijstee and Mansvelder, 2014; Lüscher and Malenka, 2011; Morales and Pickel, 2012). Much of this work has pointed toward the hypothesis that addiction results from plasticity within the same NAc circuits that drive motivated behaviors for natural rewards. Fewer studies have investigated experience-dependent plasticity in the NAc following exposure to natural rewards or punishments; however, these studies indicate that this region undergoes lasting changes in response to valenced stimuli.

For example, using extracellular recordings in awake, behaving rats, Roitman et al (2005) identified a population of NAc neurons that develop responses to cues predicting sucrose or quinine across training. The activity of these neurons tracks with behavioral evidence of learning, suggesting that their emergent activity is central to the association of the CS to a positive or negative outcome. In order for responses to emerge to the CS during training, either the threshold of these NAc neurons for responding to the CS must decrease, or upstream inputs to the NAc must themselves be amplified. Changes in AMPA/NMDA ratios and morphological evidence of plasticity support the former possibility, and indicate that the NAc undergoes experience-dependent plasticity after positive and negative experiences.

Exposure to both appetitive and aversive experiences shapes structure and activity in the NAc. Following the natural reward of sexual experience in male rats, a long-lasting reduction in AMPA/NMDA ratio is apparent within a day and persists for at least a month. This reduction in AMPAR/NMDAR results in part from an increase in surface and intracellular NMDARs (Pitchers et al, 2012). Moreover, sexual experience in males results in increased numbers of dendrites and spines in MSNs in both NAc core and shell (Pitchers et al, 2010). Similarly, housing in enriched environments leads to increased dendritic arborization and spine density in NAc MSNs (Kolb et al, 2003). Whereas chronic sucrose consumption leads to an increase in vesicular glutamate transporters in the NAc, indicating an increase in glutamatergic input to the structure, chronic pain resulting from spared nerve injury decreases levels of these transporters in the NAc (Tukey et al, 2013). Social defeat, a powerfully aversive stimulus, evokes lasting changes in the NAc—namely that even 4 weeks following the stress, BDNF levels are elevated in the NAc (Berton et al, 2006).

The specific populations of NAc neurons that undergo these plastic changes following emotionally charged experiences remains undefined and requires further study.

Flexibility of Valence Representation in the NAc

Although certain populations of NAc neurons appear to indelibly encode positive or negative valence (for example, the rostrodorsal tip of the NAc shell reliably encodes positive hedonic value), other populations in the NAc more flexibly encode valence. This flexibility is in large part based upon the motivational state of the animal. For example, the majority (77%) of sucrose-responsive NAc neurons exhibit a decrease in firing rate when sucrose is consumed. However, following conditioned taste aversion to sucrose using lithium chloride, rats evidence behavioral aversion to sucrose, which is accompanied by a remarkable shift in activity among the sucrose-responsive NAc neurons. In cases in which sucrose had been rendered aversive, the majority (69%) of sucrose-responsive neurons increase their firing rate during sucrose consumption (Roitman et al, 2010). This pattern of responding suggests that activity in NAc neurons tracks the hedonic value of a stimulus per se, although it is still unclear in this case whether a single population of neurons responded to sucrose when it was rewarding vs when it was aversive.

Complementary to these findings, NAc responses to environmental stimuli shift with shifting motivational states of the animal. Whereas a concentrated salt solution is aversive in most cases, a salt-depleted animal is highly motivated to seek out that solution, which is appetitive, given the homeostatic needs of the animal. In the NAc shell, salt-responsive neurons increase activity when salt is non-preferred (sodium replete); however, these cells decrease activity in response to sodium when it is preferred (sodium-deplete animals). In the NAc core, neurons were only responsive to sodium after the sodium balance was restored in the sodium-depleted animals (Loriaux et al, 2011). Together, these data suggest that the shell flexibly encodes the stimulus value based upon internal drives and motivational state.

Further evidence for the flexibility of valence encoding in the NAc is offered by the Berridge group, who have carefully mapped the ‘affective keyboard’ in the NAc medial shell. Whereas under normal conditions the medial shell is roughly divided into rostral and caudal portions, which signal positive and negative hedonic values, respectively, the layout of this map is sensitive to a variety of factors, including the stress level of the rodent. When animals are in familiar environments such as the home cage, the majority of the medial shell is retuned to encode positive hedonic value. By contrast, stressful environments rife with bright lights and loud music cause a rapid reorganization of the affective keyboard, such that a greater proportion of the NAc shell encodes negative valence and only the rostral-most edge of the shell persists in driving appetitive behaviors (Reynolds and Berridge, 2008; Richard and Berridge, 2011; Richard et al, 2013). Taken together, it appears as if valence encoding in the NAc is more flexible than valence encoding in the BLA.

ADVANCES IN TARGETING SPECIFIC SUBPOPULATIONS OF NEURONS

Recent advances in using viral vectors to target and express genes in specific neural populations have facilitated in asserting their role, necessity, and sufficiency in valence-learning. Here we summarize some of the modern tools available for targeting projection-target-defined and genetically defined neural populations. We will also summarize some of the tools available to selectively express genes in populations of neurons active during a specific time window, which we hope will evolve to target even more specific functional populations of neurons.

Retrograde viruses have been immensely useful for projection specific targeting—including the herpes simplex virus (HSV; Lima et al, 2009), canine adenovirus (CAV; Kremer et al, 2000), and rabies virus (RV; Wickersham et al, 2007). A dual virus recombination approach can be used to drive gene expression selectively in a projection-target-defined population of neurons. In this approach, a cre-dependent construct introduced nonspecifically into a brain region is unlocked in specific cells with a retrograde virus carrying a construct to express Cre-recombinase (Hnasko et al, 2006; Lima et al, 2009; Namburi et al, 2015; Nieh et al, 2015; Senn et al, 2014). Genetically defined populations can be targeted either using mouse lines expressing cre/flp recombinase in specific cell populations (Gong et al, 2007; Taniguchi et al, 2011) or viral delivery of constructs expressing genes under the control of a promoter that is active only in certain populations of neurons. There are also tools available to drive gene expression in populations complementary to those expressing Cre (Cre-out; Cai et al, 2014; McDevitt et al, 2014), or, more generally, in populations specified by multiple cell-type features, such as DA neurons in the VTA that do not project to mPFC (Fenno et al, 2014). Targeting populations of neurons that project to a specific subpopulation in a downstream region can be achieved using monosynaptic tracing technology employing replication-incompetent RV (Callaway and Luo, 2015; Ogawa et al, 2014; Pollak Dorocic et al, 2014; Watabe-Uchida et al, 2012; Wickersham et al, 2010).

In addition to viral approaches, the last few years have seen the development and use of μ-ILED devices implanted in the brain to target discrete subpopulations of cells (Jeong et al, 2015; Kim et al, 2013; McCall et al, 2013). These devices are tailor-made to be implanted and target a specific subset of cells with photostimulation. Al-Hasani et al (2015) show that two subpopulations of D1-dynorphin cells can be controlled independently using μ-ILED devices to drive opposing motivational behaviors.

Current activity-dependent tagging techniques involve gene expression under the control of an immediate early gene promoter, such as cFOS and/or Arc (Denny et al, 2014; Eguchi and Yamaguchi, 2009; Garner et al, 2012; Guenthner et al, 2013; Liu et al, 2012; Reijmers and Mayford, 2009; Reijmers et al, 2007). These are limited to labeling neurons whose activity is above a certain threshold within a time window that is in the order of hours. The time window for tagging is dictated either by a pharmacological agent (Guenthner et al, 2013), life time of protein degradation (Eguchi and Yamaguchi, 2009), or more recently, by light (Fosque et al, 2015). CaMPARI detects the coincidence between calcium levels in a cell (neural activity) and the presence of light (time window; Fosque et al, 2015). Current techniques are not able to label neurons inhibited by a stimulus. From the populations of neurons designated a–i in Figure 2, they are only able to selectively label (c+f+i) or (a+b+c). The ability to tag each individual population in a–i would be a considerable addition to the arsenal of tools available for circuit-based drug discovery. Their role can be appreciated in a two-step approach. First, the ability to control a precisely defined functional population of neurons (eg, reward-selective neurons, d+f) will help us determine their necessity and sufficiency during a particular behavior. Second, if the population is either determined to be necessary for a desirable behavior, or sufficient to cause an undesirable behavior, genetic dissection of this population has the ability to reveal potential drug targets as a means to the end of selectively turning a population of neurons on or off.

Figure 2
figure 2

A model for quantifying valence representation. The above figure illustrates the process for computing metrics from single-unit recording studies to examine valence representation. The right column illustrates the process of applying the model to a population of neurons. In this case, we apply the model to serotonin neurons in the dorsal raphe nucleus (DRN), based on the results from Cohen et al (2015). A neuron can be qualitatively classified into non-responsive, excited, or inhibited to a conditioned stimulus (CS). In a typical valence-conditioning paradigm, there is a positive CS, predictive of an appetitive outcome, and a negative CS, predictive of an aversive outcome. Given these definitions, there are nine possible disjoint base classes for each neuron, as illustrated above (a–i). To compare the valence representation between brain regions, we define the following classes of neurons, based on the nine base classes. Task response index is the proportion of task-responsive neurons. Valence index is the proportion of differentially responsive neurons (neurons that either have opposite responses to each valence or a selective response to one valence), among the task-responsive neurons. Valence selectivity is the number of neurons selectively responsive to reward (d,f), relative to the number of neurons selectively responsive to either valence. Valence bias is the ratio of number of neurons excited by a positive CS and/or inhibited by a negative CS to the number of neurons having distinct responses to each valence. Each computed parameter in the model is color-coded for ease of visualization, and the meaning of each color is presented in the left column.

PowerPoint slide

In addition, the advent of optical tools that confer spatiotemporal specificity of signaling will provide an additional layer of resolution. Recent efforts to utilize opto-XR receptors (modified G-protein-coupled receptors, GPCRs) to mimic endogenous neurotransmission through peptide and monoamine receptors (Airan et al, 2009; Gunaydin et al, 2014; Siuda et al, 2015) will further extend the possible selective targets for intervention in therapeutic realms. These receptors couple endogenous receptor-signaling domains to class A rhodopsin GPCRs, and upon photostimulation allow for rapid time-locked engagement of excitatory or inhibitory signaling in vivo in closed-loop behavioral models (Table 1). Further advances in other optically sensitive protein–protein interactions using CIB1/CRY domains (Konermann et al, 2013; Schindler et al, 2015; Taslimi et al, 2014; Tucker et al, 2014) for in vivo manipulations are now possible, and provide the ability to directly target native pathways with unprecedented precision and will prove useful in studies of the mechanisms of plasticity within defined neuronal populations.

These tools will accelerate the discovery of valence-signaling populations distributed throughout the brain, and will thus demand a comprehensive strategy for characterizing these newly identified populations. Comparing the extent of valence representation in these populations and contrasting valence representation between populations will be of particular value in shaping the direction of research. To this end, we propose a model containing metrics easily quantifiable from a single-unit recording study that will facilitate comparison and contrast between the multitudes of candidate populations signaling valence.

MODEL FOR VALENCE REPRESENTATION

In this section, we will develop a model to parametrize the neural responses from a population of neurons in response to cues predicting positive and negative outcomes. This model summarizes neural responses into four parameters, which can then be used to contrast valence representation between populations of neurons either within one brain region, or across brain regions. Applying this model to neural activity recorded from various brain regions in the context of valence illustrates the heterogeneity among anatomically localized populations of neurons (Figure 3a). Applying this model to either genetically or projection-target-defined subpopulations of neurons within a brain region (Figure 4b) illustrates that examining specific subpopulations can help reduce heterogeneity in their valence-signaling properties. Therefore, this model can be used to describe valence coding neural populations.

Figure 3
figure 3

Applying the model to data from nonspecific populations within several brain regions illustrates the heterogeneity of valence representation in limbic structures. (a) Metrics from the model for six brain regions, based on data from Fuster and Uyeda, 1971; Ono et al, 1986; Roitman et al, 2005. These metrics suggest responses in all these brain regions to be quite similar, illustrating the idea of distributed representation of valence. All these brain regions contain populations of neurons responsive to both fear and reward cues (valence selectivity is neither close to 0 nor 1), and they signal both positive and negative valences (valence bias is neither close to 0 nor 1). From these data, lateral hypothalamus (LH) has the highest proportion of neurons showing distinct responses to fearful and rewarding stimuli. (b) Predicted values of valence selectivity and valence bias for specific subpopulations within the basolateral amygdala (BLA) and the ventral tegmental area (VTA), based on the results of Lammel et al, 2011; Namburi et al, 2015. For the meaning of each color in this figure, please refer to the valence representation model in Figure 2. AHA, anterior hypothalamic area.

PowerPoint slide

Figure 4
figure 4

Plotting valence representation in multiple neural populations. (a) Plotting brain regions according to their task response index and valence index provides an alternative visualization to contrast them. Both brain regions and specific populations of neurons can be included in this plot. (b) Contrasting neural populations using valence selectivity and valence bias reveals a zone (blue) where one can find good candidate populations for circuit-based drug discovery because populations within this zone are relatively more selective and homogeneous in their response to positively and negatively valenced stimuli. Axes in both (a) and (b) are between 0 and 1. Axes in (b) cross at 0.5. Data for BLA, hippocampus and piriform cortex are from Fuster and Uyeda, 1971. Data for LH and AHA are from Ono et al, 1986. Data for NAc are from Roitman et al, 2005. Data for DRN, DRN 5-HT and DRN un-identified populations are from Cohen et al, 2015. Positions of BLA-NAc and BLA-CeM are based on data from Namburi et al, 2015. Positions of VTA (DA)-mNAc Sh and VTA (DA)-mPFC are based on data from Lammel et al, 2011. Data for VTA (Type 1/2/3) and VTA (DA/GABA) populations are from Cohen et al, 2012, where the authors define VTA (Type 1) as neurons that show reward prediction error like responses, VTA (Type 2) and VTA (Type 3) populations as neurons whose activity between CS and US was modulated positively and negatively by the value of the outcome.

PowerPoint slide

A Neurobiological Definition of Valence

Although the concept of valence is intuitive from a behavioral or psychological standpoint, bringing the concept of valence to neurophysiology is more challenging. The crux of the challenge arises from the psychological concept of valence occupying a single dimension, ranging from negative to positive. However, neural responses to cues of positive and negative valences can be independent of each other, and therefore occupy a two-dimensional space (Figure 2). Therefore, there is more than one plausible criterion by which a neuron can represent valence through modulation of its firing rate.

A neuron can represent valence by increasing its firing rate to a cue of one valence (eg, positive) and decreasing its firing rate to a cue of the opposite valence (eg, negative). We term this the opposing criterion. This criterion captures all neurons that represent both positive and negative valences. However, the opposing criterion in itself is limiting because it eliminates all neurons signaling only one valence.

A neuron can signal valence by modulating its firing rate to a cue of one valence (eg, increase in the firing rate to a positive cue), but not to a cue of the opposite valence (eg, no change in the firing rate to a negative cue). We term this the selective criterion. A neuron satisfying the selective criterion can be readily identified as signaling positive valence or negative valence (contrast this with a neuron satisfying the opposing criterion, where it signals both valences).

The valence of a cue can be inferred by sampling activity from a neural population containing neurons that satisfy either the opposing or selective criteria. Therefore, we propose that a neuron encodes valence if the response of the neuron satisfies either the opposing or selective criterion. In Figure 2, this translates to neurons categorized under a or i (satisfying the opposing criterion), b, d, f, or h (satisfying the selective criterion).

Finally, consider the case where a neuron responds by increasing its firing rate to cues of both positive and negative valences; however, the extent of modulation in the firing rate is different for both cues. Even though it is possible to infer the valence of the cue from the response of this neuron (meaning that it could encode valence), it is not clear whether the differential modulation in the firing rate is due to the stimulus valence (ie, positive or negative aspect of the stimulus), sensory features, or the salience of the outcome predicted by the stimulus (eg, a foot shock can be more salient relative to a drop of sucrose reward). This response profile would require additional experimental parameters to distinguish between these possibilities.

Implicit to the neurobiological definition of valence is the idea that a valence-encoding neuron signals the positive and/or negative aspect of the cue, independent of the sensory aspects of the cue. This independence criterion applies not only to the CS, where the neuron’s response should remain the same across multiple conditioned stimuli (eg, pure tone, light, smell of acetone), but also to the US, where different unconditioned stimuli predicting a negative outcome elicit the same response in the neuron (eg, foot shock vs air puff). In summary, we propose that a neuron encodes valence if its output is either oppositely or selectively modulated by the positive/negative properties of a conditioned or unconditioned stimulus.

A Model for Investigating Valence Representation

On the basis of the neurobiological definition of valence coding, we will now define four terms to facilitate the understanding of valence representation in different brain regions or within a population of interest (eg, dopaminergic neurons of the VTA, NAc-projecting neurons in the BLA, or neurons in the LH). First, we begin by categorizing each neuron into one of nine possible categories (Figure 2) based on the neuron’s response to positive and negative conditioned stimuli. Once we categorize an entire population, we use the number of neurons in each category to compute the model parameters. The meaning of each parameter is described below, and the guide to compute each metric is presented in Figure 2.

Task response index quantifies the fraction of task-responsive neurons. Valence index quantifies the proportion of task-responsive cells within a population that encode valence. A high or low valence selectivity informs whether a population is primarily responsive to rewarding stimuli or aversive stimuli, respectively. A high valence bias suggests the population to have a net excitatory output to a positive stimulus and/or inhibitory output to a negative stimulus.

The use of these proposed metrics carries multiple advantages, as described below.

Generalizability

The proposed metrics can be used to objectively compare populations of neurons within and between brain regions in the context of valence learning (Figures 3 and 4).

Reward vs aversion

A population of neurons primarily representing reward will have a valence selectivity close to 1, and a population of neurons primarily representing aversion will have a valence selectivity close to 0. The valence selectivity of the BLA is close to 0.6, which supports the idea that there are representations of both reward and aversion in this region. On the basis of the results from (Xiu et al, 2014), most of the neurons in the medial amygdala and amygdala–striatal transition zone respond to foot shock, not to morphine or chocolate rewards, and, therefore, they may have a valence selectivity close to 0. In contrast, neurons in the oval nucleus of the bed nucleus of stria terminalis and lateral subdivision of the central amygdala respond to rewards and not to foot shocks, and, therefore, may have a valence selectivity close to 1. Single-unit recordings are needed to confirm this observation.

Candidate populations for studying mechanisms of valence acquisition

A population with a high probability of finding a neuron representing valence is a good candidate for studying valence. This probability is simply the product of the task response index and valence index. If we multiply these two numbers, we get the probability of discovering a neuron representing valence within the given population. From this definition, the probability of finding a neuron representing valence in the BLA is ~0.4, LH is ~0.6, and piriform cortex is ~0.2 (Fuster and Uyeda, 1971; Ono et al, 1986).

Power analysis

The objective measures provided in this study can be used to inform the minimum number of neurons needed to find desired subpopulations within a population. Consider the BLA for example—the probability that a neuron randomly sampled from the BLA represents positive valence is at best: task response index × valence index × valence bias=0.16. Therefore, if we would like to sample neural activity from 10 positive valence-encoding neurons in the BLA (perhaps for assessing their role in a different behavioral paradigm), we would record from ~63 neurons (10/0.16).

Parallel vs opposing pathways for reward and aversion

Valence selectivity and valence bias, taken together, can inform us whether a subpopulation is differentially contributing to reward or aversion, or is having an opposing role in reward and aversion. If two subpopulations are contributing differentially to reward or fear, then valence selectivity will be close to 1 or 0, respectively, eg, based on the results from Lammel et al (2011), we may expect VTA-NAc medial shell to have a valence selectivity and valence bias close to 1 and VTA-mPFC projectors to have a valence selectivity and valence bias close to 0. If two subpopulations have opposing roles in fear and reward, then valence selectivity will be close to 0.5, but valence bias will be close to 0 or 1. Based on the results from Namburi et al, 2015, we can expect a mixture of BLA-NAc and BLA-CeM neurons to have a valence selectivity close to 0.5, with BLA-NAc having a valence bias close to 1, and BLA-CeM neurons having a valence bias close to 0 (Figure 3b).

Although the model introduced here has several advantages, it only synthesizes part of the spectrum of valence representation found in the brain. For example, DRN neurons represent valence over multiple timescales (Cohen et al, 2015) and valence representations between the hippocampus and BLA vary in their degree of flexibility (Redondo et al, 2014). Neither the timescale of valence representation, nor the flexibility of valence processing are currently captured by our model.

A circuit-based approach to identify novel drug targets

Identifying functional roles for specific circuit components offers subpopulations of neurons in which to search for novel drug targets. Within specific subpopulations of neurons that have well-characterized roles in modulating disease-relevant behaviors, we can examine the transcriptomic profiles of these cells to reveal surface receptors that could be targeted for therapeutic interventions. While each ligand would need empirical testing following the development of a novel target ‘short list,’ this could enhance the selection of promising, mechanistically novel drugs for treating neuropsychiatric disease.

The neurobiological basis of emotion has long been the focus of intense study, and early efforts succeeded in identifying the key nuclei that shape emotion through encoding valence. With the advent of modern techniques in neuroscience, we are now moving beyond evaluating the net effect of entire brain regions upon emotional behaviors, and instead beginning to parse regions into the specific populations of neurons responsible for valence processing. As reviewed here, these populations can be segregated from neighboring cells based upon anatomical, genetic, and/or functional categorizations. Because populations of cells defined by these categories can now be readily targeted and manipulated, we anticipate the imminent discovery of multiple valence-encoding populations of neurons distributed throughout the brain. These discoveries will require a unified model to serve as the basis of comparison among specific populations, to clearly delineate their unique contributions to valence coding. To meet this need, we offer here one such model, which we advance in an effort to compare and contrast the neural substrates of valence-processing and emotion distributed in the brain.

FUNDING AND DISCLOSURE

KMT is a New York Stem Cell Foundation—Robertson Investigator and McKnight Scholar, and this work was supported by funding from the JPB Foundation, PIIF, PNDRF, JFDP, Whitehall Foundation, Klingenstein Foundation, NARSAD Young Investigator Award, Alfred P Sloan Foundation, New York Stem Cell Foundation, McKnight Foundation, Whitehead Career Development Chair, NIH R01-MH102441-01 (NIMH) and NIH Director’s New Innovator Award DP2-DK-102256-01 (NIDDK). PN was supported by the Whitaker fellowship. GGC is supported by the JFDP Postdoctoral Fellowship from the JPB Foundation. This work is supported by NIDA R01 DA033396 (MRB), EUREKA grant R01DA037152 (MRB) and NIDA K99/R00 Pathway to Independence Award DA038725 (RA-H). The remaining authors declare no conflict of interest.