Conditioned preferences: Gated by experience, context, and endocrine systems ☆

Central to the navigation of an ever-changing environment is the ability to form positive associations with places and conspecifics. The functions of location and social conditioned preferences are often studied independently, limiting our understanding of their interplay. Furthermore, a de-emphasis on natural functions of conditioned preferences has led to neurobiological interpretations separated from ecological context. By adopting a naturalistic and ethological perspective, we uncover complexities underlying the expression of conditioned preferences. Development of conditioned preferences is a combination of motivation, reward, associative learning, and context, including for social and spatial environments. Both social-and location-dependent reward-responsive behaviors and their conditioning rely on internal state-gating mechanisms that include neuroendocrine and hormone systems such as opioids, dopamine, testosterone, estradiol, and oxytocin. Such reinforced behavior emerges from mechanisms integrating past experience and current social and environmental conditions. More-over, social context, environmental stimuli, and internal state gate and modulate motivation and learning via associative reward, shaping the conditioning process. We highlight research incorporating these concepts, focusing on the integration of social neuroendocrine mechanisms and behavioral conditioning. We explore three paradigms: 1) conditioned place preference, 2) conditioned social preference, and 3) social conditioned place preference. We highlight nonclassical species to emphasize the naturalistic applications of these conditioned preferences. To fully appreciate the complex integration of spatial and social information, future research must identify neural networks where endocrine systems exert influence on such behaviors. Such research promises to provide valuable insights into conditioned preferences within a broader naturalistic context.


Introduction
In navigating the complexities of natural environments, organisms face multifaceted challenges throughout their lives.Central to the navigation of such challenges is associative learning, enabling individuals to adapt to ever-changing environments that influence survivorship and reproductive success.This adaptability is informed by past experiences, incorporates current environmental factors, and facilitates predictions about future social and physical contexts.Researchers who study brains, hormones, and behavior within natural and semi-natural environments often examine the social dimensions and experiential history of an individual, for example, the identities of social actors, the setting of the experience, and the focal individual's life and reproductive stages.Each of these elements modulates associative learning and gates the expression of behaviors in ecological contexts.However, the endocrine/ neuroendocrine link between social and spatial environments, and the modulation of this connection by life and reproductive stages, represents a vast reservoir of untapped exploration.Here, we delve into essential components of associative learning leading to conditioned preferences that rely on social and locational factors in natural contexts.Our goal for this review is to link naturalistic and neuroendocrine perspectives on conditioned preferences (in particular conditioned place and social preferences) to explore the complexities of ecologically-relevant associative learning.In doing so we hope to increase 1) appreciation by behavioral neuroscientists for the naturalistic factors that gate associative learning and 2) appreciation by behavioral ecologists for neuroendocrine advances that elucidate the underpinnings of complex naturalistic behavior.Our hope is that future studies will continue to elegantly bridge neuroendocrinology and behavioral ecology.
The fundamental basis of the conditioned preference is the formation of a reinforcing association between an animal and an appetitive stimulus, such as a specific location or social partner in the animal's environment.This association enhances the probability of the animal's return to (or prolonged interaction with) that appetitive stimulus in order to enhance its survival and reproductive success.A classic example of this is the preference developed by males of some species for the location in which they encountered sexual stimuli (reviews by Adkins-Regan, 2020;Domjan and Gutiérrez, 2019;Hollis, 1990).Because space use is a reliable method for assessing classical (i.e., Pavlovian, see Domjan, 2005) conditioning, the conditioned place preference (CPP) in particular has become a standard tool in basic and translational research.CPPs are evoked not only by ethologically relevant reinforcements, such as food and matings, but also by motivating and pleasurable substances, including psychostimulants and opiates.This has led to the widespread adoption of CPPs as a method for investigating the underlying mechanisms of addiction and substance abuse in pharmacology and neuroscience.The present review shifts the focus towards an ethologically relevant approach to study the modulation of conditioned preference, integrating social and locational influences through the lens of behavioral neuroendocrinology.For example, what are the social and physiological factors that might lead a seasonally-breeding bird to return to a socially dense area at some times of the year but not others, or alter the locations to which paired and unpaired mice of a monogamous and territorial species will condition?We investigate how life history, reproductive stages, past and current social experiences, and the physical environment (e.g., familiarity) impact endocrine and neural systems, shaping the development and expression of conditioned preferences.This exploration addresses a key question: what are the natural adaptive functions of conditioned preferences from a neuroendocrine perspective?We highlight diverse species to enrich our understanding of these adaptive functions, showcasing innovative methods to investigate such preferences and answer ethologically grounded questions.
Pharmacologically-induced preferences are most often studied in social isolation.However, an increasing body of evidence is revealing the importance of social context and social endocrine systems in the regulation of reinforcement.For example, ethanol-induced social preference in female mice is enhanced by the intoxication of a peer, an effect facilitated by the presence of estradiol (Wood and Rice, 2013).Additionally, ethanol-induced social conditioning is abolished in oxytocin receptor/vasopressin receptor 1 A knockout female mice (Wood et al., 2015).Thus, social context serves as another important dimension of association and conditioning.Recent study designs have integrated concepts underlying place preference conditioning with those related to social reinforcement (e.g., the well-established partner preference test) to develop social context-dependent assessments of reinforcement, such as the conditioned social preference (CSP) and the social conditioned place preference (SCPP) (Fig. 1).This review explores 1) the conceptual overlap between the CPP, the CSP, and the SCPP; 2) a brief survey of their major neurobiological underpinnings; 3) the context-dependency of their establishment by endocrine examples especially gonadal hormones and neuropeptides.We also highlight two recent ethological examples of how the concept of a conditioned preference can be applied: 1) a set of laboratory studies with European starlings (Sturnus vulgaris) examining the use of CPPs to investigate changes in reward that lead to differences in reinforcement of social behavior and song across breeding and non-breeding seasons, and 2) field and laboratory studies using CPPs to investigate changes in the rewarding properties of testosterone pulses in bonded and non-bonded male monogamous California mice (Peromyscus californicus).

Conditioned preferences for natural experiences
Reinforcing experiences and behaviors are diverse and varied in natural settings, especially for social animals.Beyond reinforcement of resource acquisition and defense, such as foraging (Rudebeck and Izquierdo, 2022) and territoriality (X.Zhao et al., 2020), social interactions such as play (C.Zhao et al., 2020), mating (Pitchers et al., 2014), communication (Riters, 2012), bonding (Young and Wang, 2004), parental care (Mulligan et al., 2021), and even aggression (Farrell and Wilczynski, 2006;Legrand, 1970;Martínez et al., 1995;Tellegen et al., 1969;Tellegen and Horn, 1972) can be reinforcing.Associations between a place or another individual and a reinforcing experience present important opportunities for learning and adjusting to a changing environment; reinforcing stimuli likely evolved their reinforcing valence because they confer a fitness advantage.Conditioning to the context of a positive experience increases the likelihood of experiencing it again.The likelihood of engaging in a particular reinforcing experience, and thus forming a conditioned preference to its social or environmental context, is dependent on both reproductive and life

Glossary
Conditioned place preference (CPP) An animal's preference for a particular location (i.e., conditioned stimulus, CS) in which it previously formed an association between said location and a nonsocial stimulus (i.e., unconditioned stimulus, US).The typical CPP experiment involves substances of abuse (Fig. 1B, top).Social conditioned place preference (SCPP) A conditioned place preference formed by the association of a location (CS) with a social stimulus (US).The typical SCPP experiment involves sexual reward (Fig. 1B, middle).

Conditioned social preference (CSP) A non-location
conditioned preference in which the CS and/or the US are social.For example, a conspecific (CS) is associated with a reinforcing substance (US).Or, for example, an odor (CS) is associated with a reinforcing social interaction (US) (Fig. 1B, bottom).

Social and environmental Context
The classical conditioned preference is a laboratory construct.However, any conditioned preference can be modified by previous interactions with the social and ecological environment independently of a classical conditioned place preference.

Reward vs reinforcement
In brief, reward is received from a stimulus that an animal interprets as positive (e.g., satisfying or pleasurable), whereas reinforcement is the process by which an experience increases the likelihood of a particular behavioral output.A rewarding experience often serves as a positive reinforcer, but by the same measure alleviation of an aversive experience serves as a negative reinforcer.In the example of a typical CPP the US (e.g., a substance of abuse) is rewarding, which reinforces the behavioral preference for the CS (e.g., the location in which the US was administered).It is important to mention that it is possible to have reinforcement through associative learning beyond the classical context of reward (for example, see Crawford and Domjan, 1993, also see Section 3.2.Motivation (wanting)).
P.K. Monari et al. history stages.For example, in rats, play is most reinforcing during the juvenile stage (Vanderschuren et al., 1997).Additionally, sexual behavior is reinforcing (Pfaus et al., 2001), but the strength of reinforcement may vary based on social context and the ovarian cycle phase (Corona, 2011;Paredes and Alonso, 1997).For example, in seasonally breeding animals, reinforcement changes seasonally such that sexual behavior is reinforcing during the breeding but nonsexual social interactions are rewarding during the non-breeding season (Riters and Stevenson, 2022).This evidence suggests that the capacity to either experience a stimulus as reinforcing or form an association between the stimulus and the social or environmental context is dependent on plasticity within neural systems involved in reward (Berridge andDayan, 2021), social behavior (O'Connell andHofmann, 2011), and spatial processing (Vann and Albasser, 2011).Moreover, as these neural networks are enriched for both gonadal hormone and neuropeptide receptors (ISH Data: Allen Brain Atlas: Mouse Brain, n.d.), and their signaling is regulated by reproductive and life history stages, evidence is emerging that such endocrine systems play an important role in the reinforcement of context-specific naturalistic experiences.Pharmacologically-induced conditioned preference research has been an important component of understanding the underlying reinforcement neural circuitry co-opted by drugs of abuse, providing important insights for human health (Bardo and Bevins, 2000).However, these experiments are often done without consideration for social context.All brain and hormonal mechanisms evolved in natural environments; therefore, studies taking place in natural environments replete with relevant, competing, and multimodal stimuli, are essential for understanding the function of neural systems.Moreover, association of reinforcement with place and social stimuli is likely appropriate and beneficial to fitness in naturalistic conditions, and often only becomes inappropriate and maladaptive when selection pressures are lessened, such as in environments of abundance.

The conditioned place preference (CPP)
The CPP paradigm is a well-established method for investigating reinforcement learning (Tzschentke, 2007), typically in the absence of social context.The basic design of the task is to associate a particular location with the effects of a stimulus (typically a pharmacological substance) and associate a different environment with the absence of that stimulus.In brief, an animal will develop a preference for a previously neutral environment (the conditioned stimulus, CS+) if it becomes associated with a reinforcing stimulus (unconditioned stimulus, US, such as food or a drug of abuse) even after the reinforcing stimulus is no longer present.CPPs can also reflect the reward value of a stimulus, as the greater reward received from a stimulus, the greater the strength of the reinforcement (Tzschentke, 2007).
Commonly, this experiment consists of a three-chambered apparatus, with the outer chambers designed to have different features (e.g., textures, visual cues, etc.), while the central chamber is neutral (although variations exist, for a review see (Bardo et al., 1995)).A threechambered apparatus allows for some delineation between preference and avoidance.Training involves administering the potentially reinforcing drug (US) to an animal and confining their movement to one of the outer chambers (CS+) while the drug is active.The following day, the animal is placed in the opposite compartment (CS-) and is administered a control vehicle.This alternating procedure is typically repeated several times (often for 2-3 days for each bout of drug and vehicle administration).Following conditioning, the animal is tested for their location preference by allowing them to freely travel between all three chambers and recording the amount of time spent in each chamber.If the animal spends more time in the chamber where the drug was administered (CS+) than in the opposite chamber (CS-), as is often the case for drugs of abuse, it is concluded that a CPP was formed.However, if the opposite is observed, such as for drugs that induce pain or discomfort, then it is concluded that a conditioned place avoidance was formed.For examples of CPP, see Section 4.1.Testosterone.A. The conditioned place preference (CPP), conditioned social preference (CSP), and social conditioned place preference (SCPP) test preference with different combinations of social or nonsocial unconditioned stimuli (US) and conditioned stimuli (CS).B. Simplified diagrams of typical conditioned preference paradigms.For the CPP, the US is a nonsocial reinforcement and the CS+ is a location; the strength or reinforcement can be modulated by social context.For the SCPP, the US is social reinforcement (e.g.sexual experience) and the CS+ is a location.For the CSP, the US is either a social (e.g.scent of a conspecific) or nonsocial reinforcement and the CS+ is either a social stimulus (top) or a nonsocial stimulus with a social context (bottom) (CPP Examples: (Bechtholt and Cunningham, 2005;Carr and White, 1983;Figlewicz et al., 2001;Nam et al., 2019;Van Der Kooy et al., 1982); SCPP examples: (García Horsman and Paredes, 2004;Peartree et al., 2012;Thiel et al., 2008;Trezza et al., 2009;X. Zhao et al., 2020); CSP examples: (Ismail et al., 2009;Kent et al., 2014;Kippin and Pfaus, 2001;Kosaki and Watanabe, 2016;Peris et al., 2022;Quintana et al., 2019;Wood et al., 2015)).P.K. Monari et al.

Social context modulates CPPs
It is important to note that the reinforcing properties of some stimuli can be revealed or modulated by social context.Evidence is accumulating in both human and animal models that social context and social separation are capable of conferring resilience or susceptibility to the effects of pharmacologically-induced reinforcement.For example, adolescent social defeat stress strengthens adult CPP to amphetamines in male rats (Burke et al., 2011).Conversely, social interactions in male rats and mice inhibit the formation of cocaine-induced CPP (Zernig and Pinheiro, 2015).

The social conditioned place preference (SCPP)
Social interaction can be highly reinforcing in social species.Social reinforcement can serve as the US and location can serve as the CS, indicating place association with social experience.For example, paced mating produces a SCPP in female rats, indicated by a preference for the chamber in which they engaged in paced mating (Paredes and Alonso, 1997).Another instance is play; juvenile rats will prefer a location where they have previously played with other juveniles (Calcagnetti and Schechter, 1992).
Both affiliative behaviors (such as copulation and bonding behaviors) as well as aggressive behaviors (such as winning fights) are able to induce the formation of SCPPs.Regardless of the stimulus, the formation of SCPP is likely a function of classical conditioning whereby the repeated pairing of an unconditioned stimulus with a stimulus results in a new association formed with the previously conditioned stimulus.Importantly, the conditioned preferences rely on the transient effects of a substance beyond a baseline level (pulsatility), in order to associate the reinforcement with a particular context.For examples of SCPP, see Section 4.2.Estradiol.

The conditioned social preference (CSP)
By replacing the neutral environment CS+ with a neutral social context CS+ it is possible to determine if a substance increases the reinforcing nature of a particular social context, thus forming a conditioned social preference (Kent et al., 2013).The CSP paradigm combines elements of both the CPP and partner preference tests (Wood and Rice, 2013).Partner preference tests are typically used as indicators of sociosexual and pair-bond motivation, whereby the subject interacts with two tethered animals of the opposite sex and time spent near one or the other indicates preference (Paredes, 2009).However, other social preference tests, such as those for parent-offspring or same-sex affiliation are also used.In the CSP paradigm, however, the stimulus animals are instead the conditioned stimuli (CS+ and CS-) and a separate stimulus serves as the unconditioned stimulus (e.g.drugs, reinforcing social interactions).

It is important to note that a CSP can be formed with either a social or nonsocial US
For example, in female rats, multiple paced copulations with a male (US) scented with a particular odor (CS+) increased their solicitations of a novel male with the same odor, whereas nonpaced copulations did not produce the same effect (Coria-Avila et al., 2005).Similarly to females, male rats will prefer to ejaculate with an unfamiliar female with a particular scent (CS+) if that scent is associated with previous copulation and ejaculation (US), a particular type of CSP known as the conditioned ejaculatory preference (Kippin and Pfaus, 2001).In another, more complex example, male Japanese quail develop a CSP to another castrated male as a result of mating attempts, indicating that CSP can reveal plasticity in sexual preferences (Nash and Domjan, 1991).However, nonsocial reinforcing stimuli are also capable of producing CSPs.For example, ethanol consumption (US) in the presence of a particular rat (CS+) increases future time spent in the presence of that rat (Wood et al., 2015).Taken together, evidence suggests that the formation of a CSP can occur in a manner similar to the formation of a CPP, suggesting that the same reinforcement system underlies both social and location associations.For examples of CSP, see Section 5. Oxytocin.

Ethologically relevant conditioned preferences and social behavior: an example research program in European starlings
What is reinforcing can change depending on ecological variables, social experience, and resource availability.The affective states induced by social stimuli adaptively reinforce behaviors that are critical for survival and reproduction and discourage others that may be maladaptive.Here, we briefly focus on research with European starlings to highlight the overlap of social-and place-based conditioned preferences.
For female European starlings, hearing male courtship song during the breeding season is rewarding, but only for females that possess the resources necessary for reproduction (i.e., nesting sites) (Riters et al., 2013;Spool and Riters, 2017).For male European starlings, the act of producing courtship song during the breeding season does not result in a CPP to the location associated with singing (Riters and Stevenson, 2012).Instead, it is thought that for breeding male songbirds, the reward comes from successful copulation, not singing itself (Riters and Stevenson, 2012).
A different scenario occurs in starlings during the non-breeding season, in which song is produced by both males and females and is a form of non-sexual flock cohesion rather than mate attraction (Riters et al., 2019).In this social and ecological context, the act of producing song itself then becomes rewarding, where starlings develop strong SCPPs for places paired with the production of their own flock songs (Hahn et al., 2017;Riters and Stevenson, 2012;Stevenson et al., 2020).
Taken together, these data demonstrate that social context is vitally important for the development of CPPs, especially in neurobiologically plastic and socially dynamic species.We do not know what components of a European starling's own song production lead to the associated preference, or how location and social context interact.Future research is needed to explore these interesting concepts.
Here, we describe an example of considerations for developing naturalistic conditioned preferences in nonmodel species.In both model and non-model species, there are certain behaviors that cannot be assessed within the confines of the most common CPP apparatus.For such behaviors, it becomes necessary to observe the behavior and then place the animal in the respective chamber for conditioning.Using this method, the physiological consequences of the behavior rather than the factors triggering the behavior are measured.This method has achieved robust results when determining copulation-associated reward in rats (Ågmo and Berenfeld, 1990;Paredes, 2009;Pfaus et al., 2001) and songassociated reward in birds (Riters, 2012;Riters et al., 2014;Riters and Stevenson, 2012;Stevenson et al., 2020).For the latter, non-sexual songs are often only produced when stress is low.Within the CPP apparatus, birds may not sing even when housed with familiar conspecifics.One solution is to capture a bird directly from the flock immediately after song is observed and place it into the designated CPP chamber for conditioning (Riters and Stevenson, 2012).
In non-model species, it is important to adjust the design of the CPP apparatus to suit the unique needs of that species.For instance, nesting European starlings are highly motivated to gather green nesting material (Rubalcaba et al., 2016), whereas female zebra finches avoid males with green leg bands (Hunt et al., 1997), making it ill-advised to use green in CPPs in these birds.There are additional concerns with using nonterrestrial animals that exist in three-dimensional space; for example, in birds, perches of varying yet consistent heights are used to achieve a semi-naturalistic testing chamber.In addition, wild-caught animals are often neophobic (e.g.(Kelly et al., 2020)), which may result in the birds confining themselves to a single chamber.For these species, extra caution is needed to include a habituation period prior to conditioning.P.K. Monari et al. 3.An overview of the neurobiology of associative learning through the lens of conditioned preferences

Neural dopamine and opioid underpinnings
The mechanisms of learned association underlying conditioned preferences are complex and can contribute to how (i.e.strength of) and whether conditioned preferences are formed.Neurobiologically, it becomes important to disentangle the three components of reward: "wanting", "liking", and "learning" (Berridge and Kringelbach, 2015;Berridge and Robinson, 2003;Dickinson and Balleine, 2002), further conceptualized as motivation, hedonics, and cognition (Trezza et al., 2010).

Motivation (wanting)
While a number of neurotransmitters are implicated in the subcomponents of motivation, hedonics, and cognition, mesolimbic dopaminergic signaling has typically been associated with the motivational aspects of reward, with many studies conducted in rodents.Appetitive motivation and the role of dopamine in "wanting" are well-reviewed, and well-debated, elsewhere (Rieger et al., 2022;Russo and Nestler, 2013).In brief, the reinforcing mechanism of the association formed in conditioned preferences relies heavily on the mesolimbic dopamine system.Numerous studies demonstrate that a CPP can be established by dopamine receptor agonists and blocked by receptor antagonists.Moreover, dopamine signaling drives both social reinforcement and drug reinforcement, suggesting a common underlying mechanism (Burkett and Young, 2012).Regions of the brain involved in reinforcement, in particular the nucleus accumbens (Dölen et al., 2013), amygdala (Hu et al., 2021), and preoptic area (Bayless et al., 2023;Riters and Stevenson, 2022), are likely highly overlapping across types of reinforcement: a common currency reinforcing network of interacting brain regions (Berridge and Kringelbach, 2015).Activation of the mesolimbic reinforcement system mediates motivation to experience social interaction (Borland et al., 2018).Thus, reinforcement for drugs of abuse and naturalistic reinforcement such as social interactions (affiliative behaviors, successful aggressive behaviors) likely emerge from similar processes.
Naturally rewarding behaviors (like fall birdsong or rodent social play) have an associated motivation to engage or seek out contexts in which those behaviors can be produced.For example, European starlings that produce fall song in flocks are more motivated to seek out conspecifics than birds that do not sing in flocks (Maksimoski et al., 2023), testifying to the positive affect associated with producing fall song and its link with social reward and motivation.In addition, fall song is strongly correlated with increased dopamine related mRNA expression in brain regions known to regulate appetitive and consummatory reinforcement (Maksimoski et al., 2023;Merullo et al., 2016).

Hedonics (liking)
The role of opioids in the regulation of subjective pleasure (i.e., "liking"), including that from social reward, has been theorized for several decades (Berridge, 1996;Hayward et al., 2002;Panksepp et al., 1980).Opioid activity within the nucleus accumbens modulates the hedonic properties of rewards such as food, and may indirectly enhance motivation in the ventral tegmental area by stimulating mesolimbic dopaminergic signaling (Barbano and Cador, 2007;Berridge and Kringelbach, 2008).Evidence for the role of the endogenous opioid system in mediating the hedonic components of reinforcement is evidenced by opioid involvement in many affective social behaviors including parenting, sexual behavior, and play (Depue and Morrone-Strupinsky, 2005).
SCPPs resulting from sexual reward likely require endogenous opioid release in both sexes (Miller and Baum, 1987;Paredes and Martínez, 2001).Females display mate-guarding behavior towards a male following paced mating (Holley et al., 2014).Accordingly, mate guarding behavior in female rats resulting from paced mating conditioning is eliminated if an opioid antagonist is administered during conditioning (Coria-Avila et al., 2008).In male rats, ejaculation leads to opioid release (Coolen et al., 2004), and copulation-induced CPP and conditioned ejaculatory preference depend on the reward state produced by opioids (Kippin and Pfaus, 2001;Ulloa et al., 2018).Both copulation-induced SCPP and conditioned ejaculatory preference are eliminated if an opioid antagonist is administered during conditioning sessions (Quintana et al., 2019).However, some conflicting evidence exists to suggest that opioids are involved in the expression of the conditioned preference, rather than its acquisition (Mehrara and Baum, 1990).
Activation of mu opioid receptors in a number of brain regions induces reward in rodents (Berridge and Kringelbach, 2013).For example, there is substantial evidence for the role of site-specific mu opioid activity in the avian nucleus accumbens and preoptic area in stimulating reward (Riters et al., 2017;Stevenson et al., 2020) and non-sexual fall song (Maksimoski et al., 2021;Polzin et al., 2022;Riters et al., 2014).Conversely, sexually motivated song is inhibited by mu opioid receptor stimulation in the preoptic area, perhaps by supplementing the external reward otherwise received from mating, thereby negating the motivation to sing in these contexts (Riters, 2012;Riters et al., 2019).

Cognition (learning)
While motivation and hedonics are crucial for the formation of conditioned preference, so too is the contextualization that situates the reinforcement within a particular location or social context.It is important to realize that conditioned preferences are a subset of associative learning and in the context of natural functions it is likely that several associative learning processes are occurring when a preference is formed.Thus, spatial perception and social memory are necessary to induce conditioned preferences.
One caveat is the difficulty of distinguishing between the cognitive learning and the hedonic/motivational components of a conditioned preference, as behavioral output requires an animal to remember the association (Huston et al., 2013).In the case of social reinforcement associated with location, there are two cognitive systems that interact with hedonic/motivation networks to impact the expression of a conditioned preference: the social behavioral network and the spatial memory network.In order to untangle the contribution of these components to conditioned preference, it is necessary to rely on a combination of behavioral assessments such as operant, partner preference, and memory tasks.These two cognitive systems in addition to the neural reward system (Fig. 3) are enriched for hormonal receptors, suggesting that they are regulated by endocrine systems (ISH Data: Allen Brain Atlas: Mouse Brain, n.d.).This is supported by an abundance of evidence demonstrating that hormones are capable of modulating both cognitive performance and drug induced CPP as reviewed elsewhere (Skuse and Gallagher, 2009;Williams and Meck, 1991).Cognitive properties of rewards are complex and varied and are likely modulated on multiple neural levels by an array of neurotransmitters and neurohormones.

Testosterone and CPP in males
Studies across rodent species demonstrate that aggressive and sociosexual behavior can induce a CPP in males (Hughes et al., 1990;Mehrara and Baum, 1990;Miller and Baum, 1987).These effects are likely mediated by testosterone due to the role of testosterone in aggression and sexual behavior in males (Antunes and Oliveira, 2009;Hughes et al., 1990).Interestingly, conditioned cues are also able to P.K. Monari et al. elicit hormone secretion, for example male rats have been shown to secrete testosterone in anticipation of sexual activity (Graham and Desjardins, 1980).Moreover, the propensity for the rapid action of testosterone to positively reinforce place associations is wellcharacterized in male mice, rats, and hamsters (Alexander et al., 1994;Arnedo et al., 2000;Wood et al., 2004;Zhao andMarler, 2014, 2016).Subcutaneous injections of testosterone are sufficient to establish a CPP in a dose-dependent fashion (Alexander et al., 1994;De Beun et al., 1992).Central injection of testosterone into the nucleus accumbens likewise establishes a CPP, suggesting that this reinforcement is driven by nucleus accumbens activity (Packard et al., 1997).Androgen reinforcement can act through membrane androgen receptors (Wood et al., 2004) and androgen metabolites (Frye, 2007;Rosellini et al., 2001).It is likely that sub-populations of androgen receptor neurons within the nucleus accumbens are responsible for the reinforcing properties of testosterone, as testosterone metabolites such as dihydrotestosterone and 3a-androstanediol induce CPP when administered to the nucleus accumbens shell but not the core (Frye et al., 2002).
The role of testosterone on CPP may be through the dopamine and opioid reward pathways, as treatments combining estrogens and androgens influence tyrosine hydroxylase (the rate limiting enzyme for the synthesis of both dopamine and norepinephrine) labeling density in birds (Appeltants et al., 2003;LeBlanc et al., 2007) and mammals (Kritzer, 2000(Kritzer, , 2003)), indicating that steroid hormones affect dopamine synthesis.Testosterone and its metabolites in mammals can also affect enkephalin opioid densities (Simerly et al., 1988;Watson et al., 1986).In male dark-eyed juncos, opioid receptor densities in the medial preoptic area and ventral tegmental area shift seasonally in association with gonadal volume (Woods et al., 2010), suggesting seasonal changes in testosterone in seasonally-breeding birds such as European starlings may alter opioid receptor activity in medial preoptic area and ventral tegmental area (Spool et al., 2016).
Dopamine signaling is central to the reinforcing effects of testosterone in the nucleus accumbens, as peripheral or intra-accumbens administration of dopamine receptor 1 or 2 antagonists prevent a testosterone-induced CPP (Becker and Marler, 2015;Packard et al., 1998;Schroeder and Packard, 2000).In addition to the nucleus accumbens, injections of testosterone into the medial preoptic area at low doses establish a CPP (King et al., 1999), suggesting a distributed network of regions may be responsible.For example, one brain area that is expected to be important for spatial and social memory processing is the hippocampus, which processes and integrates multimodal sensory information with emotional valence to consolidate memory engrams (Kitamura et al., 2017;Yu and Moss, 2022).While further research is necessary to uncover whether androgen signaling is capable of stimulating opioid release, there is evidence that blocking opioids can lead to pulsatile secretion of testosterone (Tenhola et al., 2012) and that the opioid antagonist naltroxone blocks self-administration of testosterone and the seeking of testosterone by increasing access via nose pokes in hamsters (Peters and Wood, 2005;Wood, 2008).Such results suggest a relationship between androgen and opioid systems and raise the intriguing possibility that opioid hotspots (Castro and Berridge, 2017) may be activated by testosterone.

Testosterone, CPP, and social context
The CPP test is used frequently in non-social conditions in laboratory studies for measuring the reinforcing and addictive nature of drugs and neurochemicals.However, we understand little about the natural functions and social relevance of location preferences, including those produced by testosterone.Pulses of testosterone are evoked naturally in males during aggressive male-male and affiliative female-male encounters, suggesting the role of testosterone on SCPPs is driven by territorial and sexual behavior (Camacho et al., 2004;Gleason et al., 2009;Goymann et al., 2019).Recent studies by Zhao, Marler, and colleagues highlight a function for testosterone in ethologically relevant CPPs and social-context dependent CPPs in the monogamous and highly territorial California mice in the lab.In these California mice, Zhao and Marler (2014) investigated variation in testosterone-driven space use by examining the formation of CPP in sexually naive and pair-bonded males.Subcutaneous administration of physiologically relevant levels of testosterone induced a place preference to a novel environment in unpaired adult males (Zhao and Marler, 2014).In paired males, testosterone strengthened a CPP for the location containing the nesting area in which they may have naturally conditioned to via cues such as the scents of their mate.The effect of testosterone on place preference was shown to be socially context dependent, as testosterone did not lead to a CPP when given to males in the location containing the nesting area with other male cagemates (Zhao and Marler, 2016).Thus, testosterone is likely important for territory development in locations in which the male has had successful sexual encounters and where the male has formed a bond with the female mate.
The social context dependence of testosterone-induced CPPs is reflected in the neural plasticity of androgen receptors.Singly housed males exhibit upregulated androgen receptor expression in the preoptic area relative to same-sex group housed males, which positively correlated with the strength of the CPP (X.Zhao et al., 2020), implicating a connection of the preoptic area to CPPs.This change in androgen sensitivity in the preoptic area may be characteristic of males that have dispersed to establish their own territory, making them more sensitive to testosterone signaling.Winning and testosterone pulses can result in an increase in AR receptors in the nucleus accumbens, ventral tegmental area, and the bed nucleus of the stria terminalis when experienced in a familiar environment, which may be associated with territorial establishment, maintenance, or expansion (Fuxjager et al., 2010).Collectively, these findings support that the reinforcing nature of testosterone is dependent on the familiarity of the environment, pair bond status, and sexual experience (Fig. 2).

Ethologically relevant conditioned preferences and hormones: an example research program in California mice
There are classic studies relating testosterone CPP conditioning to the context of sexual behavior in the lab (reviews by Adkins-Regan, 2020; Domjan and Gutiérrez, 2019;Hollis, 1990).In the first test of CPP in the wild, Petric, Kalcounis-Rupell and Marler (Petric et al., 2022) confirmed the classical view of T-induced CPPs as being site specific even in a complex natural environment.Among California mice, territorial defense, parental care, and resource acquisition are carried out by both sexes, with considerable behavioral plasticity both within and between pairs (Monari et al., 2021;Rieger et al., 2019Rieger et al., , 2021) ) and compensation can occur between pair members such as time at the nest (Trainor and Marler, 2001).One mechanism for adjusting time between locations (i.e.nest versus territorial boundary) is through conditioned preferences.As described above, testosterone is rewarding and reinforcing and males can experience a pulse in testosterone after both malemale and male-female encounters (Fig. 2).The male testosterone response to female California mice represent laboratory data that are presented for the first time in this review and are critical for demonstrating male testosterone responsiveness to females in California mice (Fig. 2 and Supplement 1).Three experimentally-induced pulses of testosterone mimicking natural levels were administered to paired males in the field over five days in their nests (Petric et al., 2022).Males treated with testosterone while in traps at the nest developed an altered spatial preference, accompanied by changes in social interactions and call production.Males treated with testosterone spent more time in the nest, which likely increased parental care both because of being in the nest and because testosterone implants alone increase paternal behavior (Trainor and Marler, 2001).As described earlier for laboratory research, males also spent more time in the nest chamber previously associated with his mate when paired with testosterone pulses (Zhao and Marler, 2016).We therefore see consistency between the laboratory and field studies.Untreated female mates rapidly respond to changes in male spatial preference by spending more time away from the nest.In P.K. Monari et al. response to the female mate's absence, more vocalizations were produced (Petric et al., 2022;Timonin et al., 2018), specifically hypothesized to be associated with long distance communication in the field (Briggs and Kalcounis-Rueppell, 2011).Such vocalizations are increased when the mates are far apart but not close together, and this selectivity is likely a combination of the change in location and direct and indirect effects of testosterone.The above results with California mice illustrate that a place preference can be induced through testosterone pulses in a similar manner in the laboratory and field and that behavioral changes may occur because of both the conditioning altering location preferences and, potentially, through other direct and indirect (such as through location changes) changes induced by testosterone.A potential scenario for the functions of testosterone pulses at different stages of the life history of California mice is shown in Fig. 2.

Testosterone influences spatial context learning in males
While the above studies suggest that testosterone is capable of providing a reinforcing signal, it is also likely that it alters the capability of learning the reinforcing association.Brain regions central to learning and memory, such as the hippocampus, are highly enriched for androgen receptors (Naghdi et al., 2001), suggesting a link between the experience of testosterone-induced reinforcement and the cognitive aspects of reinforcement learning.Androgens regulate spatial memory in male rodents, although there is little consensus of the direction of this effect across studies.In some cases, androgens have been observed to improve spatial memory in tasks such as the Y-maze and the Morris water maze (Hawley et al., 2013;Khalil et al., 2005), while in other cases androgen treatment was associated with impaired spatial memory or no effect (Goudsmit et al., 1990;Naghdi et al., 2001).Testosterone Detailed methods for measurements of testosterone levels (as also described in Bester-Meredith and Marler, 2001;Davis and Marler, 2003;Trainor and Marler, 2001) and male-female encounters are provided in Supplement 1.A one-way ANOVA demonstrated a statistically significant overall impact (F(5,39) = 2.64, P = 0.03).Subsequent Dunnett's multiple comparisons test against the 0 min control group indicated a significant rise in T levels at 30 min (Adjusted P = 0.0446).Testosterone therefore peaked 30 min after a male-female encounter.C-G.Testosterone pulses and their potential contextual place preference functions.Natural conditioned place preferences (CPPs) are dependent on location, social context, and hormone signaling.C. Juveniles/adults within their natal territory do not establish a CPP to testosterone pulses.D. Testosterone pulses following maturation and dispersal from the natal territory may induce a CPP as they establish and defend a new territory.E. Sociosexual behavior induces testosterone release that elicits CPP.F. Conditioned preferences related to family unit bonding likely depend on additional hormonal signaling such as oxytocin.G. Testosterone pulses resulting from territorial defense induce a CPP that adjusts time spent at the territory boundary versus the nest based on location and social interactions with neighboring competitors.P.K. Monari et al. administered directly into the hippocampus improved spatial navigation performance (Roof and Havens, 1992), whereas flutamide, an androgen receptor antagonist, decreased spatial learning (Naghdi et al., 2001).Moreover, testosterone supplementation may improve spatial memory in men (Cherrier et al., 2001;Driscoll and Resnick, 2007;Janowsky, 2006;Sherwin, 2003).At a structural level, androgens in the hippocampus produce changes in spine density and neurogenesis in a sexually differentiated way (Atwi et al., 2016).For example, testosterone enhances the survival of immature dentate gyrus neurons in adult males via an androgen receptor mechanism (Hamson et al., 2013).Androgens likely result in complex effects on spatial memory due to their conversion to various metabolites, such as estradiol and dihydrotestosterone, in several regions including the hippocampus (Tabatadze et al., 2014;Yague et al., 2010).When considering the propensity for androgens to establish place preferences, it is important to integrate reinforcement with spatial learning.

Testosterone and CPP in females
Research is limited on the role of testosterone on CPP establishment in females.Females do not exhibit the same testosterone pulse associated with the reward of an aggressive encounter in males (Davis and Marler, 2003).Additionally, there are mixed results regarding the impact of testosterone on reward in females in general, with some studies showing that testosterone does not elicit a CPP in female rodents, while others showing that the testosterone metabolite, 3αDIOL, does induce a CPP in female rodents (De Beun et al., 1992;Jorge et al., 2005).The influence of testosterone on reward in females may be more closely related to sexual behavior rather than aggression, as one study found a prenatal blockade of androgen receptors reduced the number of intromissions necessary for paced mating to induce a CPP, highlighting a potential role of androgen receptors in the rewarding value of female sexual behavior (Domínguez-Salazar et al., 2005).Future studies should explore the complex interactions of hormonal systems, including testosterone, on reinforcement in females (Smiley et al., 2022).

Estradiol and CPP/SCPP formation in females
17β-estradiol is the most bioactive estrogen in reproductive age females (Salole, 1986; "The Smoking Estrogensa Potential Synergy between Estradiol and Benzo(a)Pyrene," 2021) and is an important contributor to the development of conditioned preferences.There is a close link between estradiol levels and sexual receptivity in females, as well as motivated sexual behavior, or sexual proceptivity (Johnson and Phoenix, 1976;Spiteri et al., 2009).Reproductive behavior may therefore be the evolutionary basis for the role of estradiol on motivation and reward and its impact on CPP formation (Corona, 2011;Frye and Rhodes, 2006;Richard et al., 2017).Female rodents exclusively engage in copulation in the estrus phase, immediately following the estradiol peak which occurs during the proestrus phase (Hardy, 1972;Jennings and de Lecea, 2020).Estradiol treatment that mimics this peak in ovariectomized female rodents is known to induce a CPP (Frye and Rhodes, 2006).
Sexual interaction in female rodents leads to conditioned preferences in a context-dependent manner.Females exposed to a male form a SCPP in the compartment where copulation occurred (Meisel and Joppa, 1994;Oldenburger et al., 1992).However, paced mating, where females have control over the sexual interaction, is necessary for the SCPP to form (Paredes and Alonso, 1997;Paredes and Martínez, 2001).Furthermore, estradiol, rather than copulation alone, may be necessary for this SCPP to occur.In a study evaluating the dose-dependent impact of estradiol on CPPs, ovariectomized female rats were treated with both high and low levels of estradiol and all females engaged in paced mating (Corona, 2011).The females treated with higher levels of estradiol preferred the chamber in which mating had occurred relative to females treated with low levels of estradiol (Corona, 2011).This highlights the importance of estradiol concentration in a SCPP formation and reward more broadly, as it suggests that male stimulus in addition to copulation was only rewarding when estradiol levels were high.The internal environment in the form of the background hormonal milieu is therefore also critical for forming conditioned preferences.
Not only does estradiol administration lead directly to a CPP, but estradiol levels also influence the ability of other rewarding substances to induce a CPP.For example, estradiol can both upregulate and downregulate non-sexual reward (Hilderbrand and Lasek, 2018;Richard et al., 2017).Motivation for sucrose decreased during phases associated with high estradiol in naturally cycling female rats, and estradiol injection in ovariectomized rats also decreased motivation for sucrose, measured via lever pressing (Richard et al., 2017).It is possible that estradiol may alter motivation from food-seeking behavior to sexual behavior through the reward system, an adaptive shift to prioritize reproductive behavior during ovulation (E.Yoest et al., 2014).
Estradiol can also influence CPP by increasing the strength of reinforcement of a reward.Estradiol treatment enhances ethanol and cocaine induced CPP in ovariectomized rodents ("Estradiol," 2010;Hilderbrand and Lasek, 2018).In women, subjective hedonic responses to amphetamines are increased during the late follicular phase, when estradiol levels are rising, compared to phases when estradiol levels are lower (Justice and de Wit, 1999), which may contribute to the increased severity of and vulnerability to substance use disorders in females relative to males (McHugh et al., 2018;Tonn Eisinger et al., 2018).
It is likely that estradiol influences CPP in non-rodent species as well.The link between estradiol and sexual motivation is not limited to rodents, suggesting certain types of behavior or social encounters may be more rewarding depending on estradiol concentration, impacting the likelihood of the formation of a CPP.Little is known about the formation of CPPs in non-human primates, but there is evidence to support an important role for estradiol in sexual motivation and reward (Bardo and Bevins, 2000).Estradiol is not necessary for copulation to occur in primates, but females exhibit greater motivation to engage in sexual behavior in menstrual phases where estradiol levels are high.Rhesus macaques are more likely to seek out males to copulate during the follicular phase ("Sex and Context," 2001).Estradiol may regulate CPP formation in birds as well, as female Japanese quail have demonstrated a cocaine-induced CPP with high estradiol, but this preference can be erased when estradiol levels are low (Gill et al., 2016).This, of course, highlights the importance of female reproductive state for the reinforcing effects of drugs.
The coupling of sexual behavior with a specific location may have reproductive advantages.In European starlings, breeding condition females develop strong natural place preferences for nesting sites.Female European starlings that are allowed to explore a nest site have higher concentrations of circulating estradiol and express higher levels of the endogenous opioid preproenkephalin in the medial preoptic area; a region in which estrogens and opioids modulate sexual motivation (Spool et al., 2018).This demonstrates that not only can estradiol given in a specific location under the context of sexual behavior lead to the formation of a CPP, but locations associated with reproduction, such as a nesting site, can also lead to release of estradiol which may both induce a preference for a nesting site and stimulate reproductive behavior (Spool et al., 2018).
To summarize, hormonal state must be considered when determining whether or not a stimulus will or will not be rewarding and lead to the formation of a CPP.Under levels of high estradiol, a sexual stimulus may be highly rewarding and lead to the formation of a CPP, while a food reward will not (Corona, 2011;Richard et al., 2017).Under conditions of low estradiol, copulation may not lead to a CPP, but a food reward will (Corona, 2011;Richard et al., 2017).

Mechanisms by which estradiol influences CPP
We have demonstrated that estrogenic activity can modulate rewards leading to a CPP in females; the mechanisms through which estradiol P.K. Monari et al. may influence CPPs likely function via estrogen receptors on neurons interacting within the reward system, such as the nucleus accumbens and ventral tegmental area.Both estrogen receptor alpha (ERα) and estrogen receptor beta (ERβ) play a role in the formation of a CPP, as ethanol in conjunction with an ERα or ERβ agonist alone was not sufficient to enhance an ethanol-induced a CPP, but activation of both enhances an ethanol-induced CPP (Hilderbrand and Lasek, 2018).However, amphetamine-induced CPPs are enhanced by activation of ERβ alone, with no effect of ERα (Silverman and Koenig, 2007).
Despite having low levels of estrogen receptors, there are rapid effects of estradiol on activity in the nucleus accumbens that have been associated with estradiol-associated CPP formation (Frye and Rhodes, 2006;Yoest et al., 2014).This may be due to the presence of estrogen receptors on extranuclear sites surrounding the nucleus accumbens core and shell, influencing GABAergic and catecholaminergic systems at the presynaptic level, and ultimately impacting dopamine transmission (Almey et al., 2022).The coupling of estradiol activity during reproductive periods and sexual motivation and reward has been linked to the action of estrogen receptors on metabotropic glutamate receptors on the nucleus accumbens (Tonn Eisinger et al., 2018).This is consistent with the finding that the formation of a CPP during paced mating is associated with estradiol levels in the nucleus accumbens (Frye and Rhodes, 2006).Estradiol also reduces expression of Regulator G-protein Signaling 9-2 protein in the nucleus accumbens shell, an important protein for the behavioral responses to rewarding substances such as amphetamines and cocaine (Silverman and Koenig, 2007).
Estrogenic activity in the ventral tegmental area also plays an important role in both social and non-social CPP formation.Activation of the ventral tegmental area is crucial for female sexual receptivity in rodents (Sumida et al., 2005), and may also play a major role in female sexual interest (Yin and Lin, 2023).Ovariectomy decreases the preference in a cocaine-induced CPP test, and decreases the amount of dopamine and serotonin present in the ventral tegmental area (Russo et al., 2003), two neurochemicals that are central to the motivation and reward signaling pathway.Dopamine neuron sensitivity in the ventral tegmental area fluctuates throughout the estrous cycle, and direct treatment with estradiol in ovariectomized mice enhances the response of dopamine neurons in the ventral tegmental area to both dopamine and ethanol (Vandegrift et al., 2017).This may help to explain how estradiol enhances ethanol-induced CPPs (Hilderbrand and Lasek, 2018).The ventral tegmental area may also be critical for the estradiolinduced suppression of food motivation during reproduction, as estradiol injected directly into the ventral tegmental area reduces sucroseinduced CPPs, suggesting an important regulatory role of the ventral tegmental area (Richard et al., 2017).

Estradiol and spatial context learning in females
It is well-established that estrogens play an important role in spatial cognition, a critical component to the formation of CPPs (Genazzani et al., 2007).Estrogen receptors are found within the hippocampus as well as in prefrontal cortices (Genazzani et al., 2007).Estradiol-induced synaptic plasticity during seasonal changes and ovarian cycling in brain regions important for cognition (García-Segura et al., 1994) suggest that estradiol modulates the response to environmental and social context in a top-down fashion.Indeed, estradiol regulates spatial memory and hippocampal plasticity (Gibbs et al., 2004;Li et al., 2004;Luine et al., 1998).Ovariectomized female rats display memory and spatial learning impairments that can be rescued via estradiol treatment (Rashidy-Pour et al., 2019).Contrary to this, naturally cycling female rhesus macaques demonstrate reduced spatial memory performance during ovulation, when estradiol concentrations are high (Lacreuse et al., 2001).To form a CPP, the animal must also learn and remember the location.Therefore, the role of estradiol on spatial learning may play a critical role in the formation of CPPs, beyond enhancing the reward.

Estradiol and CPP in males
Little is known about the direct role of estradiol and CPPs in males.However, many of the rewarding effects of testosterone may come from the conversion of testosterone into estradiol via aromatase, as aromatase knockout male mice do not form SCPPs exposed to females in estrus (Pierman et al., 2006).This demonstrates that estradiol may be similarly rewarding in both males and females.

Oxytocin
Oxytocin is an evolutionarily conserved neuropeptide, synthesized in the paraventricular nucleus and the supraoptic nucleus of the hypothalamus, that is critical for the regulation of social behavior in mammals (Lim and Young, 2006).Oxytocin receptors are distributed widely in the rodent brain (Tribollet et al., 1988(Tribollet et al., , 1992)), and oxytocin is a critical modulator of affiliative and aggressive behaviors.Campbell (2008) and others suggest that it may increase social salience (Shamay-Tsoory and Abu-Akel, 2016).The impact of oxytocin on social recognition and reinforcement is well-established and its release may temporally link the social context of mating to reward (Burkett and Young, 2012).Additionally, oxytocin receptor expression is highly linked to partner preference formation in female prairie voles (Ross et al., 2009), and social recognition is absent in oxytocin knockout mice (Carter et al., 2008).

Oxytocin and conditioned preference in females
Beyond partner and sexual preference, the ability for oxytocin to induce conditioned preferences has been primarily studied in femalefemale affiliative contexts (Wood et al., 2015).Ventral tegmental area oxytocin receptor activation is important for the reinforcing properties of social interactions in Syrian hamsters, and same-sex social interactions are more reinforcing in females than in males (Borland et al., 2018).Although activation of oxytocin receptors can be reinforcing in the absence of a social stimulus and subcutaneous oxytocin administration is able to produce a CPP, intracerebroventricular infusion of oxytocin dose-dependently induces a CSP in ovariectomized female mice given estradiol implants, suggesting that it is capable of reinforcement in both solitary and social conditions (Kent et al., 2013).Other research has found that intranasal oxytocin induced a conditioned same-sex social preference (CSP) in female mice but not a CPP (Kosaki and Watanabe, 2016).
Ethanol can modify the development of conditioned preferences.If female mice are intoxicated when exposed to a stimulus female, they will form a CSP (Wood and Rice, 2013).Oxytocin receptor and vasopressin receptor 1a knockout female mice do not form a CSP following ethanol administration, and in fact developed conditioned social avoidance, suggesting that these receptors are required for conditioning an association between a reward and a social stimulus (Wood et al., 2015).
The ability of oxytocin to lead to a CSP is an inverted U-shaped response curve, with both high and low doses showing no effect and medium doses leading to a CSP (Kent et al., 2013).This inverted U may be influenced by the presence of estradiol; estradiol upregulates oxytocin and oxytocin receptor expression, which may contribute both to sex differences in the ability of oxytocin to induce a conditioned place or social preference, as well as changes across the hormonal cycle (Borland et al., 2019;Jirikowski et al., 2018).To summarize, oxytocin appears to be an important contributor to female-female CSPs with variable results for CCPs.

Oxytocin and conditioned preference in males
For oxytocin-induced conditioned preferences in males, there is more information available for the brain regions involved.Infusion of oxytocin into the central amygdala induces a CPP in male rats (László P.K. Monari et al. et al., 2016), and both the reinforcing and anxiolytic effects of this treatment can be abolished with a dopamine receptor 2 antagonist (László et al., 2020).Additionally, oxytocin receptors are present on dopamine neurons in the ventral tegmental area that project to limbic regions (Peris et al., 2017), and mesolimbic dopamine release is modulated by exogenous oxytocin (Estes et al., 2019).These findings support the close interaction between oxytocin and the mesolimbic reinforcement system (Love, 2014).The generalizability of the reinforcing effects of oxytocin requires further investigation, as there is conflicting evidence of the ability for exogenous oxytocin to form a nonsocial CPP in males (Kosaki and Watanabe, 2016).For example, while subcutaneous injections of oxytocin can produce a CPP in male rats (Liberzon et al., 1997), other studies did not observe a nonsocial CPP when oxytocin was infused intraperitoneally in male rats (Ramos et al., 2015), intracerebroventricularly in male mice (Qi et al., 2009), or into the nucleus accumbens in male rats (Baracz et al., 2012).This continues to emphasize the importance of the social component necessary for oxytocin effects on the formation of condition place preferences.
Whether or not oxytocin is reinforcing in nonsocial contexts (rarely conducted in laboratory settings, as rodents are most often grouphoused), the social environment is an important modulator of the reinforcement.Accordingly, intraperitoneal injection of oxytocin can induce a robust CPP only if another rat is present during the conditioning sessions (Ramos et al., 2015).Another caveat is that effects are likely dependent on dose and route of administration, as other experiments failed to observe either a conditioned same sex social preference or a CPP in male rats (Kosaki and Watanabe, 2016).Sex differences in the reinforcing properties of oxytocin are consistent with prior research demonstrating that exogenous oxytocin in the absence of mating can induce an opposite-sex partner preference in female but not male prairie voles (Cushing and Carter, 2000).The different susceptibilities of male and female subjects to exogenous oxytocin are in accordance with previous studies focusing on the effect of oxytocin on different aspects of social cognition and behavior.For example, administration of oxytocin either centrally (via intracerebroventricular injection; (Insel and Hulihan, 1995)) or peripherally (via subcutaneous injection; (Cushing and Carter, 2000)) facilitates the formation of opposite sex partner preference without mating experience in female but not male prairie voles, whereas administration of vasopressin to male prairie voles enhances partner preference for a familiar female over a novel female prior to cohabitation (Cho et al., 1999).Oxytocin therefore plays a significant role in CSPs, but more information is needed regarding its effects on nonsocial conditioning.

Oxytocin and spatial context learning
Oxytocin can modulate memory processes, especially social memory (Albers, 2012;Gabor et al., 2012).Therefore, in studies of social conditioned preferences, it is necessary to disentangle the possibility that activation of oxytocin receptors influences memory rather than reward itself (Bardo and Bevins, 2000).An interesting method to dissociate memory and reward in the reinforcing effects of oxytocin was recently described (Borland et al., 2018(Borland et al., , 2019)), in which a novel operant social preference task was used to assess behavioral effort in male Syrian hamsters.Hamsters were placed in a three-chambered apparatus and given access to a social chamber containing a conspecific or a nonsocial chamber, with each chamber separated by a vertical swinging door able to change the degree of behavioral effort required to gain access to the adjacent chamber.Animals demonstrate a preference for the social chamber, and motivation to access the social chamber can be studied by the frequency of chamber entries and by increasing the behavioral effort required to access the chamber.Infusion of oxytocin into the ventral tegmental area decreased the motivation to seek social interactions, whereas the opposite was observed for an oxytocin antagonist (Borland et al., 2019), providing evidence for the hypothesis that oxytocin receptors in the ventral tegmental area are directly involved in social reinforcement.

Oxytocin-vasopressin interplay and conditioned preference
Oxytocin and vasopressin are highly similar in structure (Acher and Chauvet, 1995), as are their receptors (Gimpl and Fahrenholz, 2001).Promiscuity between these ligands and receptors has been observed (Song et al., 2016;Song and Albers, 2018), and it is therefore challenging to separate the functions of oxytocin and vasopressin signaling.However, carefully designed studies have been able to tease apart the role of vasopressin, oxytocin, and their interplay on conditioned preferences.For example, male rats given a subcutaneous injection of oxytocin developed a conditioned ejaculatory preference, but not rats given vasopressin or saline, indicating that oxytocin is able to enhance the association of odor with sexual reinforcement (Ménard et al., 2019).Additionally, through its action on oxytocin receptors rather than vasopressin receptors, vasopressin can produce a CPP when injected into the ventral tegmental area (Song et al., 2014).This evidence suggests that oxytocin receptor activity drives conditioned preference formation.
Oxytocin and vasopressin may differ in their interaction with nonsocial reward induced CPPs.For example, intracerebroventricular and nucleus accumbens shell oxytocin administration enhances the expression of a morphine-induced CPP in male rats (Moaddab et al., 2015), while lateral septum vasopressin administration decreases the expression of amphetamine-induced CPP in male rats (Gárate-Pérez et al., 2021).Moreover, vasopressin may contribute to the impact of social context on the ability of drugs of abuse to result in conditioned preferences.Both vasopressin antagonists and housing with drug-naive animals reduces the expression of a morphine-induced CPP in male mice, suggesting that the protective effect of housing on the formation of opioid-dependent CPPs may be mediated by vasopressin (Bates et al., 2018).Additionally, ethanol-induced CSPs to same-sex cagemates are reduced in oxytocin receptor/vasopressin receptor 1 A knockout female mice (Wood et al., 2015).Taken together, the present research emphasizes the complexity of the interplay between oxytocin and vasopressin and emphasizes the need for future experiments to disentangle the differential impacts of neuropeptides on conditioned preferences.

Conclusion and future directions
Associative learning is a fundamental process that can lead to location and social environment-dependent conditioned preferences in both the lab and field.These preferences are mediated by endocrine signals that interact with pathways of reward, reflecting an internal stategating.To fully grasp both the naturalistic functions of conditioned preferences and the emergence of pathological conditioned preferences, a comprehensive approach that encompasses sex differences, life history, reproductive, social, and environmental factors is essential.The studies involving European starlings and California mice serve as exemplary models, demonstrating the effective application of these holistic approaches in ethological research.Conditioned preferences, in their simplicity, offer a versatile methodology applicable in both laboratory and field settings, allowing for an integrated examination of location and social preferences at behavioral and neural levels.
We advocate for the crucial role of hormonal and neuroendocrine system signaling in mediating both top-down influences (such as life history and reproductive stage) and bottom-up factors (like social and environmental contexts) in the formation of conditioned preferences.While not discussed in-depth in this review, future research directions should better elucidate the brain networks that interact to orchestrate complex naturalistic associative preferences.In Fig. 3 we provide a broad conceptual framework for guiding such future investigations.
Given that brain networks related to social behavior, reward processing, and spatial cognition are rich in endocrine receptors, often in a sex-dependent manner, a deeper understanding of their individual contributions to the hedonic, motivational, and learning aspects of conditioned preference is invaluable.Future research into the endocrine regulation of conditioned preferences should delve into the comparative effects of endocrine signaling on the release of neurotransmitters, such as dopamine and opioids.For instance, while it is known that testosterone can trigger dopamine release, the specifics of how testosterone influences opioid signaling-and its impact on conditioned preferences via this pathway-remain largely unexplored.Investigating these mechanisms will not only enhance our understanding of the neurobiological basis of conditioned preferences in situations that are ethologically relevant but also aid in developing context-specific treatments for addiction, offering new perspectives and therapeutic avenues.
Supplementary data to this article can be found online at https://doi.org/10.1016/j.yhbeh.2024.

Fig. 1 .
Fig. 1.Location and social context are two separate and overlapping dimensions for understanding context-dependent reinforcement conditioning.A. The conditioned place preference (CPP), conditioned social preference (CSP), and social conditioned place preference (SCPP) test preference with different combinations of social or nonsocial unconditioned stimuli (US) and conditioned stimuli (CS).B. Simplified diagrams of typical conditioned preference paradigms.For the CPP, the US is a nonsocial reinforcement and the CS+ is a location; the strength or reinforcement can be modulated by social context.For the SCPP, the US is social reinforcement (e.g.sexual experience) and the CS+ is a location.For the CSP, the US is either a social (e.g.scent of a conspecific) or nonsocial reinforcement and the CS+ is either a social stimulus (top) or a nonsocial stimulus with a social context (bottom) (CPP Examples:(Bechtholt and Cunningham, 2005;Carr and White, 1983;Figlewicz et al., 2001;Nam et al., 2019;Van Der Kooy et al., 1982); SCPP examples: (García Horsman andParedes, 2004;Peartree et al., 2012;Thiel et al., 2008;Trezza et al., 2009;  X. Zhao et al., 2020); CSP examples:(Ismail et al., 2009;Kent et al., 2014;Kippin and Pfaus, 2001;Kosaki and Watanabe, 2016;Peris et al., 2022;Quintana et al., 2019;Wood et al., 2015)).

Fig. 2 .
Fig. 2. Male California mouse testosterone responses to A. A male encounter (based on Marler et al., 2005) and B. A female encounter (new data; Zhao and Marler).Detailed methods for measurements of testosterone levels (as also described in Bester-Meredith andMarler, 2001;Davis and Marler, 2003;Trainor and Marler, 2001) and male-female encounters are provided in Supplement 1.A one-way ANOVA demonstrated a statistically significant overall impact (F(5,39) = 2.64, P = 0.03).Subsequent Dunnett's multiple comparisons test against the 0 min control group indicated a significant rise in T levels at 30 min (Adjusted P = 0.0446).Testosterone therefore peaked 30 min after a male-female encounter.C-G.Testosterone pulses and their potential contextual place preference functions.Natural conditioned place preferences (CPPs) are dependent on location, social context, and hormone signaling.C. Juveniles/adults within their natal territory do not establish a CPP to testosterone pulses.D. Testosterone pulses following maturation and dispersal from the natal territory may induce a CPP as they establish and defend a new territory.E. Sociosexual behavior induces testosterone release that elicits CPP.F. Conditioned preferences related to family unit bonding likely depend on additional hormonal signaling such as oxytocin.G. Testosterone pulses resulting from territorial defense induce a CPP that adjusts time spent at the territory boundary versus the nest based on location and social interactions with neighboring competitors.

Fig. 3 .
Fig. 3.A "future directions" conceptual framework for understanding how social and location-dependent conditioned preferences may emerge from associations formed between three overlapping neural networks enriched for sex hormone and neuropeptide receptors: social, reward, and spatial networks.Top left: Internal and external factors lead to context-dependent gonadal hormone and neuropeptide signaling in the brain.Bottom left: Simplified diagrams of the overlapping networks involved in conditioned social preferences (CSP), and social and nonsocial conditioned place preferences (SCPP and CPP, respectively).Key regions are color-filled, other important regions are color-outlined.Center: Network activity interactions lead to learned associations and conditioning.Right: Plasticity results in the expression of conditioned preferences in social and nonsocial contexts.