Serotonergic activation during courtship and aggression in the brown anole, Anolis sagrei

Subjects and social condition groups

The subjects examined in this study are the same as those used in previously published vasotocin, mesotocin, and catecholamine studies and detailed methodology can be found in those papers (Kabelik et al., 2013; Kabelik & Magruder, 2014; Kabelik et al., 2014). Fifty-seven adult male brown anole lizards served as the focal subjects. These animals were placed for 72 h into one half of a terrarium (30.5 cm H × 26 cm W × 51 cm L), separated from the other half by an opaque partition. The animals were visually isolated from one another. In the case of the control group (n = 12), the other half of the terrarium contained no animal. The other half of the terrarium in the male-male aggression condition group (n = 23) contained a conspecific stimulus male up to 2 mm shorter in snout-vent length and the male–female courtship group (n = 22) contained a conspecific stimulus female. All procedures involving live animals were approved by the Rhodes College Institutional Animal Care and Use Committee (approval #101-R2).

Behavioral trials

To commence a trial, the opaque divider was removed and the behaviors of the focal and stimulus animals were observed from behind a blind and scored over a 14-minute interval. Behaviors were scored for frequency as well as intensity; intensity measurements were divided into appetitive display and pursuit behaviors, as well as consummatory biting (in aggression trials) and copulation (in courtship trials). Kabelik et al. (2013) present an ethogram describing the recorded behaviors. At the end of the trial, animals were gently separated and maintained undisturbed for 90 min, at which point they were sacrificed by rapid decapitation. This delay in sacrifice allowed for peak Fos induction, as well as for Fos degradation in cases of reduced neural activity (Herdegen & Leah, 1998).

Tissue preparation and immunohistochemistry

Immediately following decapitation, brains were dissected and submerged overnight in 4% paraformaldehyde (in 0.1 M phosphate buffer) at 4 °C. Brains were then transferred to 30% sucrose in 0.1 M phosphate-buffered saline (PBS) for another 24 h at 4 °C, frozen and stored at −80 °C. The brains were cryosectioned at 50-μm thickness, and 2 series of alternating coronal sections were kept. The caudal half of series 2 underwent immunohistochemical processing for this study; multiple immunohistochemistry runs were conducted, each containing brains from a mixture of social condition groups. Brains were first rinsed for 30 min in PBS and blocked in a solution of 2.5% donkey serum (Sigma-Aldrich, St. Louis, MO, USA) and 0.03% Triton-X-100 (Fisher Scientific, Waltham, MA, USA) in PBS for 1 h. Sections were then placed for 40 h at 4 °C in the blocking solution containing 0.2 μg/L goat anti-5-HT polyclonal antibody (Immunostar #20079) and 20 μg/L mouse anti-Fos monoclonal antibody (Santa Cruz Biotechnology #sc-166940; Santa Cruz Biotechnology, Dallas, TX, USA). The sections were rinsed again for 30 min in PBS and then incubated at room temperature for 2 h in blocking solution now containing secondary antibodies, 1:500 Alexa Fluor donkey-anti-goat 488 and 1:200 Alexa Fluor donkey-anti-mouse 555 (both from ThermoFisher Scientific, Waltham, MA, USA). Following secondary antibody incubation, sections were rinsed for 30 min in PBS. Sections were then mounted onto gelatin- and chrome-alum-subbed slides, allowed to air dry, and cleared with Xylene Substitute (Sigma-Aldrich, St. Louis, MO, USA) for 5 min. Finally, sections were coverslipped with Prolong Gold mounting medium containing DAPI (ThermoFisher Scientific, Waltham, MA, USA), and allowed to cure for 24 h. Preadsorption with 5x 5-HT blocking peptide (Immunostar, Hudson, WI, USA) eliminated the majority of signal in control tissues, while 25x blocking peptide eliminated all signal.

Image acquisition and analysis

An LSM 700 confocal microscope was used in combination with Zen 2010 software (Carl Zeiss Inc., Oberkochen, Germany) to acquire fluorescent photomicrographs of 5-HT-ir and Fos-ir neurons. Images were captured using a 20x objective in a z-stack at 10 levels, each 5 μm apart. These images were converted into a two-dimensional projection image and separate colors were exported as separate images using AxioVision 4.8 (Carl Zeiss Inc., Oberkochen, Germany). These images were viewed as stacked monochromatic layers using Adobe Photoshop (Adobe Systems Inc., San Jose, CA, USA). Each layer could be toggled on and off to observe cell and colocalization counts (Fig. 1). A single observer, blind to social condition group, analyzed each brain region. Regional divisions into the superior medial raphe nucleus (RAsm), superior lateral raphe nucleus (RAsl), medial superior reticular nucleus (Rsm), lateral superior reticular nucleus (Rsl), inferior raphe nucleus (RAi), and caudal inferior raphe nucleus (RAic) were developed based on 5-HT mapping in other reptiles (Wolters et al., 1985; Smeets & Steinbusch, 1988; Bennis et al., 1990; Challet et al., 1991; Ayala-Guerrero, Huitrón-Reséndiz & Mancilla, 1991; Kiehn, Rostrup & Møller, 1992; Ten Donkelaar, 1998; Rodrigues et al., 2008). Although not all studies make an explicit medial-lateral division in either of these nuclei, a spatial separation between medial and lateral neuron clusters in both the superior raphe and superior reticular nuclei allowed us to delineate such a division. While no previous subdivision of the inferior raphe has been made to our knowledge, the anole brains in the present study always show two neuron clusters in the inferior raphe region spatially separated by at least one 100-micrometer section of no 5-HT immunofluorescence, thus we have subdivided the region. Serotenergic cells were also seen in other brain regions, but not in sufficient numbers that would allow for analysis in this study.

Statistical analysis

We wished to compare the total number of cells expressing 5-HT-Fos colocalization across social conditions (control, courtship, and aggression), both in the entire brain and in individual brain regions (see Fig. 2 for images of analyzed brain regions). Data from brains containing missing or damaged slides in the region of interest were therefore excluded from this analysis. Because our sample size was reduced by excluding brains with missing or damaged sections, we thus also conducted analyses comparing the percentage of 5-HT cells showing colocalizing with Fos (this relative measure was not as greatly affected by a given missing section). Only brain regions with at least 5 5-HT neurons were included in the percentage analysis.

Some variables did not meet assumptions of parametric analyses so non-parametric tests were employed throughout. In all between-group analyses, non-parametric Kruskal–Wallis tests were conducted, with pair-wise post hoc Mann–Whitney tests run at α = 0.05. Mann–Whitney tests were also conducted for analyses of appetitive versus consummatory behavioral intensity. Non-parametric Spearman’s correlations were used to investigate the relationship of behavior frequency and latency within social condition groups with total number of cells expressing 5-HT-Fos colocalization, percentage of 5-HT cells colocalizing Fos, and total number of 5-HT cells. Benjamini–Hochberg corrections to the alpha level were made when correcting for multiple comparisons (Benjamini & Hochberg, 1995).

Effects of social condition on 5-HT activity

All regions combined

When looking at all of the 5-HT regions together, the number of 5-HT cells colocalizing Fos was found to significantly differ across social conditions (χ² = 7.662, df = 2, p = 0.022; Fig. 3). Please note that this analysis includes only subjects for whom we had complete datasets (N = 38), i.e., no missing sections; however, comparable results were produced even when examining the percentage of 5-HT cells colocalizing Fos (N = 57; χ² = 6.907, df = 2, p = 0.032) rather than the total number of such cells. Post hoc pair-wise Mann–Whitney tests indicated that the courtship group (N = 14) had significantly fewer 5-HT cells colocalizing Fos (U = 24.500, p = 0.008) than the control group (N = 10) and even marginally fewer counts (U = 55.500, p = 0.050) than the aggression group (N = 14). There were no significant differences between the aggression and control groups in total number of colocalized cells (U = 59.000, p = 0.519). There was also no difference between groups in total number of 5-HT cells (χ² = 2.194, df = 2, p = 0.334).

Relationship between behavior intensity and 5-HT activity

Courtship intensity (individual regions)

We found a significant decrease in the number of 5-HT cells colocalizing Fos in the superior medial raphe nucleus (RAsm) of subjects whose courtship included actual copulation relative to those that did not copulate (U = 8.000, N = 20, p = 0.018; Fig. 4). Although these differences do not meet the threshold of the Benjamini-Hochburg adjusted α of 0.007, likely due to our small sample sizes, the difference between group is infinite as no copulating animal showed any 5-HT-Fos colocalized cells. Hence, we nevertheless report the results here for the reader’s benefit. This same effect was seen even if examining percent of 5-HT-Fos colocalization (U = 8.000, N = 20, p = 0.019). We saw no other differences between copulating and non-copulating animals in any other 5-HT population.

Aggression intensity (individual regions)

In the Rsl, there was a significant reduction in number of 5-HT cells colocalized with Fos in subjects that bit the male conspecific relative to those that did not bite the conspecific (U = 13.500, N = 21, p = 0.024; Fig. 5). Although this significance level was not below the Benjamini-Hochburg adjusted α of 0.008, the difference between the group means was again great—this time 2000%. However, this time no significance was detected when examining percent colocalization (U = 16.500, N = 18, p = 0.089). We saw no other differences between biting and non-biting animals in any other 5-HT population.

5-HT correlations with behavior frequency and latency

When examining all 5-HT regions combined, there were no significant correlations between neural measures of 5-HT colocalization with Fos and behavioral frequencies or latencies in either courtship or aggression trials (p > 0.05 for all). However, correlations emerged when analyzing brain regions separately. Although none of these correlations resulted in a p value below the adjusted α for multiple comparisons, we nevertheless present them below as at least strong trends for the reader’s benefit, as they can serve to generate hypotheses for future studies.

Courtship behavior latency (individual regions)

The total number of 5-HT cells colocalizing Fos in the Rsl was found to correlate negatively with latency to first courtship display (r =  − 0.467, N = 20, p = 0.038, Fig. 6). No correlation was present when examining percent colocalization (r =  − 0.342, N = 21, p = 0.129). No other regions showed any significant correlations with behavioral frequency or latency (p > 0.05 for all).

Aggression behavior frequency (individual regions)

As with courtship latency, a significant relationship was found between behavioral measures and the count of 5-HT neurons colocalizing Fos within the Rsl. In relation to aggression, the total number of colocalized 5-HT neurons correlated negatively with the total number of exhibited aggressive behaviors (r =  − 0.445, N = 21, p = 0.043) (Fig. 7). No correlation was again found if examining percent colocalization (r =  − 0.374, N = 22, p = 0.086). No other regions showed significant correlations with aggressive behaviors.

Courtship

Source populations of 5-HT neurons that affect sexual behaviors include neurons in the raphe nuclei, which are suspected to regulate sexual behavior because lesions of the dorsal raphe nucleus (DRN) have been shown to decrease ejaculation latency in male rats as well as increase lordosis behavior in female rats (McIntosh & Barfield, 1984; Kakeyama, Negishi & Yamanouchi, 1997). The superior raphe nucleus, which we subdivide into medial (RAsm) and lateral (RAsl) parts, described in sharks and reptiles, is believed to be homologous to the mammalian DRN (Stuesse, Cruce & Northcutt, 1991; Kiehn, Rostrup & Møller, 1992). However, the lizard RAsm and RAsl do not appear to be as greatly differentiated into subnuclei as the DRN (Hale & Lowry, 2011) and the exact homology of the RAsm and RAsl to DRN subnuclei is unknown.

Furthermore, we identify the superior reticular nucleus, which we subdivide into medial (Rsm) and lateral (Rsl) parts based on spatial distribution and similar subdivision in other reptiles (Wolters et al., 1985; Smeets & Steinbusch, 1988; Challet et al., 1991; Kiehn, Rostrup & Møller, 1992). The reticular nucleus is found lateral to the superior raphe and believed by some to be homologous to the mammalian B9 nucleus (Stuesse, Cruce & Northcutt, 1991; Rodrigo-Angulo, Rodriguez-Veiga & Reinoso-Suárez, 2000). However, a separate pontomesencephalic reticular formation is also present in mammals (Hale & Lowry, 2011), and although the function of this region is not well understood, it does possesses projections to the basal forebrain. Both the B9 and the pontomesencephalic reticular formation show social behavior-related differences in gene expression (Kerman et al., 2011). Therefore, the functional associations to behavioral regulation seen in the present study (condition-dependent Fos induction in the Rsm and Fos induction related to aggression intensity and frequency, as well as courtship latency, in the Rsl) are consistent with homology with either or both of these two 5-HT groups in mammals.

Although not exhaustively explored, it has been suggested that the 5-HT system may influence courtship behaviors through connections to the bed nucleus of the stria terminalis (BNST) as well as through an interaction with the neuropeptide Y system (Finn & Yahr, 2005; Muroi & Ishil, 2016). Our study provides further support for the involvement of raphe nuclei in courtship behavior; however, we believe our study to also be unique as we know of no previous research that has linked 5-HT neurons of the reticular nucleus to courtship.

The mechanisms of 5-HT action at release sites pertinent to courtship behavior regulation are complex. Previous research has found that generally decreasing central 5-HT transmission facilitates copulatory behavior in rodents; however, 5-HT appears to have a complex regulatory role in which certain receptors activate, while others inhibit, sexual behaviors (Albinsson et al., 1996; Yonezawa et al., 2008; Carnevale et al., 2011; Uphouse, 2014). For example, 5HT _1Areceptors appear to facilitate ejaculation, while 5HT _1B receptors seem to inhibit copulatory behaviors (Fernandez-Guasti & Rodríguez-Manzo, 1992; Olivier et al., 2011). These functional differences of 5-HT based on receptor type and location could help explain our findings that a decrease in total 5-HT cells colocalizing Fos in the RAsm was found in animals that engaged in copulation, while the total number of 5-HT cells colocalizing Fos in the Rsl was nevertheless negatively correlated with latency to display courtship behavior. Projections of these two nuclei to different sets or types of post-synaptic receptors could explain the simultaneous positive relationship between courtship behaviors and neural activity in the Rsl and negative relationship with activationin the RAsm. Overall, the effects of 5-HT on sexual behaviors are still not fully understood and the effect of 5-HT on sexual behavior even appears to differ between sexes (Mariana & Kuhn, 2015).

Our study is also unique because it provides some evidence for 5-HT regulation of appetitive courtship behaviors—a correlation with latency to display of any courtship behavior (the first behavior displayed was always an appetitive display rather than the consummatory behavior of copulation). Such a link has not been well established. While 5-HT is known to primarily regulate consummatory behaviors (Dzieweczynski & Hebert, 2012; Fumero et al., 1994; Rhodes, Yu & Yamaguchi, 2007), it has been suggested that it is dopamine that plays a larger role in regulating appetitive behaviors. We have indeed confirmed the involvement of dopamine in appetitive (as well as consummatory) courtship behaviors in these same individuals within a separate study (Kabelik et al., 2014), but the present study suggests that 5-HT also has some connection to appetitive courtship display. However, the most striking results of our study are in accordance with the more well-established role of 5-HT in the regulation of consummatory behaviors, as we found a dramatic decrease in total 5-HT cells colocalizing Fos in the RAsm in animals that engaged in copulation compared to those that did not.

While most research focuses on the effects of 5-HT on social behaviors, relatively little research has examined how exposure to a social context can affect 5-HT activity. Past research along this line of inquiry has detected increased 5-HT levels in the striatum and nucleus accumbens of male rats after exposure to a female rat (Urakawa et al., 2014). An increase in 5-HT activity was also seen in the DRN of female salamanders when exposed to male pheromones (Laberge et al., 2008). Our between-group analysis adds to these findings by demonstrating the effects (decreased colocalization with Fos) of a courtship encounter on 5-HT activity regardless of the intensity or frequency of behavior elicited from the focal animal. These effects of social encounter condition were localized to the RAsl and Rsm, while two separate regions, the RAsm and Rsl, were linked to behavioral intensity and frequency—suggesting separate systems involved in perception of courtship context and in courtship behavioral expression. This finding is consistent with the idea of functional topography within subdivisions of midbrain and hindbrain serotonin-containing regions (Hale & Lowry, 2011; Commons, 2015). While the exact homology of the lizard subdivisions used in this study to mammalian subdivisions is not clear, the involvement of the more rostral regions (RAsl, Rsm, RAsm, and Rsl) and not the caudal regions (Rai and Raic) with courtship behavior found in this study is consistent with previous associations of rostral DR regions to reproductive behavior in mice (Commons, 2015).

Aggression

Execution of aggressive behaviors has generally been associated with a decrease in 5-HT activity, although the opposite has also been reported (Korzan et al., 2001; Van der Vegt et al., 2003; Haller, Tóth & Halász, 2005; Audero et al., 2013). Our results extend this involvement of 5-HT in aggression to the brown anole and, as with the courtship condition above, suggest that some 5-HT populations respond primarily to social context, while the activity of others is associated with intensities and frequencies of behavioral expression.

As with the courtship analyses above, fewer 5-HT neurons colocalizing Fos were observed in the Rsm and RAsl of male brown anoles exposed to an aggressive encounter, relative to control subjects. In contrast, our within-group comparisons indicate a strong link between displayed aggression and 5-HT activity within the Rsl, whereby subjects exhibiting the most intense behavior, biting, showed dramatically lower levels of 5-HT cell activation than subjects exhibiting non-maximal levels of aggressive display. Activation of 5-HT cells in the Rsl also correlated negatively with the number of aggressive behaviors displayed (both appetitive and consummatory behaviors combined). These results strongly suggest that the Rsl is a nucleus associated with aggressive behavior expression, whereas the Rsm and RAsl are, as in the courtship analyses, regions responding to a perception of the social context.

Although 5-HT release is generally associated with inhibition of exhibited aggression, the source nuclei for this 5-HT release are not fully clear. For instance, our study found decreased 5-HT activity in the RAsl of aggression-group animals relative to control-group subjects. Aggressive encounters were instead found to increase 5-HT activity in the DRN of hamsters; however, this was only seen in the losers of the encounter (Cooper et al., 2009), suggesting other possible functions, and the increase may have been due to DRN subnuclei not homologous with the RAsl. Interestingly, rats exposed to aggressive visual and olfactory stimuli, but not an actual physical encounter, also showed an increase in raphe 5-HT activity (Haller, Tóth & Halász, 2005). Likewise, a study of green anoles found increased 5-HT activity in the raphe nuclei following an aggressive encounter (Korzan & Summers, 2004); however, this time the effect was only seen in dominant males. Hence, evidence clearly suggests an involvement of the superior raphe 5-HT population in aggression, but there may be contextual and/or species differences present in the relationship of these neurons to aggressive behavior display, and differential activation of neurons across specific raphe subnuclei may further complicate the situation.

Unlike the raphe nuclei, we found little research examining the role of the reticular nucleus in aggression, although the medial division of the reticular nucleus has been implicated in aggression in Siamese fighting fish, although not specifically through the 5-HT system (Gorlick, 1990). As mentioned previously, the reticular nucleus is not consistently defined across classes of vertebrate. Two potential mammalian homologs of the reptile reticular nuclei include the B9 nucleus and the pontomesencephalic reticular formation—and both of these nuclei showed changes in Fos expression in response to an intruder in rats (Kerman et al., 2011).

Overall, our findings for the superior raphe somewhat contradict previous literature (though already itself contradictory), while our findings in the reticular nucleus add to the sparse knowledge of the region’s role in social behavior regulation.