Skip to main content
SpringerLink
Log in

The Role of Brain Serotonin in the Expression of Genetically Determined Defensive Behavior

  • Published:
Russian Journal of Genetics Aims and scope Submit manuscript

Abstract

The review summarizes the results of long-term studies on the role of the brain neurotransmitter serotonin in genetic predisposition to various types of defensive behavior. The involvement of the serotonergic brain system in the mechanisms of genetic control of both active and passive defensive responses has been established using silver foxes, Norway rats of S40 selection for low and high aggressiveness to humans, aggressive mice with genetic knockout of monoaminoxidase A, and S40 rats selected for predisposition to passive defensive response of freezing (catalepsy). The changes in the serotonergic 5-HT1A brain receptors of rats genetically predisposed to different strategies of defensive behavior were similar. However, the activity of the key enzyme of serotonin biosynthesis and the brain structures, in which serotonin metabolism was altered, significantly differed with regard to the preferred strategy. The conclusion was drawn that the 5-HT1A receptors and enzymes of serotonin metabolism in the brain are involved in implementing genetic control of defensive behavior. Expression of the 5-HT1A brain receptors was suggested to determine the levels of fear and anxiety and, consequently, the predisposition to defensive behavior, whereas the preferred strategy of defensive response (active or passive defensive) depends on genetically determined features of serotonin metabolism in the brain structures.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

REFERENCES

  1. Gallup, G.G., Boren, J.L., Suares, S.D., and Wallnau, L.D., The Psychopharmacology of Tonic Immobility in Chickens, The Brain and Behavior in Fowl, Okawa, T., Ed., Tokyo: Jap. Sci., 1983, pp. 43-59.

    Google Scholar 

  2. Blanchard, R.J., Attack and Defense in Rodents as Ethoexperimental Models for the Study of Emotions, Prog. Neuro-Psychopharmacol. Biol. Psychiatry, 1989, vol. 13, pp. S3-S14.

    Google Scholar 

  3. Lagerspetz, K.M. and Lagerspetz, K.Y., Genetic Determination of Aggressive Behavior, The Genetics of Behavior, van Abeleen, J., Ed., Amsterdam, 1974, pp. 321-346.

  4. Popova, N.K., Genetics of Aggressive Behavior, Izv. Sib. Otd. Akad. Nauk SSSR, 1988, no. 3, pp. 120-127.

    Google Scholar 

  5. Kulikov, A.V. and Popova, N.K., Forms of Aggressive Behavior, Usp. Sovrem. Genet., 1991, pp. 131-151.

  6. Kulikov, A.V., Kozlachkova, E.Y., Maslova, G.B., and Popova, N.K., Inheritance of Predisposition to Catalepsy in Mice, Behav. Genet., 1993, vol. 23, pp. 378-384.

    Google Scholar 

  7. Popova, N.K. and Kulikov, A.V., Genetic Analysis of Different Kinds of Aggressive Behavior, Behav. Genet., 1993, vol. 23, pp. 491-497.

    Google Scholar 

  8. Nikulina, E.M., Borodin, P.M., and Popova, N.K., Changes in Several Forms of Aggressive Behavior and Monoamine Content in the Brain in Wild Rats Selected for Reduced Aggressiveness towards Human, Zh. Vyssh. Nervn. Deyat., 1985, vol. 35,no. 4, pp. 703-708.

    Google Scholar 

  9. Popova, N.K., Naumenko, E.V., and Kolpakov, V.G., Serotonin i povedenie (Serotonin and Behavior), Novosibirsk: Nauka, 1978.

    Google Scholar 

  10. Manuck, S.B., Flory, J.D., Muldoon, M.F., and Ferrell, R.E., Central Nervous System Serotonergic Responsivity and Aggressive Disposition in Man, Physiol. Behav., 2002, vol. 77, pp. 705-709.

    Google Scholar 

  11. Moyer, K.E., Kinds of Aggression and Their Physiological Basis, Commun. Behav. Biol., 1968, vol. 2, pp. 65-87.

    Google Scholar 

  12. Belyaev, D.K., Destabilizing Selection as a Factor in Domestication, J. Hered., 1979, vol. 70, pp. 301-308.

    Google Scholar 

  13. Naumenko, E.V., Popova, N.K., Nikulina, E.M., et al., Behavior, Adrenocortical Activity, and Brain Monoamines in Norway Rats Selected for Reduced Aggressiveness towards Man, Pharmacol. Biochem. Behav., 1989, vol. 33, pp. 85-92.

    Google Scholar 

  14. Kolpakov, V.G., Barykina, N.N., and Chepkasov, I.L., Genetic Predisposition to Catatonic Behavior and Methylphenidate Sensitivity in Rats, Behav. Proc., 1981, vol. 6, pp. 269-281.

    Google Scholar 

  15. Popova, N.K., Barykina, N.N., Plyusnina, I.F., et al., Expression of the Startle Reaction in Rats Genetically Predisposed to Different Defensive Behaviors, Ros. Fiziol. Zh., 1999, vol. 85,no. 1, pp. 99-104.

    Google Scholar 

  16. Popova, N.K., Voitenko, N.N., and Trut, L.N., Changes in Brain Serotonin and 5-Hydroxyindole Acetic Acid in Silver Foxes Selected by Behavior, Dokl. Akad. Nauk SSSR, 1975, no. 223, pp. 1496-1500.

    Google Scholar 

  17. Popova, N.K., Voitenko, N.N., Kulikov, A.V., and Avgustinovich, D.F., Evidence for the Involvement of Central Serotonin in Mechanism of Domestication of Silver Foxes, Pharmacol. Biochem. Behav., 1991, vol. 40, pp. 751-756.

    Google Scholar 

  18. Kulikov, A.V., Zhanaeva, E.Yu., and Popova, N.K., Changes in Brain Tryptophan Hydroxylase Activity in Silver Foxes and Norway Rats during Behavioral Selection, Genetika (Moscow), 1989, vol. 25,no. 2, pp. 346-350.

    Google Scholar 

  19. Popova, N.K. and Zhanaeva, E.Yu., Properties, Activity Regulation, and Functional Role of Brain Tryptophan Hydroxylase, Neirokhimiya, 1988, vol. 7, pp. 274-287.

    Google Scholar 

  20. Boadle-Biber, M.C., Biosynthesis of Serotonin, Biology of Serotonergic Transmission, Chichester: Wiley, 1982, pp. 63-64.

    Google Scholar 

  21. Blanchard, D.C., Popova, N.K., Plysnina, I.F., et al., Defensive Reactions of “Wild-Type” and “Domesticated” Wild Rats to Approach and Contact by a Threat Stimulus, Aggressive Behavior, 1994, vol. 20, pp. 387-397.

    Google Scholar 

  22. Popova, N.K., Brain Serotonin in Genetically Defined Defensive Behavior, Complex Brain Functions: Conceptual Advances in Russian Neuroscience, Millar, R., Ivanitsky, A.M., and Balaban, P.M., Eds., Harwood, 1999, pp. 317-329.

  23. Popova, N.K., Kulikov, A.V., Nikulina, E.M., et al., Serotonin Metabolism and Serotonergic Receptors in Norway Rats Selected for Low Aggressiveness to Man, Aggressive Behavior, 1991, vol. 17,no. 4, pp. 207-213.

    Google Scholar 

  24. Popova, N.K., Kulikov, A.V., Avgustinovich, D.F., et al., Effect of Domestication on the Basic Enzymes of Serotonin Metabolism and Serotonin Receptors in the Silver Foxes, Genetika (Moscow), 1997, vol. 33,no. 3, pp. 370-374.

    Google Scholar 

  25. Naumenko, E.V. and Popova, N.K., Serotonin i melatonin v regulyatsii endokrinnoi sistemy (Serotonin and Malatonin in the Regulation of the Endocrine System), Novosibirsk: Nauka, 1975.

    Google Scholar 

  26. Naumenko, E.V., Popova, N.K., and Ivanova, L.N., Neuroendocrine and Naurochemical Mechanisms of Domestication of Animals, Genetika (Moscow), 1987, vol. 23,no. 6, pp. 1011-1024.

    Google Scholar 

  27. Popova, N.K., Voitenko, N.N., Pavlova, S.I., et al., Genetics and Phenogenetics of Hormonal Characteristics of Animals: VII. Correlations between Brain Serotonin and the Hypothalamic-Pituitary-Adrenal System in Emotional Stress in Domesticated and Non-Domesticated Silver Foxes, Genetika (Moscow), 1980, vol. 26,no. 10, pp. 1865-1870.

    Google Scholar 

  28. Popova, N.K., Kulikov, A.V., Kolpakov, V.G., et al., Changes in the Brain Serotonin System in Rats Genetically Predisposed to Catalepsy, Zh. Vyssh. Nervn. Deyat., 1985, vol. 35,no. 4, pp. 742-746.

    Google Scholar 

  29. Kolpakov, V.G., Kulikov, A.V., Barykina, N.N., et al., Catalepsy and Increased Tryptophan Hydroxylase Activity in Rat Striatum, Biogenic Amines, 1985, vol. 2, pp. 131-136.

    Google Scholar 

  30. Popova, N.K. and Kulikov, A.V., On the Role of Brain Serotonin in Expression of Genetic Predisposition to Catalepsy in Animal Models, Am. J. Med. Genet. Neuropsychiatric Genet., 1995, vol. 60, pp. 214-220.

    Google Scholar 

  31. Koffer, K.B., Berney, S., and Hornykievicz, O., The Role of the Corpus Striatum in Neuroleptic and Narcotic-Induced Catalepsy, Eur. J. Pharmacol., 1978, vol. 47, pp. 81-86.

    Google Scholar 

  32. Kulikov, A.V., Kozlachkova, E.Y., and Popova, N.K., Activity of Tryptophan Hydroxylase in Brain of Hereditary Predisposed to Catalepsy Rats, Pharmacol. Biochem. Behav., 1992, vol. 43, pp. 999-1003.

    Google Scholar 

  33. Kulikov, A.V. and Voronova, I.P., The Role of Reversible phosphorylation in Genetically Determined Polymorphism for Tryptophan Hydroxylase Activity of the Brain, Byull. Eksp. Biol. Med., 1995, vol. 119, pp. 67-68.

    Google Scholar 

  34. Nutt, D.J. and Glue, P., Clinical Pharmacology of Anxiolytics and Antidepressants: A Psychopharmacological Perspective, Psychopharmacology of Anxiolytics and Antidepressants, File, S.E., Ed., New York: Pergamon, 1991, pp. 1-28.

    Google Scholar 

  35. Lesh, K.-P., The Ipsapirone/5-HT1A Receptor Challenge in Anxiety Disorders and Depression, Serotonin 1A Receptors in Depression and Anxiety, Stahl, S.M. et al., Eds., New York: Raven, 1992, pp. 135-162.

    Google Scholar 

  36. Popova, N.K., Avgustinovich, D.F., and Plyusnina, I.F., Specifics of the Binding of [3H]-8-OH-DPAT in Brain Structures of Wild Rats Selected for a Lack of Aggressiveness, Neirokhimiya, 1996, vol. 13, pp. 260-264.

    Google Scholar 

  37. Hammer, R., Hori, K., Blanchard, R., and Blanchard, D.C., Domestication Alters 5-HT1A Receptor Binding in Rat Brain, Pharmacol. Biochem. Behav., 1992, vol. 42, pp. 25-28.

    Google Scholar 

  38. Popova, N.K., Avgustinovich, D.F., Shigantsov, S.N., and Kulikov, A.V., Distribution of Serotonin 5-HT1A Receptors in the Brain of Rats Genetically Predisposed to Catalepsy, Zh. Vyssh. Nervn. Deyat., 1996, vol. 46,no. 3, pp. 578-582.

    Google Scholar 

  39. Popova, N.K., Avgustinovich, D.F., Kolpakov, V.G., and Plyusnina, I.Z., Specific [3H]8-OH-DPAT Binding in Brain Regions of Rats Genetically Predisposed to Various Defense Behavior Strategies, Pharmacol. Biochem. Behav., 1998, vol. 59, pp. 793-797.

    Google Scholar 

  40. Vishnivetskaya, G.B., Plyusnina, I.Z., and Popova, N.K., Involvement of 5-HT1A Receptors in Regulation of Various Types of Aggressive Behavior, Zh. Vyssh. Nervn. Deyat., 2001, vol. 51,no. 6, pp. 715-719.

    Google Scholar 

  41. Schreiber, R., Behavioral Pharmacology of 5-HT1A Receptor Ligands, Hamburg: Proefschrift, 1992.

    Google Scholar 

  42. Graeff, F.G., Neuroanatomy and Neurotransmitter Regulation of Defensive Behaviors and Related Emotions in Mammals, Brazil. J. Med. Biol. Res., 1994, vol. 27, pp. 811-829.

    Google Scholar 

  43. Cases, O., Seif, I., Grimsby, J., et al., Aggressive Behavior and Altered Amounts of Brain Serotonin and Norepinephrine in Mice Lacking MAO-A, Science, 1995, vol. 268, pp. 1763-1766.

    Google Scholar 

  44. Brunner, H.G., Nelen, M., Breakefield, X.O., et al., Abnormal Behavior Associated with a Point Mutation in the Structural Gene for Monoamine Oxidase A, Science, 1993, vol. 262, pp. 578-580.

    Google Scholar 

  45. Brunner, H.G., MAO-A Deficiency and Abnormal Behavior: Perspectives on an Association, Genetics of Criminal and Antisocial Behavior, Chichester: Wiley, 1995, pp. 155-167.

    Google Scholar 

  46. Popova, N.K., Skrinskaya, Yu.A., Amstislavskaya, T.G., et al., Behavioral Specifics of Mice Knocked-Out in Monoamine Oxidase A, Zh. Vyssh. Nervn. Deyat., 2000, vol. 50,no. 6, pp. 991-998.

    Google Scholar 

  47. Popova, N.K., Vishnivetskaya, G.B., Ivanova, E.A., et al., Altered Behavior and Alcohol Tolerance in Transgenic Mice Lacking MAO-A: A Comparison with Effects of MAO-A Inhibitor Clorgyline, Pharmacol. Biochem. Behav., 2000, vol. 67, pp. 1-9.

    Google Scholar 

  48. Popova, N.K. and Naumenko, E.V., Dominance Relations and the Pituitary-Adrenal System in Rats, Anim. Behav., 1972, vol. 20, pp. 108-110.

    Google Scholar 

  49. Popova, N.K., Gilinsky, M.A., Amstislavskaya, T.G., et al., Regional Serotonin Metabolism in the Brain of Transgenic Mice Lacking Monoamine Oxidase A, J. Neurosci. Res., 2001, vol. 66, pp. 423-427.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Cytology and Genetics, Russian Academy of Sciences, Novosibirsk, 630090, Russia

    N. K. Popova

Authors
  1. N. K. Popova
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Popova, N.K. The Role of Brain Serotonin in the Expression of Genetically Determined Defensive Behavior. Russian Journal of Genetics 40, 624–630 (2004). https://doi.org/10.1023/B:RUGE.0000033309.92155.5a

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/B:RUGE.0000033309.92155.5a

Keywords

Search

Navigation