Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Frontostriatal circuit dysfunction leads to cognitive inflexibility in neuroligin-3 R451C knockin mice

Molecular Psychiatry (2024)Cite this article

Subjects

Abstract

Cognitive and behavioral rigidity are observed in various psychiatric diseases, including in autism spectrum disorder (ASD). However, the underlying mechanism remains to be elucidated. In this study, we found that neuroligin-3 (NL3) R451C knockin mouse model of autism (KI mice) exhibited deficits in behavioral flexibility in choice selection tasks. Single-unit recording of medium spiny neuron (MSN) activity in the nucleus accumbens (NAc) revealed altered encoding of decision-related cue and impaired updating of choice anticipation in KI mice. Additionally, fiber photometry demonstrated significant disruption in dynamic mesolimbic dopamine (DA) signaling for reward prediction errors (RPEs), along with reduced activity in medial prefrontal cortex (mPFC) neurons projecting to the NAc in KI mice. Interestingly, NL3 re-expression in the mPFC, but not in the NAc, rescued the deficit of flexible behaviors and simultaneously restored NAc-MSN encoding, DA dynamics, and mPFC-NAc output in KI mice. Taken together, this study reveals the frontostriatal circuit dysfunction underlying cognitive inflexibility and establishes a critical role of the mPFC NL3 deficiency in this deficit in KI mice. Therefore, these findings provide new insights into the mechanisms of cognitive and behavioral inflexibility and potential intervention strategies.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: NL3R451C mice exhibit deficits in behavioral flexibility.
Fig. 2: Altered encoding of decision-related stimuli by NAc neurons in NL3R451C mice.
Fig. 3: Impaired experience-dependent modulation of NAc pMSN activity in NL3R451C mice.
Fig. 4: Altered [DA]NAc dynamics signal abnormal RPE and impaired associate learning in NL3R451C mice.
Fig. 5: mPFC-NAc circuit is impaired in NL3R451C mice.
Fig. 6: NL3 re-expression in mPFC rescues inflexible behaviors and NAc alterations in NL3R451C mice.

Similar content being viewed by others

Data availability

The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.

Code availability

All custom code used for analysis in this manuscript is available on request.

References

  1. Lopez BR, Lincoln AJ, Ozonoff S, Lai Z. Examining the relationship between executive functions and restricted, repetitive symptoms of Autistic Disorder. J Autism Dev Disord. 2005;35:445–60.

    Article  PubMed  Google Scholar 

  2. Cissne MN, Kester LE, Gunn AJM, Bodner KE, Miles JH, Christ SE. Brief report: a preliminary study of the relationship between repetitive behaviors and concurrent executive function demands in children with autism spectrum disorder. J Autism Dev Disord. 2022;52:1896–902.

    Article  PubMed  Google Scholar 

  3. Chan MMY, Chan MC, Lai OL, Krishnamurthy K, Han YMY. Abnormal prefrontal functional connectivity is associated with inflexible information processing in patients with autism spectrum disorder (ASD): an fNIRS study. Biomedicines. 2022;10:1132.

    Article  PubMed  PubMed Central  Google Scholar 

  4. D’Cruz AM, Mosconi MW, Ragozzino ME, Cook EH, Sweeney JA. Alterations in the functional neural circuitry supporting flexible choice behavior in autism spectrum disorders. Transl Psychiatry. 2016;6:e916.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Uddin LQ. Brain mechanisms supporting flexible cognition and behavior in adolescents with autism spectrum disorder. Biol Psychiatry. 2021;89:172–83.

    Article  CAS  PubMed  Google Scholar 

  6. Stokes MG, Kusunoki M, Sigala N, Nili H, Gaffan D, Duncan J. Dynamic coding for cognitive control in prefrontal cortex. Neuron 2013;78:364–75.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  7. Sekutowicz M, Schmack K, Steimke R, Paschke L, Sterzer P, Walter H, et al. Striatal activation as a neural link between cognitive and perceptual flexibility. Neuroimage. 2016;141:393–8.

    Article  PubMed  Google Scholar 

  8. Macpherson T, Matsumoto M, Gomi H, Morimoto J, Uchibe E, Hikida T. Parallel and hierarchical neural mechanisms for adaptive and predictive behavioral control. Neural Netw. 2021;144:507–21.

    Article  PubMed  Google Scholar 

  9. Kehagia AA, Murray GK, Robbins TW. Learning and cognitive flexibility: frontostriatal function and monoaminergic modulation. Curr Opin Neurobiol. 2010;20:199–204.

    Article  CAS  PubMed  Google Scholar 

  10. St Onge JR, Stopper CM, Zahm DS, Floresco SB. Separate prefrontal-subcortical circuits mediate different components of risk-based decision making. J Neurosci. 2012;32:2886–99.

    Article  Google Scholar 

  11. Otis JM, Namboodiri VM, Matan AM, Voets ES, Mohorn EP, Kosyk O, et al. Prefrontal cortex output circuits guide reward seeking through divergent cue encoding. Nature. 2017;543:103–7.

    Article  CAS  PubMed  PubMed Central  ADS  Google Scholar 

  12. Sala-Bayo J, Fiddian L, Nilsson SRO, Hervig ME, McKenzie C, Mareschi A, et al. Dorsal and ventral striatal dopamine D1 and D2 receptors differentially modulate distinct phases of serial visual reversal learning. Neuropsychopharmacology. 2020;45:736–44.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  13. Britt JP, Benaliouad F, McDevitt RA, Stuber GD, Wise RA, Bonci A. Synaptic and behavioral profile of multiple glutamatergic inputs to the nucleus accumbens. Neuron. 2012;76:790–803.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  14. Li Z, Chen Z, Fan G, Li A, Yuan J, Xu T. Cell-type-specific afferent innervation of the nucleus accumbens core and shell. Front Neuroanat. 2018;12:84.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  15. Yawata S, Yamaguchi T, Danjo T, Hikida T, Nakanishi S. Pathway-specific control of reward learning and its flexibility via selective dopamine receptors in the nucleus accumbens. Proc Natl Acad Sci USA. 2012;109:12764–9.

    Article  CAS  PubMed  PubMed Central  ADS  Google Scholar 

  16. Bamford NS, Wightman RM, Sulzer D. Dopamine’s effects on corticostriatal synapses during reward-based behaviors. Neuron. 2018;97:494–510.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  17. Kim HR, Malik AN, Mikhael JG, Bech P, Tsutsui-Kimura I, Sun F, et al. A unified framework for dopamine signals across timescales. Cell. 2020;183:1600–16.e25.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  18. Cui Q, Li Q, Geng H, Chen L, Ip NY, Ke Y, et al. Dopamine receptors mediate strategy abandoning via modulation of a specific prelimbic cortex-nucleus accumbens pathway in mice. Proc Natl Acad Sci USA. 2018;115:E4890–E9.

    Article  CAS  PubMed  PubMed Central  ADS  Google Scholar 

  19. Burguiere E, Monteiro P, Mallet L, Feng G, Graybiel AM. Striatal circuits, habits, and implications for obsessive-compulsive disorder. Curr Opin Neurobiol. 2015;30:59–65.

    Article  CAS  PubMed  Google Scholar 

  20. Welch JM, Lu J, Rodriguiz RM, Trotta NC, Peca J, Ding JD, et al. Cortico-striatal synaptic defects and OCD-like behaviours in Sapap3-mutant mice. Nature. 2007;448:894–900.

    Article  CAS  PubMed  PubMed Central  ADS  Google Scholar 

  21. Jamain S, Quach H, Betancur C, Rastam M, Colineaux C, Gillberg IC, et al. Mutations of the X-linked genes encoding neuroligins NLGN3 and NLGN4 are associated with autism. Nat Genet. 2003;34:27–9.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  22. Rothwell PE, Fuccillo MV, Maxeiner S, Hayton SJ, Gokce O, Lim BK, et al. Autism-associated neuroligin-3 mutations commonly impair striatal circuits to boost repetitive behaviors. Cell. 2014;158:198–212.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  23. Cao W, Lin S, Xia QQ, Du YL, Yang Q, Zhang MY, et al. Gamma oscillation dysfunction in mPFC leads to social deficits in neuroligin 3 R451C knockin mice. Neuron. 2018;98:670.

    Article  CAS  PubMed  Google Scholar 

  24. Norris RHC, Churilov L, Hannan AJ, Nithianantharajah J. Mutations in neuroligin-3 in male mice impact behavioral flexibility but not relational memory in a touchscreen test of visual transitive inference. Mol Autism. 2019;10:42.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Burrows EL, May C, Hill T, Churliov L, Johnson KA, Hannan AJ. Mice with an autism-associated R451C mutation in neuroligin-3 show a cautious but accurate response style in touchscreen attention tasks. Genes Brain Behav. 2022;21:e12757.

    Article  CAS  PubMed  Google Scholar 

  26. Wise RA. Dopamine, learning and motivation. Nat Rev Neurosci. 2004;5:483–94.

    Article  CAS  PubMed  Google Scholar 

  27. Terra H, Bruinsma B, de Kloet SF, van der Roest M, Pattij T, Mansvelder HD. Prefrontal cortical projection neurons targeting dorsomedial striatum control behavioral inhibition. Curr Biol. 2020;30:4188–200.e5.

    Article  CAS  PubMed  Google Scholar 

  28. Spellman T, Svei M, Kaminsky J, Manzano-Nieves G, Liston C. Prefrontal deep projection neurons enable cognitive flexibility via persistent feedback monitoring. Cell. 2021;184:2750–66.e17.

    Article  CAS  PubMed  Google Scholar 

  29. Silverman JL, Yang M, Lord C, Crawley JN. Behavioural phenotyping assays for mouse models of autism. Nat Rev Neurosci. 2010;11:490–502.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  30. Peca J, Feliciano C, Ting JT, Wang W, Wells MF, Venkatraman TN, et al. Shank3 mutant mice display autistic-like behaviours and striatal dysfunction. Nature. 2011;472:437–42.

    Article  CAS  PubMed  PubMed Central  ADS  Google Scholar 

  31. McFarlane HG, Kusek GK, Yang M, Phoenix JL, Bolivar VJ, Crawley JN. Autism-like behavioral phenotypes in BTBR T+tf/J mice. Genes Brain Behav. 2008;7:152–63.

    Article  CAS  PubMed  Google Scholar 

  32. Radke AK, Kocharian A, Covey DP, Lovinger DM, Cheer JF, Mateo Y, et al. Contributions of nucleus accumbens dopamine to cognitive flexibility. Eur J Neurosci. 2019;50:2023–35.

    Article  PubMed  Google Scholar 

  33. Korn C, Akam T, Jensen KHR, Vagnoni C, Huber A, Tunbridge EM, et al. Distinct roles for dopamine clearance mechanisms in regulating behavioral flexibility. Mol Psychiatry. 2021;26:7188–99.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  34. Zessen Rv, Flores J, Eekel T, Reijen Svd, Lodder B, Omrani A, et al. Cue and reward evoked dopamine activity is necessary for maintaining learned Pavlovian associations. J Neurosci. 2021;41:5004–14.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Fischbach S, Janak PH. Decreases in cued reward seeking after reward-paired inhibition of mesolimbic dopamine. Neuroscience. 2019;412:259–69.

    Article  CAS  PubMed  Google Scholar 

  36. Rudebeck PH, Saunders RC, Prescott AT, Chau LS, Murray EA. Prefrontal mechanisms of behavioral flexibility, emotion regulation and value updating. Nat Neurosci. 2013;16:1140–5.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  37. Laurent V, Balleine BW. Factual and counterfactual action-outcome mappings control choice between goal-directed actions in rats. Curr Biol. 2015;25:1074–9.

    Article  CAS  PubMed  Google Scholar 

  38. Bari BA, Grossman CD, Lubin EE, Rajagopalan AE, Cressy JI, Cohen JY. Stable representations of decision variables for flexible behavior. Neuron. 2019;103:922–33.e7.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  39. Cox J, Witten IB. Striatal circuits for reward learning and decision-making. Nat Rev Neurosci. 2019;20:482–94.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  40. Floresco SB. The nucleus accumbens: an interface between cognition, emotion, and action. Annu Rev Psychol. 2015;66:25–52.

    Article  PubMed  Google Scholar 

  41. Atallah HE, McCool AD, Howe MW, Graybiel AM. Neurons in the ventral striatum exhibit cell-type-specific representations of outcome during learning. Neuron. 2014;82:1145–56.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  42. Soares-Cunha C, de Vasconcelos NAP, Coimbra B, Domingues AV, Silva JM, Loureiro-Campos E, et al. Nucleus accumbens medium spiny neurons subtypes signal both reward and aversion. Mol Psychiatry. 2020;25:3241–55.

    Article  CAS  PubMed  Google Scholar 

  43. Owen SF, Berke JD, Kreitzer AC. Fast-spiking interneurons supply feedforward control of bursting, calcium, and plasticity for efficient learning. Cell. 2018;172:683–95.e15.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  44. Owesson-White C, Belle AM, Herr NR, Peele JL, Gowrishankar P, Carelli RM, et al. Cue-evoked dopamine release rapidly modulates D2 neurons in the nucleus accumbens during motivated behavior. J Neurosci. 2016;36:6011–21.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  45. Golden SA, Jin M, Heins C, Venniro M, Michaelides M, Shaham Y. Nucleus accumbens Drd1-expressing neurons control aggression self-administration and aggression seeking in mice. J Neurosci. 2019;39:2482–96.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  46. Floresco SB, Yang CR, Phillips AG, Blaha CD. Basolateral amygdala stimulation evokes glutamate receptor-dependent dopamine efflux in the nucleus accumbens of the anaesthetized rat. Eur J Neurosci. 1998;10:1241–51.

    Article  CAS  PubMed  Google Scholar 

  47. Hosie S, Malone DT, Liu S, Glass M, Adlard PA, Hannan AJ, et al. Altered amygdala excitation and CB1 receptor modulation of aggressive behavior in the neuroligin-3(R451C) mouse model of autism. Front Cell Neurosci. 2018;12:234.

    Article  PubMed  PubMed Central  Google Scholar 

  48. Motiwala A, Soares S, Atallah BV, Paton JJ, Machens CK. Efficient coding of cognitive variables underlies dopamine response and choice behavior. Nat Neurosci. 2022;25:738–48.

    Article  CAS  PubMed  Google Scholar 

  49. Dabney W, Kurth-Nelson Z, Uchida N, Starkweather CK, Hassabis D, Munos R, et al. A distributional code for value in dopamine-based reinforcement learning. Nature. 2020;577:671–5.

    Article  CAS  PubMed  PubMed Central  ADS  Google Scholar 

  50. Cao W, Li JH, Lin S, Xia QQ, Du YL, Yang Q, et al. NMDA receptor hypofunction underlies deficits in parvalbumin interneurons and social behavior in neuroligin 3 R451C knockin mice. Cell Rep. 2022;41:111771.

    Article  CAS  PubMed  Google Scholar 

  51. Verharen JPH, Adan RAH, Vanderschuren L. Differential contributions of striatal dopamine D1 and D2 receptors to component processes of value-based decision making. Neuropsychopharmacology. 2019;44:2195–204.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  52. Wang X, Qiao Y, Dai Z, Sui N, Shen F, Zhang J, et al. Medium spiny neurons of the anterior dorsomedial striatum mediate reversal learning in a cell-type-dependent manner. Brain Struct Funct. 2019;224:419–34.

    Article  CAS  PubMed  Google Scholar 

  53. Martianova E, Aronson S, Proulx CD. Multi-fiber photometry to record neural activity in freely-moving animals. J Vis Exp. 2019. https://doi.org/10.3791/60278.

  54. Hamid AA, Pettibone JR, Mabrouk OS, Hetrick VL, Schmidt R, Vander Weele CM, et al. Mesolimbic dopamine signals the value of work. Nat Neurosci. 2016;19:117–26.

    Article  CAS  PubMed  Google Scholar 

  55. Schmitt LI, Wimmer RD, Nakajima M, Happ M, Mofakham S, Halassa MM. Thalamic amplification of cortical connectivity sustains attentional control. Nature. 2017;545:219–23.

    Article  CAS  PubMed  PubMed Central  ADS  Google Scholar 

  56. Ahmadi N, Constandinou TG, Bouganis CS. Estimation of neuronal firing rate using Bayesian Adaptive Kernel Smoother (BAKS). PLoS ONE. 2018;13:e0206794.

    Article  PubMed  PubMed Central  Google Scholar 

  57. Park IM, Meister ML, Huk AC, Pillow JW. Encoding and decoding in parietal cortex during sensorimotor decision-making. Nat Neurosci. 2014;17:1395–403.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  58. Lak A, Okun M, Moss MM, Gurnani H, Farrell K, Wells MJ, et al. Dopaminergic and prefrontal basis of learning from sensory confidence and reward value. Neuron. 2020;105:700–11.e6.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  59. Meyers EM. The neural decoding toolbox. Front Neuroinform. 2013;7:8.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

This work was supported by grants from the National Natural Science Foundation of China (3192010300 and U22A20306 to JHL, 31970902 to JX), Key R&D Program of Zhejiang Province (2024C03150 to JHL), Key-Area Research and Development Program of Guangdong Province (2019B030335001 to JHL), and Autism Research Special Fund of Zhejiang Foundation for Disabled Persons (2022003 to JX). We want to thank Drs. Xiang Yu and Li Yi (Peking University), and Bo Li (Xihu University) for their helpful comments.

Author information

Author notes

  1. Shen Lin

    Present address: Fujian Provincial Institutes of Brain Disorders and Brain Sciences, First Affiliated Hospital, Fujian Medical University, Fuzhou, China

  2. These authors contributed equally: Shen Lin, Cui-ying Fan, Hao-ran Wang.

Authors and Affiliations

  1. Affiliated Mental Health Center & Hangzhou Seventh People’s Hospital, School of Brain Science and Brain Medicine, Zhejiang University School of Medicine, Hangzhou, China

    Shen Lin, Cui-ying Fan, Hao-ran Wang, Xiao-fan Li, Jia-li Zeng, Pei-xuan Lan, Hui-xian Li, Junyu Xu & Jian-hong Luo

  2. Nanhu Brain-Computer Interface Institute, Hangzhou, China

    Hao-ran Wang & Jian-hong Luo

  3. Liangzhu Laboratory, MOE Frontier Science Center for Brain Science and Brain-Machine Integration, State Key Laboratory of Brain-machine Intelligence, Zhejiang University, Hangzhou, China

    Hao-ran Wang, Junyu Xu & Jian-hong Luo

  4. Key Laboratory of Novel Targets and Drug Study for Neural Repair of Zhejiang Province, School of Medicine, Hangzhou City University, Hangzhou, China

    Bin Zhang

  5. Institute for Brain Research and Rehabilitation, Key Laboratory of Brain Cognition and Education Sciences of Ministry of Education, South China Normal University, Guangzhou, China

    Chun Hu

  6. NHC and CAMS Key Laboratory of Medical Neurobiology, Zhejiang University, Hangzhou, China

    Junyu Xu & Jian-hong Luo

Authors
  1. Shen Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Cui-ying Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hao-ran Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiao-fan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jia-li Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Pei-xuan Lan
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Hui-xian Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Bin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Chun Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Junyu Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Jian-hong Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

SL, JX, and J-hL conceived the study and coordinated the experiments. SL designed experiments, performed the behavioral tests, and analyzed the electrophysiological data. C-yF validated viral tools, performed biochemical experiments, and contributed to behavioral training and fiber photometry recording. H-rW conducted the electrophysiological recordings and analyzed behavioral and electrophysiological data. X-fL, J-lZ, P-xL, H-xL, BZ, and HC performed some experiments. SL, JX, and J-hL wrote the manuscript. All authors provided comments and feedback on the manuscript.

Corresponding authors

Correspondence to Shen Lin, Junyu Xu or Jian-hong Luo.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lin, S., Fan, Cy., Wang, Hr. et al. Frontostriatal circuit dysfunction leads to cognitive inflexibility in neuroligin-3 R451C knockin mice. Mol Psychiatry (2024). https://doi.org/10.1038/s41380-024-02505-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1038/s41380-024-02505-9

Search

Advanced search

Quick links