Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Dopamine restores reward prediction errors in old age

A Corrigendum to this article was published on 21 November 2014

This article has been updated

Abstract

Senescence affects the ability to utilize information about the likelihood of rewards for optimal decision-making. Using functional magnetic resonance imaging in humans, we found that healthy older adults had an abnormal signature of expected value, resulting in an incomplete reward prediction error (RPE) signal in the nucleus accumbens, a brain region that receives rich input projections from substantia nigra/ventral tegmental area (SN/VTA) dopaminergic neurons. Structural connectivity between SN/VTA and striatum, measured by diffusion tensor imaging, was tightly coupled to inter-individual differences in the expression of this expected reward value signal. The dopamine precursor levodopa (L-DOPA) increased the task-based learning rate and task performance in some older adults to the level of young adults. This drug effect was linked to restoration of a canonical neural RPE. Our results identify a neurochemical signature underlying abnormal reward processing in older adults and indicate that this can be modulated by L-DOPA.

This is a preview of subscription content, access via your institution

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Two-armed bandit task design and performance in young and older adults.
Figure 2: Reinforcement learning model and behavior.
Figure 3: Reward prediction in the nucleus accumbens in 32 older adults.
Figure 4: Nigro-striatal tract connectivity strength and functional prediction errors.

Similar content being viewed by others

Change history

  • 22 May 2013

    In the version of this article initially published, an affiliation for author Quentin Huys read Translational Neuroimaging Unit. The correct name is Translational Neuromodeling Unit. The error has been corrected in the HTML and PDF versions of the article.

References

  1. Eppinger, B., Hämmerer, D. & Li, S.-C. Neuromodulation of reward-based learning and decision making in human aging. Ann. NY Acad. Sci. 1235, 1–17 (2011).

    Article  PubMed  Google Scholar 

  2. Mell, T. et al. Effect of aging on stimulus-reward association learning. Neuropsychologia 43, 554–563 (2005).

    Article  CAS  PubMed  Google Scholar 

  3. Schultz, W., Dayan, P. & Montague, P.R. A neural substrate of prediction and reward. Science 275, 1593–1599 (1997).

    Article  CAS  PubMed  Google Scholar 

  4. Salamone, J.D., Correa, M., Mingote, S.M. & Weber, S.M. Beyond the reward hypothesis: alternative functions of nucleus accumbens dopamine. Curr. Opin. Pharmacol. 5, 34–41 (2005).

    Article  CAS  PubMed  Google Scholar 

  5. Haber, S.N., Fudge, J.L. & McFarland, N.R. Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum. J. Neurosci. 20, 2369–2382 (2000).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  6. O'Doherty, J. et al. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304, 452–454 (2004).

    Article  CAS  PubMed  Google Scholar 

  7. O'Doherty, J.P., Dayan, P., Friston, K., Critchley, H. & Dolan, R.J. Temporal difference models and reward-related learning in the human brain. Neuron 38, 329–337 (2003).

    Article  CAS  PubMed  Google Scholar 

  8. Daw, N.D., O'Doherty, J.P., Dayan, P., Seymour, B. & Dolan, R.J. Cortical substrates for exploratory decisions in humans. Nature 441, 876–879 (2006).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  9. Knutson, B. & Gibbs, S. Linking nucleus accumbens dopamine and blood oxygenation. Psychopharmacology (Berl.) 191, 813–822 (2007).

    Article  CAS  Google Scholar 

  10. Pessiglione, M., Seymour, B., Flandin, G., Dolan, R.J. & Frith, C.D. Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans. Nature 442, 1042–1045 (2006).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  11. Rutledge, R.B. et al. Dopaminergic drugs modulate learning rates and perseveration in parkinson's patients in a dynamic foraging task. J. Neurosci. 29, 15104–15114 (2009).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  12. Bäckman, L., Nyberg, L., Lindenberger, U., Li, S.-C. & Farde, L. The correlative triad among aging, dopamine and cognition: current status and future prospects. Neurosci. Biobehav. Rev. 30, 791–807 (2006).

    Article  CAS  PubMed  Google Scholar 

  13. Düzel, E., Bunzeck, N., Guitart-Masip, M. & Duzel, S. Novelty-related motivation of anticipation and exploration by dopamine (NOMAD): implications for healthy aging. Neurosci. Biobehav. Rev. 34, 660–669 (2010).

    Article  PubMed  CAS  Google Scholar 

  14. Fearnley, J.M. & Lees, A.J. Ageing and Parkinson's disease: substantia nigra regional selectivity. Brain 114, 2283–2301 (1991).

    Article  PubMed  Google Scholar 

  15. Vaillancourt, D.E., Spraker, M.B., Prodoehl, J., Zhou, X.J. & Little, D.M. Effects of aging on the ventral and dorsal substantia nigra using diffusion tensor imaging. Neurobiol. Aging 33, 35–42 (2012).

    Article  PubMed  Google Scholar 

  16. Samanez-Larkin, G.R., Kuhnen, C.M., Yoo, D.J. & Knutson, B. Variability in nucleus accumbens activity mediates age-related suboptimal financial risk taking. J. Neurosci. 30, 1426–1434 (2010).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  17. Schott, B.H. et al. Ageing and early-stage Parkinson's disease affect separable neural mechanisms of mesolimbic reward processing. Brain 130, 2412–2424 (2007).

    Article  PubMed  Google Scholar 

  18. Cox, K.M., Aizenstein, H.J. & Fiez, J.A. Striatal outcome processing in healthy aging. Cogn. Affect. Behav. Neurosci. 8, 304–317 (2008).

    Article  PubMed  PubMed Central  Google Scholar 

  19. Behrens, T.E.J., Hunt, L.T., Woolrich, M.W. & Rushworth, M.F.S. Associative learning of social value. Nature 456, 245–249 (2008).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  20. Cools, R. & D'Esposito, M. Inverted U-shaped dopamine actions on human working memory and cognitive control. Biol. Psychiatry 69, e113–e125 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  21. Li, J. & Daw, N.D. Signals in human striatum are appropriate for policy update rather than value prediction. J. Neurosci. 31, 5504–5511 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  22. Eppinger, B., Kray, J., Mock, B. & Mecklinger, A. Better or worse than expected? Aging, learning and the ERN. Neuropsychologia 46, 521–539 (2008).

    Article  PubMed  Google Scholar 

  23. Samanez-Larkin, G.R., Wagner, A.D. & Knutson, B. Expected value information improves financial risk taking across the adult life span. Soc. Cogn. Affect. Neurosci. 6, 207–217 (2011).

    Article  PubMed  Google Scholar 

  24. Guitart-Masip, M. et al. Go and no-go learning in reward and punishment: interactions between affect and effect. Neuroimage 62, 154–166 (2012).

    Article  PubMed  Google Scholar 

  25. Dickinson, A. & Balleine, B. The Role of Learning in the Operation of Motivational Systems (John Wiley & Sons, New York, 2002).

  26. Dickinson, A., Smith, J. & Mirenowicz, J. Dissociation of Pavlovian and instrumental incentive learning under dopamine antagonists. Behav. Neurosci. 114, 468–483 (2000).

    Article  CAS  PubMed  Google Scholar 

  27. Worthy, D.A., Gorlick, M.A., Pacheco, J.L., Schnyer, D.M. & Maddox, W.T. With age comes wisdom. Psychol. Sci. 22, 1375–1380 (2011).

    Article  PubMed  Google Scholar 

  28. de Wit, S. et al. Reliance on habits at the expense of goal-directed control following dopamine precursor depletion. Psychopharmacology (Berl.) 219, 621–631 (2012).

    Article  CAS  Google Scholar 

  29. Wunderlich, K., Smittenaar, P. & Dolan, R.J. Dopamine enhances model-based over model-free choice behavior. Neuron 75, 418–424 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  30. Gläscher, J., Daw, N., Dayan, P. & O'Doherty, J.P. States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron 66, 585–595 (2010).

    PubMed  PubMed Central  Google Scholar 

  31. Jocham, G., Klein, T.A. & Ullsperger, M. Dopamine-mediated reinforcement learning signals in the striatum and ventromedial prefrontal cortex underlie value-based choices. J. Neurosci. 31, 1606–1613 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  32. Shohamy, D. & Adcock, R.A. Dopamine and adaptive memory. Trends Cogn. Sci. 14, 464–472 (2010).

    Article  CAS  PubMed  Google Scholar 

  33. Nicola, S.M., Surmeier, D.J. & Malenka, R.C. Dopaminergic modulation of neuronal excitability in the striatum and nucleus accumbens. Annu. Rev. Neurosci. 23, 185–215 (2000).

    Article  CAS  PubMed  Google Scholar 

  34. Daw, N.D. & Gershman, S.J. Seymour, B., Dayan, P. & Dolan, R.J. Model-based influences on humans' choices and striatal prediction errors. Neuron 69, 1204–1215 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  35. Simon, D.A. & Daw, N.D. Neural correlates of forward planning in a spatial decision task in humans. J. Neurosci. 31, 5526–5539 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  36. Guitart-Masip, M. et al. Action controls dopaminergic enhancement of reward representations. Proc. Natl. Acad. Sci. USA 109, 7511–7516 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  37. Choi, W.Y., Balsam, P.D. & Horvitz, J.C. Extended habit training reduces dopamine mediation of appetitive response expression. J. Neurosci. 25, 6729–6733 (2005).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  38. Klein-Flügge, M.C., Hunt, L.T., Bach, D.R., Dolan, R.J. & Behrens, T.E.J. Dissociable reward and timing signals in human midbrain and ventral striatum. Neuron 72, 654–664 (2011).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  39. Coxon, J.P., Van Impe, A., Wenderoth, N. & Swinnen, S.P. Aging and inhibitory control of action: cortico-subthalamic connection strength predicts stopping performance. J. Neurosci. 32, 8401–8412 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  40. Forstmann, B.U. et al. Cortico-subthalamic white matter tract strength predicts interindividual efficacy in stopping a motor response. Neuroimage 60, 370–375 (2012).

    Article  PubMed  Google Scholar 

  41. Le Bihan, D. & Johansen-Berg, H. Diffusion MRI at 25: exploring brain tissue structure and function. Neuroimage 61, 324–341 (2012).

    Article  PubMed  Google Scholar 

  42. Draganski, B. et al. Regional specificity of MRI contrast parameter changes in normal ageing revealed by voxel-based quantification (VBQ). Neuroimage 55, 1423–1434 (2011).

    Article  CAS  PubMed  Google Scholar 

  43. Li, S.-C. & Sikström, S. Integrative neurocomputational perspectives on cognitive aging, neuromodulation, and representation. Neurosci. Biobehav. Rev. 26, 795–808 (2002).

    Article  PubMed  Google Scholar 

  44. Schönberg, T., Daw, N.D., Joel, D. & O'Doherty, J.P. Reinforcement learning signals in the human striatum distinguish learners from nonlearners during reward-based decision making. J. Neurosci. 27, 12860–12867 (2007).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  45. Huys, Q.J.M. et al. Disentangling the roles of approach, activation and valence in instrumental and Pavlovian responding. PLoS Comput. Biol. 7, e1002028 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  46. Lutti, A., Hutton, C., Finsterbusch, J., Helms, G. & Weiskopf, N. Optimization and validation of methods for mapping of the radiofrequency transmit field at 3T. Magn. Reson. Med. 64, 229–238 (2010).

    Article  PubMed  PubMed Central  Google Scholar 

  47. Reese, T.G., Heid, O., Weisskoff, R.M. & Wedeen, V.J. Reduction of eddy-current-induced distortion in diffusion MRI using a twice-refocused spin echo. Magn. Reson. Med. 49, 177–182 (2003).

    Article  CAS  PubMed  Google Scholar 

  48. Nagy, Z., Weiskopf, N., Alexander, D.C. & Deichmann, R. A method for improving the performance of gradient systems for diffusion-weighted MRI. Magn. Reson. Med. 58, 763–768 (2007).

    Article  PubMed  PubMed Central  Google Scholar 

  49. Andersson, J.L.R., Skare, S. & Ashburner, J. How to correct susceptibility distortions in spin-echo echo-planar images: application to diffusion tensor imaging. Neuroimage 20, 870–888 (2003).

    Article  PubMed  Google Scholar 

  50. Ashburner, J. A fast diffeomorphic image registration algorithm. Neuroimage 38, 95–113 (2007).

    Article  PubMed  Google Scholar 

  51. Knutson, B. & Cooper, J.C. Functional magnetic resonance imaging of reward prediction. Curr. Opin. Neurol. 18, 411–417 (2005).

    Article  PubMed  Google Scholar 

  52. Brett, M., Anton, J.-L., Valabregue, R. & Poline, J.-B. Region of interest analysis using an SPM toolbox. Neuroimage 16, 1140–1141 (2002).

    Google Scholar 

  53. Forstmann, B.U. et al. The speed-accuracy tradeoff in the elderly brain: a structural model-based approach. J. Neurosci. 31, 17242–17249 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  54. Schwarzkopf, D.S., de Haas, B. & Rees, G. Better ways to improve standards in brain-behavior correlation analysis. Front. Hum. Neurosci. 6, 200 (2012).

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

We thank J. Medhora and L. Sasse for their assistance with data collection, and H. Barron and M. Klein-Flügge for their assistance with time course analyses. R.C. is supported by a Wellcome Trust Research Training Fellowship (WT088286MA). R.J.D. is supported by the Wellcome Trust (grant number 078865/Z/05/Z). The Wellcome Trust Centre for Neuroimaging is supported by core funding from the Wellcome Trust (091593/Z/10/Z).

Author information

Authors and Affiliations

Authors

Contributions

R.C. and M.G.-M. conducted the experiment, analyzed the data and prepared the manuscript. C.L., P.D., Q.H. and E.D. contributed to data analysis and manuscript preparation. R.J.D. contributed to data analysis and manuscript preparation and supervised the project.

Corresponding author

Correspondence to Rumana Chowdhury.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Supplementary information

Supplementary Text and Figures

Supplementary Figures 1–7 and Supplementary Tables 1–6 (PDF 4051 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Chowdhury, R., Guitart-Masip, M., Lambert, C. et al. Dopamine restores reward prediction errors in old age. Nat Neurosci 16, 648–653 (2013). https://doi.org/10.1038/nn.3364

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn.3364

This article is cited by

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing