Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Neurons in human pre-supplementary motor area encode key computations for value-based choice

Abstract

Adaptive behaviour in real-world environments requires that choices integrate several variables, including the novelty of the options under consideration, their expected value and uncertainty in value estimation. Here, to probe how integration over decision variables occurs during decision-making, we recorded neurons from the human pre-supplementary motor area (preSMA), ventromedial prefrontal cortex and dorsal anterior cingulate. Unlike the other areas, preSMA neurons not only represented separate pre-decision variables for each choice option but also encoded an integrated utility signal for each choice option and, subsequently, the decision itself. Post-decision encoding of variables for the chosen option was more widely distributed and especially prominent in the ventromedial prefrontal cortex. Our findings position the human preSMA as central to the implementation of value-based decisions.

This is a preview of subscription content, access via your institution

Access options

Buy this article

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Electrode positions, exploration task and behaviour.
Fig. 2: Encoding of action utility components in the preSMA and vmPFC.
Fig. 3: Neurons in the preSMA encode integrated utility.
Fig. 4: The PreSMA encodes decisions.
Fig. 5: Encoding selected stimulus properties.

Similar content being viewed by others

Data availability

Behavioural and neural data have been deposited in the OSF platform: https://osf.io/34b9f/?view_only=be3c529466fa444d8b97a2bab8951435.

Code availability

The code for data analysis can be found at: https://github.com/43technetium/casino_task_analysis.

References

  1. Sutton, R. S. & Barto, A. G. Reinforcement Learning: an Introduction (MIT Press, 2018).

  2. Payzan-LeNestour, E. & Bossaerts, P. Risk, unexpected uncertainty, and estimation uncertainty: Bayesian learning in unstable settings. PLoS Comput. Biol. 7, e1001048 (2011).

    CAS  PubMed  PubMed Central  Google Scholar 

  3. Payzan-LeNestour, E. & Bossaerts, P. Do not bet on the unknown versus try to find out more: estimation uncertainty and ‘unexpected uncertainty’ both modulate exploration. Front. Neurosci. 6, 150 (2012).

    PubMed  PubMed Central  Google Scholar 

  4. Gershman, S. J. Deconstructing the human algorithms for exploration. Cognition 173, 34–42 (2018).

    PubMed  Google Scholar 

  5. Wittmann, B. C., Daw, N. D., Seymour, B. & Dolan, R. J. Striatal activity underlies novelty-based choice in humans. Neuron 58, 967–973 (2008).

    CAS  PubMed  PubMed Central  Google Scholar 

  6. Cohen, J. D., McClure, S. M. & Yu, A. J. Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration. Philos. Trans. R. Soc. Lond. B Biol. Sci. 362, 933–942 (2007).

    PubMed  PubMed Central  Google Scholar 

  7. Wilson, R. C., Geana, A., White, J. M., Ludvig, E. A. & Cohen, J. D. Humans use directed and random exploration to solve the explore–exploit dilemma. J. Exp. Psychol. Gen. 143, 2074–2081 (2014).

    PubMed  PubMed Central  Google Scholar 

  8. Wallis, J. D. Orbitofrontal cortex and its contribution to decision-making. Annu. Rev. Neurosci. 30, 31–56 (2007).

    CAS  PubMed  Google Scholar 

  9. Padoa-Schioppa, C. & Cai, X. Orbitofrontal cortex and the computation of subjective value: consolidated concepts and new perspectives. Ann. N. Y. Acad. Sci. 1239, 130–137 (2011).

    PubMed  PubMed Central  Google Scholar 

  10. Grabenhorst, F. & Rolls, E. T. Value, pleasure and choice in the ventral prefrontal cortex. Trends Cogn. Sci. 15, 56–67 (2011).

    PubMed  Google Scholar 

  11. Cai, X. & Padoa-Schioppa, C. Neuronal encoding of subjective value in dorsal and ventral anterior cingulate cortex. J. Neurosci. 32, 3791–3808 (2012).

    CAS  PubMed  PubMed Central  Google Scholar 

  12. Strait, C. E., Blanchard, T. C. & Hayden, B. Y. Reward value comparison via mutual inhibition in ventromedial prefrontal cortex. Neuron 82, 1357–1366 (2014).

    CAS  PubMed  PubMed Central  Google Scholar 

  13. Rich, E. L. & Wallis, J. D. Decoding subjective decisions from orbitofrontal cortex. Nat. Neurosci. 19, 973–980 (2016).

    CAS  PubMed  PubMed Central  Google Scholar 

  14. Kepecs, A., Uchida, N., Zariwala, H. A. & Mainen, Z. F. Neural correlates, computation and behavioural impact of decision confidence. Nature 455, 227–231 (2008).

    CAS  PubMed  Google Scholar 

  15. O’Neill, M. & Schultz, W. Coding of reward risk by orbitofrontal neurons is mostly distinct from coding of reward value. Neuron 68, 789–800 (2010).

    PubMed  Google Scholar 

  16. Grabenhorst, F., Báez-Mendoza, R., Genest, W., Deco, G. & Schultz, W. Primate amygdala neurons simulate decision processes of social partners. Cell 177, 986–998 (2019).

    CAS  PubMed  PubMed Central  Google Scholar 

  17. Hirokawa, J., Vaughan, A., Masset, P., Ott, T. & Kepecs, A. Frontal cortex neuron types categorically encode single decision variables. Nature 576, 446–451 (2019).

    CAS  PubMed  PubMed Central  Google Scholar 

  18. Dias, R. & Honey, R. C. Involvement of the rat medial prefrontal cortex in novelty detection. Behav. Neurosci. 116, 498–503 (2002).

    CAS  PubMed  Google Scholar 

  19. Matsumoto, M., Matsumoto, K. & Tanaka, K. Effects of novelty on activity of lateral and medial prefrontal neurons. Neurosci. Res. 57, 268–276 (2007).

    PubMed  Google Scholar 

  20. Bourgeois, J.-P. et al. Modulation of the mouse prefrontal cortex activation by neuronal nicotinic receptors during novelty exploration but not by exploration of a familiar environment. Cereb. Cortex 22, 1007–1015 (2012).

    PubMed  Google Scholar 

  21. Chib, V. S., Rangel, A., Shimojo, S. & O’Doherty, J. P. Evidence for a common representation of decision values for dissimilar goods in human ventromedial prefrontal cortex. J. Neurosci. 29, 12315–12320 (2009).

    CAS  PubMed  PubMed Central  Google Scholar 

  22. Hare, T. A., Schultz, W., Camerer, C. F., O’Doherty, J. P. & Rangel, A. Transformation of stimulus value signals into motor commands during simple choice. Proc. Natl Acad. Sci. USA 108, 18120–18125 (2011).

    CAS  PubMed  PubMed Central  Google Scholar 

  23. Suzuki, S., Cross, L. & O’Doherty, J. P. Elucidating the underlying components of food valuation in the human orbitofrontal cortex. Nat. Neurosci. 20, 1780–1786 (2017).

    CAS  PubMed  PubMed Central  Google Scholar 

  24. Kobayashi, K. & Hsu, M. Common neural code for reward and information value. Proc. Natl Acad. Sci. USA 116, 13061–13066 (2019).

    CAS  PubMed  PubMed Central  Google Scholar 

  25. Walton, M. E., Devlin, J. T. & Rushworth, M. F. Interactions between decision making and performance monitoring within prefrontal cortex. Nat. Neurosci. 7, 1259–1265 (2004).

    CAS  PubMed  Google Scholar 

  26. Wunderlich, K., Rangel, A. & O’Doherty, J. P. Neural computations underlying action-based decision making in the human brain. Proc. Natl Acad. Sci. USA 106, 17199–17204 (2009).

    CAS  PubMed  PubMed Central  Google Scholar 

  27. Badre, D., Doll, B. B., Long, N. M. & Frank, M. J. Rostrolateral prefrontal cortex and individual differences in uncertainty-driven exploration. Neuron 73, 595–607 (2012).

    CAS  PubMed  PubMed Central  Google Scholar 

  28. Trudel, N. et al. Polarity of uncertainty representation during exploration and exploitation in ventromedial prefrontal cortex. Nat. Hum. Behav. 5, 83–98 (2021).

    PubMed  Google Scholar 

  29. Vassena, E., Krebs, R. M., Silvetti, M., Fias, W. & Verguts, T. Dissociating contributions of ACC and vmPFC in reward prediction, outcome, and choice. Neuropsychologia 59, 112–123 (2014).

    PubMed  Google Scholar 

  30. Horvitz, J. C., Stewart, T. & Jacobs, B. L. Burst activity of ventral tegmental dopamine neurons is elicited by sensory stimuli in the awake cat. Brain Res. 759, 251–258 (1997).

    CAS  PubMed  Google Scholar 

  31. Krebs, R. M., Schott, B. H., Schütze, H. & Düzel, E. The novelty exploration bonus and its attentional modulation. Neuropsychologia 47, 2272–2281 (2009).

    PubMed  Google Scholar 

  32. Kamiński, J. et al. Novelty-sensitive dopaminergic neurons in the human substantia nigra predict success of declarative memory formation. Curr. Biol. 28, 1333–1343 (2018).

    PubMed  PubMed Central  Google Scholar 

  33. Saez, I. et al. Encoding of multiple reward-related computations in transient and sustained high-frequency activity in human OFC. Curr. Biol. 28, 2889–2899 (2018).

    CAS  PubMed  PubMed Central  Google Scholar 

  34. Domenech, P., Rheims, S. & Koechlin, E. Neural mechanisms resolving exploitation–exploration dilemmas in the medial prefrontal cortex. Science 369, eabb0184 (2020).

    CAS  PubMed  Google Scholar 

  35. Nachev, P., Kennard, C. & Husain, M. Functional role of the supplementary and pre-supplementary motor areas. Nat. Rev. Neurosci. 9, 856–869 (2008).

    CAS  PubMed  Google Scholar 

  36. Passingham, R. E. & Wise, S. P. The Neurobiology of the Prefrontal Cortex: Anatomy, Evolution, and the Origin of Insight (Oxford Univ. Press, 2012).

  37. Fu, Z. et al. The geometry of domain-general performance monitoring in the human medial frontal cortex. Science 376, eabm9922 (2022).

    CAS  PubMed  PubMed Central  Google Scholar 

  38. Kamiński, J. et al. Persistently active neurons in human medial frontal and medial temporal lobe support working memory. Nat. Neurosci. 20, 590–601 (2017).

    PubMed  PubMed Central  Google Scholar 

  39. Cockburn, J., Man, V., Cunningham, W. A. & O’Doherty, J. P. Novelty and uncertainty regulate the balance between exploration and exploitation through distinct mechanisms in the human brain. Neuron 110, 2691–2702 (2022).

    Google Scholar 

  40. Gittins, J. C. & Jones, D. M. in Progress in Statistics. (J. Gani, ed.) 241–266 (North-Holland, 1974).

  41. Niño-Mora, J. Computing a classic index for finite-horizon bandits. INFORMS J. Comput. 23, 254–267 (2011).

    Google Scholar 

  42. Carpentier, A., Lazaric, A., Ghavamzadeh, M., Munos, R. & Auer, P. Upper-confidence-bound algorithms for active learning in multi-armed bandits. In Proc. International Conference on Algorithmic Learning Theory. 189–203 (Springer, 2011).

  43. Piray, P., Dezfouli, A., Heskes, T., Frank, M. J. & Daw, N. D. Hierarchical Bayesian inference for concurrent model fitting and comparison for group studies. PLoS Comput. Biol. 15, e1007043 (2019).

    CAS  PubMed  PubMed Central  Google Scholar 

  44. Hanes, D. P., Thompson, K. G. & Schall, J. D. Relationship of presaccadic activity in frontal eye field and supplementary eye field to saccade initiation in macaque: Poisson spike train analysis. Exp. Brain Res. 103, 85–96 (1995).

    CAS  PubMed  Google Scholar 

  45. Wang, A. Y., Miura, K. & Uchida, N. The dorsomedial striatum encodes net expected return, critical for energizing performance vigor. Nat. Neurosci. 16, 639–647 (2013).

    CAS  PubMed  PubMed Central  Google Scholar 

  46. Kobak, D. et al. Demixed principal component analysis of neural population data. eLife 5, e10989 (2016).

    PubMed  PubMed Central  Google Scholar 

  47. Fu, Z. et al. Single-neuron correlates of error monitoring and post-error adjustments in human medial frontal cortex. Neuron 101, 165–177 (2019).

    CAS  PubMed  Google Scholar 

  48. Goñi, J. et al. The neural substrate and functional integration of uncertainty in decision making: an information theory approach. PLoS ONE 6, e17408 (2011).

    PubMed  PubMed Central  Google Scholar 

  49. Rushworth, M. F., Kolling, N., Sallet, J. & Mars, R. B. Valuation and decision-making in frontal cortex: one or many serial or parallel systems? Curr. Opin. Neurobiol. 22, 946–955 (2012).

    CAS  PubMed  Google Scholar 

  50. Li, Y., Vanni-Mercier, G., Isnard, J., Mauguière, F. & Dreher, J.-C. The neural dynamics of reward value and risk coding in the human orbitofrontal cortex. Brain 139, 1295–1309 (2016).

    PubMed  Google Scholar 

  51. Hunt, L. T. et al. Triple dissociation of attention and decision computations across prefrontal cortex. Nat. Neurosci. 21, 1471–1481 (2018).

    CAS  PubMed  PubMed Central  Google Scholar 

  52. Averbeck, B. & O’Doherty, J. P. Reinforcement-learning in fronto-striatal circuits. Neuropsychopharmacology 47, 147–162 (2022).

    PubMed  Google Scholar 

  53. Fried, I., Mukamel, R. & Kreiman, G. Internally generated preactivation of single neurons in human medial frontal cortex predicts volition. Neuron 69, 548–562 (2011).

    CAS  PubMed  PubMed Central  Google Scholar 

  54. Fried, I. Neurons as will and representation. Nat. Rev. Neurosci. 23, 104–114 (2022).

    CAS  PubMed  Google Scholar 

  55. Minxha, J., Adolphs, R., Fusi, S., Mamelak, A. N. & Rutishauser, U. Flexible recruitment of memory-based choice representations by the human medial frontal cortex. Science 368, eaba3313 (2020).

    CAS  PubMed  PubMed Central  Google Scholar 

  56. Gazit, T. et al. The role of mPFC and MTL neurons in human choice under goal-conflict. Nat. Commun. 11, 3192 (2020).

    CAS  PubMed  PubMed Central  Google Scholar 

  57. Bonini, F. et al. Action monitoring and medial frontal cortex: leading role of supplementary motor area. Science 343, 888–891 (2014).

    CAS  PubMed  Google Scholar 

  58. Kim, J.-N. & Shadlen, M. N. Neural correlates of a decision in the dorsolateral prefrontal cortex of the macaque. Nat. Neurosci. 2, 176–185 (1999).

    PubMed  Google Scholar 

  59. Nambu, A., Tokuno, H. & Takada, M. Functional significance of the cortico–subthalamo–pallidal ‘hyperdirect’ pathway. Neurosci. Res. 43, 111–117 (2002).

    PubMed  Google Scholar 

  60. Haber, S. N. & Knutson, B. The reward circuit: linking primate anatomy and human imaging. Neuropsychopharmacology 35, 4–26 (2010).

    PubMed  Google Scholar 

  61. Ding, L. & Gold, J. I. Caudate encodes multiple computations for perceptual decisions. J. Neurosci. 30, 15747–15759 (2010).

    CAS  PubMed  PubMed Central  Google Scholar 

  62. Yartsev, M. M., Hanks, T. D., Yoon, A. M. & Brody, C. D. Causal contribution and dynamical encoding in the striatum during evidence accumulation. eLife 7, e34929 (2018).

    PubMed  PubMed Central  Google Scholar 

  63. Fan, Y., Gold, J. I. & Ding, L. Frontal eye field and caudate neurons make different contributions to reward-biased perceptual decisions. eLife 9, e60535 (2020).

    CAS  PubMed  PubMed Central  Google Scholar 

  64. Chen, W. et al. Prefrontal-subthalamic hyperdirect pathway modulates movement inhibition in humans. Neuron 106, 579–588 (2020).

    CAS  PubMed  PubMed Central  Google Scholar 

  65. Bartra, O., McGuire, J. T. & Kable, J. W. The valuation system: a coordinate-based meta-analysis of BOLD fMRI experiments examining neural correlates of subjective value. Neuroimage 76, 412–427 (2013).

    PubMed  Google Scholar 

  66. O’Doherty, J. P. The problem with value. Neurosci. Biobehav. Rev. 43, 259–268 (2014).

    PubMed  PubMed Central  Google Scholar 

  67. Wunderlich, K., Rangel, A. & O’Doherty, J. P. Economic choices can be made using only stimulus values. Proc. Natl Acad. Sci. USA 107, 15005–15010 (2010).

    CAS  PubMed  PubMed Central  Google Scholar 

  68. Walton, M. E., Behrens, T. E., Buckley, M. J., Rudebeck, P. H. & Rushworth, M. F. Separable learning systems in the macaque brain and the role of orbitofrontal cortex in contingent learning. Neuron 65, 927–939 (2010).

    CAS  PubMed  PubMed Central  Google Scholar 

  69. Noonan, M. P., Mars, R. B. & Rushworth, M. F. Distinct roles of three frontal cortical areas in reward-guided behavior. J. Neurosci. 31, 14399–14412 (2011).

    CAS  PubMed  PubMed Central  Google Scholar 

  70. Rudebeck, P. H. & Murray, E. A. Dissociable effects of subtotal lesions within the macaque orbital prefrontal cortex on reward-guided behavior. J. Neurosci. 31, 10569–10578 (2011).

    CAS  PubMed  PubMed Central  Google Scholar 

  71. Domenech, P. & Koechlin, E. Executive control and decision-making in the prefrontal cortex. Curr. Opin. Behav. Sci. 1, 101–106 (2015).

    Google Scholar 

  72. Murray, E. A. & Rudebeck, P. H. Specializations for reward-guided decision-making in the primate ventral prefrontal cortex. Nat. Rev. Neurosci. 19, 404–417 (2018).

    CAS  PubMed  PubMed Central  Google Scholar 

  73. Pratt, W. E. & Mizumori, S. J. Neurons in rat medial prefrontal cortex show anticipatory rate changes to predictable differential rewards in a spatial memory task. Behav. Brain Res. 123, 165–183 (2001).

    CAS  PubMed  Google Scholar 

  74. Gutierrez, R., Carmena, J. M., Nicolelis, M. A. & Simon, S. A. Orbitofrontal ensemble activity monitors licking and distinguishes among natural rewards. J. Neurophysiol. 95, 119–133 (2006).

    PubMed  Google Scholar 

  75. Horst, N. K. & Laubach, M. Reward-related activity in the medial prefrontal cortex is driven by consumption. Front. Neurosci. 7, 56 (2013).

    PubMed  PubMed Central  Google Scholar 

  76. Malvaez, M., Shieh, C., Murphy, M. D., Greenfield, V. Y. & Wassum, K. M. Distinct cortical–amygdala projections drive reward value encoding and retrieval. Nat. Neurosci. 22, 762–769 (2019).

    CAS  PubMed  PubMed Central  Google Scholar 

  77. Amiez, C., Joseph, J. P. & Procyk, E. Reward encoding in the monkey anterior cingulate cortex. Cereb. Cortex 16, 1040–1055 (2006).

    CAS  PubMed  Google Scholar 

  78. Matsumoto, M., Matsumoto, K., Abe, H. & Tanaka, K. Medial prefrontal cell activity signaling prediction errors of action values. Nat. Neurosci. 10, 647–656 (2007).

    CAS  PubMed  Google Scholar 

  79. Kennerley, S. W., Behrens, T. E. & Wallis, J. D. Double dissociation of value computations in orbitofrontal and anterior cingulate neurons. Nat. Neurosci. 14, 1581–1589 (2011).

    CAS  PubMed  PubMed Central  Google Scholar 

  80. Knudsen, E. B. & Wallis, J. D. Closed-loop theta stimulation in the orbitofrontal cortex prevents reward-based learning. Neuron 106, 537–547 (2020).

    CAS  PubMed  PubMed Central  Google Scholar 

  81. Hill, M. R., Boorman, E. D. & Fried, I. Observational learning computations in neurons of the human anterior cingulate cortex. Nat. Commun. 7, 12722 (2016).

    CAS  PubMed  PubMed Central  Google Scholar 

  82. Rescorla, R. & Wagner, A. A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In Classical Conditioning II: Current Theory and Research. (Black, A. H. & Prokasy, W. F., eds.) 64-99 (Appleton-Century-Crofts, 1972).

  83. Sutton, R. S. Learning to predict by the methods of temporal differences. Mach. Learn. 3, 9–44 (1988).

    Google Scholar 

  84. Rigoux, L., Stephan, K. E., Friston, K. J. & Daunizeau, J. Bayesian model selection for group studies—revisited. Neuroimage 84, 971–985 (2014).

    CAS  PubMed  Google Scholar 

  85. Rutishauser, U., Schuman, E. M. & Mamelak, A. N. Online detection and sorting of extracellularly recorded action potentials in human medial temporal lobe recordings, in vivo. J. Neurosci. Methods 154, 204–224 (2006).

    PubMed  Google Scholar 

  86. Elber-Dorozko, L. & Loewenstein, Y. Striatal action-value neurons reconsidered. eLife 7, e34248 (2018).

    PubMed  PubMed Central  Google Scholar 

  87. Harris, K. D. Nonsense correlations in neuroscience. Preprint at bioRxiv https://doi.org/10.1101/2020.11.29.402719 (2021).

  88. Jaccard, P. The distribution of the flora in the alpine zone. New Phytol. 11, 37–50 (1912).

    Google Scholar 

Download references

Acknowledgements

We thank the members of the O’Doherty and Rutishauser laboratories for discussions and feedback. We thank all participants and their families for their participation, and nurses and medical staff for their work. This work was supported by National Institutes of Health grant nos. R01DA040011 and R01MH111425 (to J.P.O.), R01MH110831 and U01NS117839 (to U.R.) and P50MH094258 (to J.P.O. and U.R.). The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.

Author information

Authors and Affiliations

Authors

Contributions

T.G.A., J.C., U.R. and J.P.O. designed the study. T.G.A. performed the experiments. T.G.A. and J.C. analysed the data. T.G.A., J.C., A.N.M., U.R. and J.P.O. wrote the paper. A.N.M. performed the surgery and supervised the clinical work.

Corresponding author

Correspondence to Tomas G. Aquino.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Human Behaviour thanks Camillo Padoa-Schioppa and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Tables 1–6 and Figs. 1–4.

Reporting Summary

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Aquino, T.G., Cockburn, J., Mamelak, A.N. et al. Neurons in human pre-supplementary motor area encode key computations for value-based choice. Nat Hum Behav 7, 970–985 (2023). https://doi.org/10.1038/s41562-023-01548-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41562-023-01548-2

This article is cited by

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing