Abstract
Termination of a painful or unpleasant event can be rewarding. However, whether the brain treats relief in a similar way as it treats natural reward is unclear, and the neural processes that underlie its representation as a motivational goal remain poorly understood. We used fMRI (functional magnetic resonance imaging) to investigate how humans learn to generate expectations of pain relief. Using a pavlovian conditioning procedure, we show that subjects experiencing prolonged experimentally induced pain can be conditioned to predict pain relief. This proceeds in a manner consistent with contemporary reward-learning theory (average reward/loss reinforcement learning), reflected by neural activity in the amygdala and midbrain. Furthermore, these reward-like learning signals are mirrored by opposite aversion-like signals in lateral orbitofrontal cortex and anterior cingulate cortex. This dual coding has parallels to 'opponent process' theories in psychology and promotes a formal account of prediction and expectation during pain.
This is a preview of subscription content, access via your institution
Access options
Subscribe to this journal
Receive 12 print issues and online access
$209.00 per year
only $17.42 per issue
Buy this article
- Purchase on Springer Link
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
References
Cabanac, M. Physiological role of pleasure. Science 173, 1103–1107 (1971).
Craig, A.D. A new view of pain as a homeostatic emotion. Trends Neurosci. 26, 303–307 (2003).
Fields, H. State-dependent opioid control of pain. Nat. Rev. Neurosci. 5, 565–575 (2004).
Solomon, R.L. & Corbit, J.D. An opponent-process theory of motivation. I. Temporal dynamics of affect. Psychol. Rev. 81, 119–145 (1974).
Konorski, J. Integrative Activity of the Brain: an Interdisciplinary Approach (Chicago, University of Chicago Press, 1967).
Schull, J. A conditioned opponent theory of Pavlovian conditioning and habituation. in The Psychology of Learning and Motivation (ed. Bower, G.) 57–90 (Academic, New York, 1979).
Grossberg, S. Some normal and abnormal behavioral syndromes due to transmitter gating of opponent processes. Biol. Psychiatry 19, 1075–1118 (1984).
Solomon, R.L. The opponent-process theory of acquired motivation: the costs of pleasure and the benefits of pain. Am. Psychol. 35, 691–712 (1980).
Solomon, R.L. Recent experiments testing an opponent-process theory of acquired motivation. Acta Neurobiol. Exp. (Wars.) 40, 271–289 (1980).
Dickenson & Dearing, M F. Appetitive-aversive interactions and inhibitory processes. in Mechanisms of Learning and Motivation. (eds. Dickinson, A. & Boakes, R.A.) 203–231 (Erlbaum, Hillsdale, New Jersey, 1979).
Daw, N.D., Kakade, S. & Dayan, P. Opponent interactions between serotonin and dopamine. Neural Netw. 15, 603–616 (2002).
Tanimoto, H., Heisenberg, M. & Gerber, B. Experimental psychology: event timing turns punishment to reward. Nature 430, 983 (2004).
Barto, A.G. Adaptive critics and the basal ganglia. in Models of Information Processing in the Basal Ganglia (eds. Houk, J.C., Davis, J.L. & Beiser, D.G.) 215–232 (MIT Press, Cambridge, Massachusetts, 1995).
Sutton, R.S. & Barto, A.G. Reinforcement Learning: an Introduction. (MIT Press, Cambridge, Massachusetts, 1998).
Montague, P.R., Dayan, P. & Sejnowski, T.J. A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J. Neurosci. 16, 1936–1947 (1996).
Schultz, W., Dayan, P. & Montague, P.R. A neural substrate of prediction and reward. Science 275, 1593–1599 (1997).
O'Doherty, J.P., Dayan, P., Friston, K., Critchley, H. & Dolan, R.J. Temporal difference models and reward-related learning in the human brain. Neuron 38, 329–337 (2003).
Seymour, B. et al. Temporal difference models describe higher-order learning in humans. Nature 429, 664–667 (2004).
Dayan, P. & Balleine, B.W. Reward, motivation, and reinforcement learning. Neuron 36, 285–298 (2002).
Schwartz, A. A reinforcement learning method for maximizing undiscounted rewards. in Proceedings of the Tenth International Conference on Machine Learning. 298–305 (Morgan Kaufmann, San Mateo, California, 1993).
Mahadevan, S. Average reward reinforcement learning: Foundations, algorithms and empirical results. Mach. Learn. 22, 1–38 (1996).
Fields, H.L. Pain modulation: expectation, opioid analgesia and virtual pain. Prog. Brain Res. 122, 245–253 (2000).
Price, D.D. Psychological Mechanisms of Pain and Analgesia (IASP, Seattle, 1999).
Tanaka, S.C. et al. Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops. Nat. Neurosci. 7, 887–893 (2004).
Ploghaus, A. et al. Dissociating pain from its anticipation in the human brain. Science 284, 1979–1981 (1999).
Jensen, J. et al. Direct activation of the ventral striatum in anticipation of aversive stimuli. Neuron 40, 1251–1257 (2003).
McClure, S.M., Berns, G.S. & Montague, P.R. Temporal prediction errors in a passive learning task activate human striatum. Neuron 38, 339–346 (2003).
Setlow, B., Schoenbaum, G. & Gallagher, M. Neural encoding in ventral striatum during olfactory discrimination learning. Neuron 38, 625–636 (2003).
Daw, N.D. & Touretzky, D.S. Long-term reward prediction in TD models of the dopamine system. Neural Comput. 14, 2567–2583 (2002).
Markowitz, H. The utility of wealth. J. Polit. Econ. 60, 151–158 (1952).
Camerer, C., Loewenstein, G. & Prelec, D. Neuroeconomics: how neuroscience can inform economics. J. Econ. Lit. (in the press).
Rogan, M.T., Leon, K.S., Perez, D.L. & Kandel, E.R. Distinct neural signatures for safety and danger in the amygdala and striatum of the mouse. Neuron 46, 309–320 (2005).
Watkins, L.R. et al. Neurocircuitry of conditioned inhibition of analgesia: effects of amygdala, dorsal raphe, ventral medullary, and spinal cord lesions on antianalgesia in the rat. Behav. Neurosci. 112, 360–378 (1998).
Holland, P.C. & Gallagher, M. Amygdala-frontal interactions and reward expectancy. Curr. Opin. Neurobiol. 14, 148–155 (2004).
O'Doherty, J., Kringelbach, M.L., Rolls, E.T., Hornak, J. & Andrews, C. Abstract reward and punishment representations in the human orbitofrontal cortex. Nat. Neurosci. 4, 95–102 (2001).
Small, D.M., Zatorre, R.J., Dagher, A., Evans, A.C. & Jones-Gotman, M. Changes in brain activity related to eating chocolate: from pleasure to aversion. Brain 124, 1720–1733 (2001).
Glascher, J. & Buchel, C. Formal learning theory dissociates brain regions with different temporal integration. Neuron 47, 295–306 (2005).
Petrovic, P., Kalso, E., Petersson, K.M. & Ingvar, M. Placebo and opioid analgesia–imaging a shared neuronal network. Science 295, 1737–1740 (2002).
Wager, T.D. et al. Placebo-induced changes in FMRI in the anticipation and experience of pain. Science 303, 1162–1167 (2004).
Logothetis, N.K., Pauls, J., Augath, M., Trinath, T. & Oeltermann, A. Neurophysiological investigation of the basis of the fMRI signal. Nature 412, 150–157 (2001).
Stefanovic, B., Warnking, J.M. & Pike, G.B. Hemodynamic and metabolic responses to neuronal inhibition. Neuroimage 22, 771–778 (2004).
Altier, N. & Stewart, J. The role of dopamine in the nucleus accumbens in analgesia. Life Sci. 65, 2269–2287 (1999).
Horvitz, J.C. Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward events. Neuroscience 96, 651–656 (2000).
Smith, A.J., Becker, S. & Kapur, S. A computational model of the functional role of the ventral-striatal D2 receptor in the expression of previously acquired behaviors. Neural Comput. 17, 361–395 (2005).
Ungless, M.A., Magill, P.J. & Bolam, J.P. Uniform inhibition of dopamine neurons in the ventral tegmental area by aversive stimuli. Science 303, 2040–2042 (2004).
Roitman, M.F., Wheeler, R.A. & Carelli, R.M. Nucleus accumbens neurons are innately tuned for rewarding and aversive taste stimuli, encode their predictors, and are linked to motor output. Neuron 45, 587–597 (2005).
Johansen, J.P. & Fields, H.L. Glutamatergic activation of anterior cingulate cortex produces an aversive teaching signal. Nat. Neurosci. 7, 398–403 (2004).
Gadd, C.A., Murtra, P., De Felipe, C. & Hunt, S.P. Neurokinin-1 receptor-expressing neurons in the amygdala modulate morphine reward and anxiety behaviors in the mouse. J. Neurosci. 23, 8271–8280 (2003).
Dayan, P. & Abbott, L.F. Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems (MIT Press, Cambridge, Massachusetts, 2001).
Buchel, C., Holmes, A.P., Rees, G. & Friston, K.J. Characterizing stimulus-response functions using nonlinear regressors in parametric fMRI experiments. Neuroimage 8, 140–148 (1998).
Acknowledgements
We wish to thank P. Dayan and N. Daw for many helpful discussions and O. Josephs, B. Johanssen and C. Rickard for technical assistance. This research was funded by The Wellcome Trust.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing financial interests.
Rights and permissions
About this article
Cite this article
Seymour, B., O'Doherty, J., Koltzenburg, M. et al. Opponent appetitive-aversive neural processes underlie predictive learning of pain relief. Nat Neurosci 8, 1234–1240 (2005). https://doi.org/10.1038/nn1527
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/nn1527
This article is cited by
-
Post-injury pain and behaviour: a control theory perspective
Nature Reviews Neuroscience (2023)
-
Evaluating an internet-delivered fear conditioning and extinction protocol using response times and affective ratings
Scientific Reports (2022)
-
Acquisition learning is stronger for aversive than appetitive events
Communications Biology (2022)
-
Complex Persistent Opioid Dependence—an Opioid-induced Chronic Pain Syndrome
Current Treatment Options in Oncology (2022)
-
A neuroimaging biomarker for sustained experimental and clinical pain
Nature Medicine (2021)