Striatal dopaminergic modulation of reinforcement learning predicts reward—oriented behavior in daily life
Introduction
Rewards are those stimuli or affective states that elicit approach behavior, increase frequency of such behavior, and thus maintain motivated action and adaptive learning (Schultz, 2010). For instance, we investigated the components of reward-oriented behavior in the everyday life of the general population, and demonstrated that positive affect experienced during physical or social activities significantly increased the odds of engaging in similar activities in the near future (Wichers et al., 2015). Importantly, deviations from the normative reward-oriented behavior can result in addiction on the one extreme (Volkow et al., 2010), and motivational deficits of depression or schizophrenia (Barch, Pagliaccio, & Luking, 2016) on the other.
Advances in molecular neuroscience allowed to examine the neurochemical modulators of reward-directed behavior and have brought the mesolimbic dopamine (DA) system to the fore. In vivo microdialysis studies revealed that DA levels were increased in the striatum upon lever pressing for rewards (e.g. food) (Schultz, Dayan, & Montague, 1997;Salamone, Cousins, McCullough, Carriero, & Berkowitz, 1994), occasioning reward learning. Corroborating positron emission tomography (PET) studies of DA D2/3 receptor binding, with [11C]raclopride displacement as an index of DA release (Laruelle, 2000) in the human striatum, showed increased DA signaling during active reward learning condition (Zald et al., 2004, Weiland et al., 2014). Meanwhile, passive reward delivery failed to evoke changes in baseline DA firing (Hakyemez, Dagher, Smith, & Zald, 2008), confirming that striatal DAergic phasic firing modulates reward learning by representing the imminent reward initiated by a cue.
In nature, however, the probability that a reward-approach behavior will result in a reward varies. The striatum has been shown to be exquisitely sensitive to the violation of the predicted outcome of a behavior. In experimental animals, unexpected reward delivery (positive prediction error) evokes DA bursts, while unexpected reward omission (negative prediction error) elicits DA dips (Schultz, 2010, Schultz et al., 1997, Schultz, 2016). Translated to humans, increased DA release to unexpected rewards was observed in the ventral striatum (VST) (Pappata, 2002, Yoder et al., 2009, Martin-Soelch et al., 2011), and medial caudate nucleus (Zald et al., 2004) of healthy volunteers. While these accounts converge on probabilistic reward-induced striatal DA release, their functional relevance in terms of associations with reward-oriented behavior remains sporadic. One study in human volunteers reported significant relationships between DA signaling change from baseline to rewarded task and reaction time in the task, and another detected that boosting DA levels using levodopa increased risky choices of potential gains, and happiness resulting from the gains (Rutledge, Skandali, Dayan, & Dolan, 2015).
While various studies have linked striatal BOLD covariates of reward prediction error to reinforcement learning, (Jonasson et al., 2014; Pessiglione, Seymour, Flandin, Dolan, & Frith, 2006), to our knowledge, no PET study to date has directly investigated the relationship between reward-induced striatal DA release and concurrent acquisition of reward contingencies in humans. Moreover, there has been little effort to establish whether either experimental measures of reinforcement learning or striatal DA release are related to daily-life reward-oriented behavior.
We therefore explored the striatal DAergic modulation of reward learning in vivo, and combined it with individual variability in reward-oriented behavior in the everyday life. Specifically, we performed a single day protocol [18F]fallypride PET scan (Alpert, Badgaiyan, Livni, & Fischman, 2003) during an active control condition and a commonly studied probabilistic reward task designed to elicit robust reward prediction errors and the associated DAergic activity in the striatum of healthy volunteers. The same participants also underwent an ecological momentary assessment (EMA) study intended to capture the extent to which the enjoyment of being active increased the odds of being active in the near future, throughout a 6-day period. Based on the existing data, we predicted a significant increase in striatal DA release from control to reward condition. Additionally, we expected the various indices of reward function to co-vary in individuals: greater reward-induced DA release will be associated with higher reward-oriented behavior, as assessed concurrently by the probabilistic reward task, and separately by EMA in the everyday life.
Section snippets
Sample and demographics
The medical ethics committee of the Maastricht University and of the RWTH Aachen University approved the study. Approval for performing the PET study was additionally granted by the national authority for radiation protection in humans in Germany (Bundesamt für Strahlenschutz, BfS). All participants signed written informed consent before entering the study.
A total of 18 healthy volunteers were recruited to participate in this study via digital and newspaper advertisements. The inclusion
Sample demographics
The sample is described in terms of demographic, psychopathology and reward sensitivity measures in Table 1. The participants endorsed minimal levels of subclinical psychopathology that could have affected the indices of reward responsiveness.
Exploratory EMA analyses revealed high compliance rate, with all participants filling out on average 84.3% of all assessment beeps, a number that far exceeds the minimal sufficient response rate (Csikszentmihalyi and Larson, 1987).
Reward-induced subcortical dopamine release
Significant amplitude and
Discussion
The present PET study combines, for the first time, the functional molecular imaging account of DA release during reward processing and real-world reward function. Specifically, we investigated the DAergic activity during probabilistic reward learning, an essential requisite of motivated action (Huys, Pizzagalli, Bogdan, & Dayan, 2013), in combination with individual variability in reward-oriented behavior in the everyday life. Firstly, we detected DA release during the task in the dorsal and
Acknowledgements
This work was supported by an ERC consolidator grant to Prof. Dr. Inez Myin-Germeys (ERC-2012-StG, project 309767—INTERACT), and by the Research Foundation—Flanders (FWO) postdoctoral fellowship to Dr. Jenny Ceccarini. The authors thank Rayyan Tutunji, Nele Soons, Dr. Siamak Mohammadkhani Shali, Dr. Ye Rong, Dr. Oliver Winz, Wendy Beuken, Bernward Oedekoven and Ron Mengelers.
References (42)
- et al.
A novel method for noninvasive detection of neuromodulatory changes in specific neurotransmitter systems
Neuroimage
(2003) - et al.
The role of dopamine in value-based attentional orienting
Current Biology
(2016) - et al.
Measuring dopamine neuromodulation in the thalamus: using F-18 fallypride PET to study dopamine release during a spatial attention task
Neuroimage
(2006) - et al.
Striatal D1 and D2 signaling differentially predict learning from positive and negative outcomes
Neuroimage
(2015) - et al.
Striatal dopamine transmission in healthy humans during a passive monetary reward task
Neuroimage
(2008) - et al.
Dopamine release in nucleus accumbens during rewarded task switching measured by (1)(1)Craclopride
Neuroimage
(2014) - et al.
Psychosocial stress is associated with in vivo dopamine release in human ventromedial prefrontal cortex: a positron emission tomography study using (1)(8)Ffallypride
Neuroimage
(2011) - et al.
Long-term potentiation of excitatory inputs to brain reward areas by nicotine
Neuron
(2000) In vivo detection of striatal dopamine release during reward: a PET study with 11 CRaclopride and a single dynamic scan approach
NeuroImage
(2002)- et al.
Combining spatial extent and peak intensity to test for activations in functional imaging
Neuroimage
(1997)
Nucleus accumbens dopamine release increases during instrumental lever pressing for food but not free food consumption
Pharmacology Biochemistry and Behavior
Relationship between impulsivity, prefrontal anticipatory activation, and striatal dopamine release during rewarded task performance
Psychiatry Research
Detection of dopamine neurotransmission in real time
Frontiers in Neuroscience
Mechanisms underlying motivational deficits in psychopathology: similarities and differences in depression and schizophrenia
Current Topics in Behavioral Neurosciences
Optimized in vivo detection of dopamine release using 18 F-fallypride PET
Journal of Nuclear Medicine: Official Publication Society of Nuclear Medicine
Validity and reliability of the experience-sampling method
The Journal of Nervous and Mental Disease
The SCL-90 and the MMPI: a step in the validation of a new self-report scale
British Journal of Psychiatry
Reinforcement learning and dopamine in schizophrenia: dimensions of symptoms or specific features of a disease group?
Frontiers in Psychiatry
Variability in dopamine genes dissociates model-based and model-free reinforcement learning
Journal of Neuroscience
By carrot or by stick: cognitive reinforcement learning in parkinsonism
Science
Comparing functional (PET) images: the assessment of significant change
Journal of Cerebral Blood Flow and Metabolism: Official Journal of the International Society of Cerebral Blood Flow and Metabolism
Cited by (37)
Basal ganglia activation localized in MEG using a reward task
2021, Neuroimage: ReportsInteraction of neurotransmitters and neurochemicals with lymphocytes
2019, Journal of Neuroimmunology