Elsevier

Biological Psychology

Volume 127, July 2017, Pages 1-9
Biological Psychology

Striatal dopaminergic modulation of reinforcement learning predicts reward—oriented behavior in daily life

https://doi.org/10.1016/j.biopsycho.2017.04.014Get rights and content

Abstract

Much human behavior is driven by rewards. Preclinical neurophysiological and clinical positron emission tomography (PET) studies have implicated striatal phasic dopamine (DA) release as a primary modulator of reward processing. However, the relationship between experimental reward-induced striatal DA release and responsiveness to naturalistic rewards, and therefore functional relevance of these findings, has been elusive.

We therefore combined, for the first time, a DA D2/3 receptor [18F]fallypride PET during a probabilistic reinforcement learning (RL) task with a six day ecological momentary assessments (EMA) of reward-related behavior in the everyday life of 16 healthy volunteers. We detected significant reward-induced DA release in the bilateral putamen, caudate nucleus and ventral striatum, the extent of which was associated with better behavioral performance on the RL task across all regions. Furthermore, individual variability in the extent of reward-induced DA release in the right caudate nucleus and ventral striatum modulated the tendency to be actively engaged in a behavior if the active engagement was previously deemed enjoyable. This study suggests a link between striatal reward-related DA release and ecologically relevant reward-oriented behavior, suggesting an avenue for the inquiry into the DAergic basis of optimal and impaired motivational drive.

Introduction

Rewards are those stimuli or affective states that elicit approach behavior, increase frequency of such behavior, and thus maintain motivated action and adaptive learning (Schultz, 2010). For instance, we investigated the components of reward-oriented behavior in the everyday life of the general population, and demonstrated that positive affect experienced during physical or social activities significantly increased the odds of engaging in similar activities in the near future (Wichers et al., 2015). Importantly, deviations from the normative reward-oriented behavior can result in addiction on the one extreme (Volkow et al., 2010), and motivational deficits of depression or schizophrenia (Barch, Pagliaccio, & Luking, 2016) on the other.

Advances in molecular neuroscience allowed to examine the neurochemical modulators of reward-directed behavior and have brought the mesolimbic dopamine (DA) system to the fore. In vivo microdialysis studies revealed that DA levels were increased in the striatum upon lever pressing for rewards (e.g. food) (Schultz, Dayan, & Montague, 1997;Salamone, Cousins, McCullough, Carriero, & Berkowitz, 1994), occasioning reward learning. Corroborating positron emission tomography (PET) studies of DA D2/3 receptor binding, with [11C]raclopride displacement as an index of DA release (Laruelle, 2000) in the human striatum, showed increased DA signaling during active reward learning condition (Zald et al., 2004, Weiland et al., 2014). Meanwhile, passive reward delivery failed to evoke changes in baseline DA firing (Hakyemez, Dagher, Smith, & Zald, 2008), confirming that striatal DAergic phasic firing modulates reward learning by representing the imminent reward initiated by a cue.

In nature, however, the probability that a reward-approach behavior will result in a reward varies. The striatum has been shown to be exquisitely sensitive to the violation of the predicted outcome of a behavior. In experimental animals, unexpected reward delivery (positive prediction error) evokes DA bursts, while unexpected reward omission (negative prediction error) elicits DA dips (Schultz, 2010, Schultz et al., 1997, Schultz, 2016). Translated to humans, increased DA release to unexpected rewards was observed in the ventral striatum (VST) (Pappata, 2002, Yoder et al., 2009, Martin-Soelch et al., 2011), and medial caudate nucleus (Zald et al., 2004) of healthy volunteers. While these accounts converge on probabilistic reward-induced striatal DA release, their functional relevance in terms of associations with reward-oriented behavior remains sporadic. One study in human volunteers reported significant relationships between DA signaling change from baseline to rewarded task and reaction time in the task, and another detected that boosting DA levels using levodopa increased risky choices of potential gains, and happiness resulting from the gains (Rutledge, Skandali, Dayan, & Dolan, 2015).

While various studies have linked striatal BOLD covariates of reward prediction error to reinforcement learning, (Jonasson et al., 2014; Pessiglione, Seymour, Flandin, Dolan, & Frith, 2006), to our knowledge, no PET study to date has directly investigated the relationship between reward-induced striatal DA release and concurrent acquisition of reward contingencies in humans. Moreover, there has been little effort to establish whether either experimental measures of reinforcement learning or striatal DA release are related to daily-life reward-oriented behavior.

We therefore explored the striatal DAergic modulation of reward learning in vivo, and combined it with individual variability in reward-oriented behavior in the everyday life. Specifically, we performed a single day protocol [18F]fallypride PET scan (Alpert, Badgaiyan, Livni, & Fischman, 2003) during an active control condition and a commonly studied probabilistic reward task designed to elicit robust reward prediction errors and the associated DAergic activity in the striatum of healthy volunteers. The same participants also underwent an ecological momentary assessment (EMA) study intended to capture the extent to which the enjoyment of being active increased the odds of being active in the near future, throughout a 6-day period. Based on the existing data, we predicted a significant increase in striatal DA release from control to reward condition. Additionally, we expected the various indices of reward function to co-vary in individuals: greater reward-induced DA release will be associated with higher reward-oriented behavior, as assessed concurrently by the probabilistic reward task, and separately by EMA in the everyday life.

Section snippets

Sample and demographics

The medical ethics committee of the Maastricht University and of the RWTH Aachen University approved the study. Approval for performing the PET study was additionally granted by the national authority for radiation protection in humans in Germany (Bundesamt für Strahlenschutz, BfS). All participants signed written informed consent before entering the study.

A total of 18 healthy volunteers were recruited to participate in this study via digital and newspaper advertisements. The inclusion

Sample demographics

The sample is described in terms of demographic, psychopathology and reward sensitivity measures in Table 1. The participants endorsed minimal levels of subclinical psychopathology that could have affected the indices of reward responsiveness.

Exploratory EMA analyses revealed high compliance rate, with all participants filling out on average 84.3% of all assessment beeps, a number that far exceeds the minimal sufficient response rate (Csikszentmihalyi and Larson, 1987).

Reward-induced subcortical dopamine release

Significant amplitude and

Discussion

The present PET study combines, for the first time, the functional molecular imaging account of DA release during reward processing and real-world reward function. Specifically, we investigated the DAergic activity during probabilistic reward learning, an essential requisite of motivated action (Huys, Pizzagalli, Bogdan, & Dayan, 2013), in combination with individual variability in reward-oriented behavior in the everyday life. Firstly, we detected DA release during the task in the dorsal and

Acknowledgements

This work was supported by an ERC consolidator grant to Prof. Dr. Inez Myin-Germeys (ERC-2012-StG, project 309767—INTERACT), and by the Research Foundation—Flanders (FWO) postdoctoral fellowship to Dr. Jenny Ceccarini. The authors thank Rayyan Tutunji, Nele Soons, Dr. Siamak Mohammadkhani Shali, Dr. Ye Rong, Dr. Oliver Winz, Wendy Beuken, Bernward Oedekoven and Ron Mengelers.

References (42)

  • J.D. Salamone et al.

    Nucleus accumbens dopamine release increases during instrumental lever pressing for food but not free food consumption

    Pharmacology Biochemistry and Behavior

    (1994)
  • B.J. Weiland et al.

    Relationship between impulsivity, prefrontal anticipatory activation, and striatal dopamine release during rewarded task performance

    Psychiatry Research

    (2014)
  • R.D. Badgaiyan

    Detection of dopamine neurotransmission in real time

    Frontiers in Neuroscience

    (2013)
  • D.M. Barch et al.

    Mechanisms underlying motivational deficits in psychopathology: similarities and differences in depression and schizophrenia

    Current Topics in Behavioral Neurosciences

    (2016)
  • J. Ceccarini et al.

    Optimized in vivo detection of dopamine release using 18 F-fallypride PET

    Journal of Nuclear Medicine: Official Publication Society of Nuclear Medicine

    (2012)
  • M. Csikszentmihalyi et al.

    Validity and reliability of the experience-sampling method

    The Journal of Nervous and Mental Disease

    (1987)
  • L.R. Derogatis et al.

    The SCL-90 and the MMPI: a step in the validation of a new self-report scale

    British Journal of Psychiatry

    (1976)
  • L. Deserno et al.

    Reinforcement learning and dopamine in schizophrenia: dimensions of symptoms or specific features of a disease group?

    Frontiers in Psychiatry

    (2013)
  • B.B. Doll et al.

    Variability in dopamine genes dissociates model-based and model-free reinforcement learning

    Journal of Neuroscience

    (2016)
  • M.J. Frank et al.

    By carrot or by stick: cognitive reinforcement learning in parkinsonism

    Science

    (2004)
  • K.J. Friston et al.

    Comparing functional (PET) images: the assessment of significant change

    Journal of Cerebral Blood Flow and Metabolism: Official Journal of the International Society of Cerebral Blood Flow and Metabolism

    (1991)
  • Cited by (37)

    View all citing articles on Scopus
    View full text