Abstract
Rationale
Optimal decision-making necessitates evaluation of multiple rewards that are each offset by distinct costs, such as high effort requirement or high risk of failure. The neurotransmitter dopamine is fundamental toward these cost-benefit analyses, and D1-like and D2-like dopamine receptors differently modulate the reward-discounting effects of both effort and risk. However, measuring the role of dopamine in regulating decision-making between options associated with distinct costs exceeds the scope of traditional rodent economic decision-making paradigms.
Objectives
We developed the effort vs probability economic conflict task (EvP) to model multimodal economic decision-making in rats. This task measures choice between two rewards of uniform magnitude associated with either a high effort requirement or risk of reward omission. We then tested the modulatory effects of systemic cocaine and D1/D2 blockade or activation on the preference between high-effort and high-risk alternatives.
Methods
In the EvP, two reinforcers of equal magnitude are associated with either (1) an effort requirement that increases throughout the session (1, 5, 10, and 20 lever presses), or (2) a low probability of reward receipt (25% of probabilistic choices). Critically, the reinforcer for each choice is comparable (one pellet), which eliminates the influence of magnitude discrimination on the decision-making process. After establishing the task, the dopamine transporter blocker cocaine and D1/D2 antagonists and agonists were administered prior to EvP performance.
Results
Preference shifted away from either effortful or probabilistic choice when either option became more costly, and this preference was highly variable between subjects and stable over time. Cocaine, D1 activation, and D2 blockade produced limited, dose-dependent shifts in choice preference contingent on high or low effort conditions. In contrast, D2 activation across multiple doses evoked a robust shift from effortful to risky choice that was evident even when clearly disadvantageous.
Conclusions
The EvP clearly demonstrates that rats can evaluate distinct effortful or risky costs associated with rewards of comparable magnitude, and shift preference away from either option with increasing cost. This preference is more tightly linked to D2 than D1 receptor manipulation, suggesting D2-like receptors as a possible therapeutic target for maladaptive biases toward risk-taking over effort.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Allain F, Minogianis EA, Roberts DCS, Samaha AN (2015) How fast and how often: the pharmacokinetics of drug use are decisive in addiction. Neurosci Biobehav Rev 56:166–179. https://doi.org/10.1016/j.neubiorev.2015.06.012
Amlung M, Yurasek A, McCarty KN et al (2015) Area under the curve as a novel metric of behavioral economic demand for alcohol. Exp Clin Psychopharmacol 23:168–175. https://doi.org/10.1037/pha0000014.Area
Andrews CM, Lucki I (2001) Effects of cocaine on extracellular dopamine and serotonin levels in the nucleus accumbens. Psychopharmacology 155:221–229. https://doi.org/10.1007/s002130100704
Baarendse PJJ, Winstanley CA, Vanderschuren LJMJ (2013) Simultaneous blockade of dopamine and noradrenaline reuptake promotes disadvantageous decision making in a rat gambling task. Psychopharmacology (Berlin) 225:719–731. https://doi.org/10.1007/s00213-012-2857-z
Bechara A (2005) Decision making, impulse control and loss of willpower to resist drugs: a neurocognitive perspective. Nat Neurosci 8:1458–1463. https://doi.org/10.1038/nn1584
Bello EP, Mateo Y, Gelman DM et al (2012) Cocaine supersensitivity and enhanced motivation for reward in mice lacking dopamine D2 autoreceptors. Nat Neurosci 14:1033–1038. https://doi.org/10.1038/jid.2014.371
Brevers D, Bechara A, Cleeremans A, Kornreich C, Verbanck P, Noël X (2014) Impaired decision-making under risk in individuals with alcohol dependence. Alcohol Clin Exp Res 38:1924–1931. https://doi.org/10.1111/acer.12447
Bryce CA, Floresco SB (2019) Alterations in effort-related decision-making induced by stimulation of dopamine D1, D2, D3, and corticotropin-releasing factor receptors in nucleus accumbens subregions. Psychopharmacology 236:2699–2712. https://doi.org/10.1007/s00213-019-05244-w
Burke CJ, Soutschek A, Weber S, Raja Beharelle A, Fehr E, Haker H, Tobler PN (2018) Dopamine receptor-specific contributions to the computation of value. Neuropsychopharmacology 43:1415–1424. https://doi.org/10.1038/npp.2017.302
Cardinal RN, Howes NJ (2005) Effects of lesions of the nucleus accumbens core on choice between small certain rewards and large uncertain rewards in rats. BMC Neurosci 6:1–19. https://doi.org/10.1186/1471-2202-6-37
Cocker PJ, Tremblay M, Kaur S, Winstanley CA (2016) Chronic administration of the dopamine D 2 / 3 agonist ropinirole invigorates performance of a rodent slot machine task, potentially indicative of less distractible or compulsive-like gambling behaviour. Psychopharmacology 56:159–166. https://doi.org/10.1007/s00213-016-4447-y
Crowley TJ, Raymond KM, Mikulich-Gilbertson SK, Thompson LL, Lejuez CW (2006) A risk-taking “set” in a novel task among adolescents with serious conduct and substance problems. J Am Acad Child Adolesc Psychiatry 45:175–183. https://doi.org/10.1097/01.chi.0000188893.60551.31
Cui R, Wang L, Liu L, Ruan H, Li X (2018) Effects of noradrenergic and serotonergic systems on risk-based decision-making and center arena activity in open field in rats. Eur J Pharmacol 841:57–66. https://doi.org/10.1016/j.ejphar.2018.09.026
Depoortere RY, Li DH, Lane JD, Emmett-Oglesby MW (1993) Parameters of self-administration of cocaine in rats under a progressive-ratio schedule. Pharmacol Biochem Behav 45:539–548. https://doi.org/10.1016/0091-3057(93)90503-L
Evenden JL, Ryan CN (1996) The pharmacology of impulsive behavior in rats of drugs on response choice with variyng delays of reinforcement. Psychopharmacology 128:161–170
Floresco SB, St. Onge JR, Ghods-Sharifi S, Winstanley CA (2008a) Cortico-limbic-striatal circuits subserving different forms of cost-benefit decision making. Cogn Affect Behav Neurosci 8:375–389. https://doi.org/10.3758/CABN.8.4.375
Floresco SB, Tse MTLL, Ghods-Sharifi S (2008b) Dopaminergic and glutamatergic regulation of effort- and delay-based decision making. Neuropsychopharmacology 33:1966–1979. https://doi.org/10.1038/sj.npp.1301565
Ford CP (2014) The Role of D2-Autoreceptors in regulating dopamine neuron activity and transmission. Neuroscience 282:13–22. https://doi.org/10.1016/j.neuroscience.2014.01.025.The
Freels TG, Gabriel DBK, Lester DB, Simon NW (2020) Risky decision-making predicts dopamine release dynamics in nucleus accumbens shell. Neuropsychopharmacology 45:266–275. https://doi.org/10.1101/572263
Gabriel DBK, Freels TG, Setlow B, Simon NW (2019) Risky decision-making is associated with impulsive action and sensitivity to first-time nicotine exposure. Behav Brain Res 359:579–288. https://doi.org/10.1016/j.bbr.2018.10.008
Grall-Bronnec M, Sauvaget A, Perrouin F, Leboucher J, Etcheverrigaray F, Challet-Bouju G, Gaboriau L, Derkinderen P, Jolliet P, Victorri-Vigneau C (2016) Pathological gambling associated with aripiprazole or dopamine replacement therapy. J Clin Psychopharmacol 36:63–70. https://doi.org/10.1097/JCP.0000000000000444
Green L, Myerson J (2004) A discounting framework for choice with delayed and probabilistic rewards. Psychol Bull 130:769–792. https://doi.org/10.1037/0033-2909.130.5.769
Kalenscher T, van Wingerden M (2011) Why we should use animals to study economic decision making - a perspective. Front Neurosci 5:1–11. https://doi.org/10.3389/fnins.2011.00082
Killcross AS, Everitt BJ, Robbins TW (1997) Symmetrical effects of amphetamine and alpha-flupenthixol on conditioned punishment and conditioned reinforcement: contrasts with midazolam. Psychopharmacology 129:141–152. https://doi.org/10.1007/s002130050174
Koffarnus MN, Kaplan BA (2018) Clinical models of decision making in addiction. Pharmacol Biochem Behav 164:71–83. https://doi.org/10.1016/j.pbb.2017.08.010
Koffarnus MN, Newman AH, Grundt P, Rice KC, Woods JH (2011) Effects of selective dopaminergic compounds on a delay discounting task. Behav Pharmacol 22:300–311. https://doi.org/10.1146/annurev.neuro.31.060407.125627.Brain
Mai B, Sommer S, Hauber W (2015) Dopamine D1/D2 receptor activity in the nucleus accumbens core but not in the nucleus accumbens shell and orbitofrontal cortex modulates risk-based decision making. Int J Neuropsychopharmacol 18:1–9. https://doi.org/10.1093/ijnp/pyv043
Mcelvain JS, Schenk JO (1992) A multisubstrate mechanism of striatal dopamine uptake and its inhibition by cocaine. Biochem Pharmacol 43:2189–2199. https://doi.org/10.1016/0006-2952(92)90178-L
Montague PR, Berns GS (2002) Neural economics and the biological substrates of valuation. Neuron 36:265–284. https://doi.org/10.1016/S0896-6273(02)00974-1
Myerson J, Green L, Warusawitharana M (2001) Area under the curve as a measure of discounting. J Exp Anal Behav 76:235–243. https://doi.org/10.1901/jeab.2001.76-235
Nunes EJ, Randall PA, Santerre JL, Given AB, Sager TN, Correa M, Salamone JD (2010) Differential effects of selective adenosine antagonists on the effort-related impairments induced by dopamine D1 and D2 antagonism. Neuroscience 170:268–280. https://doi.org/10.1016/j.neuroscience.2010.05.068
Orsini CA, Simon NW (2020) Reward/punishment-based decision making in rodents. Curr Protoc Neurosci 93:1–21. https://doi.org/10.1002/cpns.100
Orsini CA, Moorman DE, Young JW, Setlow B, Floresco SB (2015) Neural mechanisms regulating different forms of risk-related decision-making: Insights from animal models. Neurosci Biobehav Rev 58:147–167. https://doi.org/10.1016/j.neubiorev.2015.04.009
Orsini CA, Hernandez CM, Bizon JL, Setlow B (2018) Deconstructing value-based decision making via temporally selective manipulation of neural activity: insights from rodent models. Cogn Affect Behav Neurosci 19:459–476. https://doi.org/10.3758/s13415-018-00649-0
Peña-Oliver Y, Sanchez-Roige S, Stephens DN, Ripley TL (2014) Alpha-synuclein deletion decreases motor impulsivity but does not affect risky decision making in a mouse gambling task. Psychopharmacology 231:2493–2506. https://doi.org/10.1007/s00213-013-3416-y
Pierce RC, Kalivas PW (1997) A circuitry model of the expression of behavioral sensitization to amphetamine-like psychostimulants. Brain Res Rev 25:192–216. https://doi.org/10.1016/S0165-0173(97)00021-0
Rangel A, Camerer C, Montague PR (2008) A framework for studying the neurobiology of value-based decision making. Nat Rev Neurosci 9:545–556. https://doi.org/10.1038/nrn2357
Robbins TW (2005) Chemistry of the mind: neurochemical modulation of prefrontal cortical function. J Comp Neurol 493:140–146. https://doi.org/10.1002/cne.20717
Saddoris MP, Sugam JA, Stuber GD, Witten IB, Deisseroth K, Carelli RM (2015) Mesolimbic dopamine dynamically tracks, and is causally linked to, discrete aspects of value-based decision making. Biol Psychiatry 77:903–911. https://doi.org/10.1016/j.biopsych.2014.10.024
Saddoris MP, Wang X, Sugam JA, Carelli RM (2016) Cocaine self-administration experience induces pathological phasic accumbens dopamine signals and abnormal incentive behaviors in drug-abstinent rats. J Neurosci 36:235–250. https://doi.org/10.1523/JNEUROSCI.3468-15.2016
Salamone JD, Correa M, Yohn SE, Lopez Cruz L, San Miguel N, Alatorre L (2016) The pharmacology of effort-related choice behavior: dopamine, depression, and individual differences. Behav Process 127:3–17. https://doi.org/10.1016/j.beproc.2016.02.008
Schultz W (2016) Dopamine reward prediction-error signalling: a two-component response. Nat Rev Neurosci 17:1079–1084. https://doi.org/10.1038/nrn.2015.26.Dopamine
Schultz W, Stauffer WR, Lak A (2017) The phasic dopamine signal maturing: from reward via behavioural activation to formal economic utility. Curr Opin Neurobiol 43:139–148. https://doi.org/10.1016/j.conb.2017.03.013
Simon NW, Gilbert RJ, Mayse JD, Bizon JL, Setlow B (2009) Balancing risk and reward: a rat model of risky decision making. Neuropsychopharmacology 34:2208–2217. https://doi.org/10.1038/npp.2009.48
Simon NW, Montgomery KS, Beas BS, Mitchell MR, LaSarge CL, Mendez IA, Banuelos C, Vokes CM, Taylor AB, Haberman RP, Bizon JL, Setlow B (2011) Dopaminergic modulation of risky decision-making. J Neurosci 31:17460–17470. https://doi.org/10.1523/JNEUROSCI.3772-11.2011
Soutschek A, Gvozdanovic G, Kozak R, Duvvuri S, de Martinis N, Harel B, Gray DL, Fehr E, Jetter A, Tobler PN (2020) Dopaminergic D1 receptor stimulation affects effort and risk preferences. Biol Psychiatry 87:1–8. https://doi.org/10.1016/j.biopsych.2019.09.002
St. Onge JR, Floresco SB (2009) Dopaminergic modulation of risk-based decision making. Neuropsychopharmacology 34:681–697. https://doi.org/10.1038/npp.2008.121
St. Onge JR, Chiu YC, Floresco SB (2010) Differential effects of dopaminergic manipulations on risky choice. Psychopharmacology 211:209–221. https://doi.org/10.1007/s00213-010-1883-y
Stopper CM, Tse MTL, Montes DR, Wiedman CR, Floresco SB (2014) Overriding phasic dopamine signals redirects action selection during risk/reward decision making. Neuron 84:177–189. https://doi.org/10.1016/j.neuron.2014.08.033
Sugam JA, Saddoris MP, Carelli RM (2014) Nucleus accumbens neurons track behavioral preferences and reward outcomes during risky decision making. Biol Psychiatry 75:807–816. https://doi.org/10.1016/j.biopsych.2013.09.010
Tian L, Liu X, Mei X, Cui R, Li X (2019) The role of dopamine D1- and D2-like receptors related to muscarinic M1 receptors in impulsive choice in high-impulsive and low-impulsive rats. Pharmacol Biochem Behav 176:43–52. https://doi.org/10.1016/j.pbb.2018.11.005
Tremblay M, Silveira MM, Kaur S, Hosking JG, Adams WK, Baunez C, Winstanley CA (2017) Chronic D 2/3 agonist ropinirole treatment increases preference for uncertainty in rats regardless of baseline choice patterns. Eur J Neurosci 45:159–166. https://doi.org/10.1111/ejn.13332
van Haaren F, Anderson K (1994) Effects of cocaine on fixed-interval behavior and schedule-induced alcohol consumption in male and female rats. Pharmacol Biochem Behav 47:997–1002. https://doi.org/10.1016/0091-3057(94)90311-5
Varazzani C, San-Galli A, Gilardeau S, Bouret S (2015) Noradrenaline and dopamine neurons in the reward/effort trade-off: a direct electrophysiological comparison in behaving monkeys. J Neurosci 35:7866–7877. https://doi.org/10.1523/JNEUROSCI.0454-15.2015
Verdejo-García AJ, Perales JC, Pérez-García M (2007) Cognitive impulsivity in cocaine and heroin polysubstance abusers. Addict Behav 32:950–966. https://doi.org/10.1016/j.addbeh.2006.06.032
Verdejo-García A, Chong TT-J, Stout JC et al (2018) Stages of dysfunctional decision-making in addiction. Pharmacol Biochem Behav 164:99–105. https://doi.org/10.1016/j.pbb.2017.02.003
Wade TR, De Wit H, Richards JB (2000) Effects of dopaminergic drugs on delayed reward as a measure of impulsive behavior in rats. Psychopharmacology 150:90–101. https://doi.org/10.1007/s002130000402
Wall VZ, Parker JG, Fadok JP, Darvas M, Zweifel L, Palmiter RD (2011) A behavioral genetics approach to understanding D1 receptor involvement in phasic dopamine signaling. Mol Cell Neurosci 46:21–31. https://doi.org/10.1016/j.mcn.2010.09.011
Winstanley CA, Floresco SB (2016) Deciphering decision making: variation in animal models of effort- and uncertainty-based choice reveals distinct neural circuitries underlying core cognitive processes. J Neurosci 36:12069–12079. https://doi.org/10.1523/JNEUROSCI.1713-16.2016
Winstanley CA, Cocker PJ, Rogers RD (2011) Dopamine modulates reward expectancy during performance of a slot machine task in rats: Evidence for a near-miss effect. Neuropsychopharmacology 36:913–925. https://doi.org/10.1038/npp.2010.230
Yang JH, Cheng CP, Liao RM (2018) Effects of d-amphetamine on risk choice in rats depend on the manner in which the expected reward value is varied. Pharmacol Biochem Behav 171:20–29. https://doi.org/10.1016/j.pbb.2018.05.008
Yates JR (2018) Dissecting drug effects in preclinical models of impulsive choice: emphasis on glutamatergic compounds. Psychopharmacol 235:607–626. https://doi.org/10.1007/s00213-017-4825-0.Dissecting
Yohn SE, Santerre JL, Nunes EJ, Kozak R, Podurgiel SJ, Correa M, Salamone JD (2015) The role of dopamine D1 receptor transmission in effort-related choice behavior: effects of D1 agonists. Pharmacol Biochem Behav 135:217–226. https://doi.org/10.1016/j.pbb.2015.05.003
Yohn SE, Lopez-Cruz L, Hutson PH, Correa M, Salamone JD (2016) Effects of lisdexamfetamine and s-citalopram, alone and in combination, on effort-related choice behavior in the rat. Psychopharmacology 233:949–960. https://doi.org/10.1007/s00213-015-4176-7
Zhang D, Wang X, Xiang X, Chen H, Zhang J, Su Q, Hao W (2010) The dopamine D 2 partial agonist and antagonist terguride decreases heroin self-administration on fixed- and progressive-ratio schedules. Pharmacol Biochem Behav 97:222–226. https://doi.org/10.1016/j.pbb.2010.08.002
Funding
This work was supported by the National Institute on Drug Abuse (R15DA046797), a Young Investigator Award from the Brain and Behavior Research Foundation, and a Faculty Research Grant from the University of Memphis (NWS).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Gabriel, D.B.K., Liley, A.E., Freels, T.G. et al. Dopamine receptors regulate preference between high-effort and high-risk rewards. Psychopharmacology 238, 991–1004 (2021). https://doi.org/10.1007/s00213-020-05745-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00213-020-05745-z