Chemical neuromodulation of cognitive control avoidance

Why do we so often fail to exert cognitive control, even though we are in principle able to do so? In this review, we begin to address this question by considering the contribution of the major ascending neuromodulators that are often implicated in cognitive control and motivation, in particular dopamine, noradrenaline and serotonin. Accumulating evidence indicates that cognitive control is subjectively costly and people generally choose to refrain from mentally effortful tasks, despite, at times, devastating consequences. This tendency to avoid cognitive control tasks has been shown to be sensitive to catecholaminergic interventions in rodents and humans, where choices about cognitive control can be altered even in the absence of performance changes. Such effects might reflect modulation by dopamine and/or noradrenaline of a variety of mechanisms that contribute to our motivation for cognitive control. These likely include the calculation and integration into behavior of both the expected value (i.e. cost vs benefit), as well as outcome uncertainty of exerting cognitive control. In addition, serotonin might impact cognitive control avoidance by modulating specifically the computation of effort costs. Advancing our understanding of the distinct roles of the various chemical neuromodulators will help elucidate the computational mechanisms that contribute to our tendency to avoid difficult cognitive tasks.


Chemical neuromodulation of cognitive control avoidance
Monja I Frobö se 1 and Roshan Cools 1,2 Why do we so often fail to exert cognitive control, even though we are in principle able to do so? In this review, we begin to address this question by considering the contribution of the major ascending neuromodulators that are often implicated in cognitive control and motivation, in particular dopamine, noradrenaline and serotonin. Accumulating evidence indicates that cognitive control is subjectively costly and people generally choose to refrain from mentally effortful tasks, despite, at times, devastating consequences. This tendency to avoid cognitive control tasks has been shown to be sensitive to catecholaminergic interventions in rodents and humans, where choices about cognitive control can be altered even in the absence of performance changes. Such effects might reflect modulation by dopamine and/or noradrenaline of a variety of mechanisms that contribute to our motivation for cognitive control. These likely include the calculation and integration into behavior of both the expected value (i.e. cost vs benefit), as well as outcome uncertainty of exerting cognitive control. In addition, serotonin might impact cognitive control avoidance by modulating specifically the computation of effort costs. Advancing our understanding of the distinct roles of the various chemical neuromodulators will help elucidate the computational mechanisms that contribute to our tendency to avoid difficult cognitive tasks.

Introduction
Cognitive control is effortful, subjectively costly and people are generally biased to avoid it [1 ,2 ,3 ,4 ]. They prefer to perform a task with less rather than more task-switching [5] and with lower rather than higher working memory load [6]. On average, people also choose to forego a higher monetary reward to avoid a more demanding task [6,7]. This can be considered paradoxical, given the following observations. First, cognitive control is a hallmark of the human mind and the brain region commonly associated with cognitive control, the prefrontal cortex [8], is exquisitely well developed. Accordingly, we are very good at exerting cognitive control. Second, exerting cognitive control has obvious benefits for performance, and most of us are aware that failures of cognitive control can have disastrous consequences, ranging from obesity and monetary crises to murder. Finally, there is a growing consensus that cognitive control functions, are unlikely to be metabolically more costly than other functions, associated, for example, with the visual cortex [9,10 ,11], but [12]. Therefore, a key open question is why do we so often fail to exert cognitive control, even though we are in principle able to do so [13,14 ]. We begin to address this question by considering the contributions to value-based choice about cognitive control of a set of major ascending neuromodulators that have been strongly implicated in motivation, choice and cognitive control, in particular dopamine, noradrenaline and serotonin (Figure 1a). Note that few empirical studies have so far addressed this specific question. Thus, we present ideas that build on current literature, but need to be tested in future studies.

Dopamine and cognitive control avoidance
Effortful cognitive control has long been associated with optimal catecholamine transmission. For example, patients with disorders that implicate dopamine, like Parkinson's disease or attention deficit/hyperactivity disorder (ADHD), exhibit cognitive control deficits which can be remedied by dopaminergic medication [15]. Moreover, dopamine is also a key ingredient in drugs that are used to boost cognitive control in healthy adults [16]. Paradoxically, however, altering dopamine transmission by medication or by promising reward can also impair cognitive performance [17,18]. For example, in Parkinson's disease, the dopaminergic medication doses that are well established to improve motor control can contribute to the development of impulse control disorder, putatively by impairing cognitive control [19]. Here, we consider the possibility that such paradoxical effects might reflect, in part, modulation by dopamine of value-(and effort cost) based choice about whether or not to exert motor and cognitive control [20 ]. Indeed the phasic firing of midbrain dopamine neurons are well accepted to contribute to reward prediction error signaling [21,22], which drives temporal difference learning and Synthesis and projections of major ascending neuromodulators Illustration of (opposite) impulsivity-dependent methylphenidate effects on choice (i.e. task-switching avoidance -left) versus task execution (i.e. switch cost and general performance -right) on the demand selection task Hypothesized mechanism by which dopaminergic medication can improve motor, but impair cognitive control in Parkinson's disease  (a) Simplified presentation of synthesis pathway and projections of the major ascending neuromodulators dopamine, noradrenaline, and serotonin.
(b) Schematic overview of the (opposite) effects of methylphenidate on the avoidance versus execution of task-switching. Methylphenidate increased task-switching avoidance in more, relative to less impulsive participants, whereas task-switching performance was unaffected. By contrast, methylphenidate actually enhanced performance in more impulsive participants, evidenced by speeding of responses (illustration based value-based choice, not only of actions that have high value but also of valuable (while costly) cognitive tasks [20 ,23 ].
As made explicit in the expected value of control (EVC) model [14 ], one way in which dopamine might bias such value-based learning and choice about cognitive tasks is by altering the (expected) value of cognitive control, which corresponds to the benefit minus the costs of control. According to neurocomputational models of dopamine in the basal ganglia, such as the OPAL model and supportive empirical evidence [20, but 24], prolonging (striatal) dopamine likely enhances the benefit while also reducing the cost of actions by having opposite effects on the D1 (GO) and D2 (NO-GO) pathways of the basal ganglia. Thus, based on this evidence, we argue that increases in dopamine will increase the benefits, while reducing the costs of cognitive control. Based on further empirical evidence for an 'inverted U'-shaped relationship between dopamine and reward-based versus punishment-based learning [18,25], we also hypothesize that excess or supraoptimal levels of dopamine might paradoxically reduce the benefits versus the costs of cognitive control, perhaps by acting via a presynaptic mechanism of action, thus leading to a net reduction in dopamine synthesis and/or release.
The nature of the control cost is currently under active study. Some have argued that it represents an intrinsic conflict-related cost [14 ,26,27 ], while others highlight that it might correspond to an opportunity cost of time, equal to either the value of the next best alternative [10 ] or, following work on dopamine's role in motor motivation [28,29] to an average net reward per unit time [23 ].
Regardless of the origin of the putatively dopaminergic cost of cognitive control, empirical evidence for an effect of dopamine on value-based choice about cognitive control is still scarce. So far, two studies have revealed that challenging catecholamine transmission by amphetamine or methylphenidate administration, which prolongs the activity of both dopamine and noradrenaline, alters the willingness to engage in cognitive effort. Work with experimental animals revealed that administration of amphetamine motivated rodent 'slackers' (but not 'workers') to choose a perceptually more demanding option for a higher reward [30 ]. However, follow-up work from the same group suggested that this effect was mediated by changes in noradrenaline rather than dopamine transmission, as selective dopamine antagonists did not alter demand avoidance [31 ]. In parallel, work with young healthy human volunteers has shown that the administration of methylphenidate (20 mg, oral) altered the avoidance of a classic cognitive control task, taskswitching [32 ], in a demand selection paradigm previously shown to be sensitive to demand avoidance [5]. The effect of methylphenidate depended on participants' degree of trait impulsivity, a measure that has been associated with enhanced drug-induced dopamine release and reduced D2/D3 (auto-)receptor availability [33][34][35].
More impulsive participants became more demand avoidant relative to low-impulsive participants [32 ]. Intriguingly, in the latter study, methylphenidate did not alter the ability to implement task-switching, as measured during the performance of the task-switching and taskrepetition trials that followed each choice (Figure 1b), although the drug did render performance across trial types faster as well as more accurate, consistent with a general performance enhancing effect. Thus in this study methylphenidate impacted only the avoidance and not the execution of cognitive control, with methylphenidate actually undermining impulsive participants' motivation to exert control. The hypothesis that this effect reflects modulation of the cost of cognitive effort by dopamine is currently under study.
Which mechanism might underlie the paradoxical effects of methylphenidate in high-impulsive individuals, where it potentiates the avoidance of cognitive control? One possibility, as referred to above, is that the cost of cognitive control was increased, because methylphenidate elicited supraoptimal levels of dopamine in these individuals with high trait impulsivity. Trait impulsivity has been shown to be accompanied by enhanced baseline levels of striatal dopamine release and low (but perhaps more sensitive) presynaptic dopamine D2 receptor availability in the midbrain [33]. Indeed, methylphenidate has previously been argued to act presynaptically, especially in high dopamine states, by triggering a self-regulatory mechanism, thus leading to a net reduction in dopamine release [36,37].
An alternative, more speculative possibility is inspired by opportunity cost accounts of tonic dopamine's role in motivating vigor (physical effort) [28,29,38]. Generalization of this account led to the hypothesis that an increase in tonic dopamine motivates people to avoid slow cognitive control strategies because such an increase is accompanied by an increase in the opportunity cost of time [10 ]. In one account the opportunity cost of time is equal to the average reward rate of the environment [23 ].
Although one study demonstrated that dopaminergic medication effects on physical effort-based decision Chemical neuromodulation of cognitive control avoidance Frobö se and Cools 123 ( Figure 1 Legend Continued) on data presented in [32 ]). (c) Dopaminergic medication in Parkinson's disease increases dopamine levels and has been shown to remediate some motor symptoms, while at the same time, contributing, in a considerable proportion of patients, to impulse (cognitive) control disorder. Increased dopamine tone has been hypothesized to elevate the cost of time due to higher average net reward per unit time (R; [29]). This might account, in part, for the contrasting motor and cognitive effects of dopaminergic medication, which would enhance the motivation for physical vigor, yet reduce the motivation for time costly cognitive control processes.
making were independent of the possibility to save time [39], another recent study provided some preliminary supportive evidence that strategic adjustments in the degree to which people perform fast and accurately on Simon, task-switching and perceptual decision tasks do indeed depend on fluctuations in the average reward rate [40]. People with high levels of tonic dopamine might evaluate control as relatively more costly than people with lower dopamine tone because their estimate of the average reward rate in the environment is increased.
One key implication of this hypothesis is that dopamineinduced increases in an opportunity cost of time might account, in part, for the contrasting motor and cognitive effects of dopaminergic medication in Parkinson's disease, described above. According to this account, increases in tonic dopamine would be accompanied by increases in the cost of time, which would enhance the motivation for physical vigor [29], thus remediating bradykinesia, yet reduce the motivation for time costly cognitive control processes [23 ], thus potentiating impulse control problems (Figure 1c). An account of dopamine's effects in terms of time costs is particularly promising in the context of the recent observation that dopamine neurons control the judgment of time [41].
Direct empirical evidence for a role of dopamine in cognitive motivation comes from a separate line of work, indicating that effects of monetary incentive reward (the promise of a bonus) on cognitive control vary as a function of striatal dopamine levels. This was shown to be the case in patients with Parkinson's disease depending on dopamine cell loss [42], as well as in healthy volunteers depending on striatal dopamine synthesis capacity, as indexed by 6-[ 18 F]fluoro-L-m-tyrosine (FMT) positron emission tomography [43]. Intriguingly, in these studies, the relationship between striatal dopamine levels and the effect of incentives on cognitive control was negative, such that higher striatal dopamine was associated with more detrimental effects of reward on cognitive control [43]. Conversely, patients with Parkinson's disease, which is accompanied by severe dopamine depletion in the striatum, have been shown to exhibit paradoxically greater beneficial effects of reward on cognitive control than controls [17]. Although the mechanism underlying these effects on incentivized cognitive control remains unclear, they are certainly reminiscent of the pattern of paradoxical effects of methylphenidate on the avoidance of cognitive control. Indeed, changes in the value of cognitive control might surface, in these tasks, in terms of changes in (the effect of reward on) task performance [44]. This concurs with the recent finding that the effect of reward on task (-switching) performance correlated with participants' scores on the need for cognition scale [45], which had been associated with the valuation of cognitive control in earlier work [6]. In the current set of tasks, patients with Parkinson's disease might exhibit greater beneficial effects of reward on cognitive control, because there is greater cost to be offset by increases in the benefits of cognitive control.

Noradrenaline and cognitive control avoidance
Many drugs, including amphetamine or methylphenidate, prolong catecholamine transmission in a nonspecific manner by targeting both dopamine and noradrenaline transporters [46]. There are multiple reasons for thinking that such drug effects on motivated cognition reflect not just modulation by dopamine, but also noradrenaline, not least for its well-known association with arousal and fatigue.
For example, according to the classic adaptive gain theory of locus coeruleus function, task engagement is modulated by activity of the locus coeruleus, which favors either exploitation (task engagement) or exploration (task disengagement) depending on a tonic or phasic mode of action [47]. In line with this, baseline pupil diameter at trial onset, a measure that has been associated with locus coeruleus activity [48], was found to correlate with lapses of attention in a sustained attention task [49], with participant's tendency to explore in a gambling task [50], with decisions to disengage from a (discrimination) task [51] and with mental fatigue [52]. However, in contrast to predictions of the adaptive gain theory, prolonging tonic noradrenaline levels pharmacologically by administering reboxetine, a selective noradrenaline reuptake inhibitor, failed to alter task (dis)engagement or exploratory behavior despite intervention effects on nonspecific autonomic nervous system parameters [53]. Thus, the jury is still out with regard to noradrenaline's role in exploration and task engagement. One way in which the locus coeruleus-noradrenaline system might alter task engagement and demand avoidance is by encoding unexpected (outcome) uncertainty or surprise due to errors in judging uncertainty [54]. For instance, greater outcome uncertainty might elicit greater task engagement given the greater likelihood of unsigned (surprise) prediction error signals at outcome [55], and thus greater potential for new learning, knowledge acquisition and curiosity relief [56]. Conversely, greater certainty about the outcome of performance, whether it is good or bad, might elicit boredom or learned helplessness respectively, thus reducing the opportunity for new learning and task engagement. Recent empirical evidence indicates that blocking noradrenaline, by propranolol, increases participants' confidence in good performance on a dot-motion task relative to placebo [57 ]. It would be interesting to contrast directly in future studies the putative role of noradrenaline in mediating a putative link between outcome uncertainty and task engagement with a putative role of dopamine in task engagement as a function of the expected value of an outcome, thus the probability (rather than uncertainty) of performing well.

Serotonin and cognitive control avoidance
Like the catecholamines, serotonin is a major neuromodulator that is strongly implicated in both motivation and cognitive (impulse) control. Serotonin transmission is perhaps best known for its association with (learning about) aversive outcomes, waiting and behavioral inhibition [58,59], although there is also extensive evidence for a complementary role in appetitive processing and reward [60,61]. In line with the idea that serotonin also plays a role in (the learning about time and/or effort) costs, the optogenetic activation of serotonergic neurons in the midbrain dorsal raphe nucleus reduced the cost of waiting. Timed activation decreased premature responding in a delayed reward task, promoting animals' patience to wait for a reward. Relatedly, an 8-week selective serotonin reuptake inhibitor intervention (escitalopram) in healthy humans improved decision-making about reward and (physical) effort costs by reducing specifically effort costs, leaving unaffected the weight of monetary incentives [62 ]. A key question for future work is whether such a dissociation extends from the domain of physical effort to that of cognitive effort.

Conclusions
In this review, we highlight the potential contribution of the major ascending neuromodulators, in particular dopamine, noradrenaline and serotonin, to our tendency to avoid cognitive control. We suggest that these chemical neuromodulators might alter cognitive control by altering not just the ability but also the willingness to exert cognitive control. In line with this hypothesis, catecholaminergic challenges, like amphetamine and methylphenidate, have been shown to alter demand avoidance while leaving unaltered the ability to perform well on a cognitive control task. Based on accumulating evidence from chemical and functional neuroimaging studies for a role for striatal dopamine in our motivation for cognitive control, we hypothesize that these catecholaminergic effects reflect in part modulation of striatal dopamine. Striatal dopamine might alter choices about cognitive control (avoidance) by modulating (learning about) the expected value (i.e. cost) of cognitive task performance. However, we also consider the role of noradrenaline in cognitive control (avoidance), and speculate that noradrenaline might contribute by modulating, instead, our uncertainty or confidence in the outcome of performance. Lastly, we hypothesize that serotonin might affect the motivation for cognitive control by modulating (time and/or effort) costs, specifically. Overall, this review highlights the relevance of advancing our understanding of the various cognitive computations carried by the different ascending neuromodulators for elucidating the basis of our tendency to avoid cognitive control.

Conflict of interest statement
Nothing declared.

References and recommended reading
Papers of particular interest, published within the period of review, have been highlighted as: of special interest of outstanding interest 1.
Botvinick, Braver: Motivation and cognitive control: from behavior to neural mechanism. Annu Rev Psychol 2015, 66: 83-113. A comprehensive review summarizing psychological and neuroscientific work on the interaction between motivation and cognitive control, the cost of cognitive control and relevant accounts, including reward-based models of the selection of cognitive computations.

10.
Kurzban R, Duckworth A, Kable JW, Myers J: An opportunity cost model of subjective effort and task performance. Behav Brain Sci 2013, 36:661-726. This extensive review describes the qualia of mental effort and associated reduction of task performance to arise from a limited capacity mechanism and to correspond with an opportunity cost, namely the value of the next best alternative.