Cognitive control, motivation and fatigue: A cognitive neuroscience perspective

The present article provides a unified systematic account of the role of cognitive control, motivation and dopamine pathways in relation to the development of fatigue. Since cognitive fatigue is considered to be one aspect of the general control system that manages goal activity in the service of motivational requirements (Hockey, 2011), our focus is also broader than fatigue itself. The paper shall therefore first focus on the motivation-control interactions at the level of networks of the brain. A motivational control network is argued to play a critical role in shaping goal-directed behavior, in conjunction with dopamine systems that energize the network. Furthermore, motivation-control interactions as implemented in networks of the brain provide an important element to elucidate how decision making weighs both the anticipated benefits and costs of control operations, in optimal and suboptimal conditions such as mental fatigue. The paper further sketches how fatigue affects the connectivity of large-scale networks in the brain during effortful exercition, in particular the high-cost long striatal-cortical pathways, leading to a global reduction of integration in the brain's network architecture. The resulting neural state within these networks then enters as interoceptive information to systems in the brain that perform cost-benefit calculations. Based on these notions we propose a unifying cost-benefit model, inspired by influential insights from the current neuroscience literature of how fatigue changes the motivation to perform. The model specifies how the reward value, effort costs and fatigue aspects of task performance converge in the medial prefrontal cortex to calculate the net motivation value of stimuli and select the appropriate actions.


Introduction
Although everyone knows by intuition what fatigue is, the concept as such has too many meanings to justify a univocal definition, analogous to the frequently quoted example of attention (Hommel et al., 2019). Does mental fatigue perhaps reflect a process similar to physical fatigue, but on the level of the brain? The time-on-task effects of cognitive and physical fatigue indeed share the common feature of depending on the level of difficulty required in a task (van Cutsem et al., 2017). But although mental and physical fatigue do overlap to a certain extent, since the domains of the body and mind share important characteristics at the level of the brain, mental fatigue has also a unique signature.
Mental fatigue (henceforth fatigue) occurs after prolonged periods of demanding cognitive activity, but also after sleep loss. In its more extreme form fatigue may lead to mental exhaustion, or a state of chronic fatigue ('Chronic Fatigue Syndrome' CFS; Lang et al., 2005). CFS is a name for a group of complaints that are dominated by persistent fatigue, a state that is not due to physical exercise and is not relieved by rest (Cook et al., 2007). Taken together, the major symptoms of fatigue in daily life are a loss of concentration, of quality of life and of work productivity. Fatigue has also been described as 'brain fog': a collection of symptoms involving memory problems, lack of mental clarity, poor concentration, inability to focus attention and feelings of being without energy or even 'washed-out'.
To date, there has been no unifying framework to describe fatigue as it is manifested, not as a symptom of neuropathological states but of suboptimal working conditions. The Motivated control Model described by Hockey (2011Hockey ( , 2013) was a first systematic attempt to describe how performance under demanding conditions such as stress or performing mentally demanding tasks depend on mobilizing required cognitive resources. If task goals are deemed sufficiently important, allocation of such resources can be channeled through exertion of compensatory effort.
Reward based decision making is another promising avenue that could help in clarifying the role of fatigue (Massar et al., 2018). The field has received considerable support from neuroscience studies, helping to further specify areas and circuits in the brain involved in reward evaluation, effort allocation and decision making (Botvinick and Braver, 2015). More recently, this further took shape in the theory that effortbased decision processes are mediated by a distributed hierarchical neural network that incorporates different regions of the frontal lobes, the amygdala, the basal ganglia, and the dopamine (DA) system (Stenhav et al., 2013;Massar et al., 2018;Müller and Apps, 2019). This integrated approach has also helped to elucidate how fatigue changes the motivation to perform, as well as to test predictions from fatigue models at the neural level. Fatigue research could thus greatly benefit from insights derived from decision neuroscience, considering that fatigue reflects a state of the organism that would directly affect the balance between the costs one is willing to endure to obtain the benefit of achieving a goal. Available evidence further suggest that motivational effects in control reflect interactions between two large-scale brain networks, one involved in representing reward value and the other involved in implementing control function (Pessiglione et al., 2018). Their interface could potentially incorporate a core mechanism underlying the theme of reward maximization, that is keeping a balance between mental effort and rewards in decision making. Since concepts as cognitive control and motivation can take on different meanings, the following paragraphs will first focus in more detail on their characteristics (See Box 1).

Control and motivation systems in the brain
Cognitive control and motivation are often treated as a dichotomy. Cognitive control (or simply: control) is taken to refer to a set of functions that regulate basic attention-, memory-, language-, and actionrelated faculties and coordinate their activity in the service of specific tasks. It has emerged as a core research topic in cognitive neuroscience, since its early introduction (Miller, 2000;Posner and Snyder 1975). Motivation in turn, refers to the orienting and invigorating impact on behavior and cognition, of prospective reward that is both extrinsic and intrinsic and tied to the satisfaction of self-relevant behavioral goals. Recent research on cognitive control has increasingly focused on its interaction with motivation, a field referred to as motivated control (Hockey, 2011;Braver et al., 2014;Botvinick and Braver, 2015;Pessiglione et al., 2018). Motivated control implies the coordination of behavior to achieve affectively valenced outcomes or goals, which in turn implies a distinction between control and motivational processes, mapped to distinct sections of the brain (Pezzulo et al., 2018). Within the framework of theories of control, motivation is typically considered as a modulatory system that affects the efficiency of working memory and attentional functions in the cognitive control system, perhaps also by enhancing perceptual discrimination via dopamine (DA) (Popescu et al., 2016). Taken together, reward evaluation, motivated control and expectation are considered to be the core processes underlying the joint operation of motivation and cognitive control at the level of the PFC (see Box 1).

Functional anatomy
Control over behavior has traditionally been assigned to two subdivisions of the prefrontal cortex (PFC) of the brain, the dorsolateral PFC (dlPFC) and the ventromedial PFC (vmPFC), located respectively above and below the principal sulcus of the macaque prefrontal brain (homologous to the inferior frontal sulcus of the adult human; see Box 2 and Fig. 1 for a summary of terms and involved structures). The dlPFC is allocated a role in the maintenance of representations of a task set to guide temporally integrated, goal-directed behavior, in order to bias processes that depend on the posterior brain area (Botvinick et al., 2009). The ventral part of the lateral PFC, called the vlPFC, in turn has dense reciprocal connections with the temporal cortex of the posterior brain, necessary for processing of object information. The dlPFC is part of the Central Executive Network (CEN) notably responsible for the control of attention and working memory functions, recruiting also the posterior parietal cortex (PPC), responsible for the integration of spatial relationships. The same network has also been referred to as the Frontoparietal Control Network (Ridderinkhof et al., 2004;Kok et al., 2006;Kok, 2019;Harding et al., 2015). The engagement of effort seems closely tied to engagement of the CEN.
The vmPFC is a broad area of the frontal lobe at the bottom of the two cerebral hemispheres (Sallet et al., 2011). The vmPFC is also a central region in the Default Mode Network of the brain, associated with a state of leisure, contrasting its function with that of the dlPFC as part of the Central Executive Network. Functionally, the vmPFC plays a vital role as the backbone in the limbic system. In association with the medial part of the orbitofrontal cortex (OFC) and the anterior cingulate cortex (ACC) it participates in performance monitoring, action evaluation and detection of events that indicate the need for behavioral adaptation and action revaluation (Brown and Braver, 2005). Together with two lower sections, the anterior insula (AI) and the ventral part of the striatum (VS), it will further be referred to as the motivation system: a network set to the coordination of behavior to achieve affectively meaningful outcomes or goals (Fig. 1).

Box 1
Mental effort and the brain.
Mental effort is a central concept in cost-benefit analysis models, where the current and anticipated costs of mental effort for a particular activity are balanced against the anticipated benefits (Székely and Michael, 2020). Importantly, performance decline due to fatigue may depend on a reduced desire to exert further effort, suggesting that under fatigue, the integrated effort-value function would be shifted toward a diminished preference for effort (Kanfer, 2011;Massar et al., 2018). Although satisfactory definitions of mental effort are still lacking, Inzlicht et al. (2018) characterized as a rough starting point mental effort as 'the subjective intensification of mental and/or physical activity in the service of meeting some goal'. Under demanding conditions (e.g., stress, fatigue), and if task goals are deemed sufficiently important, effort implies the mobilizing of cognitive resources (Hockey, 2011). Yet, this comes at the expense of increased discomfort and fatigue (Massar et al., 2018). The question how mental effort enhances cognition on the level of the brain remains largely unanswered, but has come increasingly to center stage, with a surge of new studies with a particular focus on neuroscience research examining the interface of motivation and cognitive control (Botvinick and Braver, 2015). One hypothesis is that effort improves the signal-to-noise ratio in the neural coding of task rules within the prefrontal cortex (PFC), thus increasing the efficacy and precision of cognitive control performance. Such an account would be consistent with experimental and computational studies of the effects of dopamine (DA) on activation of the PFC (Yee and Braver, 2018). Accordingly, neurotransmitters have been assigned a central role as energetic modulators or enhancers of these processes (Aarts et al., 2011).
Notice that the human amygdala, in particular the basolateral nucleus (BLA) has also been implicated in adaptive goal-directed behavior, positing it as a core element in the reinforcement learning networks involved in the appraisal of negative as well as positive reinforcers (Bechara et al., 1999;Basten et al., 2010;O'Neill et al., 2018). The BLA and orbitofrontal cortex (OFC) are two reciprocally connected key nodes in the circuitry controlling outcome-guided behaviors that have much in common. Considering that both areas play a role in evaluating the affective or rewarding character of stimuli, but that these evaluation processes take place at a higher level in the OFC, the focus of the following sections will be primarily on the (medial) OFC and vmPFC assuming that they also incorporate some of the appraisal functions of the amygdala in reward evaluation and decision making.

Motivated control: Linking the motivation and cognition control systems
Working on a thesis a student uses cognitive control in the sense of mobilizing attentional and working memory functions in order to concentrate optimally on the content of the work. But problems with concentration or maintaining an optimal focus of control (i.e. the classical executive functions) are likely to occur when the student does not actually want to concentrate or finds the work boring. Putting it differently: control is driven, powerfully and fundamentally by the student's desires and goals. This example exemplifies that both motivation and cognitive control play critical roles in shaping goal-directed behavior, but only recently has scientific interest focused around the question of motivation-control interactions (Ridderinkhof et al., 2004;Hockey, 2011;Botvinick and Braver, 2015;Yee and Braver, 2018). In short: control is motivated, in the sense that 'motivational factors fuel executive functions' (Botvinick and Braver, 2015).
When performing a pleasant and intrinsically motivating task in optimal working conditions, cognitive control may be allocated in a 'flow', that is a state of subjectively effortless automatic processing (Csikszentmihalyi, 1990, for introducing the flow concept, and Westbrook and Braver, 2015). In an unpleasant task however, performed Box 2 Glossary of anatomical terms as used in the present paper.
• Cognitive system (dlPFC, PPC, SMA, THAL, dACC): a hierachical system functionally similar to the Central Executive Network. Also referrred to as control system. • Motivation system (vmPFC, mOFC, ACC, amygdala, AI, VS): a hierarchical system set to the coordination of behavior to achieve affectively meaningful outcomes or goals. Overlaps with the limbic system. • Dorsal PFC (dlPFC, dmPFC) • Ventral PFC (vlPFC, vmPFC, OFC) • Medial PFC (vmPFC, dmPFC, mOFC, ACC) • Motivated control: the coordination of behavior to achieve affectively meaningful outcomes or goals via the interaction between the control and motivation systems. Also defined as the system that drives executive control. Fig. 1. Networks of the cognitive system (upper part) and the motivation system (lower part) of the brain. The dotted circles in the lower part represent the medial locations of the motivation and cognitive systems. The amygdala, sensory and motor cortices are not shown for the sake of parsimony. The solid lines represent reciprocal connections between the nonoverlapping locations within each network. SMA supplementary motor area, VS ventral striatum, DS dorsal striatum, AI anterior insula, mOFC medial orbitofrontal cortex, THAL thalamus, ACC Anterior cingulate cortex, dACC dorsal ACC (upper part ACC). Notice that the term vmPFC is also used to reflect the interconnected and overlappigng network of regions in the lower medial and orbital prefrontal cortices (thus also incorporating the mOFC).
under suboptimal conditions (e.g. fatigue, environmental noise) where performance does not meet the desired levels, effort will likely be allocated voluntarily and probably also experienced as unpleasant. In short: motivated control may be allocated automatically (or effortless), when task demands and the desired levels of performance are in balance, or they may require the application of mental effort when these two task variables are not in balance.
The motivation system, as displayed in Fig. 1, must also interact with the cognitive control system at various levels of their respective hierarchies. A challenging but still unresolved problem is how the motivational and cognitive control systems communicate with another, and which neural structures or pathways link them together (Braver et al., 2014). A set of structures at the interface of the two systems have been suggested as possible candidates to control the cross-linking between vital neural structures within each hierarchy (Kouneiher et al. 2009;Alexander and Brown, 2011; see also some recent simulation studies using a computational framework of hierarchical reinforcement learning, to interpret recent empirical findings (Botvinick and Weinstein, 2014;Holroyd and McClure, 2015;Pezzulo et al., 2018).
The model in Fig. 1 does not pretend to provide a quantitative computational model of control-motivation interactions. Rather it serves as a global integrative framework of neural components and their interactions involved in motivated control, to guide the more detailed discussion of empirical findings in subsequent paragraphs.
Bidirectional pathways running through the cingulum could further provide the structural links of their interface. The cingulum is a long curved bundle of nerve fibers of the neurons in the cingulate gyrus that connects structures in the limbic system such as the ACC with systems in the dorsal medial PFC (Petrides and Pandya, 2002). These structural connections would ideally be implemented in the anatomically interfacing structures such as the dorsomedial PFC and dACC, connected via pathways of the cingulum, extending from the posterior to anterior sections of the medial PFC (see Bubb et al., 2018, for a review). In this way they would effectively link the cognitive control system with the motivation system. This view would also be in accordance with a pivotal function of the dACC in the hierarchical organization of effortful behavior as suggested in various studies (e.g. Holroyd and McClure, 2015) and, more specifically, with the prediction and evaluation of behavioral outcomes (Alexander and Brown, 2011).
A second interactive link between the motivational and cognitive control systems, to be discussed in the following section, is striatalprefrontal dopamine (DA), that functionally connects these systems, in particular with respect to the stability of task control (Floresco et al., 2003;Aarts et al., 2011;Cools, 2016;Yee and Braver, 2018).

The role of dopamine in motivated control
DA is produced in the ventral tegmentum area (VTA) of the midbrain and projected to the ventral striatum (VS, also referred to as nucleus accumbens in animal studies) in the basal ganglia of the forebrain. Some dopamine neurons encode motivational value, supporting brain networks for seeking, evaluation, and value learning (see Box 3).
Others encode motivational salience, supporting brain networks for orienting, cognition and general motivation (Bromberg-Martin et al., 2010). Motivation-cognition interactions have long been proposed to reflect dopamine-dependent interfacing between different parallel frontostriatal circuits associated with motivation and cognition (Aarts et al., 2011;Yee and Braver, 2018).
The neural basis for these interactions are long-ranging looping parallel circuits between the striatum and prefrontal cortex, that globally can be subdivided in a motor control section, running from the substantia nigra (SN) through the dorsal striatum and projecting to the motor cortices, and a motivational control section, running from the ventral tegmental area and ventral striatum (VTA/VS; see Box 2) and projecting to the medial frontal structures, like the ACC and the vmPFC (Haber and Knutson, 2010).

Mental fatigue and the brain
Prolonged active engagement in cognitively demanding tasks not only requires effort, but often leads to a subjective state labeled cognitive fatigue (Meijman, 1991;Lorist et al., 2005). Fatigue is often accompanied by a gradual decline in the level of performance, with is more conspicuous in more difficult or prolonged tasks (Boksem and Tops, 2008). Unlike symptoms of state fatigue occurring in reflecting temporary costs of high energy expenditure, chronic fatigue relates to long lasting conditions of structural and functional impairments such as the Chronic Fatigue Syndrome (CFS, Lang et al., 2005, Cook et al., 2007.

Neurotransmitters and fatigue
Catecholamines and neurotransmitters seem to be the most likely candidates for an invigorating mechanism to control task performance in suboptimal conditions like fatigue (Cools, 2016). Indeed, fatigue, whether chronic or acute, appears to involve a neuronal circuitry which depends on the accumulation of specific neurotransmitters in the synapse, as well as on their sensitivity to bind with post-synaptic receptors. This involves in particular monoamine neurotransmitters, which include serotonin (5-HT), noradrenalin (NA) and dopamine (DA). All of these, except DA, are distributed to all cortical areas: DA is distributed to frontal and cingulate areas only. It has been further shown to interact with acetylcholine (ACh) in the ventral striatum in the control of GABA (an inhibitory neurotransmitter), with opposite roles in balancing approach and avoidance reactions (Hoebel et al., 2007).
DA and fatigue With respect to the regulatory role of DA in fatigue,

Box 3
Two dopamine systems.
Of particular importance are two pathways in which dopamine (DA) is released in the ventral tegmentum area (VTA), that project to different prefrontal regions and their related DA firing patterns. In the mesocortical system DA potentiates stable tonic patterns of neuronal firing in widespread regions of the prefrontal cortex via D1 receptors. The mesocortical system interacts with the mesolimbic system that produces more fast-acting phasic patterns of neuronal firing. This system projects from the VTA to D2 receptors in the ventral striatum (VS) that subsequently enhances information on the reward value of stimuli. Furthermore, fast, 'phasic' burst like firing induces massive synaptic DA release, which is rapidly removed by reuptake before escaping the synaptic cleft, whereas increased population activity modulates slower 'tonic' extrasynaptic DA levels that are less influenced by reuptake (Floresco et al., 2003;Cools, 2016). Notice that the pathways running from the PFC to the striatum form a closed loop with the ascending striatal-PFC pathways (Van Schouwenburg et al., 2010;Van Schouwenburg et al., 2012). Suggesting that DA in the VS could be released in a direct (bottom up) as well as indirect (top down) fashion.
several studies have corroborated a DA balance hypothesis, implying that the facilitatory effects of DA are absent or attenuated with too much or too little DA (see Dobryakova et al., 2013 and, for a review). Accordingly, feelings of fatigue might then be more likely to emerge in conditions where there is either too little or too much of DA. The DA balance proposal is further supported by pharmacological studies on the effect of amphetamines and their agonists (Heal et al., 2013). For example, the DA agonist methylphenidate was shown to boost the functioning of brain regions with low baseline levels of DA more readily than that of brain regions with already optimized DA levels. These results further suggested that while fatigue decreases with DArgic medication, the effect appears to be dose-dependent (Johansson et al., 2014). In addition, with advancing age there are alterations of DA receptors and DA synthesis, with receptors generally showing reduction, and synthesis demonstrating increases (Bäckman et al., 2006;Berry et al., 2016). The role of serotonin/DA interactions in fatigue Studies of fatigue in clinical as well as normal populations have pointed to the role of DA, possibly in interaction with serotonin (5HT), as an important contributing factor. 5-HT is produced in the raphe nuclei that connect with the insula as well as the ACC. There further seems to be a general consensus that while DA serves to promote behavioral activation to seek rewards, 5-HT serves to inhibit actions when punishment may occur (Floresco and Tse, 2007;Cools et al., 2011). It was also hypothesized that in this respect 5-HT could play a similar role as DA in trading off the costs and benefits of waiting to avoid punishment . Others have argued since both DA and 5HT affect the striatum, their balance, e.g. high (DA++/5HT+) versus low (DA+/5HT++) would respectively lead to a stronger or weaker DA output from the striatal-cortical pathway. A low ratio of 5-HT to DA would then favor increased motivation, improved performance and positive feelings, and a high ratio to decreased motivation and promoting lethargy and/or fatigue, resulting in decreased performance (Meeusen et al., 2006). Accordingly, Davis and Bailey (1997) argued that not 5-HT, but the interaction between 5-HT and DA influences central nervous system fatigue. This implies that a 5-HT-to-DA ratio is more relevant for determining fatigue than analyzing or manipulating only one of the two transmitters (Boureau and Dayan, 2011;Cordeiro et al., 2017).
Taken together, despite the large body of evidence showing the involvement of midbrain-to-PFC DA pathways in the regulation of fatigue, it seems justified to assume that 5-HT-DA balance might be the key neuromodulating principle in fatigue-related motivation-cognition interactions (Dobryakova et al., 2015). The primary function of high 5-HT in the 5HT-DA balance (such as D+/5-HT++) would then be to suppress DA facilitatory effects when the non-appetitive quality of reinforcing stimuli starts to dominate their appetitive quality, concomitantly triggering feelings of fatigue.

Fatigue affects networks in the brain
At the neural level cognitively demanding tasks are shown to elicit changes in several components of the networks associated with motivated control (Fig. 1). Most structures in these networks are densely interconnected, and would also be sensitive to cognitive fatigue. This was confirmed in studies using functional neuroimaging (fMRI) that identified several regions related to cognitive fatigue and potentially comprise a "fatigue network" (Wylie et al., 2020). These include the striatum of the basal ganglia, the dorsolateral prefrontal cortex (dlPFC), the dorsal anterior cingulate cortex (dACC), the ventro-medial prefrontal cortex (vmPFC) and the anterior insula (AI).
In addition, studies of the reorganization of the brains functional networks, using techniques allowing to map the overall architecture of the brain denoted as the 'connectome' (Sporns et al., 2005;Bullmore and Sporns, 2012) have shed new light on how connectivity of networks in the brain are modulated by conditions of prolonged mental work and fatigue. This research can be globally subdivided in fMRI and EEG/MEG connectivity studies, to be briefly discussed below. fMRI connectivity studies based on graph theory (a mathematical framework to characterize topological properties of the brains networks) globally supported the view that fatigue leads to a reduced global integration of the brain networks. This became manifest in a higher path length, lower global efficiency, and an increased local segregation (clustering coefficient and/or local efficiency) in the fatigue state (Mueller et al, 2014, Esposito et al., 2014Zhao et al., 2017;Qi et al., 2019;Baran et al., 2019;Farahani et al., 2019;Wylie et al., 2020). For example, after sleep restriction, the characteristic path length significantly increased and 'small-worldness' (a property of a functional network with many dense short-ranging connections and more sparse long ranging connections) significantly decreased. Taken together, these studies supported the theory of global functional workspace (Dehaene and Naccache, 2001;Baars, 2002) that an integrated network topology is needed to support high task demands. In contrast, repeated use of cognitive resources during prolonged time-on-task (TOT) is shown to lead to disintegrated and more segregated network architecture of the brain affecting in particular long-connection striatal cortical pathways.
EEG/MEG studies. Neural oscillations and their inter-areal synchronization as manifested in the electrical (EEG) as well as magnetic field (MEG) recordings have an essential function in regulating neuronal communication in distributed networks. In particular cross-frequency interactions among oscillations have been proposed as a likely candidate mechanism for such integration. EEG/MEG connectivity studies suggested that fatigue was associated with a state of lower alertness of the brain as reflected in increasing power of the background EEG, lower dopamine (DA) levels as well as with a change of the functional network topology. These studies broadly reported that with increasing time-ontask (TOT) levels of fatigue were accompanied by increases in lower occipital alpha (i.e., 8-10 Hz) and frontal theta (i.e., 4-7 Hz) band power. This was taken to reflect a shift toward a more economic, but less efficient configuration, implying a) low wiring costs and b) disruption of the effective interactions between and across cortical regions during mental fatigue (Ten Caat et al., 2008;Kitzbichler et al., 2011;Sun et al., 2014;Wascher et al., 2014;Clayton et al., 2015;Zhao et al., 2017).
In sum, the latter studies suggested that fatigue was associated with a state of lower alertness of the brain as reflected in increasing power of the background EEG, lower DA levels as well as with a change of the functional network topology. This was taken to reflect a shift toward a more economic, but less efficient configuration, implying a) low wiring costs and b) disruption of the effective interactions between and across cortical regions during mental fatigue.

Reward based decision making
Reward based decision making is a general theoretical framework based on a cost-benefit decisions system that uses evaluation of rewards and mental effort as inputs. It has its origin in economic theories, where concepts like utility and disutility of effort are balanced with the incentive properties of the contract and monetary rewards (Hare, 1951;Kaplan and Schulhofer-Wohl, 2018). Benefits can vary between the payoffs associated with successful performance in the target task (e.g. bonus payments or positive reward prediction errors, i.e. when an event is better than expected). Methods and insights from the field of effortbased decision making have thrown a new light on how fatigue changes the motivation to perform (Pessiglione et al., 2018;Massar et al., 2018).

The dynamics of cost-benefit decisions: The costs of control
Value-based decision-making involves trading off the cost associated with an action against its expected reward. Research has shown that both physical and mental effort constitute such subjective costs, biasing choices away from effortful actions, and discounting the value of obtained rewards. Elementary principles of decision theory assume that when considering whether or not to take a course of action, agents contrast costs and benefits to get a net value. Note that the net value or goal can include both positive and negative elements, which may be called gains and losses or rewards and punishments (Pessiglione et al., 2018). If the action is compared to doing nothing, then it is engaged only when its net value is positive (Fig. 2). If it is compared to a set of alternative actions, then it is engaged only when its net value is above the other. For example, the positive side of a certain task goal like gaining an influential position in an organization is more important than the negative side such as the many involved responsibilities or being the object of criticism. If goals can be reduced to the rewards that actions provide, and if actions can be reduced to the amount of effort that they involve, then net value can be simplified to the equation: with V(Ei) being the net value of producing effort Ei, R(Ei) the reward associated to effort Ei and C(Ei) the cost of producing effort Ei (Pessiglione et al., 2018). The net value should also incorporate the effects of fatigue that could change the perception of the values of reward as well of invested effort (Müller and Apps, 2019). From the earliest definitions mental effort, like physical effort, was assumed to carry intrinsic disutility, meaning that people spontaneously seek to minimize it (Stenhav et al., 2013). The overall results of this principle appear consistent with the law of 'least work', or 'least mental effort' the idea that, all else being equal, actions tend to be selected to minimize cognitive demands in attention-demanding tasks (Hull, 1943;Kool et al., 2010Kool et al., , 2017. Minimizing task demands will determine the amount of effort invested and, everything else equal, will inevitably result in performance reductions (Kurzban et al., 2013).
A further suggestion from these studies is that not the task demands as such but the balance between the cost of spending energy and the tasks -extrinsic or intrinsic-incentives that drive our motivation to pursue the current task goals.

The role of fatigue
Under fatigue, the disutility of effort would increase, leading to less allocation of effort, and thus a lower level of performance (Kanfer, 2011). A similar view was expressed in the framework proposed by Müller and Apps (2019), that a greater intensity control signal would be required to overcome this cost to simultaneously maintain task performance and the vigor of actions. The result is a reduction in the willingness to exert effort and concurrent devaluation of the rewards associated with acting (see also Müller al., 2021, for a recent discussion of this issue). Kurzban and coworkers (2013; further argued that the subjective feelings of effort and fatigue can be understood as the felt output of the cost-benefit computations. This comes close to the theory of Hockey (2011) stating that fatigue is "an adaptive state that signals a growing conflict in control activity between what is being done and what else might be done". He similarly posits an 'effort monitor which functions to evaluate the value of pursuing the current goal relative to alternate goals'. Mental or cognitive fatigue is often characterized by a reduced motivation for effortful activity that creates a problem of the management of control rather than of energy (Hockey, 2011). Importantly, fatigue may also have a unique signature and arise as an intrinsic physiological state within circuits or networks that are recruited during effortful exercition (as described in section 3.2). These changes in activity subsequently enter as interoceptive information in areas processing internal bodily signals, which in turn connect to systems in the brain that guide motivated behavior (Müller and Apps, 2019).

Reward based decision making from a neural perspective
In the earlier paragraphs of this paper we described two large-scale brain networks, one centrally involved in representing reward value, and the other involved in implementing control. We now move to the question how the neural structures in these networks may provide a causal mechanism underlying reward based decision making. More specific, we shall focus on how the trade-off between effort and reward in cost-benefit analysis are implemented in specific cortical, subcortical and neuromodulatory systems.
The role of DA in decision making The generally accepted view is that DA-dependent mechanisms enhance reinforcement learning signals in the ventral striatum (VS), and in turn sharpen representations of associative values, and produce positive feelings. For example, Rutledge and coworkers found that by boosting DA levels using levodopa (L-DOPA), human subjects increasingly reported positive feelings in a decision making task (Rutledge et al., 2015).
The coarseness of neuromodulators like DA would however be difficult to reconcile with a direct role in creating specific patterns of synaptic connections, such as those committed to precise encoding and retrieval of representations at content addressable memory locations (Schultz, 2019). DA released in the striatum may then not reflect the behavioral selection and decision selection process themselves, but rather a bias or commitment to a cortically developed choice (see Cisek and Kalaska, 2010;Cisek and Pastor-Bernier, 2014 for similar suggestions). Thus, DA would more likely affect the decision parameter indirectly by creating a bias, lending more weight to positive goal values or rewards. Instead, structures more upstream in the brain, such as the ACC, supported by the VS and possibly also the ventral medial PFC and the dlPFC, would be more suitable neural candidates for a specific selective role in a reward decision system (Walton et al., 2006;Grabenhorst and Rolls, 2011). Which could include processes such as monitoring, a decision to act or not to act, or even to select the appropriate action (Ballard et al., 2011;Holroyd and McClure, 2015).
Neural components involved in cost-benefit calculations Changes in the blood oxygen level dependent (BOLD) signals have been shown to reflect the accumulation of a cost-benefits difference signal from the vmPFC (Basten et al., 2010). The vmPFC, together with the ventral striatum (VS) are key players in the reward network, processing the goal (reward) values of performance. In addition, Stenhav et al., (2013) proposed that the dorsal anterior cingulate area dACC, acts as a critical area to integrate information about the rewards and costs expressed in the Expected Value of Control (EVC), a measure representing the net value associated with allocating control to a given task. In particular, they proposed that the dACC registers the costs of control in a manner Fig. 2. A simplified view of cost-benefit analysis with major inputs and outcome, resulting in a net value leading to a decision to act or not to act. The theoretical option is that the brain weighs costs such as invested effort (energetical demands) against benefits (rewards) by combining neural benefit and cost signals into a single, difference-based neural representation of net value. The weighing process is accumulated over time until the individual decides to accept or reject an option. Effects of fatigue can change the perception of the net value of reward as well as effort, leading to lowering of the current tasks goals.
that is proportional to the intensity of control, and that it specifies control signals in a way that is sensitive to such costs. As discussed earlier, the dACC could play a pivotal rol in linking motivational and cognitive elements of cost-benefit calculations, via pathways of the cingulum connecting the ventral and dorsal sections of the medial PFC, with the dmPFC in turn coactivating the dlPFC (Fig. 1).
The insula is another neural component related to cost-benefit calculations, where its core function is to mark salient or alerting novel events for additional processing and to initiate appropriate control signals (Menon and Udin, 2010;Uddin et al., 2017). In addition to its function as signaling salient events, the anterior dorsal insula (AI) has been typically described as an integrative hub receiving interoceptive stimuli from other parts in the body, in particular autonomic afferents of the brain, which are then relayed to the ACC in the medial PFC. Interoceptive stimuli also incorporate bodily signals of incipient fatigue and effortful performance (Perry et al., 2019), dubbed 'somatic markers' (Bechara et al., 1999;Damasio, 1996). As such the AI could fulfill a function of monitoring effort and fatigue, which evaluates the value of pursuing the current goal relative to alternate goals (Hockey, 2011(Hockey, , 2013. Multicomponent networks Instead of searching for specific areas in the brain, other recent studies have proposed multicomponent networks to perform cost-benefit calculations. For example, in their review of current neurophysiological and neuroimaging research Müller and Apps (2019) suggested a 'domain general' system with common functional properties comprising the lateral PFC (dlPFC), anterior insula (AI) and dorsal anterior cingulate cortex (dACC), and functioning as core areas to perform as a circuit evaluating the costs and benefits of exerting effort. This supposition aligns with functional brain imaging studies identifying approximately identical core regions that potentially comprise a 'fatigue network', with the striatum, the dlPFC, the AI and the vmPFC functioning as central nodes (Wylie et al., 2020). Furthermore, as suggested by Müller and Apps, during demanding physical and cognitive tasks fatigue signals could arise in widespread areas in the brain, such as the motor, premotor, SMA, somatosensory areas and posterior insula feeding as interoceptive stimuli into these core areas. The resulting bodily signals would then be experienced as a depletion of mental energy and incorporated in the evaluation of tasks motivational values and costs of pursuing cognitive labor (Inzlicht et al., 2014).
A similar multicomponent cost-benefit network was suggested by Pessiglione et al., (2018) on the basis of a meta-analysis of functional MRI studies. In their dual network model, effort and reward are processed along separate neural pathways, with the goal value (rewards) conveyed by ventral fronto-striatal circuits (vmPFC and VS), and the effort cost transmitted by the AI (Fig. 3; see also Müller et al., 2021 for a similar view regarding the role of the frontal-striatal system).
The two pathways converge in the dACC, which integrates their information in a net value. The AI and ACC cortices have a close functional relationship, such that they may be considered together as input and output regions of the same functional system (Medford and Critchley, 2010).
Moreover, it seems a plausible assumption that effort and fatigue covary in the sense that fatigue increases as effort is exerted and partially decreases with rest. Accordingly, in the model of Fig. 3 the AI would then transfer these interoceptive fatigue signals (that could arise in various part of the brain, as suggested earlier) in parallel with those of effort costs via the dACC to the dlPFC functioning as a key node in the control network initiating motor commands.
Control processes involved in cost-benefit decisions are also modulated by DA, which involves prefrontal as well as striatal DA (Cools, 2016). The association between dopamine and the incentive effects of rewards places dopamine in a key position to promote reward-directed action (Fig. 3). With respect to the relationship between DA and effort there seems to be a general consensus across studies that DA primarily codes for future reward but is less sensitive to anticipated effort cost (Walton and Bouret, 2019). Guitart-Masip et al. (2012), using a task that explicitly dissociated action and reward value, showed that DA enhanced the neural representation of rewarding actions, without significantly affecting the representation of reward value as such. They suggested that response vigor, regulated by the dorsal striatum could be an important dimension of DA working in interaction with reward value depending on the VS.
Finally, other neurotransmitters such as noradrenalin (NA) and possibly also serotonin (5HT) might further contribute to modulate functions of the dACC and the anterior insula respectively.
Dynamics of neural decisions. Cost-benefit calculations implemented in networks of the brain, as displayed in Fig. 3, would likely recruit the pathways that function as the interface between cognitive control and Fig. 3. Plausible anatomical locations for the components of a network involved in cost-benefit calculations. The pivotal region would be the dACC, which integrates the goal value conveyed by ventral frontostriatal circuits (vmPFC, VS), and the effort cost transmitted by the anterior insula. Neurotranmitters associated with each structure are presented below. The dACC would then send the net value to the dlPFC, further elaborating motor commands in the premotor (SMA) and motor areas. Adapted from Pessiglione et al., (2018). The upper and lower shaded areas correspond with the motivation and control networks respectively; see text for clarification. motivation, that is: the dense lateral connections between the medial ventral and lateral dorsal sections of the PFC.
The computation of a net motivational value-to action signal, would then take form as a gradual build-up of neural evidence, rather than a 'discrete' (all-or none) process. The affordance model of Cisek (2007), developed for sensory-motor interactions, states that the brain processes sensory information to specify, in parallel, several potential actions that are currently available. These potential actions compete against each other for further processing, while information is collected to bias this competition until a single response is selected. Following Ciseks model as a guiding principle for motivation-control interactions, the reciprocal integration of control and motivational processes in cost-benefit calculations would afford lower levels in the motivation hierarchy start to affect or bias decision making in a fairly early stage of processing. Accordingly, motivation-control interactions would be taking place in a cascade between various stations of the value-to-action pathway of the cingulum, with the spatial-temporal pattern of activations reflected at different (shorter to longer) timescales (Kouneiher et al., 2009). In particular the dACC would be a suitable structure to link valuative information to action centers in the medial and dorsal sections of the prefrontal cortex (Rushworth et al., 2004. With the vmPCF on the apex of the motivation hierarchy supplementing the ACC, and providing the interface with the premotor areas to initiate action programs via its lateral connections with the dlPFC. Summing up, in the studies cited above the dACC appears to function as a pivotal structure in a triangular network serving integration of goal value (rewards) conveyed by ventral fronto-striatal circuits (vmPFC and VS) and the effort costs and incipient fatigue signals transmitted by the anterior insula. Importantly, output of the dACC may also incorporate aspects of specification that is a decision on which task or action should be undertaken, and on how intensively it should be pursued (Stenhav et al., 2013;Holroyd and McClure, 2015). The option suggested by Stenhav et al., is that the specified control signals (representing the 'Expected Value of Control') are implemented by the PFC and associated structures, which are assumed to be responsible for the regulative function of action controlthat is, actually effecting the changes in processing required to perform the task. This, in turn, could be the implemented in more distant regions like the dlPFC or the premotor areas such as the supplementary area (SMA, see also Fig. 3).

Conclusions and discussion
In the present article, motivated control is conceived of as the product of cognition-motivation interactions, implemented globally in two networks underlying the motivation and cognition control systems (Fig. 1). These networks have recently been be conceptualized as two separate but interacting control hierarchies, each having several levels of organization (Botvinick and Braver, 2015;Pezzulo et al., 2018).
Considering the complexity of this paper, dealing with general cognitive and neural systems underlying motivation and control, as well as the modulation of neural architecture of these systems under conditions of fatigue, we shall systematically discuss its major implications in separate sections.
Resource allocation versus cost-benefit frameworks More than a century ago the concept of mental energy or cognitive resource was introduced by William James (1907). Despite its criticism (Inzlicht and Smeichel, 2012;Inzlicht et al., 2014) the notion of resources as a limited cognitive energy supply has received new impulses from studies of the brain, in particular those that emphasized the role of prefrontal structures in controlling cognitive operations.
In the present article motivated control (defined as the product of interactions between the cognitive control and motivation systems) was proposed as an alternative for resources (Box 4). Within this framework resources are interpreted to reflect the functional state of widespread networks in the brain reflecting the dynamic interaction between the cognitive and motivation systems serving task control. Prolonged and effortful task performances are shown to be reflected in changes in the physiological state of these networks referred to as a 'fatigue network' (Wylie et al., 2020). More specific, a loss of connectivity between components of the network and their coherent activation would create a suboptimal functional state that falls short in providing the necessary energetic input to cognitive control systems. The anterior insula would be equivalent to that of an 'effort monitor', a mechanism suggested by Hockey (2011Hockey ( , 2013 signaling the costs of effort. Subjective fatigue as an important neural correlate of mental effort would then enter as separate interoceptive information to the insula as input from the fatigue network (Fig. 3).
Dopamine Effects of DA on cortical decision process depend on the balance between the mesocortical (tonic DA) and mesolimbic (phasic DA) pathways, connecting with D1 and D2 receptors in the PFC, respectively. In the motivation hierarchical system DA does 'double duty' in energizing cognitive effort by modulating the functional parameters of working memory circuits, and by mediating value-learning and decision-making in conditions of highly demanding cognitive action (Westbrook and Braver, 2016). Note also that working memory is a motivated process par excellence, implying that the effect of DA on working memory circuits would likely be mediated via the motivationcontrol network, by energizing goal values in the vmPFC and VS (see Fig. 3).
Other neurotransmitters in the brain such as serotonin (5-HT) also participate in the complex pattern of interactions between brain structures and conditions of fatigue, and contribute to the cost-benefit calculations. The primary function of high 5-HT in the 5HT-DA balance (such as D+/5-HT++) would then be to suppress DA facilitatory effects Box 4 Resources redefined.
A decline in the quality of performance during prolonged and demanding cognitive tasks has traditionally been interpreted in terms of limited mental resources -or capacities-falling short to meet the demands of a cognitive tasks (Kahneman, 1973). In defining resources some have even used the analogy of a fuel that feeds our mental processes (Gailliot et al., 2007). Resources in terms of energy have however been difficult to translate directly into global indices of brain metabolism, such as consumption of oxygen or glucose. Increased metabolic activity associated with neural activity appears to occur mainly in nerve terminals, such as axon terminals and dendrites, rather than in the cell bodies (Raichle and Gusnard, 2002;Gusnard and Raichle, 2001;Inzlicht and Smeichel, 2012). In the present review resources are interpreted to reflect the functional state of networks in the brain reflecting the dynamic interaction between the cognitive and motivation systems serving task control. Incipient feelings of fatigue would then indicate that a change has occurred in the functionality of the involved network. For example, a loss of connectivity between components of the network, or of their coherent activation would create a suboptimal functional state that falls short to provide the necessary energetic input to cognitive control systems.
when the non-appetitive quality of reinforcing stimuli starts to dominate their appetitive quality, concomitantly triggering feelings of fatigue. Cost-benefit analysis as a multicomponent network The present review casted cognitive control in terms of a decision-making problem, requiring integration of the expected payoff and the mental effort cost of controlled processing, to determine whether to allocate cognitive control (Cools, 2016). This allowed describing the effects of fatigue on mental performance as resulting from the evaluation of predicted rewards and costs of effort allocation, leading to an adaptation of invested effort. Fatigue increases as effort is exerted and partially decreases with rest, with higher levels of effort increasing the rate of fatigue build-up.
Fatigue may also build up in widespread areas in the brain, such as the motor, premotor, SMA, somatosensory areas and posterior insula feeding as interoceptive stimuli into core areas (Müller and Apps, 2019). In addition, according to a number of neuroimaging as well as EEG studies fatigue resulting from prolonged cognitive activity leads to more segregated network architecture of the brain (QI et al., 2019), affecting in particular long-connection striatal cortical pathways. The resulting neural signals would then also be incorporated in the evaluation of the tasks motivational values and costs of pursuing cognitive labor (Inzlicht et al., 2014).
Following Pessiglione et al., (2018) we proposed that cost-benefit analysis can be conceptualized as a dual system with the dACC linking reward and effort networks. In this model the dACC is conceived as a pivotal hub area integrating the goal value (and rewards) conveyed by ventral fronto-striatal circuits (vmPFC and VS), and effort/fatigue signals transmitted by the anterior insula (AI). Following Stenhav et al., (2013) we further suggested that the specified control signals representing the motivational values are implemented in areas of action control, involving more distant regions like the dlPFC and the premotor areas, including the supplementary motor area (SMA, see Fig. 3).
From the present perspective the decision-to-act process might be attributed to relay structures in the pathways that function specifically as the interface between cognitive control and motivation, that is: the dense lateral connections between the medial ventral and lateral dorsal sections of the PFC. The affordance model of Cisek (2007), developed for sensory-motor interactions, states that the brain processes sensory information to specify, in parallel, several potential actions that are currently available. These potential actions compete against each other for further processing, while information is collected to bias this competition until a single response is selected. Accordingly, cognitionmotivation interactions involving reciprocal integration of control and motivational processes afford lower levels in the motivation hierarchy to bias decision making in a fairly early stage of processing. Taken together, this would likely take form in an accumulation or drifting function reflecting the gradual build-up of neural evidence in the involved structures such as the dACC and vmPFC.