Trends in Cognitive Sciences
ReviewDeciding How To Decide: Self-Control and Meta-Decision Making
Section snippets
The Choice To Exercise Control
Smart people constantly fail to ‘do the right thing’. We procrastinate, eat unhealthy food, and generally defeat our own goals. But why? Such behaviors are particularly vexing for influential normative and decision theoretic perspectives on cognition, which conceptualize decision making as maximizing long-term obtained reward. If we are optimizing, why should we ever be ‘of two minds’ about anything?
We advance here a unifying normative perspective on a range of situations involving conflict or
Normative Meta-Decision Making
A typical laboratory model of self control asks how long you will keep your hand in unpleasantly cold icewater [15]. Doing so might ensure that the experimenter pays you, but requires you to constantly inhibit the urge to escape the noxious sensation by withdrawing your hand. The core dilemma of this and many other control tasks is, in effect, whether to simplify the choice process. Often, as here, this simplification consists of releasing an automatized, prepotent response (e.g., removing your
The Benefits of Control
Investing resources can lead to a better outcome in many contexts, for instance by inhibiting maladaptive fear [21] or inappropriate prepotent responses [18], thus allowing more accurate estimates of outcome value leading to ultimately more rewarding choices [22], or by keeping in mind contextual information that helps to make responses both faster and more accurate [19].
A common theme across these examples is that choosing the most rewarding option depends on accurately knowing the value of
The Costs of Control
Viewing failure to exert control as irrational often focuses solely on the gain side of a more costly decision, while overlooking the cost [37].
Although cognitive resources do have intrinsic costs (e.g., the metabolic cost of firing spikes) – as the old economic adage goes, all costs are ultimately opportunity costs. In other words, the cost equates to what one could have obtained by spending the same resource some other way (Box 1). Thus the crucial computations of resource cost usually hinge
Balancing the Costs and Benefits of Control
Figuring out the exact costs and benefits of control requires difficult computations and knowledge of all the contingencies of the task at hand. This raises a problem of infinite regress, particularly because such meta-decisions themselves are supposed to assess whether or not it is worthwhile to simplify decision computations.
One shortcut is to suppose that the brain might use rough quantities that approximately capture the costs and benefits of control. Two candidates for such approximate
Goal-Directed versus Habitual Decision Making
Probably the best understood example of rational meta-decision making, empirically and theoretically, is for decisions about whether to deliberate. The expected value of an action can be retrieved from aggregate past experience or worked out more prospectively by enumerating the consequences of the action. These two strategies are termed model-free and model-based decision-making, and are closely related to the categories of habitual and goal-directed responding from behavioral psychology 22, 59
Fatigue and Cognitive Control
Some theories of fatigue, defined as unwanted changes in performance owing to continued activity, propose that the sensation of fatigue indicates rising conflict between current and competing goals, and therefore tracks opportunity costs of executive resource use 56, 57. A related proposal views self-control erosion as stemming from a similar shift of motivation 83, 84 that could also cleanly map to an increase in the estimated cost of the resource. However, the timescales involved are very
Estimating the Benefit of Control: Controllability and Learned Helplessness
Early work made a distinction between ‘resource-limited’ and ‘data-limited’ response functions to resource investment, according to whether a task responds favorably to increased resource allocation 31, 87. In the data-limited regime, no resource should be applied, regardless of its current cost. How can the agent estimate the return on resource investment – that is, the value of control [3]?
As we suggested, 4, 28, 30 this could be identified by estimating the amount of controllability in the
Concluding Remarks
Many decision contexts involve a higher-order automaticity tradeoff that requires deciding what amount of resources to invest into the decision itself. We have suggested that analogous considerations arise in several different experimental circumstances broadly related to self-control, and that emerging theories and results in many of these areas reflect strikingly similar mechanisms for rational cost–benefit tradeoffs, although important outstanding questions remain (see Outstanding
Acknowledgments
This work was partially supported by a Junior Fellow award from the Simons Foundation (Y.L.B.), an award in understanding human cognition from the James S. McDonnell Foundation (N.D.), and support from Google DeepMind (N.D.). We thank Peter Dayan for helpful discussions.
Glossary
- Central executive
- a flexible system regulating and coordinating cognitive processes.
- Decision controller
- a system that chooses actions, typically taking into account environmental circumstances (sensory inputs, state of the world, etc.).
- Ego depletion
- an experience-dependent impairment of performance in tasks requiring self-control or cognitive control, usually observed after having performed a preliminary demanding task.
- Habitual controller
- a decision controller that maps a context or a stimulus to a
References (100)
The expected value of control: an integrative theory of anterior cingulate cortex function
Neuron
(2013)How to set the switches on this thing
Curr. Opin. Neurobiol.
(2012)Three ways to resist temptation: the independent contributions of executive attention, inhibitory control, and affect regulation to the impulse control of eating behavior
J. Exp. Soc. Psychol.
(2009)- et al.
A Bayesian approach to relevance in game playing
Artif. Intell.
(1997) - et al.
An information theoretical approach to prefrontal executive function
Trends Cogn. Sci.
(2007) Context-dependent decisions among options varying in a single dimension
Behav. Processes
(2012)Executive functions and self-regulation
Trends Cogn. Sci.
(2012)Optimal prey selection in the great tit (Parus major)
Anim. Behav.
(1977)- et al.
A Bayesian formulation of behavioral control
Cognition
(2009) The misbehavior of value and the discipline of the will
Neural Netw.
(2006)
Goals and habits in the Brain
Neuron
Motivation, personal beliefs, and limited resources all contribute to self-control
J. Exp. Soc. Psychol.
The neural bases of emotion regulation: reappraisal and suppression of negative emotion
Biol. Psychiatry
Cognitive and emotional influences in anterior cingulate cortex
Trends Cogn. Sci.
Why self-control seems (but may not be) limited
Trends Cogn. Sci.
On data-limited and resource-limited processes
Cognit. Psychol.
Helpful self-control: autonomy support, vitality, and depletion
J. Exp. Soc. Psychol.
On feeling in control: a biological theory for individual differences in control perception
Brain Cogn.
Deciding between conflicting motivations: what mice make of their prefrontal cortex
Behav. Brain Res.
An opportunity cost model of subjective effort and task performance
Behav. Brain Sci.
Cognitive effort: a neuroeconomic approach
Cogn. Affect. Behav. Neurosci.
Speed/accuracy trade-off between the habitual and the goal-directed processes
PLoS Comput. Biol.
When to use which heuristic: a rational solution to the strategy selection problem
Prefrontal cortex, cognitive control, and the registration of decision costs
Proc. Natl. Acad. Sci. U.S.A.
Decision making and the avoidance of cognitive demand
J. Exp. Psychol. Gen.
The decision to engage cognitive control is driven by expected reward-value: neural and behavioral evidence
PLoS ONE
A labor/leisure tradeoff in cognitive control
J. Exp. Psychol. Gen.
Thinking, Fast and Slow
The neural costs of optimal control
Optimality and some of its discontents: successes and shortcomings of existing models for binary decisions
Top. Cogn. Sci.
The Computational and Neural Basis of Cognitive Control: Charted Territory and New Frontiers
Cogn. Sci.
Self-control: factors enhancing tolerance of noxious stimulation
J. Pers. Soc. Psychol.
Adaptive strategy selection in decision making
J. Exp. Psychol. Learn. Mem. Cogn.
Deciding when to decide: time-variant sequential sampling models explain the emergence of value-based decisions in the human brain
J. Neurosci.
Reward motivation enhances task coding in frontoparietal cortex
Cereb. Cortex
Ego depletion – is it all in your head? Implicit theories about willpower affect self-regulation
Psychol. Sci.
Emotion regulation: conceptual foundations
Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control
Nat. Neurosci.
The mixed instrumental controller: using value of information to combine habitual choice and mental simulation
Front. Psychol.
The physics of optimal decision making: a formal analysis of models of performance in two-alternative forced-choice tasks
Psychol. Rev.
A neuronal model of a global workspace in effortful cognitive tasks
Proc. Natl. Acad. Sci. U.S.A.
Multi-Armed Bandit Allocation Indices
Exploration from generalization mediated by multiple controllers
Humans use directed and random exploration to solve the explore–exploit dilemma
J. Exp. Psychol. Gen.
On the analysis of performance operating characteristics
Psychol. Rev.
Self-regulation and the problem of human autonomy: does psychology need choice, self-determination, and will?
J. Pers.
Context-dependent preferences
Manag. Sci.
Context-dependent preferences in starlings: linking ecology, foraging and choice
PLoS ONE
Older adults learn less, but still reduce metabolic cost, during motor adaptation
J. Neurophysiol.
Rationalizable Irrationalities of Choice
Top. Cogn. Sci.
Cited by (127)
Neurocognitive basis of model-based decision making and its metacontrol in childhood
2023, Developmental Cognitive NeuroscienceAssociative learning and high-level cognitive processes in the control of food-related behaviors
2022, Current Opinion in Behavioral SciencesFilling the gaps: Cognitive control as a critical lens for understanding mechanisms of value-based decision-making
2022, Neuroscience and Biobehavioral Reviews