Review
Deciding How To Decide: Self-Control and Meta-Decision Making

https://doi.org/10.1016/j.tics.2015.08.013Get rights and content

Trends

Meta-decisions are decisions about how decisions are made. Many recent models in different domains have conceptualized meta-decision dilemmas as pitting more carefully computed decisions against automatic defaults, including goal-directed versus habitual responses, deliberative versus heuristic choices, and controlled versus impulsive actions.

These recent models show that many puzzling decision patterns as well as phenomena of self-control and conflict can be understood as rational arbitration that balances the potentially better outcomes of more considered decisions against the higher costs of such consideration.

A central cost of deliberation across many seemingly separate domains has been proposed to be the opportunity cost of occupying shared resources over time. Common decision variables and mechanisms may guide these allocations across many such domains.

Many different situations related to self control involve competition between two routes to decisions: default and frugal versus more resource-intensive. Examples include habits versus deliberative decisions, fatigue versus cognitive effort, and Pavlovian versus instrumental decision making. We propose that these situations are linked by a strikingly similar core dilemma, pitting the opportunity costs of monopolizing shared resources such as executive functions for some time, against the possibility of obtaining a better outcome. We offer a unifying normative perspective on this underlying rational meta-optimization, review how this may tie together recent advances in many separate areas, and connect several independent models. Finally, we suggest that the crucial mechanisms and meta-decision variables may be shared across domains.

Section snippets

The Choice To Exercise Control

Smart people constantly fail to ‘do the right thing’. We procrastinate, eat unhealthy food, and generally defeat our own goals. But why? Such behaviors are particularly vexing for influential normative and decision theoretic perspectives on cognition, which conceptualize decision making as maximizing long-term obtained reward. If we are optimizing, why should we ever be ‘of two minds’ about anything?

We advance here a unifying normative perspective on a range of situations involving conflict or

Normative Meta-Decision Making

A typical laboratory model of self control asks how long you will keep your hand in unpleasantly cold icewater [15]. Doing so might ensure that the experimenter pays you, but requires you to constantly inhibit the urge to escape the noxious sensation by withdrawing your hand. The core dilemma of this and many other control tasks is, in effect, whether to simplify the choice process. Often, as here, this simplification consists of releasing an automatized, prepotent response (e.g., removing your

The Benefits of Control

Investing resources can lead to a better outcome in many contexts, for instance by inhibiting maladaptive fear [21] or inappropriate prepotent responses [18], thus allowing more accurate estimates of outcome value leading to ultimately more rewarding choices [22], or by keeping in mind contextual information that helps to make responses both faster and more accurate [19].

A common theme across these examples is that choosing the most rewarding option depends on accurately knowing the value of

The Costs of Control

Viewing failure to exert control as irrational often focuses solely on the gain side of a more costly decision, while overlooking the cost [37].

Although cognitive resources do have intrinsic costs (e.g., the metabolic cost of firing spikes) – as the old economic adage goes, all costs are ultimately opportunity costs. In other words, the cost equates to what one could have obtained by spending the same resource some other way (Box 1). Thus the crucial computations of resource cost usually hinge

Balancing the Costs and Benefits of Control

Figuring out the exact costs and benefits of control requires difficult computations and knowledge of all the contingencies of the task at hand. This raises a problem of infinite regress, particularly because such meta-decisions themselves are supposed to assess whether or not it is worthwhile to simplify decision computations.

One shortcut is to suppose that the brain might use rough quantities that approximately capture the costs and benefits of control. Two candidates for such approximate

Goal-Directed versus Habitual Decision Making

Probably the best understood example of rational meta-decision making, empirically and theoretically, is for decisions about whether to deliberate. The expected value of an action can be retrieved from aggregate past experience or worked out more prospectively by enumerating the consequences of the action. These two strategies are termed model-free and model-based decision-making, and are closely related to the categories of habitual and goal-directed responding from behavioral psychology 22, 59

Fatigue and Cognitive Control

Some theories of fatigue, defined as unwanted changes in performance owing to continued activity, propose that the sensation of fatigue indicates rising conflict between current and competing goals, and therefore tracks opportunity costs of executive resource use 56, 57. A related proposal views self-control erosion as stemming from a similar shift of motivation 83, 84 that could also cleanly map to an increase in the estimated cost of the resource. However, the timescales involved are very

Estimating the Benefit of Control: Controllability and Learned Helplessness

Early work made a distinction between ‘resource-limited’ and ‘data-limited’ response functions to resource investment, according to whether a task responds favorably to increased resource allocation 31, 87. In the data-limited regime, no resource should be applied, regardless of its current cost. How can the agent estimate the return on resource investment – that is, the value of control [3]?

As we suggested, 4, 28, 30 this could be identified by estimating the amount of controllability in the

Concluding Remarks

Many decision contexts involve a higher-order automaticity tradeoff that requires deciding what amount of resources to invest into the decision itself. We have suggested that analogous considerations arise in several different experimental circumstances broadly related to self-control, and that emerging theories and results in many of these areas reflect strikingly similar mechanisms for rational cost–benefit tradeoffs, although important outstanding questions remain (see Outstanding

Acknowledgments

This work was partially supported by a Junior Fellow award from the Simons Foundation (Y.L.B.), an award in understanding human cognition from the James S. McDonnell Foundation (N.D.), and support from Google DeepMind (N.D.). We thank Peter Dayan for helpful discussions.

Glossary

Central executive
a flexible system regulating and coordinating cognitive processes.
Decision controller
a system that chooses actions, typically taking into account environmental circumstances (sensory inputs, state of the world, etc.).
Ego depletion
an experience-dependent impairment of performance in tasks requiring self-control or cognitive control, usually observed after having performed a preliminary demanding task.
Habitual controller
a decision controller that maps a context or a stimulus to a

References (100)

  • R.J. Dolan et al.

    Goals and habits in the Brain

    Neuron

    (2013)
  • K.D. Vohs

    Motivation, personal beliefs, and limited resources all contribute to self-control

    J. Exp. Soc. Psychol.

    (2012)
  • P.R. Goldin

    The neural bases of emotion regulation: reappraisal and suppression of negative emotion

    Biol. Psychiatry

    (2008)
  • G. Bush

    Cognitive and emotional influences in anterior cingulate cortex

    Trends Cogn. Sci.

    (2000)
  • M. Inzlicht

    Why self-control seems (but may not be) limited

    Trends Cogn. Sci.

    (2014)
  • D.A. Norman et al.

    On data-limited and resource-limited processes

    Cognit. Psychol.

    (1975)
  • M. Muraven

    Helpful self-control: autonomy support, vitality, and depletion

    J. Exp. Soc. Psychol.

    (2008)
  • C.H. Declerck

    On feeling in control: a biological theory for individual differences in control perception

    Brain Cogn.

    (2006)
  • S. Granon et al.

    Deciding between conflicting motivations: what mice make of their prefrontal cortex

    Behav. Brain Res.

    (2012)
  • R. Kurzban

    An opportunity cost model of subjective effort and task performance

    Behav. Brain Sci.

    (2013)
  • A. Westbrook et al.

    Cognitive effort: a neuroeconomic approach

    Cogn. Affect. Behav. Neurosci.

    (2015)
  • M. Keramati

    Speed/accuracy trade-off between the habitual and the goal-directed processes

    PLoS Comput. Biol.

    (2011)
  • F. Lieder et al.

    When to use which heuristic: a rational solution to the strategy selection problem

  • J.T. McGuire et al.

    Prefrontal cortex, cognitive control, and the registration of decision costs

    Proc. Natl. Acad. Sci. U.S.A.

    (2010)
  • W. Kool

    Decision making and the avoidance of cognitive demand

    J. Exp. Psychol. Gen.

    (2010)
  • M.L. Dixon et al.

    The decision to engage cognitive control is driven by expected reward-value: neural and behavioral evidence

    PLoS ONE

    (2012)
  • W. Kool et al.

    A labor/leisure tradeoff in cognitive control

    J. Exp. Psychol. Gen.

    (2014)
  • D. Kahneman

    Thinking, Fast and Slow

    (2011)
  • S. Gershman et al.

    The neural costs of optimal control

  • P. Holmes et al.

    Optimality and some of its discontents: successes and shortcomings of existing models for binary decisions

    Top. Cogn. Sci.

    (2014)
  • M.M. Botvinick et al.

    The Computational and Neural Basis of Cognitive Control: Charted Territory and New Frontiers

    Cogn. Sci.

    (2014)
  • F.H. Kanfer et al.

    Self-control: factors enhancing tolerance of noxious stimulation

    J. Pers. Soc. Psychol.

    (1973)
  • J.W. Payne

    Adaptive strategy selection in decision making

    J. Exp. Psychol. Learn. Mem. Cogn.

    (1988)
  • S. Gluth

    Deciding when to decide: time-variant sequential sampling models explain the emergence of value-based decisions in the human brain

    J. Neurosci.

    (2012)
  • J.A. Etzel

    Reward motivation enhances task coding in frontoparietal cortex

    Cereb. Cortex

    (2015)
  • V. Job

    Ego depletion – is it all in your head? Implicit theories about willpower affect self-regulation

    Psychol. Sci.

    (2010)
  • J.J. Gross et al.

    Emotion regulation: conceptual foundations

  • N.D. Daw

    Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control

    Nat. Neurosci.

    (2005)
  • G. Pezzulo

    The mixed instrumental controller: using value of information to combine habitual choice and mental simulation

    Front. Psychol.

    (2013)
  • R. Bogacz

    The physics of optimal decision making: a formal analysis of models of performance in two-alternative forced-choice tasks

    Psychol. Rev.

    (2006)
  • S. Dehaene

    A neuronal model of a global workspace in effortful cognitive tasks

    Proc. Natl. Acad. Sci. U.S.A.

    (1998)
  • J. Gittins

    Multi-Armed Bandit Allocation Indices

    (1989)
  • P. Dayan

    Exploration from generalization mediated by multiple controllers

  • R.C. Wilson

    Humans use directed and random exploration to solve the explore–exploit dilemma

    J. Exp. Psychol. Gen.

    (2014)
  • D.A. Norman et al.

    On the analysis of performance operating characteristics

    Psychol. Rev.

    (1976)
  • R.M. Ryan et al.

    Self-regulation and the problem of human autonomy: does psychology need choice, self-determination, and will?

    J. Pers.

    (2006)
  • A. Tversky et al.

    Context-dependent preferences

    Manag. Sci.

    (1993)
  • M. Vasconcelos

    Context-dependent preferences in starlings: linking ecology, foraging and choice

    PLoS ONE

    (2013)
  • H.J. Huang et al.

    Older adults learn less, but still reduce metabolic cost, during motor adaptation

    J. Neurophysiol.

    (2014)
  • P. Dayan

    Rationalizable Irrationalities of Choice

    Top. Cogn. Sci.

    (2014)
  • Cited by (127)

    View all citing articles on Scopus
    View full text