Adaptive neural coding: from biological to behavioral decision-making

doi:10.1016/j.cobeha.2015.08.008

Current Opinion in Behavioral Sciences

Volume 5, October 2015, Pages 91-99

https://doi.org/10.1016/j.cobeha.2015.08.008 Get rights and content

Highlights

•
Empirical decision-making often violates normative choice theory.
•
Suboptimal choice can arise from contextual value coding in neural activity.
•
Adaptive computations can maximize information coding in constrained neural circuits.
•
Biological choice reflects a tradeoff between performance and the cost of computation.

Empirical decision-making in diverse species deviates from the predictions of normative choice theory, but why such suboptimal behavior occurs is unknown. Here, we propose that deviations from optimality arise from biological decision mechanisms that have evolved to maximize choice performance within intrinsic biophysical constraints. Sensory processing utilizes specific computations such as divisive normalization to maximize information coding in constrained neural circuits, and recent evidence suggests that analogous computations operate in decision-related brain areas. These adaptive computations implement a relative value code that may explain the characteristic context-dependent nature of behavioral violations of classical normative theory. Examining decision-making at the computational level thus provides a crucial link between the architecture of biological decision circuits and the form of empirical choice behavior.

Introduction

Normative choice theories are the foundation of many modern approaches to decision-making, describing how the ideal or optimal chooser should make choices. In economics and psychology, rational choice models assume that choosers act to maximize a subjective measure of satisfaction termed expected utility [1]. In ecology, optimal foraging theory similarly assumes that organisms act to maximize an internal currency ultimately related to reproductive fitness [2]. Despite the rigorous mathematical framework and intuitive appeal of standard choice theories, empirical choice behavior violates the predictions of these optimality models in a wide range of species 3, 4, 5••, 6, 7, 8, 9]. In particular, biological choosers demonstrate context-dependent preferences, where decisions depend on additional (often irrelevant) information beyond the values of the given alternatives. Here, we review how recent work on the neural representation of value information offers a biological rationale for these apparent violations of rationality. Consideration of such computational principles suggests that choice behavior reflects a utility optimization process operating under intrinsic biological constraints.

Section snippets

Context-dependent violations of rationality

A key principle of nearly all optimal decision-making theories is that preferences rely on a stable, independent valuation of each choice alternative. Given the assumption that individual alternatives are evaluated independently, decisions should be unaffected by uninformative contextual factors such as the previous history or the structure of the choice set. However, many of the documented behavioral deviations from optimality suggest that value is determined in a relative rather than absolute

The neural representation of value

A critical step in understanding context-dependent choice behavior is identifying how neural circuits represent value-related information. While many brain areas are modulated by rewards, electrophysiological and neuroimaging studies have identified specific neural circuits that represent the subjective values of choice alternatives (see 19, 20] for recent reviews). Value coding is prominent in a network of areas linked to action selection, including sensorimotor circuits in prefrontal cortex,

Contextual modulation in value coding

Context-dependent choice behavior implies that value is constructed in a comparative manner, relative to the spatial or temporal background. Guided by these behavioral effects, recent experiments have begun to examine how contextual factors influence neural activity in identified value coding circuits (Figure 2).

The effect of spatial context on value coding has been primarily examined in sensorimotor regions of frontal and parietal cortex underlying action selection and execution. In these

Neural computation and biological constraints

The widespread prevalence of both spatial and temporal forms of contextual modulation suggests a conservation of function across different circuits and systems [37]. In particular, contextual processing is thought to be critical for the maximization of information coded in spiking activity. Neural systems face a number of constraints, imposed by factors such as the energetic cost of spiking activity, irreducible noise, and maximum firing rates. To explain how neural systems can maximize

Linking computation and choice behavior

Despite extensive documentation in the ecology and psychology literature, little is known about the neural mechanisms underlying context-dependent preferences. In sensory processing, both spatial and temporal forms of contextual modulation are linked to corresponding perceptual phenomena. Recent evidence suggests a similar linkage may exist between adaptive value coding computations and context-dependent decision-making.

A recent study quantified spatial context effects predicted by the

Attributes and the valuation process

A critical issue for future research concerns how overall decision values are constructed from separate alternative attributes and how contextual modulation affects this process. The electrophysiological and behavioral evidence discussed above focuses on the integrated value of choice alternatives, but the primary examples of context-dependence in the decision literature involve multi-attribute choice, where alternatives differ in multiple dimensions 3, 4, 6, 8, 9, 10, 63]. However, given the

Conclusion

Understanding how the brain represents behaviorally relevant variables is a key step in linking behavior to the underlying neural mechanisms. Classic approaches to decision-making rely on a characterization of choice behavior, but recent interest has turned to identifying the neural basis of valuation and choice. Emerging evidence for a context-dependent neural representation of value, and its relevance for context-dependent violations of rationality, underscores the role of information

Conflict of interest

None declared.

References and recommended reading

Papers of particular interest, published within the period of review, have been highlighted as:

• of special interest
•• of outstanding interest

Acknowledgement

PWG is supported by the National Institute on Drug Abuse through grant R01-DA038063. KL is supported by the National Institute for Mental Health through grant R01-MH104251.

References (63)

M. Bateson
Context-dependent foraging choices in risk-sensitive starlings
Anim Behav
(2002)
K.V. Morgan et al.
Context-dependent decisions among options varying in a single dimension
Behav Process
(2012)
R.M. Roe et al.
Multialternative decision field theory: a dynamic connectionist model of decision making
Psychol Rev
(2001)
A. Pastor-Bernier et al.
Neural correlates of biased competition in premotor cortex
J Neurosci
(2011)
M.N. Shadlen et al.
Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey
J Neurophysiol
(2001)
W.E. Vinje et al.
Natural stimulation of the nonclassical receptive field increases information transmission efficiency in V1
J Neurosci
(2002)
T. Ohshiro et al.
A normalization model of multisensory integration
Nat Neurosci
(2011)
C. Padoa-Schioppa et al.
The representation of economic value in the orbitofrontal cortex is invariant for changes of menu
Nat Neurosci
(2008)
J. Von Neumann et al.
Theory of games and economic behavior
(1944)
D.W. Stephens et al.
Foraging theory
(1986)

J. Huber et al.

Adding asymmetrically dominated alternatives: violations of regularity and the similarity hypothesis

J Consum Res

(1982)

K. Louie et al.

Normalization is a general neural mechanism for context-dependent decision making

Proc Natl Acad Sci U S A

(2013)

S. Shafir et al.

Context-dependent violations of rational choice in honeybees (Apis mellifera) and gray jays (Perisoreus canadensis)

Behav Ecol Sociobiol

(2002)

I. Simonson et al.

Choice in context — tradeoff contrast and extremeness aversion

J Market Res

(1992)

A. Tversky

Elimination by aspects — a theory of choice

Psychol Rev

(1972)

A. Tversky et al.

Context-dependent preferences

Manage Sci

(1993)

T.A. Waite

Background context and decision making in hoarding gray gays

Behav Ecol

(2001)

S. Bhargava et al.

Contrast effects in sequential decisions: evidence from speed dating

Rev Econ Stat

(2014)

U. Simonsohn

New Yorkers commute more everywhere: contrast effects in the field

Rev Econ Stat

(2006)

U. Simonsohn et al.

Mistake #37: the effect of previously encountered prices on current housing demand

Econ J

(2006)

S. Cheadle et al.

Adaptive gain control during human perceptual choice

Neuron

(2014)

M. Usher et al.

Loss aversion and inhibition in dynamical models of multialternative choice

Psychol Rev

(2004)

A. Soltani et al.

A range-normalization model of context-dependent choice: a new model and evidence

PLoS Comput Biol

(2012)

K. Louie et al.

Efficient coding and the neural representation of value

Ann N Y Acad Sci

(2012)

O. Bartra et al.

The valuation system: a coordinate-based meta-analysis of BOLD fMRI experiments examining neural correlates of subjective value

Neuroimage

(2013)

L. Ding et al.

Comparison of reward modulation in the frontal eye field and caudate of the macaque

J Neurosci

(2006)

M.L. Platt et al.

Neural correlates of decision variables in parietal cortex

Nature

(1999)

K. Samejima et al.

Representation of action-specific reward values in the striatum

Science

(2005)

L.P. Sugrue et al.

Matching behavior and the representation of value in the parietal cortex

Science

(2004)

S.E. Morrison et al.

The convergence of information about rewarding and aversive stimuli in single neurons

J Neurosci

(2009)

C. Padoa-Schioppa et al.

Neurons in the orbitofrontal cortex encode economic value

Nature

(2006)

Cited by (65)

Recent Opioid Use Impedes Range Adaptation in Reinforcement Learning in Human Addiction
2024, Biological Psychiatry
Drugs like opioids are potent reinforcers thought to co-opt value-based decisions by overshadowing other rewarding outcomes, but how this happens at a neurocomputational level remains elusive. Range adaptation is a canonical process of fine-tuning representations of value based on reward context. Here, we tested whether recent opioid exposure impacts range adaptation in opioid use disorder, potentially explaining why shifting decision making away from drug taking during this vulnerable period is so difficult.
Participants who had recently (<90 days) used opioids (n = 34) or who had abstained from opioid use for ≥ 90 days (n = 20) and comparison control participants (n = 44) completed a reinforcement learning task designed to induce robust contextual modulation of value. Two models were used to assess the latent process that participants engaged while making their decisions: 1) a Range model that dynamically tracks context and 2) a standard Absolute model that assumes stationary, objective encoding of value.
Control participants and ≥90-days-abstinent participants with opioid use disorder exhibited choice patterns consistent with range-adapted valuation. In contrast, participants with recent opioid use were more prone to learn and encode value on an absolute scale. Computational modeling confirmed the behavior of most control participants and ≥90-days-abstinent participants with opioid use disorder (75%), but a minority in the recent use group (38%), was better fit by the Range model than the Absolute model. Furthermore, the degree to which participants relied on range adaptation correlated with duration of continuous abstinence and subjective craving/withdrawal.
Reduced context adaptation to available rewards could explain difficulty deciding about smaller (typically nondrug) rewards in the aftermath of drug exposure.
A test of attribute normalization via a double decoy effect
2023, Journal of Mathematical Psychology
We report a “Double Decoy” experiment designed to separate two competing accounts of the asymmetric dominance effect. The experiment places an additional decoy alternative within the range of existing alternatives, which should leave choice behaviour unaltered if attributes are weighted by their range. Instead, we observe a decrease in the relative proportion of targets chosen, particularly for subjects who exhibited an initial decoy effect. We also observe considerably more variation in individual behaviour than expected. We therefore consider an alternative theory in which attributes values are compared with diminishing sensitivity (via divisive normalization) and assess its performance in an additional discrete choice experiment previously used in the discrete choice literature. We find that divisive normalization captures behaviour better than range normalization and the linear additive Logit model typically used in applied settings. We therefore propose divisive normalization as both a neuro-computational explanation for context effects and a useful empirical tool for applied researchers.
Expected subjective value theory (ESVT): A representation of decision under risk and certainty
2023, Journal of Economic Behavior and Organization
We present a descriptive model of choice derived from neuroscientific models of efficient value representation in the brain. Our basic model, a special case of Expected Utility Theory, can capture a number of behaviors predicted by Prospect Theory. It achieves this with only two parameters: a time-indexed “payoff expectation” (reference point) and a free parameter we call “predisposition”. A simple extension of the model outside the domain of Expected Utility also captures the Allais Paradox. Our models shed new light on the computational origins and evolution of risk attitudes and aversion to outcomes below reward expectation (reference point). It delivers novel explanations of the endowment effect, the observed heterogeneity in probability weighting functions, and the Allais Paradox, all with fewer parameters and higher descriptive accuracy than Prospect Theory.
Medial and orbital frontal cortex in decision-making and flexible behavior
2022, Neuron
Citation Excerpt :
VmPFC/mOFC value comparison signals reflect many other influences that impact on the way that options are valued during decision-making. For instance, the presence of a third option may impact the way that two other options are valued and compared and thus affects which choice is likely to be taken (Chau et al., 2020; Dumbalska et al., 2020; Louie et al., 2015; Webb et al., 2020). In addition, the presence of a less valuable item within a compound option is known to reduce the estimated value of the compound relative to the more valuable item alone, in the “less-is-more” effect displayed by both human and non-human primates (Kralik et al., 2012; List, 2002).
The medial frontal cortex and adjacent orbitofrontal cortex have been the focus of investigations of decision-making, behavioral flexibility, and social behavior. We review studies conducted in humans, macaques, and rodents and argue that several regions with different functional roles can be identified in the dorsal anterior cingulate cortex, perigenual anterior cingulate cortex, anterior medial frontal cortex, ventromedial prefrontal cortex, and medial and lateral parts of the orbitofrontal cortex. There is increasing evidence that the manner in which these areas represent the value of the environment and specific choices is different from subcortical brain regions and more complex than previously thought. Although activity in some regions reflects distributions of reward and opportunities across the environment, in other cases, activity reflects the structural relationships between features of the environment that animals can use to infer what decision to take even if they have not encountered identical opportunities in the past.
Context-dependent outcome encoding in human reinforcement learning
2021, Current Opinion in Behavioral Sciences
A wealth of evidence in perceptual and economic decision-making research suggests that the subjective assessment of one option is influenced by the context. A series of studies provides evidence that the same coding principles apply to situations where decisions are shaped by past outcomes, that is, in reinforcement-learning situations. In bandit tasks, human behavior is explained by models assuming that individuals do not learn the objective value of an outcome, but rather its subjective, context-dependent representation. We argue that, while such outcome context-dependence may be informationally or ecologically optimal, it concomitantly undermines the capacity to generalize value-based knowledge to new contexts — sometimes creating apparent decision paradoxes.
Adaptation of utility functions to reward distribution in rhesus monkeys
2021, Cognition
Citation Excerpt :
Likewise, it has long been known in psychology and neuroscience that distribution-adaptation is an inherent feature of the brain (Louie & De Martino, 2013). In sensory systems, for example, neuron's maximize their efficiency by tuning their firing rates to match the distribution of sensory signals (Carandini & Heeger, 2012; Laughlin, 1981) – the same is thought to occur, to varying degrees, in the brain areas that encode value (Burke, Baddeley, Tobler, & Schultz, 2016; Kobayashi, Pinto de Carvalho, & Schultz, 2010; Louie, Glimcher, & Webb, 2015; Padoa-Schioppa, 2009; Tobler, Fiorillo, & Schultz, 2005; Tremblay & Schultz, 1999). Specifically, and supporting the idea of distribution-dependent utility, neurons in the primate prefrontal cortex have recently been recorded adapting their firing rate to different reward distributions in a way similar to our animals' utility curves.
This study investigated how the experience of different reward distributions would shape the utility functions that can be inferred from economic choice. Despite the generally accepted notion that utility functions are not insensitive to external references, the exact way in which such changes take place remains largely unknown. Here we benefitted from the capacity to engage in thorough and prolonged empirical tests of economic choice by one of our evolutionary cousins, the rhesus macaque. We analyzed data from thousands of binary choices and found that the animals' preferences changed depending on the statistics of rewards experienced in the past (up to weeks) and that these changes could reflect monkeys' adapting their expectations of reward. The utility functions we elicited from their choices stretched and shifted over several months of sequential changes in the mean and range of rewards that the macaques experienced. However, this adaptation was usually incomplete, suggesting that – even after months - past experiences held weight when monkeys' assigned value to future rewards. Rather than having stable and fixed preferences assumed by normative economic models, our results demonstrate that rhesus macaques flexibly shape their preferences around the past and present statistics of their environment. That is, rather than relying on a singular reference-point, reference-dependent preferences are likely to capture a monkey's range of expectations.

View all citing articles on Scopus

View full text

Adaptive neural coding: from biological to behavioral decision-making

Highlights

Introduction

Section snippets

Context-dependent violations of rationality

The neural representation of value

Contextual modulation in value coding

Neural computation and biological constraints

Linking computation and choice behavior

Attributes and the valuation process

Conclusion

Conflict of interest

References and recommended reading

Acknowledgement

Anim Behav

Behav Process

Psychol Rev

J Neurosci

J Neurophysiol

J Neurosci

Nat Neurosci

Nat Neurosci

Theory of games and economic behavior

Foraging theory

Adding asymmetrically dominated alternatives: violations of regularity and the similarity hypothesis

J Consum Res

Normalization is a general neural mechanism for context-dependent decision making

Proc Natl Acad Sci U S A

Context-dependent violations of rational choice in honeybees (Apis mellifera) and gray jays (Perisoreus canadensis)

Behav Ecol Sociobiol

Choice in context — tradeoff contrast and extremeness aversion

J Market Res

Elimination by aspects — a theory of choice

Psychol Rev

Context-dependent preferences

Manage Sci

Background context and decision making in hoarding gray gays

Behav Ecol

Contrast effects in sequential decisions: evidence from speed dating

Rev Econ Stat

New Yorkers commute more everywhere: contrast effects in the field

Rev Econ Stat

Mistake #37: the effect of previously encountered prices on current housing demand

Econ J

Adaptive gain control during human perceptual choice

Neuron

Loss aversion and inhibition in dynamical models of multialternative choice

Psychol Rev

A range-normalization model of context-dependent choice: a new model and evidence

PLoS Comput Biol

Efficient coding and the neural representation of value

Ann N Y Acad Sci

The valuation system: a coordinate-based meta-analysis of BOLD fMRI experiments examining neural correlates of subjective value

Neuroimage

Comparison of reward modulation in the frontal eye field and caudate of the macaque

J Neurosci

Neural correlates of decision variables in parietal cortex

Nature

Representation of action-specific reward values in the striatum

Science

Matching behavior and the representation of value in the parietal cortex

Science

The convergence of information about rewarding and aversive stimuli in single neurons

J Neurosci

Neurons in the orbitofrontal cortex encode economic value

Nature