Adaptive neural coding: from biological to behavioral decision-making
Introduction
Normative choice theories are the foundation of many modern approaches to decision-making, describing how the ideal or optimal chooser should make choices. In economics and psychology, rational choice models assume that choosers act to maximize a subjective measure of satisfaction termed expected utility [1]. In ecology, optimal foraging theory similarly assumes that organisms act to maximize an internal currency ultimately related to reproductive fitness [2]. Despite the rigorous mathematical framework and intuitive appeal of standard choice theories, empirical choice behavior violates the predictions of these optimality models in a wide range of species 3, 4, 5••, 6, 7, 8, 9]. In particular, biological choosers demonstrate context-dependent preferences, where decisions depend on additional (often irrelevant) information beyond the values of the given alternatives. Here, we review how recent work on the neural representation of value information offers a biological rationale for these apparent violations of rationality. Consideration of such computational principles suggests that choice behavior reflects a utility optimization process operating under intrinsic biological constraints.
Section snippets
Context-dependent violations of rationality
A key principle of nearly all optimal decision-making theories is that preferences rely on a stable, independent valuation of each choice alternative. Given the assumption that individual alternatives are evaluated independently, decisions should be unaffected by uninformative contextual factors such as the previous history or the structure of the choice set. However, many of the documented behavioral deviations from optimality suggest that value is determined in a relative rather than absolute
The neural representation of value
A critical step in understanding context-dependent choice behavior is identifying how neural circuits represent value-related information. While many brain areas are modulated by rewards, electrophysiological and neuroimaging studies have identified specific neural circuits that represent the subjective values of choice alternatives (see 19, 20] for recent reviews). Value coding is prominent in a network of areas linked to action selection, including sensorimotor circuits in prefrontal cortex,
Contextual modulation in value coding
Context-dependent choice behavior implies that value is constructed in a comparative manner, relative to the spatial or temporal background. Guided by these behavioral effects, recent experiments have begun to examine how contextual factors influence neural activity in identified value coding circuits (Figure 2).
The effect of spatial context on value coding has been primarily examined in sensorimotor regions of frontal and parietal cortex underlying action selection and execution. In these
Neural computation and biological constraints
The widespread prevalence of both spatial and temporal forms of contextual modulation suggests a conservation of function across different circuits and systems [37]. In particular, contextual processing is thought to be critical for the maximization of information coded in spiking activity. Neural systems face a number of constraints, imposed by factors such as the energetic cost of spiking activity, irreducible noise, and maximum firing rates. To explain how neural systems can maximize
Linking computation and choice behavior
Despite extensive documentation in the ecology and psychology literature, little is known about the neural mechanisms underlying context-dependent preferences. In sensory processing, both spatial and temporal forms of contextual modulation are linked to corresponding perceptual phenomena. Recent evidence suggests a similar linkage may exist between adaptive value coding computations and context-dependent decision-making.
A recent study quantified spatial context effects predicted by the
Attributes and the valuation process
A critical issue for future research concerns how overall decision values are constructed from separate alternative attributes and how contextual modulation affects this process. The electrophysiological and behavioral evidence discussed above focuses on the integrated value of choice alternatives, but the primary examples of context-dependence in the decision literature involve multi-attribute choice, where alternatives differ in multiple dimensions 3, 4, 6, 8, 9, 10, 63]. However, given the
Conclusion
Understanding how the brain represents behaviorally relevant variables is a key step in linking behavior to the underlying neural mechanisms. Classic approaches to decision-making rely on a characterization of choice behavior, but recent interest has turned to identifying the neural basis of valuation and choice. Emerging evidence for a context-dependent neural representation of value, and its relevance for context-dependent violations of rationality, underscores the role of information
Conflict of interest
None declared.
References and recommended reading
Papers of particular interest, published within the period of review, have been highlighted as:
• of special interest
•• of outstanding interest
Acknowledgement
PWG is supported by the National Institute on Drug Abuse through grant R01-DA038063. KL is supported by the National Institute for Mental Health through grant R01-MH104251.
References (63)
Context-dependent foraging choices in risk-sensitive starlings
Anim Behav
(2002)- et al.
Context-dependent decisions among options varying in a single dimension
Behav Process
(2012) - et al.
Multialternative decision field theory: a dynamic connectionist model of decision making
Psychol Rev
(2001) - et al.
Neural correlates of biased competition in premotor cortex
J Neurosci
(2011) - et al.
Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey
J Neurophysiol
(2001) - et al.
Natural stimulation of the nonclassical receptive field increases information transmission efficiency in V1
J Neurosci
(2002) - et al.
A normalization model of multisensory integration
Nat Neurosci
(2011) - et al.
The representation of economic value in the orbitofrontal cortex is invariant for changes of menu
Nat Neurosci
(2008) - et al.
Theory of games and economic behavior
(1944) - et al.
Foraging theory
(1986)
Adding asymmetrically dominated alternatives: violations of regularity and the similarity hypothesis
J Consum Res
Normalization is a general neural mechanism for context-dependent decision making
Proc Natl Acad Sci U S A
Context-dependent violations of rational choice in honeybees (Apis mellifera) and gray jays (Perisoreus canadensis)
Behav Ecol Sociobiol
Choice in context — tradeoff contrast and extremeness aversion
J Market Res
Elimination by aspects — a theory of choice
Psychol Rev
Context-dependent preferences
Manage Sci
Background context and decision making in hoarding gray gays
Behav Ecol
Contrast effects in sequential decisions: evidence from speed dating
Rev Econ Stat
New Yorkers commute more everywhere: contrast effects in the field
Rev Econ Stat
Mistake #37: the effect of previously encountered prices on current housing demand
Econ J
Adaptive gain control during human perceptual choice
Neuron
Loss aversion and inhibition in dynamical models of multialternative choice
Psychol Rev
A range-normalization model of context-dependent choice: a new model and evidence
PLoS Comput Biol
Efficient coding and the neural representation of value
Ann N Y Acad Sci
The valuation system: a coordinate-based meta-analysis of BOLD fMRI experiments examining neural correlates of subjective value
Neuroimage
Comparison of reward modulation in the frontal eye field and caudate of the macaque
J Neurosci
Neural correlates of decision variables in parietal cortex
Nature
Representation of action-specific reward values in the striatum
Science
Matching behavior and the representation of value in the parietal cortex
Science
The convergence of information about rewarding and aversive stimuli in single neurons
J Neurosci
Neurons in the orbitofrontal cortex encode economic value
Nature
Cited by (65)
Recent Opioid Use Impedes Range Adaptation in Reinforcement Learning in Human Addiction
2024, Biological PsychiatryA test of attribute normalization via a double decoy effect
2023, Journal of Mathematical PsychologyExpected subjective value theory (ESVT): A representation of decision under risk and certainty
2023, Journal of Economic Behavior and OrganizationMedial and orbital frontal cortex in decision-making and flexible behavior
2022, NeuronCitation Excerpt :VmPFC/mOFC value comparison signals reflect many other influences that impact on the way that options are valued during decision-making. For instance, the presence of a third option may impact the way that two other options are valued and compared and thus affects which choice is likely to be taken (Chau et al., 2020; Dumbalska et al., 2020; Louie et al., 2015; Webb et al., 2020). In addition, the presence of a less valuable item within a compound option is known to reduce the estimated value of the compound relative to the more valuable item alone, in the “less-is-more” effect displayed by both human and non-human primates (Kralik et al., 2012; List, 2002).
Context-dependent outcome encoding in human reinforcement learning
2021, Current Opinion in Behavioral SciencesAdaptation of utility functions to reward distribution in rhesus monkeys
2021, CognitionCitation Excerpt :Likewise, it has long been known in psychology and neuroscience that distribution-adaptation is an inherent feature of the brain (Louie & De Martino, 2013). In sensory systems, for example, neuron's maximize their efficiency by tuning their firing rates to match the distribution of sensory signals (Carandini & Heeger, 2012; Laughlin, 1981) – the same is thought to occur, to varying degrees, in the brain areas that encode value (Burke, Baddeley, Tobler, & Schultz, 2016; Kobayashi, Pinto de Carvalho, & Schultz, 2010; Louie, Glimcher, & Webb, 2015; Padoa-Schioppa, 2009; Tobler, Fiorillo, & Schultz, 2005; Tremblay & Schultz, 1999). Specifically, and supporting the idea of distribution-dependent utility, neurons in the primate prefrontal cortex have recently been recorded adapting their firing rate to different reward distributions in a way similar to our animals' utility curves.