Computational theory-driven studies of reinforcement learning and decision-making in addiction: what have we learned?

Highlights

• Computational psychiatry holds promise for mechanistic discovery in addiction.

• This approach captures latent factors driving behavioral differences from health.

• Emerging support also for capturing variation defining addiction cycles and states.

• Research needs to better account for the heterogeneous, dynamic nature of addiction.

• Expanding the parameter space examined and duration of observation will be key.

Computational psychiatry provides a powerful new approach for linking the behavioral manifestations of addiction to their precise cognitive and neurobiological substrates. However, this emerging area of research is still limited in important ways. While research has identified features of reinforcement learning and decision-making in substance users that differ from health, less emphasis has been placed on capturing addiction cycles/states dynamically, within-person. In addition, the focus on few behavioral variables at a time has precluded more detailed consideration of related processes and heterogeneous clinical profiles. We propose that a longitudinal and multidimensional examination of value-based processes, a type of dynamic ‘computational fingerprint’, will provide a more complete understanding of addiction as well as aid in developing better tailored and timed interventions.

Introduction

Reinforcement learning and decision-making — collectively, ‘value-based decision-making’ [1] — are integral to adaptive behavior in everyday life. Value-based decision-making comprises a feedback loop whereby the values of candidate actions are learned and updated through experience, and used to guide behavior that maximizes utility (and minimizes disutility). Disruption in value-based decision-making is considered a key factor in the development and maintenance of addiction [2, 3, 4], across people with substance use disorders (SUD) [5] and laboratory animals exposed to drugs of abuse [6,7], but the specific contributing mechanisms remain unknown. Decision-making biases in addiction may be due to disruption in distinct components of learning, such as error encoding or value updating, or subjective preferences that are not readily observable in coarse behavioral performance measures. The nascent field of computational psychiatry applies formal models to understand the precise mechanisms (or ‘failure modes’) that give rise to pathological behavior in psychiatric conditions [8,9,10^••]. While there is no consensus on what qualifies as computational psychiatry, here we take this term to mean a mathematically rigorous understanding of the latent drivers of behavior. Findings from theory-driven computational psychiatry [11] suggest models that focus on algorithmic processes of value-based decision-making (Box 1) are well-suited to identify the specific components of reinforcement learning and decision-making that characterize SUD. This is exciting as such mechanistic research can bridge the behavioral manifestations of SUD with underlying neurobiology, providing fertile ground for cross-species translation [12, 13, 14, 15, 16]. Computational theoretical models thus hold promise as tools to provide additional mechanistic insight into SUD diagnosis and prognosis, and to help guide personalized treatments based on the latent variables governing individual behavior.

Here, we review recent theory-driven computational psychiatry studies of SUD primarily conducted with human subjects, highlighting the ways in which these studies have extended and refined our understanding of value-based decision-making processes in addiction. We focus on two key objectives of this work: to identify deviations from health (via case-control comparisons), and to map specific SUD symptoms and clinically relevant states onto specific model variables — the latter aimed at moving closer to understanding the most defining yet most elusive aspect of the disorder: its dynamic, cyclical course. We conclude by outlining two directions for future research. We propose that a holistic approach that expands the typical parameter space examined within the same individual, and the duration of observation, may better serve these critical objectives and significantly enhance the clinical impact of computational psychiatry for addiction applications.