Introduction

Neuromodulation has been receiving progressively more attention in computational literature1. While neuronal network dynamics and their coding possibilities as defined by classical neurotransmission and the properties of the constituent cells have been extensively studied theoretically (references are too numerous to list, yet see2, 3 for general treatment) the dynamical and computational consequences of the extra-synaptic neuromodulation are just beginning to be explored1. Neuromodulatory action has been implicated in a wide variety of processes related to cognition, psychiatric disorders and drug addiction4. Existing computational models of neuromodulation can be roughly split into two classes of approaches: (i) top-down functional approaches proposing algorithmic theories of the neuromodulator action, and (ii) neurodynamical approaches looking at the consequences of neuromodulator action on the behavior of neuronal networks. Both of these approaches give complementary information about the consequences of neuromodulator action and address distinct issues. The top-down models, based on machine learning techniques, such as reinforcement learning5 and/or Bayesian inference6, treat questions about “What do the various neuromodulatory signals mean?” and “Why are particular dynamics of neuromodulator signaling seen in various behavioral tasks?”. The second approach asks the question of “How does neuromodulation produce its actions on neuronal systems involved in a behavioral task?”.

A full review of all the neuromodulators and their actions is clearly beyond the scope of this short review, hence we focus on a few examples of computational modeling approaches studying neuromodulators acetylcholine (ACh) and nicotine (Ni, that may be thought of as an exogenous neuromodulator). Studies of ACh actions historically focused on the dynamics of learning and long-term memory storage and recall7. More recently a computational theory for ACh, based on machine learning considerations, proposed a computational role for ACh as tracking specific kinds of uncertainty in behavioral situations8, 9. Notably only a very limited number of computational studies has treated issues of receptor identity (eg for dopamine see review in10) and to our knowledge virtually none had combined receptor dynamics and neuronal dynamics together (see an overview of the role of dopamine receptors in working memory11).

We give a brief overview of the main examples of modeling neuromodulation by endogenous acetylcholine and exogenous nicotine from phenomenological - top-down - to biological realistic - bottom-up - approaches. In the following two sections we discuss models from functional to network-level and the involvement of the various receptors in controlling the neuromodulatory effects. We interpret these studies with an eye towards effects of nicotine, and discuss a recently introduced computational framework for nicotine-dopamine interactions that is capable of simulating several aspects of nicotine addiction. We then describe a novel approach that combines ligand effects on nicotinic acetylcholine receptors with neurodynamics of local circuitry and show its potential to understand the control of dopaminergic (DA) signaling in the ventral tegmental area (VTA) by acetylcholine and nicotine. Taking into account pharmacodynamics and neuronal dynamics yields a class of mesoscopic models that combine detailed knowledge of the action of the endogenous neuromodulators, such as acetylcholine (ACh) and exogenous substances such as nicotine, or other addictive drugs as well as medical compounds, with neuronal circuit organization and neural properties. This approach potentially allows to draw detailed conclusions about neuromodulator actions on the functioning of neuronal systems and possibly behavior.

A top-down computational view of cholinergic neuromodulation: uncertainty and ACh

Top-down functional approaches to modeling the consequence of neuromodulators have come from machine learning literature. Primarily, theories have built on the reinforcement learning algorithms that are capable of modeling behavioral conditioning5. A top-down methodology has been recently applied to action of ACh. Yu and Dayan have proposed that ACh reflects the variability of stimuli and choices inherent in a particular task or an environment8, 9. They call it the 'expected' uncertainty and differentiate it from the unexpected uncertainty that may be encoded by norepinephrine. For example, levels of ACh signal the expectation of how informative are the various stimuli about potential choices in a learned task. This would be something that the animal will have previously learned. Norepinephrine on the other hand would encode the uncertainty about the identity of the stimuli and their compatibility with a given task-set and/or environment. In this way ACh signals the learned, internally represented, cognitive demands of a give task-set/environment. ACh could signal the attentional requirements necessary in a given environment that may be dependent on the expected uncertainty. Alternatively attentional demands could be also due to an unexpected change in the uncertain identity of stimuli or equivalently their importance for the appropriate responding. This information would be signalled by norepinephrine. Interestingly this model gives a formal relationship between two neuromodulatory systems and can account for a number of pharmacological manipulations of behavioral performance. For example, the model shows that subjects with ACh blocked, should underestimate the uncertainty about how well a cue predicts required response. This should lead to faster than normal reaction times for valid cues and slower than normal for invalid cues. This is what is seen in experiments with scopolamine, a muscarinic receptor antagonist, and is called the “validity” effect12. Interestingly nicotinic agonists (such as nicotine) reduced the validity effect by speeding up the reaction times after invalidly cued targets. Again this computational model, implemented as an abstract statistical inference algorithm, has been quite powerful in explaining a number of behavioral results and even proposed a theory for functional interactions between the different neurmodulatory systems (ACh and norepinephrine). As intriguing and powerful as this computational account of ACh action may be it offers no mechanistic treatment. The receptor level or even circuit level mechanisms were not clearly analyzed; however Hasselmo drew an analogy between this abstract model and the neurodynamical models of ACh actions (see below)13.

The above example views ACh as an information carrying channel rather than a general neuromodulator. The proposed action of ACh is to signal specific information (uncertainty, attention, etc) and to instruct the inference process that in turn affects the behavior rather than modulate the neuronal circuits that would code the information in their intrinsic dynamics. This is similar to the reinforcement learning treatments of dopamine action14. It is also interesting to note, that the above computational model, despite being abstract, already offers a chance to address actions of nicotine at least in part. If we look at the Yu and Dayan theory of ACh functional role, and assume that at least part of the ACh signaling expected uncertainty is carried through the nicotinic ACh receptors, the nicotine would boost the expected uncertainty signal pharmacologically12. This would imply that subjects would perceive a higher complexity for tasks and environments under nicotine, hence possibly dispatch more attention to their performance and become more vigilant in the environment. This speculation might give a clue to some of the cognitive effects of nicotine, at least in the short term. However, by not treating the biophysical mechanisms of ACh and nicotine action directly, the abstract models would be hard-pressed to provide a framework for understanding the specific mechanisms that lead to the changes in the circuits involved and hence behavior.

A dynamics view of cholinergic neuromodulation of neuronal network function: memory read-in and recall

Computational approaches to studying how ACh influences neural population dynamics have recently started to receive significant attention in the literature. ACh has been equated with attention and at times network models of attentional modulation make a heuristic argument that the parameter changes reflecting attentional modulation (eg15) are due to increases in ACh inputs to the cortex. Notably, attentional modulation has been modeled as an increase in the excitability of local neuronal populations, arguing that such increase is due to muscarinic AChR-dependent down-regulation of slow potassium channels (eg the IM current)16. Alernatively attentional modulation was modeled as an increase in a top-down excitatory input17. In addition to the above mechanisms, cholinergic influence was suggested to be carried by the selective nicotinic boost of glutamatergic neurotransmission18. Models of olfactory processing19, learning in the piriform cortex20 and memory formation and learning in the entorhinal/hippocampal complex21 form probably the most developed body of literature addressing dynamical effects of ACh on neuronal populations. In these studies the action of ACh at muscarinic and nicotinic receptors was well delineated and separable roles in dynamics and information processing were proposed for the two receptor-based mechanisms. The main hypothesis is that high acetylcholine levels enhance the feedforward afferent connections to the cortical areas and selectively inhibit the recurrent cortico-cortical connectivity through presynaptic muscarinic inhibition of excitatory feedback. On the other hand, low acetylcholine levels result in a weaker influence of afferent input relative to the strength of local excitatory feedback. Analysis of neural networks showed that learning in the recurrent connections is necessary for formation of associative memories22. However once a number of memories in the networks are formed they tend to interfere with formation of new memories and responses of the network to incoming sensory inputs. Hence, Hasselmo and colleagues proposed that memory recall and memory reconsolidation (by hippocampal replay for example) take place under low ACh regime (eg23, 21). High ACh regime on the other hand ensures the network being dominated by the feed-forward “sensory” information and enhances the coding of new memories. The dynamical theory makes a functional connection between the changes in ACh levels during the wake-sleep cycle and the memory acquisition and consolidation during that cycle: acquisition during wakefullness and consolidation during sleep. The framework further implies that networks dominated by afferents respond to sensory stimuli with higher fidelity - hence they “pay more attention”. Simulations showed that such cholinergic actions in the network may model effects of attentional focus on cortical neuronal responses and in particular the increases in signal-to-noise ratio in neuronal responses24. In the body of modeling work discussed here ACh modulation was accounted for by changes in model parameters - increases and decreases in synaptic strengths and/or changes in the gain of the input/output relationships of the neuronal populations or changes in maximal activation of the ACh-sensitive intrinsic cellular conductances. The issues of receptor identity, beyond muscarinic vs nicotinic, as well as their dynamics were not addressed.

It is interesting to speculate what would be the effects of nicotine within the “encoding/recall” framework for ACh modulation? In the models above, nicotinic action is to enhance the afferent inputs, while muscarinic is to (i) decrease the recurrent and top down inputs, and (ii) increase the excitability of the neuronal populations. Hence nicotine, if it activates the nicotinic ACh receptors, would pathologically lock the network in the encoding stage or at least enhance the encoding. Certainly this is compatible with the attention enhancing effects associated with nicotine (in tobacco smoke)25: encoding and acquisition enhancement functionally are equated with attentional effects in the models. In order to fully understand the impact of nicotine on the models described in this section it is crucial to consider the dynamics of receptor activation as well as the dynamics of the neuronal population to get the full picture of nicotine's effect.

Below we review two related and complementary efforts to model nicotinic effects on dopaminergic and cholinergic signaling and how these may lead to addiction. The first is a global framework, starting from large scale neuronal dynamics, combining these with heuristic models of receptor response to nicotine and showing the outcomes on a simulated self-administration task. The second is a much closer look at the circuitry responsible for the dopaminergic signaling in the ventral tegmental area, asking pointed questions about the key mechanisms by which nicotine usurps the normal cholinergic action in this system.

Large-scale neurodynamical framework for nicotine action in DA signaling and choice-making

Gutkin and colleagues introduced a neuro-computational framework for nicotine addiction that integrated nicotine effects on the dopaminergic (DA) neuron population at the receptor population level (signaling the reward-related information), together with a simple model of action-selection (Figure 1)26. This model also incorporated a novel dopamine-dependent learning rule that gives distinct roles to the phasic and tonic dopamine neurotransmission. The authors strove to tease out the relative roles of the positive (rewarding) and opponent processes in the acquisition and maintenance of drug taking behavior, and the development of such behavior into a rigid habit. The details of the mathematical methods, equations and simulation details can be found in26, below we give an overview of the model and the major results.

Figure 1
figure 1

Global neurodynamical framework for nicotine addiction. Schematic of the large-scale neurodynamical framework for simulating nicotine self-administration. Top: Functional circuitry in the initial stages of nicotine exposure. Here the behavior is motivated by positive effects of nicotine and stably acquired due to nicotine-dependent dopamine gated learning in the action-selection circuit. Bottom: Functional circuitry in the long term nicotine self administration when the drug-recruited opponent process has effectively cut the link between the dopaminergic sub-system and the action-selection machinery.

Full size image

The major hypothesis for their approach is that the nicotine effects on dopamine signaling in the ventral tegmental area (VTA) provoke the onset of nicotine addiction by biasing glutamatergic (Glu) learning processes in the dorsal striatum-related structures that are responsible for behavioral choice. The authors apply the model to simulate a standard animal behavioral assay for addiction: self-administration in a two-choice maze task27. In this task, the animal is previously implanted with a cannula through which the drug or a control saline dose can be injected. The cannula can be placed to deliver intravenously or into the brain (eg as in28 where the mouse is implanted with a cannula into the ventral tegmental area). The animal then is introduced into the behavioral apparatus where it is free to make choices. In the maze task simulated, the mouse should chose to navigate between the left and the right arms of a maze. The behavior is either (i) reinforced with a drug injection or (ii) results in injection of a control vehicle solution that is seen as “non-rewarding”. In the maze task, the mouse running into a selected arm, eg the right arm, results in an automatic injection of a nicotine dose (see 28 for a fuller description of the technique). The substance, such as nicotine, is said to be self-administrated, and hence reinforcing or addictive, when the choices are preferentially and persistently biased toward the behavior resulting in the drug injection. Monitoring the speed with which the choices are made further measures the reinforcing properties of the drug; a speed-up of the drug-reinforced choice is said to reflect the motor-activational properties of the drug. As reviewed below the model of Gutkin et al simulates both of these features of nicotine self-administration.

Gutkin et al26 specifically hypothesized that nicotine affects the DA response through a three-time scale model of drug action on the nAChRs: (i) the phasic nicotine dependent activation of nicotinic ACh receptors, (ii) slower nicotine dependent upregulation of nAChRs, (iii) and subsequent upregulation-evoked opponent homeostatic down-regulation of nAChRs (and hence their responses to nicotine). In the model, nicotine causes activation and up-regulation of nicotinic acetylcholine receptors (nAChRs) in the VTA29, 30, that in turn changes the gain of the DA signaling. Hence, a dose of nicotine can potentiate the phasic DA response to rewarding stimuli and evokes such a signal by itself29, 31, 32. The phasic DA in turn instructs the learning in the action-selection machinery33, 34 identified with dorsal nigro-striatal-cortical loops35, 36. This plasticity is governed in the model by a Hebbian learning rule that is gated by the tonic DA. Long-term presence of nicotine causes depression in the tonic DA through a receptor down-regulation driven opponent process.

During a self-administration task, nicotine is contingent on a specific action choice (encoded in the model as activity of a specific neuronal population). This choice-dependent injection of nicotine potentiates the DA signal so as to gate plasticity in the action-selection machinery causing the excitatory synaptic weights of the corresponding neural population to increase. Such potentiation in turn biases the action-selection towards the action that brings nicotine. With prolonged self-administration, the influence of the DA signal diminishes due to the opponent process; the behavioral bias for the action leading to nicotine becomes “stamped in”. This disrupts DA neurotransmission blocking the plasticity. Hence, nicotine self-administration progressively escapes from the control of the ventral DA signal26. This effectively models the ventral-dorsal progression of long-term addiction37. Drug seeking behavior becomes routinized, and inelastic to the motivational value of nicotine or the cost which is associated with hypodopaminergic withdrawal38. The authors speculate that this effect on action-selection learning may be the reason why nicotine has reportedly high addictive liability despite its limited hedonic impact.

As all computational models, the framework of Gutkin and colleagues26 has a number of strengths and shortcomings. The major strength of the model framework is that it neatly integrates the various processes involved in nicotine self-administration identifying the various functional effects with biological mechanisms and brain structures. The modular structure of the framework makes it easy to potentially incorporate additional structures and mechanisms to test their effects.

The framework predicts a hierarchy of thresholds for the progressive stages of addiction. The dose and duration of the exposure to nicotine for the initial sensitization by the drug is below that for the acquisition of the self-administration, followed by higher thresholds for the stabilization of the self-administration and for the transfer to habit-like rigidity. At low doses/short duration, nicotine may lead to apparent behavioral sensitization, but not self-administration. Following that, drug-related behaviors may be acquired due to the action of the positive “reinforcement” or “reward” DA-related process. Hence, the acquisition of self-administration would be under motivational control. The subsequent development of rigidity in actions is a major point of the neuro-computational framework proposed by Gutkin et al26. Hence subsequent to long-term self-administration of nicotine animals should show deficits in re-adjusting their behavior in the face of changing behavioral contingencies (see39 for possible experimental equivalent) even when the environment is enriched by new rewarding stimuli.

A general challenge is to develop models that integrate the neurodynamical approach with the algorithmic reinforcement learning models, and further to understand how to apply the neurodynamical framework to situations that are more complex than the simple two choice self-administration task. For example it is not clear if the framework as phrased in Gutkin et al26 can account for accommodation of the phasic DA signal as the animal learns to predict a natural reward and/or the temporal shift in the DA signal from the reward delivery to the time of the stimulus that is predictive of that reward. Hence additional mechanisms may need to be introduced into the framework in order to remedy this shortcoming. At the more mechanistic level, the global framework is rather vague on the specific identity of the opponency: Gutkin and colleagues assigned it to homeostatic down-regulation of receptors, however it may be due to influence of a further non-dopaminergic process. In addition, the global model does not pin-point the specific local mechanisms by which nicotine may bias the DA signaling. We pursued this issue by building circuit level models of dopaminergic circuitry.

Circuit level model of nicotine action in the VTA

In order to build a local circuit model that includes both the neuronal dynamics and the details of receptor dynamics in responses to acetylcholine and nicotine one must start with key information on the receptor properties. In particular their kinetics and their interactions with the relevant ligands. We review these facts briefly before launching into the model describing the VTA dynamics.

nACh receptor kinetics upon ligand interaction The first step in studying how acetylcholine and nicotine influence neuronal circuits is to examine in detail the nicotinic acetylcholine receptor (nAChR) response to both neuromodulators. nAChRs belong to a family of ligand-gated ion channels that bind neurotransmitter molecules as well as exogenous ligands and mediate fast signal transmission40. Upon agonist binding at subunit interfaces, the ligand-binding domain of the nAChR undergoes conformational rearrangements that propagate to the membrane-spanning domain leading to the opening of the receptor pore. Four functional states have been described in ACh receptors: the resting (closed) state (R), the open state (O), the fast-onset desensitized (closed) state (I), and the slow-onset desensitized (closed) state (D, Figure 2A31). The resting state is the most stable state in the absence of agonists, and the slow-onset desensitized state is the most stable state in the presence of agonists41. The open and the fast-onset desensitized state are transient metastable states, their concentrations rise transiently on the time scale of μs in the presence of agonits and reach very low values at equilibrium. Recovery from the desensitization happens on time scales of seconds to minutes42. nAChRs are allosteric, that is, they contain multiple agonist-binding sites, non-competitive-antagonist sites, and gates that interact at a distance through changes in the quaternary structure of the receptor. Their behavior can therefore be described by the Monod-Wyman-Changeux (MWC) model of allosteric interactions43.

Figure 2
figure 2

Transitions between states of the nAChR and scheme of the ventral tegmental area. (A) All potential interactions amongst the four allosteric states of the nAChR. 'R' refers to resting state, 'O' is the open state with an intrinsic conductance, 'I' and 'D' are fast- and slow-onset desensitized states, respectively. Predominant kinetic pathways are highlighted in red. Note that all the states exist with none, one or two agonists bound (adapted from 47). (B) State model of nicotinic acetylcholine receptors. Activation (horizontal) and desensitization (vertical) of nAChRs are two independent transitions in the model, ie the receptor can exist in four different states: (i) inactivated/sensitive (up-left), (ii) activated/sensitive (up-right), (iii) inactivated/desensitized (down-left), and (iv) activated/desensitized (down-right). Activation is driven by Ni and ACh and induces a transition from the deactivated/sensitive to the activated/sensitive state (green), the only open state in which the receptor mediates an excitatory current. Desensitization is driven by Ni only. a and s characterize the fraction of nAChRs in the activated and the sensitive state, respectively (modified from 44). C, Afferent inputs and circuitry of the ventral tegmental area. The GABAergic neuron population (red) and the dopaminergic neuron population (green) receive excitatory glutamatergic (blue) cholinergic projections (cyan). nAChRs are found at presynaptic terminals of glutamatergic projections (α7-containing receptors), on GABAergic neurons (α4β2 nAChRs) and DA neurons (α4β2 nAChRs). Dopaminergic efferents (green) project, amongst others, to the NAcc and the PFC (see text for more details).

Full size image

Functional models of nAChR kinetics have been developed following the progress in knowledge of the properties of nAChRs. Early descriptions account for the opening in response to the receptor-ligand interaction44 (R←→ O). This formulation was extended to account for desensitization, leading to a cyclic reaction scheme45 (R←→O←→I ←→R). This model proposes distinct resting (R) and desensitized (I) states in the absence of agonits (resting is referred to as “effective”, and desensitized as “refractory” in45). The reaction scheme was subsequently expanded to incorporate two agonist binding cites46, 47. A tetrahedron model with four conformational states, R, O, I, and D, introducing the slowly desensitized state, D, was proposed to account for the kinetic reactions observed in experiments48, 49. According to this scheme, all interaction pathways are in principle possible (Figure 2A). A simplified form of the tetrahedral model, based on the predominant kinetic pathways (red in Figure 2A), reduces the tetrahedral arrangement to a linear cascade (R←→O←→I←→D←→R)41. Based on the observation that the receptor desensitizes most readily from the open-state and returns to the resting state without opening, a two-gate mechanism is proposed by which activation and desensitization are mediated by two distinct, but interrelated, gates in the ion permeation pathway50. The asymmetrical heteromeric structure of the nAChR inspired an uncoupled model which allows the binding sites to switch between functional states independently of each other, ie one binding site can be in the resting and the other in the desensitized state51. Several additional, desensitized state have been suggested (D3, D4, ...) to account for the observation that the recovery from desensitization depends on agonist exposure time and agonist type52.

Methodological approach to constructing the circuit model of the VTA In order to study nicotine- and acetylcholine mediated neuromodulation in the VTA, we choose the following approach to construct a circuitry model which accounts for relevant experimental data and allows to study questions of Ni and ACh neuromodulation. The activity of VTA neurons is collectively accounted for by a population activity model which tracks average changes in neuronal firing rates in response to afferent inputs and in combination with the local connectivity. The proposed circuitry can be seen as a global description of the VTA or as a model of a local computational unit of neurons within the VTA. The circuit is endowed with a simple model of nACh receptors mediating Ni and ACh action. The nAChR implementation accounts for subtype-specific properties such as kinetics in response to Ni and ACh and locations within the VTA circuit. Finally, the model is parameterized to account for key in vitro and in vivo data on nicotine-evoked responses in the VTA. To reiterate, the key to our approach is to glean the relevant facts and properties from existing data to come up with a simple, heuristically minimal circuit description of the VTA. Note that necessarily certain level of detail is lost in the process and the whole purpose of the enterprise is to discover how much data can be explained without including this detail and what predictions can be made. For example, we summarize the inputs to our VTA circuit in two classes: glutamatergic and cholinergic, remaining agnostic about their specific origin. We do not account for recurrent interactions between the VTA and the nucleus accumbens (NAcc) or the ventral palladium53. Despite those simplifications, the simple model proves to be sufficient to reproduce and study cholinergic as well as nicotinic effects in the VTA on time scales ranging from milliseconds up to one hour, thereby remaining a powerful tool to shed light on the mechanisms of nicotine action.

The circuit model of the VTA discussed here accounts for afferent inputs to the VTA, local circuitry and the location as well as activation/desensitization properties of nAChRs. The VTA contains DAergic and GABAergic cells that receive major excitatory glutamatergic (Glu) inputs from the prefrontal cortex (PFC) and the tegmental nuclei in the brainstem54, 55. The laterodorsal tegmental nuclei and the pendunculopontine tegmental nuclei furthermore innervate the VTA with cholinergic projections56. GABAergic neurons in the VTA furnish local inhibitory connections57 (see Figure 2C).

Various nAChR subtypes are expressed on DA neurons, GABAergic neurons and on glutamatergic terminals in the VTA58. There are 12 known types of vertebrate neuronal ACh receptor subunit: α2-α10 and β2-β4, amongst which α7, α8, and α9 can from functional homopentamers, whereas α2-α6 and α10 form functional complexes only when coexpressed with β-subunits40. The various subtypes of nAChRs have distinct activation/desensitization properties and expression targets. We accounted for the two main classes of nAChRs responsible for nicotine evoked responses in the VTA: (i) high affinity slowly desensitizing (α4β2-type), (ii) and low affinity rapidly desensitizing nAChRs (α7-type)29, 59, 30, 60. DA neurons express both α4- and α6-containing nAChRs, while nicotine-evoked responses are predominantly mediated by α4β2 nAChRs30. The GABA neurons express mostly α4β2 nAChRs. The α7 nAChRs are found on terminals of the glutamatergic projections to the VTA61 (Figure 2C). Although nAChRs are found in many brain regions, those located in the VTA dominantly mediate the rewarding effects of nicotine62.

It is important to note that ACh and Ni show distinctly different time courses of presence at the receptor site. Behaviorally relevant stimuli evoke ACh release into the VTA, causing nearly synchronous activation of nAChRs63. The rapid delivery and breakdown of ACh precludes significant nAChR desensitization. In contrast, nicotine concentrations remain elevated (∼500 nmol/L) for about 10 min in the blood of smokers64. This activates and desensitizes nAChRs within seconds to minutes65.

Based on the above facts, we developed a mean-field circuit model of the VTA reflecting average activities of the DA and GABAergic neuron populations with respect to the local connectivity, afferent inputs as well as the localization and activation/desensitization kinetics of nAChR subtypes. The temporal behavior of the neuron populations is characterized by generic mean-field equations (see66 for a derivation of mean-field equations, and67 for an example). nAChR activation and desensitization are transitions between two independent state variables which yields four different states (see Figure 2B). Our model of nAChR dynamics is modified from Katz and Thesleff45. For simplicity, we collapse rapidly and slowly desensitized states into one sate. Using known activation and desensitization parameters for human α4β2 and α7 nAChRs, we verify that, despite the simplifications, the nAChR model reproduces experimental whole-cell current recordings (eg from oocytes, human embryonic kidney 293 cells, neurons) in response to ACh and Ni (Figure 3: 68, 69, 42, 70, 71, 72). Note that this parameterization of the nAChR model allows us to account for subtype- and agonist specific receptor responses.

Figure 3
figure 3

Nicotinic acetylcholine receptor responses to nicotine and acetylcholine. Response properties of α4β2 (panels A, C) and α7 nAChRs (panels B, D) to nicotine and acetylcholine. A & B, Maximal response and net charge mediated by α4β2- (A) and α7-containing receptors (B) in response to Ni and ACh. Full lines show the peak current and the dashed lines show the normalized net charge mediated by the receptor during a 200 ms exposure to the respective agonist concentration. The responses to nicotine (acetylcholine) exposures are depicted in blue (red). Realistic nicotine concentrations are indicated with the arrow. Example currents evoked by Ni (blue lines) and ACh (red lines) are shown on the top of the panel for different agonist concentrations (indicated in μmol/L). C & D, Dose-response curves of α4β2- (C) and α7-containing nAChRs (D) in response to ACh in the presence of Ni. The normalized peak current (full lines) and the normalized net charge (dashes lines) evoked by ACh (green lines) are shown. A constant concentration of Ni=0.5 μmol/L is present during the 200 ms exposures to ACh. The red lines depict the responses evoked by the respective ACh concentration in the absence of Ni (as depicted in A and B). The net charge is normalized to 163 unit current times ms for α4β2 nAChRs (panel A and C) and to 73 unit current times ms for α7 nAChRs (panel B and D).

Full size image

VTA circuit model and nicotine control of DA signaling Using the model based on VTA circuitry and the dynamical properties as well as locations of the nAChRs subtypes, we investigate the dynamical response of DA and GABAergic neurons to nicotine exposures and to transient afferent inputs. We furthermore study how transient input integration in the VTA is affected in the presence of nicotine. Note the distinct time scales of nicotine exposures and transient inputs, ie nicotine stays elevated for minutes in the blood of smokers64, whereas behaviorally relevant stimuli evoke DA activity changes lasting for tens to hundreds of milliseconds73, 74.

Figure 4A shows the biphasic response of the GABA and DA neuron populations to nicotine exposure, ie DA activity is increased above baseline for about 20 min after a short, initial inhibition. This behavior is dictated by the activity of the GABA neuron population which shows the inverse time course, ie activation followed by inhibition. GABA neurons are predominantly shaping the DA signal in the shown scenario for two reasons: (i) the expression ratio of α4β2 nAChRs between DA and GABA neuron populations is 1/4 for all the results shown here, and (ii) nicotinic effects are mainly mediated by α4β2 nAChRs due to their high affinity as compared to α7 nAChRs. We assume that ongoing cholinergic afferent input activates the α4β2 nACh receptor on average in vivo. In turn, nicotine predominantly drives the receptor in the desensitized state, ie GABA neuron activity drops below baseline due to the loss of excitatory cholinergic drive. Hence, DA neurons are disinhibited.

Figure 4
figure 4

VTA response to nicotine and transient inputs. The respective scenario is illustrated at the top of each column, the middle row shows the temporal dynamics of the normalized activity of the VTA GABA neuron population and the lower row of the DA neuron population. A, Time course of DA and GABA neuron activity in response to 1 μmol/L nicotine for 2 min. The nicotine exposure time is indicated by the magenta bar. Note that the increase in DA activity outlasts the time of Ni exposure. B, Temporal dynamics in response to afferent input increases. Glu (blue lines, νGlu 0=0.1, νGlu app=0.2), ACh (cyan lines, ACh0=1.77 μmol/L, AChapp=5 μmol/L) and both inputs are augmented for 20 ms after which the input level goes back to baseline. Note the different time course as compared to A. C, Temporal dynamics in response to afferent input increases in the presence of 1 μmol/L nicotine. Same scenario as in B except that 1 μmol/L nicotine is present throughout the simulation.

Full size image

The DA and GABA neuron population activity changes in response to short-lasting increases of afferent inputs are shown in Figure 4B. The same simulation is repeated in the presence of 1 μmol/L nicotine (Figure 4C). An increase in afferent Glu activity has an excitatory impact on GABA cells and evokes a biphasic response with DA cells, as typically observed in feedforward inhibition circuits. The VTA circuit response to purely glutamatergic input is not significantly affected by nicotine (compare Figures 4B and C). This reflects the fact that realistic nicotine concentrations neither activate nor desensitize α7 nAChRs present on glutamatergic terminals. On the contrary, cholinergic input is strongly affected by nicotine. The ACh input in the VTA is mediated to a large part by the α4β2 nAChR and this receptor subtype is significantly desensitized already at low nicotine concentrations. In effect, the ACh input decreases the DA activity, again due to the larger α4β2 nAChR population on GABA cells (Figure 4B). Nicotine however desensitizes α4β2 nAChR reducing the inhibitory effect of afferent cholinergic input on DA activity (Figure 4C). Simultaneous Glu and ACh afferent input can be constructed from the linear superposition of the individual responses (Figure 4B and C).

Discussion

In this short review we gave several illustrative examples of how computational modeling can address questions related to the actions of the endogenous neuromodulator ACh and exogenous nicotine, that acts through cholinergic mechanisms (the nicotinic ACh receptors) and modulates the actions of ACh.

We present a range of philosophies to model ACh and nicotine action: from top-down to bottom-up. In the top-down approach, the computational models reflect, formally simulate and analyse the computational function of the cholinergic signal. In the example given, an abstract algorithmic model interprets ACh as encoding expected uncertainty within a task or an environment and elegantly explains a number of observed behavioral results. In the second approach, the biophysical effects of acetylcholine are gathered in a neuronal network model designed to perform a specific behavior: learning, encoding and recalling associative memories. Here it is shown that changing the network parameters in a manner compatible with experimentally observed muscarinic and nicotinic ACh effects, results in biasing the network toward one or the other of its functions. Interestingly, even though the two models do not treat effects of nicotine (and hence nAChR mechanisms) explicitly, they give a way of thinking what such effects might be within the formal framework of each of the models. The large scale neurocomputational framework for nicotine addiction we review above, showed how behavioral outcomes of nicotine - stable self-administration – can arise from nicotine working through receptor-level mechanisms and resulting in action-selection bias. This global approach, gave a formal idea how nicotine progressively usurps DA-signaling and leads to habitual drug-seeking. Yet the large-scale model also pointed out that in order to truly understand how nicotine wrestles the control of the DA circuitry away from the normal endogenous mechanisms (glutamatergic and cholinergic), we should turn our attention to the local circuitry, the properties of nAChRs and their interactions with nicotine at the circuit dynamics level. This is precisely the model we summarise in the last section of this review.

To understand the mechanisms of nicotine action on the DA machinery we discussed a computational circuit that formally implements a combination of a population activity model of the VTA with a detailed model of nAChR kinetics. Identifying the specific functional targets of nicotine action has potential direct implication for developing nicotine treatments, eg for designing replacement drugs. Hence a clear advantage of this approach is its potential applicability to translational research. However, the model as briefly described above is far from being complete. We have focused only on afferent input and the local circuitry of the VTA and did not address the possibly recurrent involvement of other neuronal structures involved in DA-signaling. Treating a dynamical situation, where inputs signal behaviorally relevant stimuli, remains a key challenge to the local circuit modeling approach. There are two possible complementary directions to address this challenge. First is to understand how the VTA circuit model would respond to transient inputs, ie signaling reward delivery, expectation of reward or appearance of a behaviorally relevant stimulus. Posing the question in more functional terms: what might be the computations that the VTA circuitry performs on its inputs, and are such compatible with the reinforcement learning accounts of DA signaling? Second and complementary approach is to incorporate the local circuit model of the VTA in a computational framework capable of simulating behavior, and examine if the specific mechanisms we propose are likely to lead to the behavioral outcomes observed under the influence of nicotine. Finally, a more general challenge to the circuit model is whether the model generalizes to drugs of addiction other than nicotine?

A major challenge for computational modeling is how to integrate the top-down and the bottom-up approaches for the ACh signaling and their influence by nicotine. This would require developing new hybrid approaches that combine both the algorithmic function of the natural ligand with dynamical models reflecting the biophysics of the neuromodulatory action. These combined models can then be used to understand how an exogenous substance, such as nicotine, would act through the same biophysical pathways to usurp their normal function. In particular such models would be an extremely useful tool in synthesizing the detailed receptor mechanisms with the longer time scale behavioral effects of the neuromodulators and the exogenous ligands, possibly leading to a deeper understanding of drug addiction as well as improving and targeting pharmacological drug design. The efforts we reviewed in the latter sections of the manuscript represent first steps in this challenging direction and remain a subject for active investigation and future work. Specifically one may propose that the detailed models of the VTA circuitry may be embedded into the large-scale framework to simulate the two-choice self-administration task. This framework can also be modified and extended to include cortical circuitry explicitly, to model learning and memory, as well as subjected to simulate and reproduce behavior in environments with changing certainty of action-strategy/reward contingencies.