Cortical circuits for transforming whisker sensation into goal-directed licking

Animals can learn to use sensory stimuli to generate motor actions in order to obtain rewards. However, the precise neuronal circuits driving learning and execution of a specific goal-directed sensory-to-motor transformation remain to be elucidated. Here, we review progress in understanding the contribution of cortical neuronal circuits to a task in which head-restrained water-restricted mice learn to lick a reward spout in response to whisker deflection. We first examine 'innate' pathways for whisker sensory processing and licking motor control, and then discuss how these might become linked through reward-based learning, perhaps enabled by cholinergic-gated and dopaminergic-gated plasticity. The aim is to uncover the synaptically connected neuronal pathways that mediate reward-based learning and execution of a well-defined sensory-to-motor transformation.


Introduction
An essential function of the brain is to interpret incoming sensory information in the context of behavioral demands and learned associations in order to drive appropriate motor output. In this review, we focus specifically on how 'innate' circuits in the mouse brain processing whisker sensory information can be linked to 'innate' circuits controlling tongue and jaw movements through rewardbased learning, such that a whisker sensory stimulus drives licking motor output in order to obtain a reward. Here, we first discuss recent studies in 'naïve' mice showing brain-wide processing of whisker information downstream of primary sensory cortex (Figure 1), and cortical pathways involved in licking motor control ( Figure 2). Subsequently, we consider how these sensory and motor circuits may become linked during rewardbased learning (Figure 3), perhaps via acetylcholinegated and dopamine-gated synaptic-plasticity mechanisms in cortex and striatum ( Figure 4).
The extensive, direct axonal projections from wS1 suggest a rapid spread of tactile-evoked activity across the mouse brain. The spatiotemporal dynamics of this spread

Current Opinion in Neurobiology
Whisker primary somatosensory cortex signals to many downstream targets. (a) The whiskers on the snout are laid out in a stereotypical pattern. Deflection of a whisker initiates sequential activity in neurons of the trigeminal ganglion (Tg), principal trigeminal nucleus (Pr5) in the brainstem, ventral posterior medial nucleus (VPM) of the thalamus, and the whisker primary somatosensory barrel cortex (wS1). Neurons in wS1 project to many cortical and subcortical targets including: whisker secondary somatosensory cortex (wS2), whisker primary and secondary motor cortex (wM1/2), perirhinal cortex (PRh), orbitofrontal cortex (OFC), secondary visual areas (V2), posterior parietal cortex (PPC), dysgranular zone (DZ), across dorsal cortex can be visualized through voltagesensitive dye imaging (Figure 1b). After a brief deflection of a single whisker, the first depolarizing response was induced in a highly localized spot in wS1 within 10 ms, and followed immediately by depolarization of wS2, wM1 and secondary whisker motor cortex (wM2) within a 20 ms window [22]. Consistent with a possible role of cortico-cortical projections from wS1 in distributing whisker information to other cortical areas, local inactivation of wS1 reduced whisker deflection-evoked responses in wM1 [22]. Anatomical and functional differentiation between cortico-cortical projections from L2/3 neurons in wS1 suggests two segregated pathways for whisker sensory processing (Figure 1c) [23,24], analogous to the dorsal and ventral streams in primate vision [25]. The dorsal stream -from wS1 to wM1 -appears to be highly responsive to brief passive stimuli or to the first contact in a sequence of active whisker-object contacts ( Figure 1c) [23]. The dorsal stream could, therefore, serve as detector for unexpected contacts with an object, which would trigger attentional searches and exploratory whisking thus informing about the 'Where' and 'When' aspects of the encountered object. On the other hand, the ventral stream -from wS1 to wS2 -more faithfully transmits the input from whiskers by responding to each touch within a sequence of active contacts, and thus might provide more detailed information about the 'What' aspect of the encountered object ( Figure 1c) [23].
In parallel to cortico-cortical pathways, subcortical pathways are also likely to play important roles. For example, a direct pathway from wS1 to dorsolateral striatum appears to prominently transmit whisker sensory information ( Figure 1d) [26,27]. Striatum is a part of the basal ganglia, which plays an important role in action selection and initiation during motivated behavior [28]. The pathway from wS1 to dorsolateral striatum might thus help link a sensory input to an appropriate goal-directed motor output.
In summary, whisker sensory processing occurs in a highly distributed manner in the mouse brain. Even a brief deflection of a single whisker can evoke signals in many brain regions downstream of wS1. As we will discuss later, these brain-wide signals are likely to be useful for learning abstract sensory-to-motor transformations, for example, converting sensory signals into goal-directed motor commands, such as licking for a reward.

Cortical control of licking
Licking provides the subjective report of sensory perception in many head-restrained tasks for rodents [29,30]. It is therefore of great interest to investigate the neuronal circuits responsible for tongue and jaw movements, and, here, we begin by considering cortical aspects of licking motor control. Optogenetic motor mapping of dorsal cortex in naïve awake mice identified a focal tongue-jaw-related region of primary motor cortex (tjM1) (Figure 2a) [31 ]. This same cortical region also appears to receive tongue and jaw sensory information, likely, at least in in part, mediated by direct long-range axonal input from the primary somatosensory area for tongue and jaw (tjS1) (Figure 2a) [31 ,32 ]. A mirrorreflected map of innervation from sensory cortex onto frontal cortex appears to be a key organizing principal of dorsal mouse cortex [22,31 ,33]. For the specific case of tongue and jaw motor control, it is likely that touch and proprioceptive somatosensory information from tjS1 is of immediate importance for sensory-guided movements via the direct projection to tjM1.
It is likely that tjM1 also receives diverse cortical information from other sensory and motor regions (Figure 2a). In a recent study [32 ], retrograde labeling of monosynaptic afferent inputs revealed that the primary orofacial motor area, including tjM1, receives direct inputs from orofacial S1/S2 and from an anterolateral part of the secondary motor area. We thus propose that tjM1 is likely to receive input from secondary tongue and jaw motor cortex (tjM2), lying within the anterolateral motor area (ALM) [34] (Figure 2a). Secondary motor areas may integrate diverse modalities of information for motor planning, and send output to various cortical and subcortical regions, including tjM1. Neurons in deep layers of tjM1 and tjM2/ALM have direct projections to brainstem premotor nuclei (Figure 2b,c) [35 ] that in turn innervate   Neural circuits for controlling licking. (a) Optogenetic motor mapping by stimulating different parts of dorsal cortex revealed a frontal region (black crosses) evoking jaw movements filmed using a high-speed camera (left). Green crosses mark primary somatosensory tongue/jaw region (tjS1). Wide-field calcium imaging of dorsal cortex upon mechanical stimulation of the tongue revealed two sequentially activated spots: tjS1 (green crosses) and tjM1 (black crosses) (middle). Monosynaptic retrograde tracing of afferent inputs to tjM1 revealed projections from tjS1/S2 as well as tjM2/ALM area (right, green dots). In contrast, retrograde labelling from wM2 showed a distinct input map (right, red dots). (b) Pyramidal tract motor neurons in the hypoglossal motor nucleus (Hg, Mo12) and trigeminal motor nucleus (Mo5) that control tongue and jaw muscles, respectively (Figure 2d) [36,37 ,38].
In summary, it appears likely that cortical control of licking might be most directly mediated by pyramidal tract neurons in tjM1 [31 ] projecting to brainstem central pattern generators. However, other cortical areas such as tjM2/ALM are also likely to contribute importantly, especially to motor planning [301 , 34,35 ].
Pathway for transforming whisker sensory input into licking motor output Having described 'innate' circuits for whisker sensory processing ( Figure 1) and licking motor control (Figure 2), we now turn our attention to how whisker sensory information can become linked to licking motor output through reward-based learning. One of the simplest paradigms to study such a goal-directed sensory-to-motor transformation is to train water-restricted head-restrained mice to lick a reward spout upon detecting a brief whisker deflection (whisker detection task; Figure 3a). A whisker deflection has been reported to evoke biphasic neuronal responses in wS1 with a relatively invariant primary response, and a variable secondary response which was larger in trials where the mouse successfully licked (hit trials) compared to trials in which the mouse failed to lick (miss trials) (Figure 3b) [39][40][41][42]. In addition, inactivation of wS1 reduced the probability of hit trials [31 ,39,42-44,45 ], whereas activation of wS1 substituted for the whisker deflection [39]. Neuronal activity in wS1, therefore, appears to participate in the conversion of whisker sensory input into licking motor output during execution of the whisker detection task and it is thus important to investigate downstream targets of wS1.
Two important projection targets of L2/3 neurons in wS1 are wS2 and wM1, as described earlier (Figure 1c) [15,16 ,17,23]. After learning the whisker detection task, neurons in wS1 projecting to wS2 (wS1 ! wS2 neurons; but not those projecting to wM1) gain a prominent secondary late excitation in hit trials (Figure 3c) [40]. Furthermore, wS1 ! wS2 neurons (but not wS1 ! wM1 neurons) depolarize before spontaneous false alarm licking in expert mice but not naïve mice (Figure 3c). Learning of the whisker detection task, therefore, appears to enhance the flow of sensory signals from wS1 towards wS2, and the activity in this pathway could contribute to the initiation of licking. High-resolution two-photon calcium imaging has furthermore revealed enhanced bidirectional signaling between wS1 and wS2 during correct performance of the whisker-detection task [42,44,45]. An early step in execution of the whisker detection task may, therefore, be reciprocal excitatory interactions between wS1 and wS2, which are apparently enhanced through reward-based learning. Recurrent excitation between wS1 and wS2 could amplify the behaviorally relevant transient input of a whisker deflection by generating a long-lasting cortical activity, which might further contribute to recruiting neurons in appropriate motor areas for initiating tongue and jaw movements (Figure 3d). Importantly, inactivation of wS1, wS2 and tjM1 results in reduced hit rates in the whisker detection task, whereas inactivation of wM1 and the primary forepaw somatosensory cortex (fpS1) have little effect upon hit rates [31 ,39,43,44,46 ].
Neither wS1 nor wS2 are thought to connect directly to brain areas involved in licking motor control, such as tjM1 [31 ], but one possible pathway might be through connectivity within secondary motor cortex between wM2 and tjM2/ALM. Consistent with such a hypothesis, voltage-sensitive dye imaging of task-performing mice revealed highly dynamic activity in dorsal cortex evoked by a whisker deflection (Figure 3e) [41]. The earliest responses (within 50 ms of whisker deflection) differed little comparing hit and miss trials, but at later times cortex was more depolarized in hit compared to miss trials, with the strongest differences occurring in frontal brain regions, including wM2, tjM1, and tjM2/ALM, which might thus contribute to decision-making and initiation of licking motor output.
Other higher-order cortical areas also appear to contribute to the learning and execution of goal-directed sensorimotor transformations. In particular, neurons in the medial prefrontal cortex (mPFC) and the dorsal part of hippocampal CA1 region (dCA1) are recruited through rewardbased learning of sensory detection or discrimination 42 Whole-brain interactions between neural circuits neurons in motor cortex project to premotor nuclei of the brainstem (including reticular nuclei) likely conveying lick commands. (c) Premotor neurons in the brainstem in turn project to motor neurons of the hypoglossal nucleus (Hg) and motor trigeminal nucleus (Mo5), which respectively control tongue and jaw muscles. Simultaneous monosynaptic rabies tracing of genioglossus tongue muscle (Gg, green) and masseter jaw muscle (Mas, red) premotor neurons revealed the spatial distribution of the premotor neurons within brainstem, with many premotor neurons being located in the reticular nuclei. (d) We propose tjM1 as a critical cortical node in control of licking by integrating different cortical inputs and projecting to downstream premotor nuclei involved in the control of tongue and jaw movements. These premotor nuclei integrate motor commands and innervate motor neurons in Hg and Mo5, which control tongue and jaw muscles respectively. Jaw-closer muscles, Mas and temporalis (Tem), and jaw-opener muscle, digastric (Dig), are shown together with tongue-protrudor muscle, Gg, and tongue-retractor muscles, hyoglossus (Hg) and styloglossus (Sg).  tasks ( Figure 3f) and inactivation of these areas strongly impairs the task performance [46 ,47-50].
Therefore, many brain areas appear to participate in the conversion of whisker stimulus into goal-directed licking, but the neural circuits and plasticity mechanisms linking these various brain regions remain to be determined.

Connecting sensory and motor circuits through reward-based learning
As a hypothesis for the basis of future experimental investigations, we propose that the highest order cortical regions such as mPFC and hippocampus might contribute to rule learning (Figure 4a), imposing context-dependent routing of sensory signals from wS1 and wS2 to frontal areas such as wM2, which might then communicate with tjM2/ALM and tjM1 to drive licking motor output. Subcortical regions such as thalamus and basal ganglia are also likely to contribute importantly to initiating goal-directed licking, and might also contribute to exciting frontal motor-related cortex in response to whisker stimuli ( Figure 4a). An important question relates to how the underlying synaptically connected neuronal circuits change across task learning through reward-based feedback. Reward signals in the brain have been prominently linked to phasic dopamine increases in striatum [51,52], and more recently also to cholinergic signals in cortex [53], which we explore below in the context of learning the whisker detection task.
Cholinergic signaling has long been implicated in cortical plasticity [54,55]. Cholinergic neurons have been shown to increase firing rate in response to various reinforcement signals, including unexpected rewards [53]. Such cholinergic reward signals in cortex during task learning could help drive neocortical circuit plasticity through disinhibition ( Figure 4b) [56,57]. In particular, acetylcholine might excite a type of GABAergic inhibitory neuron characterized by expression of vasoactive intestinal peptide (VIP) and nicotinic receptors. Cholinergic excitation of VIP neurons could drive disinhibition of local excitatory cortical microcircuits because VIP neurons prominently inhibit other inhibitory neurons expressing somatostatin (SST) or parvalbumin (PV) [58][59][60], which in turn strongly inhibit excitatory neurons. Electrophysiological recordings from VIP neurons show a long-lasting whisker-evoked excitation during whisker-detection tasks (Figure 4b) [61], consistent with their possible involvement. Disinhibition mediated by VIP neurons could lead to prolonged depolarization and burst firing of excitatory neurons, which could drive long-term synaptic potentiation between synaptically connected co-active neurons. Reward-driven synaptic plasticity might help strengthen excitatory connectivity between wS1 and wS2, which as discussed above, might contribute to enhance sensory processing of the relevant whisker stimulus across learning. The recurrent excitation could serve as a short-term memory of recent important sensory input, which might be useful for driving motor circuits downstream of wS1 and wS2. Acetylcholine could have a similar binding function for the various cortical areas active during task execution, resulting in enhanced connectivity through fire-together-wire-together synaptic plasticity. Future experiments should test these hypotheses.
Midbrain dopamine neurons show some of the most prominent reward-related signals in the brain [51,52]. Unexpected rewards evoke a transient increase in the activity of midbrain dopaminergic neurons, which project prominently to striatum. Dopaminergic signaling can influence synaptic plasticity in striatum by acting on dopamine type 1 receptors (D1Rs) to promote long-term potentiation of glutamatergic synaptic input onto direct pathway striatonigral projection neurons (dSPNs) helping to reinforce rewarded behaviors (Figure 4c) [62][63][64][65]. Consistent with this hypothesis, investigating the whisker detection task, enhanced sensory signals were found in D1R-expressing dSPNs compared to the D2R-expressing indirect pathway striatopallidal neurons (iSPNs) (Figure 4c) [66]. Furthermore, optogenetic stimulation of dSPNs, but not iSPNs, could substitute for the whisker stimulus during task execution, suggesting a possible causal role for dSPNs in goal-directed sensorimotor transformation [66]. One hypothesis to account for rewardbased learning in the whisker detection task is, therefore, that a dopamine reward signal strengthens corticostriatal synapses from wS1 onto D1R-expressing dSPNs, which in turn would enhance the whisker-deflection evoked excitation of these neurons. Increased activity of dSPNs could directly inhibit tonically active GABAergic neurons in substantia nigra pars reticulata (SNr). Reduced activity in two inhibitory pathways from SNr could contribute to 44 Whole-brain interactions between neural circuits Reward-based learning of the detection task appears to enhance activity of wS1 and wS2, but how these activities reach a licking motor region such as tjM1 remains unknown. (e) Voltage-sensitive dye imaging shows more wide-spread depolarization in hit trials compared to miss trials during a late period (>50 ms after whisker deflection). Frontal (motor) regions are more strongly modulated than posterior (sensory) regions comparing hit versus miss trials. (f) Local field potential recordings reveal sequential whisker deflection-evoked activity in wS1, wS2, wM1, parietal association cortex (PtA), medial prefrontal cortex (mPFC) and dorsal hippocampal area CA1 (dCA1) (above). Action potential firing in mPFC on average increases in response to whisker stimulation in trained mice, but not in mice which have been exposed to the same whisker stimuli uncoupled to reward. Panels  Putative cholinergic and dopaminergic reward signals might help wire circuits for goal-directed sensorimotor transformation through synaptic plasticity. (a) Complex interactions between different brain areas likely underlie learning and execution of the whisker-detection task. Acetylcholine reward signals could be important for learning and plasticity in cortex, whereas dopamine reward signals could play a major role in the striatum (NAc, nucleus accumbens). (b) Cholinergic reward signals might enhance cortico-cortical plasticity through disinhibition. Acetylcholine might excite VIP-expressing GABAergic neurons, in turn inhibiting SST-expressing and PV-expressing GABAergic neurons, and thus causing disinhibition of nearby excitatory neurons (left). Disinhibition could help mediate 'fire-together-wire-together' plasticity of co-active synaptically connected cortical regions, such as wS1 and wS2. Consistent with this hypothesis, juxtacellular electrophysiological recordings show that VIP neurons increase firing rate in response to whisker deflection during detection task performance (right). (c) Dopaminergic reward signals could promote strengthening of cortical glutamatergic input onto D1-receptor-expressing striatonigral neurons of the direct pathway (dSPNs), and weakening of cortical glutamatergic input onto D2-receptor-expressing striatopallidal neurons of the indirect pathway (iSPNs) (left). Consistent with this hypothesis, dSPNs depolarized more rapidly in response to whisker stimulation than iSPNs (right). Transient activation of dSPNs is sufficient to initiate licking, and this GO signal could thus contribute importantly to task execution. initiation of licking. Firstly, SNr projects to brainstem [67], and reduced SNr activity might thus release tonic inhibition of licking-related brainstem areas (Figure 4a,c). Secondly, SNr projects to motor thalamus, and reduced SNr activity might disinhibit thalamocortical circuits contributing to licking [68 ,69].

Future perspectives
In this review, we focused on current understanding of cortical neural circuit mechanisms underlying the rewardbased learning of a well-defined goal-directed sensorimotor transformation. In addition to the basal ganglia, many other subcortical regions are also likely to participate and much work remains to be done before a complete understanding is uncovered, including likely important contributions of midbrain, brainstem, colliculus and cerebellum [70 ]. In addition, investigation of more complex tasks involving delays and directional licking [71] will help tease apart sensory processing, decision-making and motor commands.

Conflict of interest statement
Nothing declared.