Role of the basal ganglia in innate and learned behavioural sequences

3.1 Rhythmic behaviours

Many of the most basic behaviours that are in our behavioural repertoire, such as breathing, and others that do not seem so basic, such as walking, are formed of rhythmic sequences of movements. Many of these patterns involve temporally organised sequences of muscle activations that are regulated subcortically by neural networks called central pattern generators (CPGs) (Bucher et al. 2015). CPGs are neural networks that control many innate automatic behavioural sequences that form some of the most basic behaviours in an animal’s repertoire, something that has been called the motor infrastructure (Grillner and Wallen 2004). CPGs are present in many species, both in invertebrates, controlling behaviours such as crawling in leeches and swimming in molluscs (Cacciatore et al. 2000; Sakurai and Katz 2016); and in vertebrates, underlying locomotion and other behaviours in several species, such as lampreys, zebrafish, turtles and cats, to mention a few (Berkowitz et al. 2010; Marder 2000). Although most studies of CPGs have been performed on small invertebrates and lower vertebrates, due to their reduced number of neurons, several shared characteristics across many species have been found (Bass 2014; Grillner and Wallen 2002; Grillner et al. 2005).

The main characteristic of CPGs is that they can generate continuous rhythmic activity without external tonic timing inputs or sensory feedback (Marder and Bucher 2001; Satterlie 1985). Thus, as their name indicates, their pattern of activity can be produced intrinsically, both by synaptic connections and through neuromodulation (Bucher et al. 2015; Marder and Bucher 2001). This means that the basic behavioural patterns controlled by CPGs can still be found after deafferentation. For example, both leeches and cats can still produce somewhat normal coordinated locomotion after deafferentation and, in the case of leeches, even after complete decerebration (Cacciatore et al. 2000; Frigon and Grossard 2010).

Although sensory feedback is not necessary for the production of the basic activity pattern, depending on the behaviour, some characteristics of the CPG’s dynamics do depend on sensory feedback to different degrees. For example, the rhythmic wing sequence of movements executed by some insects to fly can still be produced after sensory deafferentation, but it is considerably slowed down and the inter-segmental coordination is affected (Pearson and Wolf 1987). Other CPGs, like the ones controlling swimming in the crayfish, can still produce basic rhythmicity and coordination after all sensory afferents have been cut (Hughes and Wiersma 1960). The dependency on sensory feedback is believed to be subject to how stable or variable the environment was in which the behaviours evolved (Cacciatore et al. 2000).

3.2 Neural bases of CPGs

A CPG unit has been described as “a group of neurons that can generate recurrent bursts” (Grillner 2006). This bursting activity pattern in vertebrates arises from relatively simple designs formed mostly of motoneurons and glutamatergic (excitatory) and glycinergic (inhibitory) interneurons located in the brainstem and spinal cord (Grillner 2003). There are two main neural mechanisms by which a CPG network is able to produce rhythmic sequential activity. First, some CPGs have neurons with intrinsic oscillatory properties, referred to as “pacemakers”, which are able to impose rhythm on a network that by itself does not burst periodically (Marder and Bucher 2001). These types of networks usually control behaviours where the rhythm needs to be present for prolonged times or even at all times, such as respiration in mammals and swimming in jellyfish (Marder and Bucher 2001; Rekling and Feldman 1998; Satterlie and Nolan 2001).

Nonetheless, it is more common that the patterned activity of a CPG is the result of its synaptic interactions, and that the resulting behaviour can be started and stopped at will, as in locomotion, which can be initiated and ended in a goal-directed manner (Grillner 2006; Grillner and Wallen 2004). Thus, in cases where there are no pacemaker neurons, the rhythmic activity of the networks emerges from the connections between its neurons and the descending afferents that reach the CPGs (Bucher et al. 2015; Satterlie 1985). In general terms, it is believed that CPG reconfiguration depends significantly on neuro-modulatory projections, both from descending projections and from sensory feedback (Marder 2000; Ramakrishnan et al. 2014). The basal ganglia output nuclei send projections to brainstem nuclei, which project back to the striatum through the thalamus, thus, putting the basal ganglia in a position to modulate the selection of motor plans encoded in brainstem and spinal CPGs via descending commands (Grillner et al. 2005; McHaffie et al. 2005; see Figure 2).

Figure 2:

Central pattern generator (CPG) selection. The cortex in mammals and the corresponding structures in lower vertebrates (pallium), alongside the thalamus, provide an excitatory (red) input to striatum. The striatum consists primarily of inhibitory (blue) GABAergic neurons, which in turn inhibit the output nuclei of the basal ganglia (pallidum, and in mammals, the substantia nigra pars reticulata [not shown]), as part of the basal ganglia’s direct pathway. The output nuclei again consist of GABAergic neurons, and their high level of baseline activity keeps CPGs elsewhere in the nervous system under tonic inhibition. This is released when the striatum is activated and inhibits the output nuclei, resulting in a disinhibition of the CPGs. Dopamine (DA) affects the responsiveness of striatal neurons. Reproduced with permission from Grillner et al. (2006).

In conclusion, CPGs are neuronal networks that control basic sequences of movements, and are able to sustain their basic firing pattern without external inputs, although, in some instances they can be modified by sensory feedback and descending influences, giving them some flexibility (Grillner 2006). Those descending influences, for example from the basal ganglia, are also in a position to select appropriate CPGs as required (Grillner et al. 2005; McHaffie et al. 2005). In the following section we will review grooming sequences, an innate behaviour that – although controlled by CPGs in the brainstem – its sequential patterning seems to come from the basal ganglia.

3.3 Grooming behaviour

Grooming behaviours, such as body licking, face washing and paw licking, are innate behaviours present in many species and can be rich in structure. Berridge et al. (1987) discovered that among the several behaviours that are performed within a grooming bout, rodents execute grooming chains with a very specific order, both spontaneously and triggered by certain stimuli (e.g. water on the rodent’s fur). These stereotypical grooming chains are present in many species – such as squirrels, guinea pigs, gerbils and hamsters – that differ up to 65 million years in their evolution, suggesting that the implementation of this patterned behaviour, and possibly the mechanism underlying it, is highly conserved (Berridge 1990).

The stereotypical grooming chain is different in each species of rodent. Nonetheless, in all of them, around four sequential phases with a hierarchical structure can be found (Berridge 1990). In rats, the grooming chain consists of four phases executed in a specific order, with a cephalo-caudal direction (Kalueff et al. 2007). The first phase consists of a series of very fast and small elliptical strokes around the nose. This is followed by a series of unilateral strokes around the mystacial vibrissae below the eye and a set of large bilateral and symmetrical strokes that usually go over the ears. The chain is finished when animals turn to their flank and begin to lick their body. The completion of the first three phases takes between 3 and 5 s, while the last element’s length can vary, lasting up to 30 s (Berridge 1990; Berridge et al. 1987; see Figure 3).

Figure 3:

Phases of the grooming chain. Bottom: Drawings showing the four phases (A–D) of the grooming chain in their order of appearance; Top: The movement path of the forelimbs over time, showing the distance from midline (vertical dimension). The inset diagram to the left shows a rat’s face as viewed from below on the video monitor on which the distances were determined. The movements from midline are measured from the midline to the centre of the forepaw (Y dimension on this drawing), with the base of the vibrissae, eyes, and ears as landmarks. Reproduced with permission from Aldridge and Berridge (1998); © 1998 The Society for Neuroscience.

It has been calculated that the appearance of this grooming chain is 13,000 times more likely than would be expected by chance, representing an exceptional case of serial order (Berridge et al. 1987). Thus, it is believed that the execution of the grooming chain is not the result of some random process, but rather the product of an active sequential mechanism. However, it is also possible to observe the behaviours that are part of the grooming chain performed in an unstructured way during grooming bouts. This has been used as a way to compare whether a treatment has an effect only in the context of the grooming chain or on the whole grooming bout (Berridge 1990). Importantly, the grooming chain is executed as a unit. Once the first element of the chain is performed, the probability that the rest of the elements will be completed in the same order is around 0.9 or higher (Berridge and Whishaw 1992). Therefore, the execution of the first element, the elliptical strokes, is usually taken as a reliable criterion to identify the presence of a grooming chain. The high degree of stereotypy and the fact that the elements of the chain are easily distinguishable, have made it a useful behavioural model to study the implementation of sequential organisation (Kalueff et al. 2007).

3.4 Neural substrate underlying the grooming chain

Berridge, Aldridge and collaborators exploited the grooming chain as a behavioural model to investigate the neural structures underlying the implementation of action sequences. Through different lesion and electrophysiological studies, they have found that the basal ganglia are a key network in the performance of this sequential innate behaviour. Accordingly, it has been found that disruptions in the execution of the grooming chain are present in animal models of Tourette’s syndrome (Taylor et al. 2010), obsessive compulsive disorder (Berridgem et al. 2005) and Huntington’s disease (Tartaglione et al. 2016), all of them disorders related to dysfunction of the basal ganglia.

From several studies, the striatum has emerged as a key region for the implementation of the sequential order of the grooming chain. Striatal damage decreases the probability of completing the grooming chain and increases its duration, without actually damaging the ability of the rats to perform each behaviour individually (Berridge and Fentress 1987b; Tartaglione et al. 2016). Lesions to brain structures known to have a role in motor control, such as the cerebellum, primary and secondary motor cortex or the entire neocortex, do not seem to produce any lasting effects on the sequential organisation of the grooming chain (Berridge and Whishaw 1992). Furthermore, lesions to other structures in the rat, such as globus pallidus (equivalent to the primate GPe) or ventral pallidum do not affect grooming chain serial performance either (Cromwell and Berridge 1996).

In more detailed studies, it has been found that only lesions in the anterior dorsolateral striatum (DLS) disrupt the serial execution of the grooming chain, with only around 24 % of the chains being completed correctly. Excitotoxic lesions in the dorsomedial, ventromedial or ventrolateral striatum do not disrupt its execution (Cromwell and Berridge 1996). This is interesting, given that the DLS has been related to the performance of habitual behaviours, suggesting a possible overlap of mechanisms (Yin and Knowlton 2006). Again, interestingly, sensory deafferentation of the face also has no effect on grooming chain performance, suggesting that the serial order of this behavioural pattern is not based on somatosensory feedback, but rather on some central mechanism possibly implemented or at least modulated by the striatum (Berridge and Fentress 1987a). This independence from sensory feedback and the fact that the order of the grooming chain seems to be independent of timing from cortical inputs has led to the suggestion that CPGs in the brainstem are modulated by the striatum, which contributes to the sequential pattern (Cromwell and Berridge 1996).

Electrophysiological studies have revealed that neurons in the DLS display higher firing rates when each of the chain behaviours are executed in the context of the ordered grooming chain than when these same behaviours are performed in an unordered fashion. This is not observed in the ventromedial striatum, where neuronal activity seems to be more strongly related to the initiation of the grooming sequence (Aldridge and Berridge 1998; Aldridge et al. 1993). Neurons in the SNr have also been found to show distinctive firing patterns according to whether the grooming behaviours are performed inside or outside the grooming chain. Neurons in SNr are excited during the initiation of the chain, and significantly more inhibited as the grooming chain is performed (Meyer-Luehmann et al. 2002). The distinctive increase in activity observed when the grooming chain is executed has also been observed in the execution of sequences that, although not as stereotyped, follow a certain order. Aldridge et al. (2004) analysed what they called the “warm-up sequence”, a sequence of behaviours that occurs when rats transition from periods of immobility to periods of movement. This sequence is composed of resting, head and torso movements and locomotion, and it tends to occur in this particular order, although not in such a fixed way as the grooming chain. Nevertheless, a similar increase of striatal activity is observed during its execution. Although this increased firing is at a lower level than the one observed in the grooming chain, these results suggest that the striatum could be involved in sequential action organisation in a general way.

Finally, damaging dopaminergic nigrostriatal projections can lead to a decrease in the percentage of correct completions of the grooming chain, suggesting a role for dopamine in sequence implementation (Berridge 1989; Pelosi et al. 2015). Accordingly, increasing dopamine levels in the mouse brain renders the performance of the grooming chain more rigid, making hyper-dopaminergic mice more resistant to disruptions of the chain (Berridge et al. 2005). Likewise, D1-type receptor agonists generate super-stereotypy in the grooming chain; and co-administration of D2-type receptor antagonists decrease these effects, suggesting that both D1-type and D2-type dopamine receptors are involved in the implementation of sequential stereotypy (Taylor et al. 2010).

In summary, studies carried out with the grooming chain as a model have suggested that the basal ganglia, and in particular the DLS, play a crucial role in the implementation of these sequential patterns, with increased firing rates only during ordered sequences both in striatum and SNr, with an important role for dopamine. It has been proposed that the role of the striatum might lie in selecting and allowing the CPGs in charge of the grooming chain to gain access to the motor system in the correct order, while inhibiting other behaviours (Berridge and Wishaw 1992). In the following section we will discuss some results that have been found in another innate sequential behaviour linked to the basal ganglia, birdsong, which unlike grooming, has a learning phase.

3.5 Birdsong

Many species of birds sing songs in order to reproduce and defend territory. These songs are arrays of complex sequences of syllables that display long-range correlations that can extend up to 10 s over time (Markowitz et al. 2013); thus, they have also been used as a behavioural model to study sequence codification. Much like the grooming chain, birdsongs are arrangements of syllables that are not random – they tend to follow a predictable order and they recurrently start and finish in the same way (Gil and Slater 2000). However, unlike other innate behaviours, birds go through a learning period before they crystallise their song structures, which has suggested that birdsong arose from the relaxation of an innate mechanism (Gardner et al. 2005).

The process of vocal learning in birds is usually divided into two general phases: (1) a sensory phase in which birds listen to the songs of more experienced “tutor” birds, and (2) a sensorimotor learning phase, in which birds practice the memorised songs and perfect them. After these phases, songs are crystallised into highly stereotyped patterns (Williams 2004). Therefore, birdsongs are an interesting sequential behavioural pattern to review, because both innate and learning mechanisms are at play in the development of the songs’ structure (Gardner et al. 2005).

Depending on the species, birds develop different song repertoires, from Bengalese finches performing only one stereotyped song, to nightingales that produce over 200 different song types. To deal with the serial information present in the songs, it has been shown that many species of birds, such as canaries, zebra finches and nightingales, produce phrases of syllables, sometimes called “motifs”, which themselves can be grouped into songs (Hultsch and Todt 1989). This arrangement of syllables is believed to be done in a hierarchical manner, which, much like chunks in the motor domain, allows an efficient way to process and store the large number of syllables a bird can come to produce (Markowitz et al. 2013).

These syllable phrases have been described as “subsets of sequentially associated items”. They are characterised by having large transition probabilities between elements of the same phrase or chunk, and low transition probabilities between phrase boundaries. Furthermore, these syllable phrases are separated from each other by long silent intervals. The timing of these intervals correlates with how the syllables are sequenced, with high transition probabilities associated with short silent intervals and low transition probabilities with long ones (Matheson and Sakata 2015; Takashi et al. 2010), suggesting a similar sequence representation as the one described in mammals, who display short inter-response times between actions within a chunk, and long inter-response times between chunks (Sakai et al. 2003).

This syllable organisation gives rise to highly consistent songs. The high degree of song stereotypy is believed to have been selected by evolution, since female birds prefer males that sing more stereotyped syllable sequences (Sakata and Vehrencamp 2011). Indeed, it is known that some aspects of the syntactic organisation of songs is imposed by innate mechanisms, given that birds reared in isolation are able to develop some structured songs (Liu and Nottebohm 2007). Moreover, even if birds are exposed to incorrect tutor songs, they still develop structured songs. For example, birds tutored with songs without the species-typical first element, tend to invent their own initial element (Hultsch and Todt 1989). Interestingly, creating chunks of syllables also seems to be, at least partly, an innate characteristic, since birds exposed to really long tutor songs (i.e. with less and shorter boundaries) have a tendency to spontaneously segment the songs into smaller segments, even though they were not explicitly tutored to do it (Hultsch 1992).

Nonetheless, although birds exposed to these “incorrect” tutor songs or birds reared alone (i.e. untutored) preserve some characteristics of normal songs, they do display odd structures, with decreased sequential stereotypy and smaller song repertoires (Hultsch 1992; Hughes et al. 2002), meaning that learning in birdsongs plays an important role for the development of their structure. During the learning period, sensory feedback is very important to refine the precise execution of the song, but once learned, a song’s rendition becomes very stable, and it does not change even if birds are exposed to new tutor songs (Brainard and Doupe 2000). However, after crystallisation, if auditory feedback is disturbed by external noises, birdsongs display disruptions both in sequencing and timing aspects, with less stereotyped songs and with slower tempo (Sakata and Brainard 2006). This seems to be in contrast with chain grooming, a much less flexible behaviour, which can be carried out normally without any sensory feedback (Berridge and Fentress 1987a). Thus, more flexible mechanisms play an important role in the execution of crystallised syllable sequences.

3.6 Neural circuits underlying birdsong

The system in charge of producing songs in birds involves the avian cortex, basal ganglia, thalamus and brainstem nuclei. This song system has been typically divided into two pathways: (1) the motor or posterior pathway which is necessary for acquisition and execution, and (2) the anterior pathway which is only necessary for acquisition (Nottebohm 2005). In the motor pathway, neurons from the high vocal centre (HVC) send projections to the robust nucleus of the arcopallium (RA), which in turn is directly connected to brainstem and midbrain neurons that control the vocal and respiration muscles (Bertram et al. 2014). On the other hand, in the anterior pathway, neurons from the HVC send projections to Area X, a structure homologous to the mammalian striatum. Area X in turn sends exclusive inhibitory projections to a portion of the dorsolateral anterior thalamic nucleus, which in turn sends exclusive projections to the lateral magnocellular nucleus of the anterior nidopallium (LMAN; Bottjer et al. 1989), which by projecting back to the RA closes the cortico-basal ganglia-thalamocortical loop. HVC and RA are analogous to the premotor and motor cortex in mammals, respectively (Brainard and Doupe 2013), thus, the posterior pathway is mostly a motor cortical circuit, while the anterior pathway resembles the cortico-thalamo-basal ganglia loop observed in mammals (Jarvis et al. 1998; Mooney 2009). A comparison is shown in Figure 4.

Figure 4:

Comparison of the mammalian and avian basal ganglia network. LMAN: The lateral magnocellular nucleus of the anterior nidopallium, HVC: High vocal centre, RA: robust nucleus of the arcopallium. Excitatory connections are in red and inhibitory connections are in blue. Yellow/semi-circles indicate mixed excitatory/inhibitory connections.

The roles of these two pathways differ at the different stages of song acquisition. The motor pathway, as its names indicates, is fundamental for motor control. Lesions to this pathway lead to the complete loss of singing (Nottebohm et al. 1976). Furthermore, the highly stereotyped performance of crystallised songs is believed to come from activity in this pathway, since lesions to the HVC (i.e. homologue of the premotor cortex) lead to disruptions in the songs’ stereotypy and timing (Long and Fee 2008; Thompson and Johnson 2007). Recordings made from HVC neurons have shown that these neurons fire only once per song phrase or motif, suggesting a sparse hierarchical coding of song phrases in this area (Hahnloser et al. 2002). Neurons in the HVC are connected in a chain fashion, believed to be responsible for producing the ordered sequences of syllables (Long et al. 2010), which shares some similarities with the neuronal chains believed to orchestrate sequential activation for crawling patterns in leeches (Cacciatore et al. 2000).

Lesions to the anterior pathway (i.e. the avian basal ganglia) only disrupt the songs’ structure when they are performed during the phase of song acquisition. If lesions to the LMAN are made during acquisition, the stereotypy of the songs significantly increases, producing highly repetitive patterns from very early on, indicating that one of the main roles of this pathway is to introduce variability, a key component for learning (Ölveczky et al. 2005). This variability is apparently not completely random, since it has been shown that stimulating the LMAN can bias the song towards specific goals (Kao et al. 2005). On the other hand, birds that are lesioned in Area X, equivalent to the mammalian striatum, are never able to develop a structured song, displaying longer than normal syllables and less sequence stereotypy (Scharff and Nottebohm 1991). When birds are already adults, lesions to Area X slow down the song production, increasing the inter-syllable intervals, although unlike the chain grooming and the mammalian striatum, sequence production is preserved (Chen et al. 2014). That said, similar to the mammalian striatum, Area X in birds receives a dopaminergic input. Optogenetic inactivation or excitation of these dopaminergic terminals leads to online changes in the songs’ structure, suggesting that dopamine serves as a key teaching signal shaping the songs (Xiao et al. 2018). Finally, the medial portion of the avian dorsolateral anterior thalamic nucleus seems to be more implicated in song initiation, since birds lesioned in this area, although still able to produce certain calls, hardly sing and when they do, they show disrupted rhythm, possibly showing a deficit in initiating and pacing of syllable sequences (Chen et al. 2014).

In conclusion, it seems that developing chunks or motifs is a strategy that has been used by several species to deal with large amounts of serial information, with a hierarchical representation being favoured. However, in contrast to the grooming chain and other innate behaviours, in birdsong there is an added learning process, in which both sensory feedback and variability are two fundamental aspects for the development of stereotyped sequences. In terms of the neural circuits involved, while in the grooming chain the striatum along with its downstream targets are apparently enough for its sequential implementation, in birdsong, damage to the homologous striatum, Area X, preserves the ability to produce a sequentially ordered song once learnt. However, it renders birds unable to learn new structured songs. Syllable sequencing with learnt songs seems to be imposed by the cortical pathway, with the avian cortico-basal ganglia network playing a key role in the sensorimotor learning of the songs. Furthermore, the flexibility in song production suggests that as behaviours become more complex, such that they incorporate a learned component, CPGs in the brainstem and spinal cord seem to become more dependent on descending signals. In the following section, we take learning one step further and review some of the findings in scenarios in which completely new sequences of behaviours have to be acquired.

4.1 Neural substrate underlying sequence learning and performance

Just as the cortico-basal ganglia network is necessary for learning sequences of syllables in birds, it has also been found to be fundamental for sequential learning and chunking in mammals (Boyd et al. 2009; Fee and Scharff 2010; Graybiel 1998). In this section, studies involving lesions, electrophysiological recordings, pharmacological interventions and optogenetic manipulations of the basal ganglia during sequential learning tasks are reviewed.

First of all, as with the innate grooming chain, the striatum has been found to be a key region in learned action sequences, with different roles for the medial and lateral aspects. Lesions to the DLS, but not the DMS during the early stage of learning have been found to selectively disrupt action sequence acquisition, without actually producing any deficit in single action learning, suggesting that the DLS might have a very specific role in action concatenation (Geddes et al. 2018; Yin 2010). This has also been reported in humans, in which evidence also suggests that sequencing is a task of the striatum, while premotor areas of the cortex and cerebellum are more involved in other motor and cognitive aspects of the task (Janacsek et al. 2020; Wymbs et al. 2012).

Furthermore, electrophysiological recordings have revealed that as a sequence is learned a bracketing activity at the beginning and end of the sequence emerges and remains even after devaluation, suggesting that this activity pattern might represent the action sequence as a unit (Jin and Costa 2010; Jog et al. 1999; Smith and Graybiel 2013). This start/stop activity is expressed both in direct and indirect pathway MSNs, with direct pathway MSNs firing both at the beginning and end of a sequence, and indirect pathway MSNs firing preferentially at the beginning of the sequence (Jin et al. 2014). Direct pathway MSNs also display sustained firing during the complete execution of a learned action sequence, while indirect pathway MSNs have been found to display inhibited firing (Jin and Costa 2010; Jin et al. 2014). Importantly, this seems to be a specific characteristic of the DLS, since this bracketing activity is not found in the DMS (Martiros et al. 2018). It has also been reported that fast spiking interneurons (FSIs) in the striatum are important for habitual behaviour (O’Hare et al. 2017). FSI interneurons play a crucial role in regulating the output of striatal circuits by forming strong connections with MSNs, particularly those belonging to the direct pathway, thereby serving as one of the primary mediators of feedforward inhibition within the striatum (Gittits et al. 2010). In action sequences, FSIs have been found to develop specific firing patterns as an action sequence is learned, firing mostly in the middle of a learned action sequence (Martiros et al. 2018). The start/stop activity in MSNs alongside the activity of FSIs in the middle of learned action sequences is presumably at least in part facilitated by the inhibitory impact of FSIs on MSNs (Mallet et al. 2005). Importantly, these activity patterns, both in MSNs and interneurons, are only observed when the sequences are performed correctly, indicating that these different patterns, possibly encoding the action sequences as a unit, emerge in the basal ganglia as a consequence of RL (Martiros et al. 2018).

The striatum is not only important during early-stage sequence learning, as in birdsong, but also once the sequence has been well learned, with specific roles for the direct and indirect pathways. Completely ablating MSNs of the direct pathway in the dorsal striatum has been found to completely disrupt the performance of a crystallised sequence, with animals showing a return to initial performance, becoming unable to correctly complete the sequence. On the other hand, ablating indirect MSNs produces a deficit in switching between elements of the sequence (Geddes et al. 2018; Rothwell et al. 2015). Importantly, these findings have been shown not to be the result of disrupted locomotion, motivation or general switching, but rather they seem to be indicating a specific deficit in sequential performance.

With the development of optogenetics, transient activation or inactivation at particular time points is now possible, making manipulations very specific. This has allowed further differentiation of the roles of direct and indirect pathway MSNs during the performance of learned sequences. Transient optogenetic stimulation of direct pathway MSNs performed in the middle of a learned sequence facilitates behaviour by adding actions to the sequence; whereas transient stimulation of indirect pathway MSNs leads to elimination of ongoing actions, making the sequences shorter (Geddes et al. 2018). Accordingly, Tecuapetla et al. (2016) found that activating DLS indirect pathway MSNs in the middle of a well learned sequence leads animals to abort the ongoing sequence and switch to other unrelated behaviours. On the other hand, if optogenetic activation or inactivation of each pathway in the DLS is performed right before an action sequence is started, this leads to increased latency to the first element of the sequence (Tecuapetla et al. 2016). Thus, it seems that very specific activity patterns of striatal MSNs are critical for action sequence acquisition and performance.

Given that it is known that striatal MSNs are quiescent much of the time, these findings have led to the question of what is driving these striatal firing patterns. One of the main excitatory inputs to the striatum comes from the cortex (Wall et al. 2013), and it has been shown that NMDA- and AMPA-dependent plasticity at these synapses are necessary for acquisition of sequential stepping patterns on a rotarod (Dang et al. 2006; Nakamura et al. 2017; Yin et al. 2009). Thus, one proposal is that cortico-striatal plasticity is one of the mechanisms that shapes MSN activity during action sequence learning (Jin and Costa 2015; Tremblay et al. 2010). That said, studies investigating the role of cortical inputs in action sequences has yielded mixed results. During the initial learning phase, lesions to the primary and secondary motor cortices render animals unable to learn action sequences (Kawai et al. 2015). In line with these findings, Rothwell et al. (2015) have also reported that the during acquisition of a two-action sequence, the synapses between secondary motor cortex (M2) and striatum are strengthened, and that these synapses are fundamental for action sequence initiation, even after crystallisation of the learned action sequence. Finally, the bracketing firing pattern found in striatum during the execution of a learned motor sequence has also been observed in the prefrontal cortex during the performance of oculomotor sequences (Fuji and Graybiel 2003).

However, Ostlund et al. (2009) report that damage to the dorso-medial prefrontal cortex (i.e. M2) does not impair rats in learning to perform an action sequence, but it does prevent sequence-level representations to form, only noticeable in a devaluation test, not in performance itself. Furthermore, others have reported that once an action sequence has been learned, bilateral lesions to primary and secondary motor cortex have no effect in its performance (Dhawale et al. 2021; Kawai et al. 2015). Recordings made in the primary motor cortex by Martiros et al. (2018) seem to confirm this, since their results revealed that although the cortex represents the individual actions of a sequence, this is regardless of their reinforcement history. Moreover, optogenetically inhibiting these cortical neurons has no effect on the sequence performance, or in the bracketing activity of the striatum. Thus, it seems that some parts of the cortex might be necessary for learning, playing a tutor role to the striatum, but not for storing or performing a well learned action sequence (Dhawahle et al. 2021).

So, it seems that as a sequence is learned and progressively chunked, it can be executed without inputs from the cortex. This is associated with a more automatic performance, as indicated by a reduction in inter-response times (Sakai et al. 2003), and a decrease in the sensitivity to the environmental feedback resembling a characteristic of CPG networks (Dezfoulli et al. 2014; Grillner 2006). This has led to the proposal that, once a sequence is learned, its underlying neuronal representation might resemble a CPG network (Yin et al. 2009). This is in line with the proposal that CPG-like structures could be found in other parts of the central nervous system. Indeed, it has been suggested that there are similarities between the CPG network arrangements found in the spinal cord and brainstem, and neural networks found in cortex, both with similar oscillatory properties (Yuste et al. 2005). Although the CPG-like structures suggested to be in cortex would largely be more flexible than those found subcortically, Yuste et al. (2005) propose that a basic CPG-like neuronal organisation could be found throughout the CNS, in which excitatory recurrent networks are ingrained in inhibitory circuits, with neurons displaying oscillations between up states (depolarized) and down states (hyperpolarized). Interestingly, in vitro studies have shown that the striatum has neuronal ensembles that display spatiotemporal activity patterns with similar characteristics to those found in CPGs, displaying recurrent and synchronised activity patterns (Carrillo-Reid et al. 2008).

Furthermore, it is most likely that other structures besides the cortex that send projections to the striatum are also important for the organisation of sequences, and that the process of learning and performing a behavioural sequence is really distributed in several areas (Penhune and Steele 2012). The pedunculopontine and laterodorsal tegmental nuclei provide a cholinergic input to the striatum (Dautan et al. 2016), however the thalamus is the major subcortical source of innervation, with the PF having the highest density of striatally-projecting neurons of any subcortical structure (Mandelbaum et al. 2019). In a recent study, Diaz-Hernandez et al. (2018) have shown that activity in the thalamic reticular nucleus is modulated by the initiation and performance of an action sequence, and that optogenetic inhibition of these neurons delays the beginning and execution of a learned action sequence. A similar function for sequence initiation has been found in birdsong (Chen et al. 2014). This makes sense, given that motor information from subcortical-basal ganglia loops goes through a thalamic relay before reaching the striatum (McHaffie et al. 2005).

Finally, it is known that dopamine is a main modulator of cortico-striatal synaptic plasticity (Reynolds and Wickens 2002). Thus, not surprisingly, dopamine has also been implicated in chunking of action sequences. Accordingly, rats lesioned in the SNc display an abnormal temporal structure of open field behavioural sequences (Casarrubea et al. 2019), and blocking D2-type dopamine receptors in the striatum during sequence learning in monkeys disrupts motor chunking (Levesque et al. 2007). As an action sequence is learned, it has been reported that preferential dopamine release moves from the last element to the first element of the sequence (Collins et al. 2016; Wassum et al. 2012), thus possibly contributing to the bracketing activity found in the DLS. Accordingly, difficulties in chunking have been well reported in PD patients, who suffer from dopamine depletion in the striatum. PD patients are known to have difficulties initiating, performing and ending action sequences, and in particular, they show difficulties switching between two different actions (Georgiou et al. 1994; Harrington and Haaland 1991). This seems to be dopamine dependent, given that only when PD patients are off their medication are they unable to chunk actions, as evidenced by an inability to reduce the inter-response times after extended training (Tremblay et al. 2010). Furthermore, PD patients do not show an increase in striatal activity that is normally found in healthy individuals when executing automatic action sequences. Instead, they show greater cortical activity than controls, suggesting that the cortex never stops playing its tutor role (Wu et al. 2010). PD patients not only have sequencing deficits in the motor domain, but they have also been found to display disrupted cognitive sequencing (i.e. in a serial prediction task), which correlates with decreased striatal activity (Schönberger et al. 2015). Overall, these findings have led to the suggestion that PD patients are unable to shift control from cortical to subcortical areas, which might be why they cannot chunk actions together (Tremblay et al. 2010).

Taken together, experimental and clinical evidence suggests a complex role for the cortico-basal ganglia network in learned action sequences, with distinctive roles for the direct and indirect pathways and their inputs. As in chain grooming, learned action sequences also display specific striatal activity patterns; however, unlike chain grooming, which can occur without the whole cortex, the acquisition and possibly some aspects of the performance of a learned action sequence seem to require different parts of the cortex, with cortico-striatal plasticity believed to play a central role in acquisition, mediated partially by dopamine. Interestingly, this is similar to findings in birdsong, in which the cortex plays a central role, and dopamine is also believed to be fundamentally involved in the plasticity needed for learning syllable sequences. However, in spite of a recent surge in research, the mechanisms in the striatum that lead to action–action associations are still not fully understood. That said, some clues are beginning to emerge from closer study of the striatal microcircuit, especially in relation to striatal neurotransmission.

5.1 Substance P

SP is part of a family of neuropeptides referred to as tachykinins that is present both in the central and peripheral nervous systems. Its effects are mediated primarily through the NK1 receptor, a G-protein coupled receptor, but it also binds to NK2 and NK3 receptors in a lesser degree (Rupniak and Kramer 2002). In the central nervous system, NK1 receptors and SP fibres can be found in the basal ganglia, nucleus accumbens (NAc), amygdala, thalamus and hypothalamus, amongst other areas. In the basal ganglia specifically, NK1 receptors and SP fibres can be found in SNr, globus pallidus, NAc and striatum, however, cell bodies containing SP are only present in striatum and NAc (Ribeiro-da-Silva and Hökfelt 2000; Shults et al. 1984). In the striatum, SP is mainly released by direct pathway MSNs, and SP boutons mainly target other MSNs, primarily at the dendritic shafts and spines; though they also contact striatal interneurons (Bolam and Izzo 1988; Bolam et al. 1986). Accordingly, NK1 receptors can be found both postsynaptically on cholinergic and GABAergic striatal interneurons, and presynaptically on axon terminals contacting MSNs, most likely afferents from cortex or thalamus (Chen et al. 2001, 2003; Jakab and Goldman-Rakic 1996). The conditions under which SP is released are still under research. However, MSNs are known to often fire in bursts of 3–10 action potentials (Stern et al. 1997) and antidromically activating globus pallidus-projecting MSNs in a bursting pattern has a greater SP-mediated facilitatory effect on glutamatergic excitation than single spike activation (Blomeley et al. 2009). Furthermore, it has been found that high frequency but not low frequency activation of direct pathway MSNs causes the release of SP in the NAc (Francis et al. 2019), which makes sense given that high frequency stimulation is known to be required for some peptide release (Hökfelt et al. 2000). This evidence indirectly suggests that SP may be preferentially released during high frequency bursts. Given that SP is co-released with GABA, burst-induced release may be necessary to counteract the effects of co-released GABA.

SP influences neuronal activity through different pathways. First of all, although NK1 receptors have not been reported on MSNs directly, it has been demonstrated that SP can directly elicit depolarization of MSNs (Blomeley and Bracci 2008). This is believed to be mediated by presynaptic effects, since SP has been shown to facilitate the response of neighbouring MSNs to glutamatergic inputs in the rat, through presynaptic NK1 receptors (Blomeley et al. 2009). SP facilitation of glutamatergic inputs to MSNs leads on to the second way that SP influences neuronal activity. As shown in Figure 5, in some dual-cell recordings in the striatum, if an MSN is repeatedly activated before a cortical input arrives to a second connected neighbouring MSN, the response amplitude in the second MSN increases over time, suggesting some kind of long-term plasticity mediated by SP. This could mean that SP connections between MSNs might encode the order in which two neurons are repeatedly activated by cortical inputs. A similar finding has been shown in the spinal cord of lampreys. It has been reported that SP facilitates the response to descending reticulospinal inputs by potentiating glutamatergic transmission, which ultimately leads the network to a more stable and higher frequency of bursting, which behaviourally would lead to faster and “better” swimming in the lamprey (Parker et al. 1998). Whether this is a long-term effect in the spinal cord is not known.

Figure 5:

Long term plasticity mediated by SP. An example of data from a paired recording experiment in a striatal slice in which glutamatergic afferents to striatal neurons were electrically activated. Five spikes were evoked by current injection in medium spiny neuron 1 (MSN1) 100 ms before every other afferent stimulation. The amplitude of the response to glutamatergic afferent activation in a second connected medium spiny neuron (MSN2) are shown preceded (right) or not preceded (left) by spikes in the first medium spiny neuron. GABA_A, GABA_B and broad spectrum opioid receptor antagonists were present throughout. While no significant change over time was observed for responses not preceded by spikes, a significant linear trend (p < 0.001) was present for responses preceded by spikes. Blomeley and Bracci (unpublished data).

Besides directly affecting MSNs, either post or presynaptically, applying SP to the striatum has also been found to produce excitatory responses in cholinergic interneurons, increasing acetylcholine (Ach) levels in freely moving rats (Anderson et al. 1993; Aosaki and Kawaguchi 1996). Furthermore, it has also been reported that SP released by direct pathway MSNs causes a long-lasting potentiation of indirect pathway MSNs through cholinergic interneurons in the NAc, suggesting that SP might play a fundamental role in communication between the direct and indirect pathways through interneuron networks, at least in the NAc (Francis et al. 2019). Finally, several studies have found a modulatory effect of SP on dopamine. Although there is no consensus on whether SP increases or decreases dopamine levels in the striatum (Gauchy et al. 1996; Gygi et al. 1993; Kraft et al. 2001; Tremblay et al. 1992), Brimblecombe and Cragg (2015) have proposed that the mixed results concerning SP and dopamine are due to different effects of SP on the matrix and striosomes, two biochemical compartments of the striatum (Crittenden and Graybiel 2011). Their results suggest that SP upregulates dopamine only in striosomes, inhibits it at the striosome-matrix boundaries and leaves it unaltered in matrix.

These results suggest that SP’s effects on striatal output are manifold. Thus, not surprisingly, studies in which SP, NK1 agonists or antagonists have been injected, either locally or systemically, have produced numerous effects on behaviour. In terms of general locomotion, systemic injections of SP have been reported to increase behavioural output, with increased locomotion, grooming, scratching and rearing having been reported (Hall et al. 1987; Katz and Gelbart 1978; Van Wimersma Greidanus and Maigret 1988). Accordingly, blocking SP has been found to inhibit stereotypical behaviours (Duffy et al. 2002). However, others have reported that mice injected with an NK1 antagonist and mice lacking NK1 receptors actually display hyperactivity or no effect on locomotion (Kertes et al. 2010; Porter et al. 2015; Yan et al. 2010). Either way, these effects of SP on behavioural output have been suggested to be partially regulated by dopamine, since intrastriatally blocking NK1 receptors decreases the locomotion induced by D1-type dopamine receptor agonists or dopamine-related drugs like amphetamine (Duffy et al. 2002; Gonzalez-Nicolini and McGinty 2002; Krolewski et al. 2005). Furthermore, administration of an NK1 receptor antagonist significantly diminishes cocaine-induced DA release (Kraft et al. 2001).

The possibility that SP plays a role in the serial organisation of behaviour has recently been proposed by Buxton et al. (2017) using a computational modelling approach. According to Buxton et al.’s (2017) model, SP, being an excitatory neuropeptide co-released by direct pathway MSNs, contributes to striatal activity, allowing sustained selection of actions, and facilitates the response of neighbouring neurons, aiding subsequent actions to be selected in the correct order. Interestingly, this is similar to the proposal of Cacciatore et al. (2000), who suggest that the sequential coordination of leeches’ body segments could be achieved with a neural chain, in which neurons from one unit directly excite the next unit, spreading activity in an orderly fashion. In summary, this computational model suggests that directed release of SP in the striatum improves action selection performance, both in ordered and unordered sequences of actions.

The potential role of SP in the serial organisation of behaviour has received further support from experimental studies. Earlier studies have used the 5-choice serial reaction time task, a task that uses random sequences of nose pokes guided by light. Using this task, it has been found that mice lacking NK1 receptors display a greater percentage of omissions in the sequence (i.e. they fail to respond), perseverations and premature responses, and they take longer times to retrieve the reward (Porter et al. 2015; Weir et al. 2013; Yan et al. 2011). Overall, these results suggest that mice lacking NK1 receptors display disrupted action selection and attentional deficits in a sequential unordered task. Although interesting, the structure of the task (i.e. random sequences with guiding stimuli) means that the mice were not able to develop integrated sequences.

More recently, we have examined role of SP in innate and learned action sequences by examining the behavioural impact of the systemic administration of an SP antagonist. Looking first at chain grooming (innate behaviour), we found that blocking SP receptors made the highly fixed transitions inside the grooming chain and the overall grooming bout transition structure significantly more variable and less diverse than in the control group injected with saline (Favila et al. 2021). Overall, this suggest that blocking SP led to a general break down in the fluency of behavioural patterns, making them more variable and simpler. Interestingly, when rats are decerebrated at the metencephalic and mesencephalic level they are still able to produce a few complete grooming chains (Berridge 1989), although with a decreased efficiency. Therefore, it is plausible to think that the results obtained after blocking SP were due to effects at the striatal level.

Moving to learned action sequences, using a simple left-right (L-R) lever behavioural chunk, we found that blocking SP receptors had the effect of making learning a new sequence (e.g. R-L after L-R), and simultaneously extinguishing an overlearned old sequence, faster than in the control group, whereas blocking SP receptors had no effect on the stable performance of a well learned sequence (Favila et al. 2023). The results obtained with the SP antagonist appeared surprising at first, given that the grooming chain results seem to intuitively suggest that injecting the SP antagonist should have had a detrimental effect on learning a new sequence. Taking a closer look at the results, the effect of the antagonist seems to have been on the extinguishing process of the first learned sequence, which disintegrated faster when SP was blocked, allowing the rats to learn a new sequence faster. These experiments suggest that the effect of blocking SP was on the initial phase when the contingencies change, by particularly affecting the speed at which an overlearned sequence was extinguished. As a consequence, it is tempting to propose that the role of the SP could be to consolidate action sequence representation by facilitating cortico-striatal plasticity (Figure 5), which would result in the orderly spreading of activity among striatal neurons. That said, computational modelling based on our behavioural results highlights the possibility that an interaction at the striatal level between SP and dopamine affecting the reward prediction error, an important concept in reinforcement learning, could also make a contribution (Favila et al. 2023).

5.2 Enkephalin

Enkephalin is an endogenous opioid neuropeptide that acts mainly through δ and µ opioid receptors, both G-protein coupled receptors. Enkephalin is widely expressed in the nervous system, with high concentrations in the amygdala, NAc, periaqueductal grey and hypothalamus, amongst others. In the basal ganglia in particular, its highest concentration can be found at the striatum and globus pallidus, and in both structures cell bodies containing enkephalin can be observed (Ingham et al. 1991; Mallet et al. 2012; Miller and Cuatrecasas 1978). Although computational modelling suggests that enkephalin facilitates action sequence production (Buxton et al. 2017), that possibility has been poorly explored experimentally, with the exception of Horner et al. (2012), who reported that intra-striatal treatment with the µ opioid receptor agonist DAMGO in the striatum led to an increase in repetition, frequency, duration and spatial distribution of stereotypic behaviours induced by methamphetamine.