Contextual effects in sensorimotor adaptation adhere to associative learning rules

Traditional associative learning tasks focus on the formation of associations between salient events and arbitrary stimuli that predict those events. This is exemplified in cerebellar-dependent delay eyeblink conditioning, where arbitrary cues such as a tone or light act as conditioned stimuli (CSs) that predict aversive sensations at the cornea (unconditioned stimulus [US]). Here, we ask if a similar framework could be applied to another type of cerebellar-dependent sensorimotor learning – sensorimotor adaptation. Models of sensorimotor adaptation posit that the introduction of an environmental perturbation results in an error signal that is used to update an internal model of a sensorimotor map for motor planning. Here, we take a step toward an integrative account of these two forms of cerebellar-dependent learning, examining the relevance of core concepts from associative learning for sensorimotor adaptation. Using a visuomotor adaptation reaching task, we paired movement-related feedback (US) with neutral auditory or visual contextual cues that served as CSs. Trial-by-trial changes in feedforward movement kinematics exhibited three key signatures of associative learning: differential conditioning, sensitivity to the CS-US interval, and compound conditioning. Moreover, after compound conditioning, a robust negative correlation was observed between responses to the two elemental CSs of the compound (i.e. overshadowing), consistent with the additivity principle posited by theories of associative learning. The existence of associative learning effects in sensorimotor adaptation provides a proof-of-concept for linking cerebellar-dependent learning paradigms within a common theoretical framework.


INTRODUCTION
Sensorimotor adaptation refers to the gradual adjustment of movements in response to changes 26 in the environment or body. In laboratory adaptation tasks, the introduction of perturbed sensory 27 feedback results in a sensorimotor prediction error. This error signal is used to update a model of 28 an internal sensorimotor mapping, thus ensuring that the sensorimotor system remains well-29 calibrated (Shadmehr and Krakauer, 2008;Wolpert et al., 1995;Wolpert and Ghahramani, 2000). 30 The integrity of the cerebellum is essential for this form of error-based learning (Donchin et al., by the movement segment that precedes or follows the perturbed segment of a reach (Howard et 43 al., 2012(Howard et 43 al., , 2015Sheahan et al., 2016). In these situations, the contextual cues are hypothesized 44 to be effective because the cues are incorporated into the motor plan (Howard et al., 2012(Howard et al., , 2013. 45 Less clear is the efficacy of arbitrary contextual cues, ones that are not directly related to 46 movement. In general, arbitrary cues do not appear to be effective for defining distinct motor 47 memories. For example, participants show large interference between opposing perturbations 48 that are signaled by color cues (Howard et al., 2012(Howard et al., , 2013Gandolfo et al., 1996;Forano et al., 49 2021). This interference can be reduced with multiple days of practice, although even after 10 50 days of training, residual interference remains on trials immediately following a switch in context 51 (Addou et al., 2011; see also Osu et al., 2004). 52 The ineffectiveness of arbitrary contextual cues in shaping sensorimotor adaptation is 53 surprising when one considers another popular model task of cerebellar-dependent sensorimotor 54 learningdelay eyeblink conditioning. In this form of classical conditioning, a sensory cue 55 (conditioned stimulus, CS, e.g., a tone) is repeatedly paired with an aversive event (unconditioned 56 stimulus, US, e.g., an air puff to the cornea). By itself, the aversive event naturally causes an 57 immediate unconditioned response (UR, e.g., an eye blink in response to the air puff). After just 58 a short training period with a reliable CS, organisms as diverse as turtles and humans produce 59 an adaptive conditioned response (CR, e.g., an eye blink that anticipates the aversive air puff). 60 Notably, there is little constraint on the features of the predictive cues: The conditioned response 61 can be readily acquired in response to arbitrary conditioned stimuli, such as a tone or a light flash. 62 However, there is an important temporal constraint, at least with respect to cerebellar-dependent 63 eyeblink conditioning: The cue must appear before the aversive event and the two stimuli must 64 be in close temporal proximity (Schneiderman and Gormezano, 1964;Smith et al., 1969).   During a 600-trial acquisition phase, CS+ and CS-trials were randomly interleaved. 140 Participants exhibited a marked change in movement direction during this phase, reaching an 141 8 asymptote of ~15° (Fig. 2B). The observed rapid adaptation is consistent with previous adaptation 142 studies, particularly those in which the target appears at a single fixed location (Bond and Taylor  The main analysis centered on trial-by-trial changes in heading angle. The change in 163 heading angle from trial n-1 to trial n is normally dictated by the feedback experienced in trial n-164 1. Thus, following experience with an error on CS+ trials, participants should show increased 165 adaptation (a positive change in heading angle), and following no error on CS-trials, decreased 166 adaptation (i.e., extinction). We refer to these trial-by-trial changes as the "adaptation effect", the 167 standard measure of learning in sensorimotor adaptation tasks. There was a robust adaptation 168 effect (Fig. 2C CS+ trials resulted in learning that carried over to the next trial, whereas the absence of an error 173 on a previous CS-trial resulted in a relative reversion to baseline (extinction). This is the canonical 174 signature of incremental sensorimotor adaptation. In summary, the observed effects of context on implicit sensorimotor adaptation in both 233 the acquisition and probe phases in Experiment 1 are consistent with differential conditioning 234 effects. Feedforward implicit sensorimotor adaptationhere operationalized as a type of CR -235 was differentially modulated by arbitrary sensory CSs, with a greater response to the CS+, the 236 cue that was paired with a visuomotor error. 237

238
The Rescorla-Wagner model for context-dependent sensorimotor adaptation 239 One influential model that has been used to describe contextual effects in associative learning 240 tasks is the Rescorla-Wagner model (Rescorla and Wagner, 1972 where V represents the associative strengths between the US and the CS. It is updated based on 246 the sensory prediction error (SPE) presented on trial n-1. The SPE is defined as the difference 247 between the maximum conditioning (asymptotic) level for the US (λ) and the associative strength 248 13 on the given trial. β is the learning rate parameter of the US and α represents the salience of the 249 CS. We note that the Rescorla-Wagner model is one of many computational frameworks for 250 associative learning (Courville et al., 2006;Gershman, 2015), and does not provide a mechanistic 251 account for the error-correction process itself (e.g., the fact that the motor system "knows" to 252 update movements in the direction opposite of the error). For simplicity, we assume that the sign 253 of the change in movement direction is coded in specialized neural mechanisms for reducing   where SPE is the sensory prediction errorthe difference between the predicted and the actual 275 sensory feedbackexperienced on trial n-1, A is the retention factor, and B is the learning rate. 276 The state-space model is broadly similar to the Rescorla-Wagner (e.g., both models share the 277 Markov property, produce exponential-family learning curves, etc.). However, the basic state-278 space model is unable to account for context effects: Unlike the Rescorla-Wagner model, it does 279 not include parameters that allow the updating of separate states that are associated with distinct 280 contexts. It predicts the change in behavior based on the outcome of the previous trial (i.e., the 281 adaptation effect); it cannot account for variation in heading angle due to the CS on the current 282 trial (i.e., the Pavlovian effect; Fig

16
The sum of squares residuals (SSR) and Akaike Information Criterion (AIC) difference between Rescorla-294 Wagner (RW) and the state-space (SS) models for each participant of Experiment 1.

296
An additional analysis provided further support for an associative learning account of the 297 results of Experiment 1. The Rescorla-Wagner model not only provides a framework for 298 understanding how associations can be formed with arbitrary stimuli, but it can also capture how 299 the strength of these associations is constrained by the relevance of the cues. For instance, 300 gustatory cues are much more likely to be associated with an internal bodily state (e.g., nausea) 301 than a visual cue (Garcia and Koelling, 1966). In the current study, the clamped feedback (the 302 US) is a highly relevant stimulus for reaching; as such, we should expect it to have an immediate 303 strong influence on motor behavior. In contrast, the imperative cues (the tone and light CSs) have 304 no natural relevance for reaching; as such, their contribution to the CR should initially be quite 305 modest, but gradually increase with experience ( Fig. 5A). To test this prediction, we examined 306 the time-course of the adaptation and Pavlovian effects during acquisition using a linear mixed 307 model analysis. As expected, at early stages of acquisition, the adaptation effect emerged rapidly 308 whereas the contribution of the Pavlovian effect was small. Over experience, the relative 309   reaching a mean asymptote of ~25° (Fig. 6B). The asymptotic values in this experiment were 366 higher than that seen in Experiment 1, possibly due to differences in the experimental setup (see 367  (Fig. 6D). These results suggest that extending the CS-US interval by 376 approximately 1,000 ms rendered arbitrary sensory contextual cues ineffective in separating 377 distinct motor memories. 378

Discussion
One possible concern with our method of increasing the CS-US interval is that, by using 379 a different stimulus as the imperative (the target color change), the salience of the tone and light 380 CSs may have been reduced since the task no longer required the participants to attend to these 381 cues. To address this, in Experiment 3 we used the same CSs (tone and light) and imperative 382 signal (color change of the target) as in Experiment 2, but now made the onsets of the CS and 383 imperative simultaneous on each trial (Fig. 7A). In this way, the CS-US interval is much shorter 384 ([Mean RT ± STD]: 306±57.9 ms), similar to Experiment 1. We also increased the inter-trial 385 interval in Experiment 3 to roughly match that of Experiment 2 given evidence showing that the 386 rate of associative events impacts classical conditioning (Gallistel and Gibbon, 2000).  The time course of adaptation in Experiment 3 was similar to that observed in Experiment 402 2, with a mean asymptote around ~25° (Fig. 7B) (Fig. 7D). 413 Taken together, the results of Experiments 2 and 3 support our conjecture that the efficacy 414 of arbitrary stimuli in serving as contextual cues for sensorimotor conditioning is subject to strong 415 temporal constraints, a key feature of cerebellar-dependent learning. Even when the tone and 416 light no longer required attention, they proved effective for differential conditioning when the 417 interval between these CSs and the US was short (around 300 ms), but not when the interval was 418 extended (around 1,000 ms). 419 420 Additivity principle in response to compound stimuli is observed in sensorimotor adaptation 421

23
The results of Experiments 1-3 demonstrate that implicit sensorimotor adaptation displays 422 two prominent features of associative learningthe associability of sensorimotor feedback with 423 arbitrary sensory cues, and the key role of CS-US timing in the formation of those associations. 424 In Experiment 4, we tested another core feature of associative learning, the principle of additivity 425 (Mackintosh, 1976;Pavlov, 1927;Rescorla and Wagner, 1972). This principle is based on the 426 idea that there is an associative capacity for a given US-the V term in the Rescorla-Wagner 427 equation. That is, multiple CSs can become associated with a given US, but the combined 428 associative strength is bounded by V. As a result of this capacity constraint, CSs effectively 429 compete with one another, with the associative strength split among multiple cues. 430 The classic method to examine additivity is compound conditioning, where two or more 431 stimuli are presented simultaneously to form a "compound" CS (Eq. 3). When paired with a US, 432 this compound CS will come to elicit CRs. Importantly, the associative strength of  to the first prediction of the additivity principle (Fig. 8C, 8D). 472 We note that the median trial-by-trial change in heading for the tone-alone and light-alone 473 trials is negative. This is because this measure reflects modulations in behavior around the mean 474 heading angle, and this mean decreases over time due to extinction given the absence of the 475 error feedback. This general trend should also vary for the different CSs depending on their 476 relative salience (Eq.3) (Kamin, 1967). Crucially, the additivity principle posits that there should 477 be a negative correlation between the associative strengths of competing CSs (Rescorla and 478 Wagner, 1972). That is, if a strong associative bond is formed between one CS and the US, this 479 will come at the expense of the associative strength accrued by competing CSs (Eq. 3) given the 480 26 capacity limit on associability (V). This prediction was strikingly confirmed in an analysis of the 481 heading angle changes on tone-alone and light-alone trials: Participants who were more sensitive 482 to the tone stimulus were less sensitive to the light stimulus, and vice versa (Fig. 8E, Pearson  483 correlation: r=-0.72, p<0.001, BF10=163.2). 484 As in Experiments 1-3, since the probe phase of Experiment 4 consisted of different types 485 of CSs presented randomly across trials, the behavior for a given trial should reflect not only the 486 CS on trial n but also the motor state on trial n-1 (which is itself also influenced by the CS on that The present study takes a first step toward a more parsimonious framework that might 509 account for these two canonical forms of cerebellar-dependent motor learning. We were 510 motivated by the apparent paradox concerning whether arbitrary stimuli can act as sufficient 511 contextual cues for the establishment of distinct motor memories during sensorimotor adaptation. 512 In attempting to resolve this paradox, we conceptualized adaptation as analogous to associative 513 learning. Seen from this perspective, we recognized that prior studies failing to observe adaptation 514 in response to arbitrary context cues had not considered the short, precise temporal relationship 515 between these cues and sensorimotor feedback. By modifying a visuomotor adaptation task to 516 conform to timing constraints similar to those required for delay eyeblink conditioning, we showed 517 that adaptation exhibited the hallmarks of both differential conditioning (Experiments 1 and 3) and 518 compound conditioning (Experiment 4). These results provide the first evidence, to our 519 knowledge, that pairing neutral stimuli (i.e., tones and lights) with distinct visuomotor outcomes 520 can differentially influence implicit feedforward sensorimotor adaptation in a manner consistent 521 with core principles of associative learning.  (Howard et al., 2012(Howard et al., , 2013. 534 In the current study, we found compelling contextual effects using arbitrary sensory cues. 535 Importantly, like delay eyeblink conditioning, these effects were subject to strong temporal 536 constraint on the interval between the CS and US, similar to what is observed in other forms of 537 cerebellar-dependent sensorimotor learning. Robust differential conditioning behavior was 538 observed when the CS-US interval was short (Experiments 1 and 3); increasing the interval by 539 just ~1,000 ms abolished the Pavlovian effect (Experiment 2). These results are also consistent 540 with prior evidence showing that the efficacy of movement-related cues is highly sensitive to 541 timing (Howard et al., 2012). 542 These temporal constraints are likely built into standard adaptation tasks: The target is a 543 salient stimulus that defines the task goal and movement plan, and its onset usually serves as 544 the imperative for movement initiation. RTs in these tasks are typically below 500 ms (Kim et al.,545 2018, 2019; Avraham et al., 2021). As such, a tight temporal link is established between target 546 appearance and movement, echoing (perhaps inadvertently) the CS-US temporal constraints 547 29 essential for cerebellar-dependent conditioning. Under these conditions, the target can be viewed 548 as a highly effective contextual cue. 549 More precisely, we propose that it is the movement plan itself that constitutes the primary, 550 strongest CS. While the movement plan and target appearance usually coincide spatially (i.e., 551 people aim to the target), this is not always the case. there is a 1:1 correspondence between the distance moved and the extent of the cursor 589 movement. For the web-based platform, the stimuli and movement occur in roughly orthogonal 590 planes and there is a gain factor such that the cursor movement is larger than the hand movement. 591 We expect that the laboratory setup may be much better suited for providing reliable information 592 concerning proprioception, information that may limit the extent of adaptation (Salomonczyk et  To formally relate eyeblink conditioning and adaptation, we implemented the  Wagner model, a classic associative learning model that has been widely employed in the 598 classical and operant conditioning literature (Rescorla and Wagner, 1972). The success of this 599 model to capture the general features of the current data sets should not be surprising since the 600 model was developed to account for phenomena such as differential conditioning and compound 601 conditioning. Thus, it was somewhat overdetermined that the Rescorla-Wagner model would 602 provide better fits than the standard state-space model in our study, given that the latter cannot 603 capture contextual effects. In theory, a standard state-space model could be modified such that 604 different sensory cues become associated with different independent states to allow for context-605 While the basic Rescorla-Wagner model is unable to account for higher order effects such 644 as spontaneous recovery or savings, we envision that complimenting it with more sophisticated, 645 Bayesian inference models could readily accommodate these phenomena. In fact, models that 646 33 combine associative mechanisms with some form of a Bayesian inference process that carves 647 the world into distinct contexts have successfully captured a range of complex phenomena in 648 classical conditioning and reinforcement learning (Collins and Frank, 2013;Courville et al., 2006;649 Gershman, 2015;Kruschke, 2008). We believe that our results lay the foundation for adopting a 650 similar approach to study implicit sensorimotor adaptation and perhaps a more general account 651 that captures the operation and interaction of multiple learning processes. 652 Motivated by the cerebellar literature, we limited our focus to delay eyeblink conditioning 653 in considering how associative learning mechanisms can account for contextual effects in 654 sensorimotor adaptation. We recognize that conditioning effects can be much more complex than 655 the behavioral changes that arise from the pairing of two stimuli (Rescorla, 1988). These include 656 associations between a CS and an action (Skinner, 1938) for the error correction process (e.g., the directional change of reaching movements given rotated 671 feedback, or the precise timing of an eyeblink response to a predicted air puff). Rather, it describes 672 the association between a contextual cue and a salient event (e.g., shock, reward, etc.). 673 Speculatively, an alternative idea is that sensorimotor adaptation may operate as a lookup table  674 of context-outcome associations learned by a system designed to keep the sensorimotor system 675 calibrated. A model-free conception like this could bring sensorimotor adaptation closer to 676 classical associative models of cerebellar learning and plasticity (Albus, 1971;Ito, 1984;Marr, 677 1969 For Experiments 2 and 3, we used a web-based platform that was created for conducting 699 web-based studies in sensorimotor learning (Tsay et al., 2020b). Since participants performed the 700 experiment remotely with their personal computer, the test apparatus varied across participants. 701 Based on self-report data, 71 participants used an optical mouse, 55 used a trackpad, and 2 used 702 trackball. Monitor sizes varied between 11 and 30 inches. The sizes of the stimuli were adjusted 703 to the monitor size. Below we refer to the stimuli magnitudes in the lab-based experimental setup 704 (Experiments 1 and 4). 705 At the beginning of each trial, a white circle (0.5 cm diameter) appeared at the center of 706 the black screen, indicating the start location (Fig. 1A). The participant moved the stylus to the 707 start location. Feedback of hand position (i.e., the stylus position) was indicated by a white cursor 708 (0.3 cm diameter), provided only when the hand was within 1 cm of the start location. A single 709 blue target (0.6 cm diameter) was positioned 8 cm from the start location. In most studies of 710 adaptation, the appearance of the target specifies both the movement goal (where to reach) and 711 serves as the imperative (when to reach). From a classical conditioning perspective, the target 712 should constitute a very salient CS given that its onset is temporally contingent with the US, the 713 visual feedback associated with the movement (see below). To eliminate this temporal 714 contingency, the target remained visible at the same location during the entire experiment. For 715 each participant, the target was placed at one of four locations, either 45°, 135°, 225°, and 315°, 716 and this location was counterbalanced across participants. 717 The task included the presentation of neutral (non-spatial) CS(s). We used two different 718 CSs, a tone and a light, both of which have no inherent association with the US. The tone CS was 719 36 a pure sine wave tone with a frequency of 440 Hz. The light CS was a white rectangular frame 720 [39.4 cm X 26.2 cm] that spanned the perimeter of the visual workspace. The large frame was 721 selected to provide a salient visual stimulus, but one that would not be confused with the target. 722 Depending on the specific experimental protocol and task phase, the CSs could appear 723 alone or together on a trial. In Experiments 1 and 4, the onset of the CS served as the imperative 724 signal, with the participant instructed to rapidly reach directly towards the target, slicing through 725 the target. The onset of the CS occurred following a pseudo-random and predetermined delay 726 after the hand was positioned at the start location. This was done to mitigate predictions regarding 727 the timing of the CS onset. The delay ranged between 800-1,200 ms (in steps of 100 ms), drawn 728 from a uniform distribution. While the CSs were also presented in Experiments 2 and 3, a different 729 signal was used for the movement imperative (see below). In all experiments, the CS was 730 terminated when the hand reached the radial distance to the target (Fig. 1A). In delay eyeblink 731 conditioning, learning is optimal when the US is presented 100 to 500 ms after the CS 732 (Schneiderman and Gormezano, 1964;Smith et al., 1969). To minimize the delay between the 733 onset of the CS (or the other imperative cue) and the US, the auditory message "start faster" was 734 played whenever a reaction time (RT) exceeded 400 ms. RT was operationalized as the interval 735 between imperative onset and the time required for the radial distance of the hand to exceed 1 736 cm. Pilot work showed that RTs with the variable imperative onset time were always greater than 737 100 ms; as such, we did not set a minimum RT bound. Given our objective to test the link between 738 feedforward adaptation and classical conditioning, we sought to eliminate online feedback 739 corrections. Participants were instructed to make rapid movements and the auditory message 740 "move faster" was played whenever movement time (MT) exceeded 300 ms. The end of the 741 movement was operationalized as the point where the radial distance of the hand reached 8 cm. 742 Experiments 2 and 3 were designed to test the hypothesis that the time interval between 743 the arbitrary cues and the sensorimotor feedback is critical for driving Pavlovian effects. To this 744 end, we did not use the CS onset as the imperative signal; rather, we introduced a new cue, a 745 color change of the target from gray to blue, to serve as the imperative signal (Figs. 6A, 7A). The 746 difference between the two experiments was the interval between the onsets of the CS (tone or 747 light) and imperative. In Experiment 2, the imperative was delayed with respect to CS onset, with 748 the delay drawn from a uniform distribution ranging from 800-1,200 ms (in steps of 100 ms). As 749 such, we added a mean of 1,000 ms to the CS-US interval. The delay distribution that we used 750 added variance to the CS-US interval, allowing us to make sure that, like Experiment 1, 751 participants could not anticipate the onset of the imperative. This was important since 752 predictability of the imperative timing could have decreased reaction times, and thus effectively 753 decrease the delay between the CS and the feedback. We also note that previous evidence from 754 eyeblink conditioning shows that the conditioned response is minimally affected by the variability 755 in the CS-US interval (Patterson, 1970). In Experiment 3, the CS and imperative were 756 simultaneous (similar to the timing in Experiment 1). Trial durations were made comparable 757 between Experiments 2 and 3 by setting the mean interval from the trial onset to the imperative 758 in Experiment 3 to match that of Experiment 2. 759 For the unconditioned stimulus (US), we used task-irrelevant clamped feedback 760 (Morehead et al., 2017). With clamped feedback, the radial position of the visual cursor is matched 761 to the radial position of the hand. However, the angular position of the cursor is fixed. The 762 participant thus controlled the speed and radial distance of the cursor, but not its direction. When 763 designed to produce a prediction error and elicit implicit sensorimotor adaptation, the clamp 764 followed a path that deviated from the target by 15°, with the direction, i.e., clockwise (CW) or 765 counterclockwise (CCW), counterbalanced across participants. We also included no-error trials 766 (Experiments 1-3) by presenting clamped feedback that followed a path directly to the target (0° 767 clamp; Fig. 2A). The nature of the clamp manipulation was described in detail to the participant, 768 and they were explicitly instructed strictly to ignore the feedback, aiming their reach directly toward 769 On trial n=1, before experiencing the US, all associative strengths are initialized to 0. 913

END FOR 936 937
During the probe phase in compound conditioning, the associative strengths are updated based 938 on the CS presented on each trial, either as a compound, which is simulated the same as the 939 above pseudocode for compound conditioning, or alone, and then the algorithm uses the same 940 implementation as in differential conditioning. We set US=0 in all the trials of the probe phase. 941 To illustrate the predictions of the model in Figures 2E, 2F