Abstract
One of the characteristic features of the human motor system is redundancy—i.e., the ability to achieve a given task outcome using multiple coordination patterns. However, once participants settle on using a specific coordination pattern, the process of learning to use a new alternative coordination pattern to perform the same task is still poorly understood. Here, using two experiments, we examined this process of how participants shift from one coordination pattern to another using different reinforcement schedules. Participants performed a virtual reaching task, where they moved a cursor to different targets positioned on the screen. Our goal was to make participants use a coordination pattern with greater trunk motion, and to this end, we provided reinforcement by making the cursor disappear if the trunk motion during the reach did not cross a specified threshold value. In Experiment 1, we compared two reinforcement schedules in two groups of participants—an abrupt group, where the threshold was introduced immediately at the beginning of practice; and a gradual group, where the threshold was introduced gradually with practice. Results showed that both abrupt and gradual groups were effective in shifting their coordination patterns to involve greater trunk motion, but the abrupt group showed greater retention when the reinforcement was removed. In Experiment 2, we examined the basis of this advantage in the abrupt group using two additional control groups. Results showed that the advantage of the abrupt group was because of a greater number of practice trials with the desired coordination pattern. Overall, these results show that reinforcement can be successfully used to shift coordination patterns, which has potential in the rehabilitation of movement disorders.
Similar content being viewed by others
References
Berniker M, Kording K (2008) Estimating the sources of motor errors for adaptation and generalization. Nat Neurosci 11:1454–1461. https://doi.org/10.1038/nn.2229
Bernstein NA (1967) The co-ordination and regulation of movements. Pergamon Press Ltd, Oxford
Cirstea MC, Levin MF (2000) Compensatory strategies for reaching in stroke. Brain 123:940–953. https://doi.org/10.1093/brain/123.5.940
Criscimagna-Hemminger SE, Bastian AJ, Shadmehr R (2010) Size of error affects cerebellar contributions to motor learning. J Neurophysiol 103:2275–2284. https://doi.org/10.1152/jn.00822.2009
Diedrichsen J (2007) Optimal task-dependent changes of bimanual feedback control and adaptation. Curr Biol 17:1675–1679. https://doi.org/10.1016/j.cub.2007.08.051
Dingwell JB, John J, Cusumano JP (2010) Do humans optimally exploit redundancy to control step variability in walking? PLoS Comput Biol 6:e1000856. https://doi.org/10.1371/journal.pcbi.1000856
Ferster CB, Skinner BF (1957) Schedules of reinforcement. Appleton-Century-Crofts, Inc., New York
Ganesh G, Haruno M, Kawato M, Burdet E (2010) Motor memory and local minimization of error and effort, not global optimization, determine motor behavior. J Neurophysiol 104:382–390. https://doi.org/10.1152/jn.01058.2009
Gera G, Freitas SMSF., Scholz JP (2016a) Relationship of diminished interjoint coordination after stroke to hand path consistency. Exp Brain Res 234:741–751. https://doi.org/10.1007/s00221-015-4500-9
Gera G, McGlade KE, Reisman DS, Scholz JP (2016b) Trunk muscle coordination during upward and downward reaching in stroke survivors. Mot Control 20:50–69. https://doi.org/10.1123/mc.2014-0038
Huang VS, Shadmehr R (2009) Persistence of motor memories reflects statistics of the learning event. J Neurophysiol 102:931–940. https://doi.org/10.1152/jn.00237.2009
Izawa J, Shadmehr R (2011) Learning from sensory and reward prediction errors during motor adaptation. PLoS Comput Biol 7:e1002012. https://doi.org/10.1371/journal.pcbi.1002012
Kagerer FA, Contreras-Vidal JL, Stelmach GE (1997) Adaptation to gradual as compared with sudden visuo-motor distortions. Exp Brain Res 115:557–561
Klassen J, Tong C, Flanagan JR (2005) Learning and recall of incremental kinematic and dynamic sensorimotor transformations. Exp Brain Res 164:250–259. https://doi.org/10.1007/s00221-005-2247-4
Konczak J, vander Velden H, Jaeger L (2009) Learning to play the violin: motor control by freezing, not freeing degrees of freedom. J Mot Behav 41:243–252. https://doi.org/10.3200/JMBR.41.3.243-252
Latash ML (2012) The bliss (not the problem) of motor abundance (not redundancy). Exp Brain Res 217:1–5. https://doi.org/10.1007/s00221-012-3000-4
Latash ML, Scholz JP, Schöner G (2002) Motor control strategies revealed in the structure of motor variability. Exerc Sport Sci Rev 30:26–31
Levin MF, Kleim JA, Wolf SL (2009) What do motor “recovery” and “compensation”. Mean in patients following stroke? Neurorehabil Neural Repair 23:313–319. https://doi.org/10.1177/1545968308328727
Martin JR, Zatsiorsky VM, Latash ML (2011) Multi-finger interaction during involuntary and voluntary single finger force changes. Exp Brain Res 208:423–435. https://doi.org/10.1007/s00221-010-2492-z
Mehler DMA, Reichenbach A, Klein J, Diedrichsen J (2017) Minimizing endpoint variability through reinforcement learning during reaching movements involving shoulder, elbow and wrist. PloS One 12:e0180803. https://doi.org/10.1371/journal.pone.0180803
Michaelsen SM, Dannenbaum R, Levin MF (2006) Task-specific training with trunk restraint on arm recovery in stroke: randomized control trial. Stroke 37:186–192. https://doi.org/10.1161/01.STR.0000196940.20446.c9
Müller H, Sternad D (2004) Decomposition of variability in the execution of goal-oriented tasks: three components of skill improvement. J Exp Psychol Hum Percept Perform 30:212–233. https://doi.org/10.1037/0096-1523.30.1.212
Newell KM (1991) Motor skill acquisition. Annu Rev Psychol 42:213–237. https://doi.org/10.1146/annurev.ps.42.020191.001241
Newell KM, van Emmerik REA (1989) The acquisition of coordination: preliminary analysis of learning to write. Hum Mov Sci 8:17–32. https://doi.org/10.1016/0167-9457(89)90021-3
Newell KM, Kugler PN, Van Emmerik RE, McDonald PV (1989) Search strategies and the acquisition of coordination. Adv Psychol 61:85–122
Nikooyan AA, Ahmed AA (2015) Reward feedback accelerates motor learning. J Neurophysiol 113:633–646. https://doi.org/10.1152/jn.00032.2014
Ranganathan R, Scheidt RA (2016) Organizing and reorganizing coordination patterns. In: Laczko J, Latash ML (eds) Progress in motor control. Springer International Publishing, Berlin, pp 327–349
Ranganathan R, Wang R, Gebara R, Biswas S (2017) Detecting compensatory trunk movements in stroke survivors using a wearable system. In: Proceedings of the 2017 workshop on wearable systems and applications. ACM, New York, NY, USA, pp 29–32
Rosenbaum DA, Jorgensen MJ (1992) Planning macroscopic aspects of manual control. Hum Mov Sci 11:61–69. https://doi.org/10.1016/0167-9457(92)90050-L
Shadmehr R, Smith MA, Krakauer JW (2010) Error correction, sensory prediction, and adaptation in motor control. Annu Rev Neurosci 33:89–108. https://doi.org/10.1146/annurev-neuro-060909-153135
Sutton RS, Barto AG (1998) Reinforcement leanring: an introduction. MIT Press, Cambridge
Thorndike EL (1927) The law of effect. Am J Psychol 39:212–222
Thorp EB, Kording KP, Mussa-Ivaldi FA (2016) Using noise to shape motor learning. J Neurophysiol. https://doi.org/10.1152/jn.00493.2016
Todorov E, Jordan MI (2002) Optimal feedback control as a theory of motor coordination. Nat Neurosci 5:1226–1235. https://doi.org/10.1038/nn963
van der Wel RPRD., Fleckenstein RM, Jax SA, Rosenbaum DA (2007) Hand path priming in manual obstacle avoidance: evidence for abstract spatiotemporal forms in human motor control. J Exp Psychol Hum Percept Perform 33:1117–1126. https://doi.org/10.1037/0096-1523.33.5.1117
Vereijken B, Emmerik REA van, Whiting HTA, Newell KM (1992) Free(z)ing degrees of freedom in skill acquisition. J Mot Behav 24:133–142. https://doi.org/10.1080/00222895.1992.9941608
Wolpert DM, Ghahramani Z, Flanagan JR (2001) Perspectives and problems in motor learning. Trends Cogn Sci 5:487–494. https://doi.org/10.1016/S1364-6613(00)01773-3
Wolpert DM, Diedrichsen J, Flanagan JR (2011) Principles of sensorimotor learning. Nat Rev Neurosci. https://doi.org/10.1038/nrn3112
Yang J-F, Scholz JP, Latash ML (2007) The role of kinematic redundancy in adaptation of reaching. Exp Brain Res 176:54–69. https://doi.org/10.1007/s00221-006-0602-8
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Lin, TH., Denomme, A. & Ranganathan, R. Learning alternative movement coordination patterns using reinforcement feedback. Exp Brain Res 236, 1395–1407 (2018). https://doi.org/10.1007/s00221-018-5227-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00221-018-5227-1