Abstract
Recent human motor learning studies revealed that punishment seemingly accelerated motor learning but reward enhanced consolidation of motor memory. It is not evident how intrinsic properties of reward and punishment modulate the potentially dissociable effects of reward and punishment on motor learning and motor memory. It is also not clear what causes the dissociation of the effects of reward and punishment. By manipulating probability of distribution, a critical property of reward and punishment, the present study demonstrated that probability had distinct modulation on the effects of reward and punishment in adapting to a sudden visual rotation and consolidation of the adaptation memory. Specifically, two probabilities of monetary reward and punishment distribution, 50 and 100%, were applied during young adult participants adapting to a sudden visual rotation. Punishment and reward showed distinct effects on motor adaptation and motor memory. The group that received punishments in 100% of the adaptation trials adapted significantly faster than the other three groups, but the group that received rewards in 100% of the adaptation trials showed marked savings in re-adapting to the same rotation. In addition, the group that received punishments in 50% of the adaptation trials that were randomly selected also had savings in re-adapting to the same rotation. Sensitivity to sensory prediction error or difference in explicit process induced by reward and punishment may likely contribute to the distinct effects of reward and punishment.
Similar content being viewed by others
References
Abe M, Schambra H, Wassermann EM, Luckenbaugh D, Schweighofer N, Cohen LG (2011) Reward improves long-term retention of a motor memory through induction of offline memory gains. Curr Biol 21(7):557–562
Au Yeung ST, Colagiuri B, Lovibond PF, Colloca L (2014) Partial reinforcement, extinction, and placebo analgesia. Pain 155(6):1110–1117
Balliet D, Mulder LB, Van Lange PA (2011) Reward, punishment, and cooperation: a meta-analysis. Psychol Bull 137(4):594–615
Barberis NC (2013) Thirty years of prospect theory in economics: a review and assessment. J Econ Perspect 27(1):173–196
Brown RM, Robertson EM (2007) Inducing motor skill improvements with a declarative task. Nat Neurosci 10(2):148
Buch ER, Young S, Contreras-Vidal JL (2003) Visuomotor Adaptation in Normal Aging. Learn Mem 10(1):55–63
Carnagey NL, Anderson CA (2005) The effects of reward and punishment in violent video games on aggressive affect, cognition, and behavior. Psychol Sci 16(11):882–889
Chen X, Haith AM, Mohr K, Galea JM (2017) Predicting explorative motor learning using decision-making and motor noise. PLoS Comput Biol 13(4):e1005503
Crockett MJ, Clark L, Robbins TW (2009) Reconciling the role of serotonin in behavioral inhibition and aversion: acute tryptophan depletion abolishes punishment-induced inhibition in humans. J Neurosci 29(38):11993–11999
Dayan E, Averbeck BB, Richmond BJ, Cohen LG (2014) Stochastic reinforcement benefits skill acquisition. Learn Mem 21(3):140–142
Fiorillo C, Tobler P, Schultz W (2003) Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299(5614):1898–1902
Galea JM, Mallia E, Rothwell J, Diedrichsen J (2015) The dissociable effects of punishment and reward on motor learning. Nat Neurosci 18:597–602
Haith AM, Huberdeau DM, Krakauer JW (2015) The influence of movement preparation time on the expression of visuomotor learning and savings. J Neurosci 35(13):5109–5117
Haselgrove M, Aydin A, Pearce JM (2004) A partial reinforcement extinction effect despite equal rates of reinforcement during pavlovian conditioning. J Exp Psychol Anim Behav Process 30(3):240–250
Huang VS, Haith A, Mazzoni P, Krakauer JW (2011) Rethinking motor learning and savings in adaptation paradigms: model-free memory for successful actions combines with internal models. Neuron 70(4):787–801
Imhof LA, Fudenberg D, Nowak MA (2007) Tit-for-tat or Win-stay, Lose-shift? Journal of Theoretical Biology 247(3):574–580
Izawa J, Shadmehr R (2011) Learning from sensory and reward prediction errors during motor adaptation. PLoS Comput Biol 7(3):e1002012
Jenkins WO, Stanley JC (1950) Partial reinforcement: a review and critique. Psychol Bull 47(3):193–234
Leow L-A, Loftus A, Hammond G (2012) Impaired savings despite intact initial learning of motor adaptation in Parkinson’s disease. Exp Brain Res 218(2):295–304
Mazzoni P, Krakauer JW (2006) An implicit plan overrides an explicit strategy during visuomotor adaptation. J Neurosci 26(14):3642–3645
McDougle S, Bond K, Taylor J (2015) Explicit and implicit processes constitute the fast and slow processes of sensorimotor learning. J Neurosci 35(26):9568–9579
Morehead JR, Qasim SE, Crossley MJ, Ivry R (2015) Savings upon re-aiming in visuomotor adaptation. J Neurosci 35(42):14386–14396
Moulton EA, Elman I, Pendse G, Schmahmann J, Becerra L, Borsook D (2011) Aversion-related circuitry in the cerebellum: responses to noxious heat and unpleasant images. J Neurosci 31(10):3795–3804
Nakatani Y, Matsumoto Y, Mori Y, Hirashima D, Nishino H, Arikawa K, Mizunami M (2009) Why the carrot is more effective than the stick: different dynamics of punishment memory and reward memory and its possible biological basis. Neurobiol Learn Mem 92(3):370–380
Nikooyan AA, Ahmed AA (2015) Reward feedback accelerates motor learning. J Neurophysiol 113(2):633–646
Oldfield R (1971) The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia 9(1):97–113
Padilla A (1967) A few acquisition trials: effects of magnitude and percent reward. Psychon Sci 9(5):241–242
Prados J, Sansa J, Artigas A (2008) Partial reinforcement effects on learning and extinction of place preferences in the water maze. Learn Behav 36(4):311–318
Sacchetti B, Scelfo B, Strata P (2009) Cerebellum and emotional behavior. Neuroscience 162(3):756–762
Schlerf J, Ivry RB, Diedrichsen J (2012) Encoding of sensory prediction errors in the human cerebellum. J Neurosci 32(14):4913–4922
Shadmehr R, Smith MA, Krakauer JW (2010) Error correction, sensory prediction, and adaptation in motor control. Annu Rev Neurosci 33:89–108
Shizgal P (1997) Neural basis of utility estimation. Curr Opin Neurobiol 7(2):198–208
Song S (2009) Consciousness and the consolidation of motor learning. Behav Brain Res 196(2):180–186
Steel A, Silson EH, Stagg CJ, Baker CI (2016) The impact of reward and punishment on skill learning depends on task demands. Sci Rep 6:36056
Thorndike EL (1933) A proof of the law of effect. Science 77:173–175
Tobler P, Fiorillo C, Schultz W (2005) Adaptive coding of reward value by dopamine neurons. Science 307:1642–1645
Tseng Y-W, Diedrichsen J, Krakauer JW, Shadmehr R, Bastian AJ (2007) Sensory prediction errors drive cerebellum-dependent adaptation of reaching. J Neurophysiol 98(1):54–62
van der Kooij K, Overvliet K (2016) Rewarding imperfect motor performance reduces adaptive changes. Exp Brain Res. doi:10.1007/s00221-015-4540-1
Vaswani PA, Shadmehr R (2013) Decay of motor memories in the absence of error. J Neurosci 33(18):7700–7709
Wächter T, Lungu OV, Liu T, Willingham DT, Ashe J (2009) Differential effect of reward and punishment on procedural learning. J Neurosci 29(2):436–443
Worthy D, Hawthorne M, Otto A (2013) Heterogeneity of strategy use in the Iowa gambling task: A comparison of win-stay/lose-shift and reinforcement learning models. Psychon Bull Rev 20(2):364–371
Zarahn E, Weston GD, Liang J, Mazzoni P, Krakauer JW (2008) Explaining savings for visuomotor adaptation: linear time-invariant state-space models are not sufficient. J Neurophysiol 100(5):2537–2548
Acknowledgements
We thank a reviewer for constructive feedback on an earlier version of the manuscript.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no financial disclosures or potential conflicts of interests.
Rights and permissions
About this article
Cite this article
Song, Y., Smiley-Oyen, A.L. Probability differently modulating the effects of reward and punishment on visuomotor adaptation. Exp Brain Res 235, 3605–3618 (2017). https://doi.org/10.1007/s00221-017-5082-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00221-017-5082-5