Abstract
A recent theory holds that the anterior cingulate cortex (ACC) uses reinforcement learning signals conveyed by the midbrain dopamine system to facilitate flexible action selection. According to this position, the impact of reward prediction error signals on ACC modulates the amplitude of a component of the event-related brain potential called the error-related negativity (ERN). The theory predicts that ERN amplitude is monotonically related to the expectedness of the event: It is larger for unexpected outcomes than for expected outcomes. However, a recent failure to confirm this prediction has called the theory into question. In the present article, we investigated this discrepancy in three trial-and-error learning experiments. All three experiments provided support for the theory, but the effect sizes were largest when an optimal response strategy could actually be learned. This observation suggests that ACC utilizes dopamine reward prediction error signals for adaptive decision making when the optimal behavior is, in fact, learnable.
Article PDF
Similar content being viewed by others
References
Akkal, D., Bioulac, B., Audin, J., & Burbaud, P. (2002). Comparison of neuronal activity in the rostral supplementary and cingulate motor areas during a task with cognitive and motor demands. European Journal of Neuroscience, 15, 887–904.
Amiez, C., Joseph, J.-P., & Procyk, E. (2005). Anterior cingulate error related activity is modulated by predicted reward. European Journal of Neuroscience, 21, 3447–3452.
Baker, R., Krigolson, O. E., & Holroyd, C. B. (2006). Examining the feedback error-related negativity using predictive stimuli. Psychophysiology, 43, S22.
Baker, T. E., & Holroyd, C. B. (in press). Which way do I go? Neural activation in response to feedback and spatial processing in a virtual T-maze. Cerebral Cortex.
Behrens, T. E. J., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. S. (2007). Learning the value of information in an uncertain world. Nature Neuroscience, 9, 1214–1221.
Botvinick, M. M., Braver, T. S., Barch, D. M., Carter, C. S., & Cohen, J. D. (2001). Conflict monitoring and cognitive control. Psychological Review, 108, 624–652.
Bush, G., Vogt, B. A., Holmes, J., Dale, A. M., Greve, D., Jenike, M. A., & Rosen, B. R. (2002). Dorsal anterior cingulate cortex: A role in reward-based decision making. Proceedings of the National Academy of Sciences, 99, 523–528.
Butterfield, B., & Mangels, J. A. (2003). Neural correlates of error detection and correction in a semantic retrieval task. Cognitive Brain Research, 17, 793–817.
Cohen, M. X., & Ranganath, C. (2007). Reinforcement learning signals predict future decisions. Journal of Neuroscience, 27, 371–378.
Compton, R. J., Carp, J., Chaddock, L., Fineman, S. L., Quandt, L. C., & Ratliff, J. B. (2007). Anxiety and error monitoring: Increased error sensitivity or altered expectations? Brain & Cognition, 64, 247–256.
Donkers, F. C. L., Nieuwenhuis, S., & van Boxtel, G. J. M. (2005). Mediofrontal negativities in the absence of responding. Cognitive Brain Research, 25, 777–787.
Donkers, F. C. L., & van Boxtel, G. J. M. (2005). Mediofrontal negativities to averted gains and losses in the slot-machine task: A further investigation. Journal of Psychophysiology, 19, 256–262.
Dunning, J. P., & Hajcak, G. (2007). Error-related negativities elicited by monetary loss and cues that predict loss. NeuroReport, 18, 1875–1878.
Emeric, E. E., Brown, J. W., Leslie, M., Pouget, P., Stuphorn, V., & Schall, J. D. (2008). Performance monitoring local field potentials in the medial frontal cortex of primates: Anterior cingulate cortex. Journal of Neurophysiology, 99, 759–772.
Eppinger, B., Kray, J., Mock, B., & Mecklinger, A. (2008). Better or worse than expected? Aging, learning, and the ERN. Neuropsychologia, 46, 521–539.
Falkenstein, M., Hohnsbein, J., Hoormann, J., & Blanke, L. (1990). Effects of errors in choice reaction tasks on the ERP under focused and divided attention. In C. Brunia, A. Gaillard, and A. Kok (Eds.), Psychophysiological Brain Research (pp. 192–195). Tilburg, The Netherlands: Tilburg University Press.
Gehring, W. J., Goss, B., Coles, M. G. H., Meyer, D. E., & Donchin, E. (1993). A neural system for error detection and compensation. Psychological Science, 4, 385–390.
Gibson, J., Krigolson, O. E., & Holroyd, C. B. (2006). Sensitivity of the feedback error-related negativity to reward probability. Psychophysiology, 43, S41-S42.
Grant, D. A., Hake, H. W., & Hornseth, J. P. (1951). Acquisition and extinction of verbal expectations in a situation analogous to conditioning. Journal of Experimental Psychology, 42, 1–5.
Gratton, G., Coles, M. G. H., & Donchin, E. (1983). A new method for off-line removal of ocular artifact. Electroencephalography & Clinical Neurophysiology, 55, 468–484.
Hajcak, G., Holroyd, C. B., Moser, J. S., & Simons, R. F. (2005). Brain potentials associated with expected and unexpected good and bad outcomes. Psychophysiology, 42, 161–170.
Hajcak, G., Moser, J. S., Holroyd, C. B., & Simons, R. F. (2007). It’s worse than you thought: The feedback negativity and violations of subjective expectancy. Psychophysiology, 44, 905–912.
Halgren, E., Boujon, C., Clarke, J., Wang, C., & Chauvel, P. (2002). Rapid distributed fronto-parieto-occipital processing stages during working memory in humans. Cerebral Cortex, 12, 710–728.
Hewig, J., Trippe, R., Hecht, H., Coles, M. G. H., Holroyd, C. B., & Miltner, W. H. R. (2007). Decision-making in Blackjack: An electrophysiological analysis. Cerebral Cortex, 17, 865–877.
Hewig, J., Trippe, R. H., Hecht, H., Coles, M. G. H., Holroyd, C. B., & Miltner, W. H. R. (2008). An electrophysiological analysis of coaching in Blackjack. Cortex, 44, 1197–1205.
Holroyd, C. B., & Coles, M. G. H. (2002). The neural basis of human error processing: Reinforcement learning, dopamine, and the error-related negativity. Psychological Review, 109, 679–709.
Holroyd, C. B., & Coles, M. G. H. (2008). Dorsal anterior cingulate cortex integrates reinforcement history to guide voluntary behavior. Cortex, 44, 548–559.
Holroyd, C. B., & Krigolson, O. E. (2007). Reward prediction error signals associated with a modified time estimation task. Psychophysiology, 44, 913–917.
Holroyd, C. B., Nieuwenhuis, S., Mars, R., & Coles, M. G. H. (2004). Anterior cingulate cortex, selection for action, and error processing. In M. Posner (Ed.), Cognitive neuroscience of attention (pp. 219–231). New York: Guilford Publishing, Inc.
Holroyd, C. B., Nieuwenhuis, S., Yeung, N., & Cohen, J. D. (2003). Errors in reward prediction are reflected in the event-related brain potential. NeuroReport, 14, 2481–2484.
Holroyd, C. B., Nieuwenhuis, S., Yeung, N., Nystrom, L., Mars, R., Coles, M. G. H., & Cohen, J. D. (2004). Dorsal anterior cingulate cortex shows fMRI response to internal and external error signals. Nature Neuroscience, 7, 497–498.
Holroyd, C. B., Pakzad-Vaezi, K. L., & Krigolson, O. E. (2008). The feedback correct-related positivity: Sensitivity of the event-related brain potential to unexpected positive feedback. Psychophysiology, 45, 688–697.
Holroyd, C. B., Yeung, N., Coles, M. G. H., & Cohen, J. D. (2005). A mechanism for error detection in speeded response time tasks. Journal of Experimental Psychology: General, 134, 163–191.
Isomura, Y., Ito, Y., Akazawa, T., Nambu, A., & Takada, M. (2003). Neural coding of “attention for action” and “response selection” in primate anterior cingulate cortex. Journal of Neuroscience, 23, 8002–8012.
Isomura, Y., & Takada, M. (2004). Neural mechanisms of versatile functions in primary anterior cingulate cortex. Reviews in the Neurosciences, 15, 279–291.
Ito, S., Stuphorn, V., Brown, J. W., & Schall, J. D. (2003). Performance monitoring by the anterior cingulate cortex during saccade countermanding. Science, 302, 120–122.
Kennerley, S. W., Walton, M. E., Behrens, T. E. J., Buckley, M. J., & Rushworth, M. F. S. (2006). Optimal decision making and the anterior cingulate cortex. Nature Neuroscience, 9, 940–947.
Klein, T. A., Neumann, J., Reuter, M., Hennig, J., von Cramon, D. Y., & Ullsperger, M. (2007) Genetically determined differences in learning from errors. Science, 318, 1642–1645.
Krigolson, O. E., & Holroyd, C. B. (2007). Predictive information and error processing: The role of medial-frontal cortex during motor control. Psychophysiology, 44, 586–595.
Krigolson, O. E., Mathewson, K., Baker, T. E., Baker, R., & Holroyd, C. B. (2006). The role of medial-frontal cortex in sequence learning. Psychophysiology, 43, S54-S55.
Krigolson, O. E., Pierce, L. J., Holroyd, C. B., & Tanaka, J. W. (in press). Learning to become an expert: Reinforcement learning and the acquisition of perceptual expertise. Journal of Cognitive Neuroscience.
Larson, M. J., Kelly, K. G., Stigge-Kaufman, D. A., Schmalfuss, I. M., & Perlstein, W. M. (2007). Reward context sensitivity impairment following severe TBI: An event-related brain potential experiment. Journal of the International Neuropsychological Society, 13, 615–625.
Lee, K., Krigolson, O. E., & Holroyd, C. B. (2006). Modulation of fERN amplitude by reward expectancy. Psychophysiology, 43, S57-S58.
Mars, R. B., Coles, M. G. H., Grol, M. J., Holroyd, C. B., Nieuwenhuis, S., Hulstijn, W., & Toni, I. (2005). Neural dynamics of error processing in medial frontal cortex. NeuroImage, 28, 1007–1013.
Matsumoto, M., Matsumoto, K., Abe, H., & Tanaka, K. (2007). Medial prefrontal cell activity signaling prediction errors of action values. Nature Neuroscience, 10, 647–656.
Matsumoto, K., Suzuki, W., & Tanaka, K. (2003). Neuronal correlates of goal-based motor selection in the prefrontal cortex. Science, 301, 229–232.
Matsumoto, K., & Tanaka, K. (2004). The role of medial prefrontal cortex in achieving goals. Current Opinion in Neurobiology, 14, 178–185.
Michelet, T., Bioulac, B., Guehl, D., Escola, L., & Burbaud, P. (2007). Impact of commitment on performance evaluation in the rostral cingulate motor area. Journal of Neuroscience, 27, 7482–7489.
Miltner, W. H. R., Braun, C. H., & Coles, M. G. H. (1997). Event-related brain potentials following incorrect feedback in a time-estimation task: Evidence for a “generic” neural system for error detection. Journal of Cognitive Neuroscience, 9, 788–798.
Montague, P. R., Hyman, S. E., & Cohen, J. D. (2004). Computational roles for dopamine in behavioral control. Nature, 431, 760–767.
Morris, S. E., Heerey, E. A., Gold, J. M., & Holroyd, C. B. (2008). Learning-related changes in brain activity following errors and performance feedback in schizophrenia. Schizophrenia Research, 99, 274–285.
Nakamura, K., Roesch, M. R., & Olson, C. R. (2005). Neuronal activity in macaque SEF and ACC during performance of tasks involving conflict. Journal of Neurophysiology, 93, 884–908.
Nieuwenhuis, S., Holroyd, C. B., Mol, N., & Coles, M. G. H. (2004). Reinforcement-related brain potentials from medial frontal cortex: Origins and functional significance. Neuroscience & Biobehavioral Reviews, 28, 441–448.
Nieuwenhuis, S., Nielen, M. M., Mol, N., Hajcak, G., & Veltman, D. J. (2005). Performance monitoring in obsessive-compulsive disorder. Psychiatry Research, 134, 111–122.
Nieuwenhuis, S., Ridderinkhof, K. R., Talsma, D., Coles, M. G. H., Holroyd, C. B., Kok, A., & Van der Molen, M. W. (2002). A computational account of altered error processing in older age: Dopamine and the error-related negativity. Cognitive, Affective, & Behavioral Neuroscience, 2, 19–36.
Nieuwenhuis, S., Slagter, H. A., von Geusau, N. J. A., Heslenfeld, D., & Holroyd, C. B. (2005). Knowing good from bad: Differential activation of human cortical areas by positive and negative outcomes. European Journal of Neuroscience, 21, 3161–3168.
Niki, H., & Watanabe, M. (1979). Prefrontal and cingulate unit activity during timing behavior in the monkey. Brain Research, 171, 213–224.
Nishijo, H., Yamamoto, Y., Ono, T., Uwano, T., Yamashita, J., & Yamashima, T. (1997). Single neuron responses in the monkey anterior cingulate cortex during visual discrimination. Neuroscience Letters, 227, 79–82.
Paulus, M. P., Hozack, N., Frank, L., & Brown, G. G. (2002). Error rate and outcome predictability affect neural activation in prefrontal cortex and anterior cingulate during decision-making. NeuroImage, 15, 836–846.
Potts, G. F., Martin, L. E., Burton, P., & Montague, P. R. (2006). When things are better or worse than expected: The medial frontal cortex and the allocation of processing resources. Journal of Cognitive Neuroscience, 18, 1112–1119.
Procyk, E., Tanaka, Y. L., & Joseph, J. P. (2000). Anterior cingulate activity during routine and nonroutine sequential behaviors in macaques. Nature Neuroscience, 3, 502–508.
Quilodran, R., Rothe, M., & Procyk, E. (2008). Behavioral shifts and action valuation in the anterior cingulate cortex. Neuron, 57, 314–325.
Ridderinkhof, K. R., Ullsperger, M., Crone, E. A., & Nieuwenhuis, S. (2004). The role of medial frontal cortex in cognitive control. Science, 306, 443–447.
Roesch, M. R., Calu, D. J., & Schoenbaum, G. (2007). Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards. Nature Neuroscience, 10, 1615–1624.
Rushworth, M. F. S., Walton, M. E., Kennerley, S. W, & Bannerman, D. M. (2004). Action sets and decisions in the medial frontal cortex. Trends in Cognitive Sciences, 8, 410–417.
Schultz, W. (2002). Getting formal with dopamine and reward. Neuron, 36, 241–263.
Shidara, M., & Richmond, B. J. (2002). Anterior cingulate: Single neuronal signals related to degree of reward expectancy. Science, 31, 1709–1711.
Shima, K., & Tanji, J. (1998). Role for cingulate motor area cells in voluntary movement selection based on reward. Science, 282, 1335–1338.
Ullsperger, M., & von Cramon, Y. (2003). Error monitoring using external feedback: Specific roles of the habenular complex, the reward system, and the cingulate motor area revealed by functional magnetic resonance imaging. Journal of Neuroscience, 23, 4308–4314.
Van Veen, V., Holroyd, C. B., Cohen, J. D., Stenger, A., & Carter, C. (2004). Errors without conflict: Implications of performance monitoring theories for anterior cingulate cortex. Brain & Cognition, 56, 267–276.
Wang, C., Ulbert, I., Schomer, D. L., Marinkovic, K., & Halgren, E. (2005). Responses of human anterior cingulate cortex microdomains to error detection, conflict monitoring, stimulus-response mapping, familiarity and orienting. Journal of Neuroscience, 25, 604–613.
Williams, Z. M., Bush, G., Rauch, S. L., Cosgrove, G. R., & Eskandar, E. N. (2004). Human anterior cingulate neurons and the integration of monetary reward with motor responses. Nature Neuroscience, 7, 1370–1375.
Yasuda, A., Sato, A., Miyawaki, K., Kumano, H., & Kuboki, T. (2004). Error-related negativity reflects detection of negative reward prediction error. NeuroReport, 15, 2561–2565.
Yeung, N., Holroyd, C. B., & Cohen, J. D. (2005). ERP correlates of feedback and reward processing in the presence and absence of response choice. Cerebral Cortex, 15, 535–544.
Author information
Authors and Affiliations
Corresponding author
Additional information
This research was supported in part by Natural Sciences and Engineering Research Council Discovery Grant RGPIN 312409-05, the Michael Smith Foundation for Health Research Fellowship ST-SGS-349 (05-1) CLIN, the University of Victoria Fellowship Program, and a Petch Research Scholarship.
Rights and permissions
About this article
Cite this article
Holroyd, C.B., Krigolson, O.E., Baker, R. et al. When is an error not a prediction error? An electrophysiological investigation. Cognitive, Affective, & Behavioral Neuroscience 9, 59–70 (2009). https://doi.org/10.3758/CABN.9.1.59
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.3758/CABN.9.1.59