Abstract
This paper juxtaposes the probability matching paradox of decision theory and the magnitude of reinforcement problem of animal learning theory to show that simple classifier system bidding structures are unable to match the range of behaviors required in the deterministic and probabilistic problems faced by real cognitive systems. The inclusion of a variance-sensitive bidding (VSB) mechanism is suggested, analyzed, and simulated to enable good bidding performance over a wide range of nonstationary probabilistic and deterministic environments.
Article PDF
Similar content being viewed by others
References
Flood, M.M. (1954). Environmental non-stationarity in a sequential decision-making experiment. In R.M.Thrall, C.H.Coombs, & R.L.Davis (Eds.), Decision processes. New York: Wiley.
Goldberg, D.E. (1983). Computer-aided gas pipeline operation using genetic algorithms and rule learning. Doctoral dissertation, University of Michigan. Dissertation Abstracts International, 44, 3174B. (University Microfilms No. 8402282).
Goldberg, D.E. (1989). Genetic algorithms in search, optimization, and machine learning. Reading, MA: Addison-Wesley.
Goldberg, D.E., & Richardson, J.J. (1987). Genetic algorithms with sharing for multimodal function optimization. Genetic algorithms and their applications: Proceedings of the Second International Conference on Genetic Algorithms. (pp. 41–49).
Goodnow, J.J. (1955). Determinants of choice-distribution in two-choice situations. American Journal of Psychology, 68, 106–116.
Holland, J.H. (1971). Processing and processors for schemata. In E.L.Jacks (Ed.), Associative information processing. New York: American Elsevier.
Holland, J.H. (1973). Genetic algorithms and the optimal allocation of trials. SIAM Journal of Computing, 2, 88–105.
Holland, J.H. (1975). Adaptation in natural and artificial systems. Ann Arbor, MI: University of Michigan Press.
Holland, J.H., & Reitman, J.S. (1978). Cognitive systems based on adaptive algorithms. In D.A.Waterman & F.Hayes-Roth (Eds.), Pattern directed inference systems. New York: Academic Press.
Lee, W. (1971). Decision theory and human behavior. New York: John Wiley & Sons.
McCracken, J., Osterhout, C., & Voss, J.F. (1962). Effects of instruction in probability learning. Journal of Experimental Psychology, 64, 267–271.
Mackintosh, N.J. (1974). The psychology of animal learning. London: Academic Press.
Siegel, S. (1959). Theoretical models of choice and strategy behavior: Stable state behavior in the two-choice uncertain outcome situation. Psychometrika, 24, 303–316.
Simon, H.A. (1956). A comparison of game theory and learning theory. Psychometrika, 21, 267–272.
Wilson, S.W. (1987). Classifier systems and the Animat problem. Machine Learning, 2, 199–228.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Goldberg, D.E. Probability matching, the magnitude of reinforcement, and classifier system bidding. Mach Learn 5, 407–425 (1990). https://doi.org/10.1007/BF00116878
Issue Date:
DOI: https://doi.org/10.1007/BF00116878