Skip to main content
SpringerLink
Log in

The anticipatory classifier system and genetic generalization

  • Published:
Natural Computing Aims and scope Submit manuscript

Abstract

The anticipatory classifier system (ACS)combines the learning classifier system frameworkwith the cognitive learning theory ofanticipatory behavioral control. The result is an evolutionary system thatbuilds a complete and generalized predictiveenvironmental model. Reinforcement learningtechniques are applied to form a behavioralpolicy represented in the model. After providingsome background as well as outlining the objectives of the system, we explainin detail all involved current processes. Furthermore, we analyze thedeficiency of over-specialization in the anticipatory learning process (ALP),the main learning mechanism in the ACS. Consequently, we introduce a geneticalgorithm (GA) to the ACS that is meant for generalization of over-specializedclassifiers. We show that it is possible to form a symbiosis between a directedspecialization and a genetic generalization mechanism achieving a learningmechanism that evolves a complete, accurate, and compact description of theperceived environment. Results in three different environmental settingsconfirm the usefulness of the genetic algorithm in the ACS. Finally, we discuss future research directions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Booker LB, Goldberg DE and Holland JH (1989) Classifier systems and genetic algorithms. Artificial Intelligence 40: 235–282

    Google Scholar 

  • Butz MV, Goldberg DE and Stolzmann W (2000a) Introducing a genetic generalization pressure to the anticipatory classifier system: Part 1-theoretical approach. In: Whitely D, Goldberg DE, Cantu-Paz E, Spector L, Parmee I and Beyer H-G (eds) Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2000), pp. 34–41. Morgan Kaufmann, San Francisco, CA

    Google Scholar 

  • Butz MV, Goldberg DE and Stolzmann W (2000b) Introducing a genetic generalization pressure to the anticipatory classifier system: Part 2-performance analysis. In: Whitely D, Goldberg DE, Cantu-Paz E, Spector L, Parmee I and Beyer H-G (eds) Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2000), pp. 42–49. Morgan Kaufmann, San Francisco, CA

    Google Scholar 

  • Butz MV, Goldberg DE and Stolzmann W (2000c) Investigating genetic generalization in the anticipatory classifier system. In: Schoenauer M, Deb K, Rudolph G, Yao X, Lutton E, Merely JJ and Schwefel H-P (eds) Parallel Problem Solving from Nature-PPSN VI, LNCS 1917, pp. 735–744. Springer-Verlag, Berlin

    Google Scholar 

  • Butz MV, Goldberg DE and Stolzmann W (2001) Probability-enhanced predictions in the anticipatory classifier system. In: Lanzi PL, Stolzmann Wand Wilson SW (eds) Advances in Learning Classifier Systems-Third International Workshop, IWLCS-200, LNAI 1996, pp. 38–52. Springer-Verlag, Berlin. in press

    Google Scholar 

  • Cliff D and Ross S (1994) Adding temporary memory to ZCS. Adaptive Behavior 3(2): 101–150

    Google Scholar 

  • Dorigo M and Colombetti M (1998) Robot Shaping, an Experiment in Behavior Engineering, Intelligent Robotics and Autonomous Agents. MIT press, Cambridge, MA

    Google Scholar 

  • Drescher GL (1991) Made-upMinds, a Constructivist Approach to Artificial Intelligence. MIT Press, Cambridge, MA

  • Goldberg D and Richardson J (1987) Genetic algorithms with sharing for mulitmodal function optimization. In: Grefenstette JJ (ed) Proceedings of the Second International Conference on Genetic Algorithms, pp. 41–49. Lawrence Erlbaum Associates, Hillsdale, NJ

    Google Scholar 

  • Goldberg DE (2002) The Design of Innovation: Lesssons from and for Competent Genetic Algorithms, Genetic Algorithms and Evolutionary Computation. Kluwer Academic Publishers, Boston, MA

    Google Scholar 

  • Goldberg DE and Deb K (1991) A comparative analysis of selection schemes used in genetic algorithms. Foundations of Genetic Algorithms 1: 69–93 (Also TCGA Report 90007)

    Google Scholar 

  • Grefenstette JJ (1988) Credit assignment in rule discovery systems based on genetic algorithms. Machine Learning 3: 225–245.

    Google Scholar 

  • Hoffmann J (1993) Vorhersage und Erkenntnis [Anticipation and Cognition]. Hogrefe, Goettingen, Germany Holland JH (1985) Properties of the bucket brigade algorithm. In: John J. Grefenstette (ed) Proceedings of an International Conference on Genetic Algorithms and their Applications, pp. 1–7. Carnegie-Mellon University, Pittsburgh, PA

    Google Scholar 

  • Holland JH (1990) Concerning the emergence of tag-mediated lookahead in classifier systems. In: Forrest S (ed) Emergent Computation, pp. 188–201. North-Holland

  • Holland JH (1992) Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence (Complex A).MIT press

  • Holland JH and Reitman JS (1978) Cognitive systems based on adaptive algorithms. In: Waterman DA and Hayes-Roth F (eds) Pattern Directed Inference Systems, pp. 313–329. Academic Press, New York

    Google Scholar 

  • James W (1981) (orig. 1890) The Principles of Psychology (Vol. 2). Harvard University Press, Cambridge, MA

    Google Scholar 

  • Kovacs T (1996) Evolving Optimal Populations with XCS Classifier Systems. Master's thesis, School of Computer Science, University of Birmingham, Birmingham, UK

    Google Scholar 

  • Kovacs T (1997) XCS classifier system reliably evolves accurate, complete, and minimal representations for boolean functions. In: Roy Chawdhry and Pant (eds) Soft Computing in Engineering Design and Manufacturing, pp. 59–68. Springer-Verlag, London

    Google Scholar 

  • Kuvayev L and Sutton RS (1996) Model-based reinforcement learning with an approximate, learned model. In: Proceedings of the Ninth Yale Workshop on Adaptive and Learning Systems, pp. 101–105. New Have, CT

  • LaBerge D (1995) Attentional Processing, the Brain's Art of Mindfulness. Harvard University Press, Cambridge, MA

    Google Scholar 

  • Lanzi PL (1999) An analysis of generalization in the XCS classifier system. Evolutionary Computation 7(2): 125–149

    Google Scholar 

  • Lanzi PL (2000a) Adaptive agents with reinforcemen learning and internal memory. In: Meyer JA, Berthoz A, Floreano D, Roitblat H and Wilson SW (eds) From Animals to Animats 6: Proceedings of the Sixth International Conference on Simulation of Adaptive Behavior, pp. 333–342. MIT Press, Cambridge, MA

    Google Scholar 

  • Lanzi PL (2000b) Learning Classifier Systems from a Reinforcement Learning Perspective. Technical Report 00–03, Dipartimento di Elettronica e Informazione, Politecnico di Milano

    Google Scholar 

  • Lanzi PL and Wilson SW (2000) Toward optimal classifier system performance in non-markov environments. Evoultionary Computation 8(4): 393–418

    Google Scholar 

  • Riolo RL (1991) Lookahead planning and latent learning in a classifier system. In: Meyer JA and Wilson SW (eds) From Animals to Animats: Proceedings of the First International Conference on Simulation of Adaptive Behavior, pp. 316–326. MIT Press, Cambridge, MA

    Google Scholar 

  • Rosen R (1985) Anticipatory Systems. Pergamon, Oxford, UK

    Google Scholar 

  • Rosen R (1991) Life Itself (Complexity in Ecological Systems Series). Columbia University Press, New York, USA

    Google Scholar 

  • Stolzmann W (1997) Antizipative Classifier Systeme [Anticipatory Classifier Systems]. Shaker Verlag, Aachen, Osnabrueck, Germany

    Google Scholar 

  • Stolzmann W (1998) Anticipatory classifier systems. In: Genetic Programming '98, pp. 658–664. University of Wisconsin, Morgan Kaufmann, Madison, Wisconsin

    Google Scholar 

  • Stolzmann W (2000) An introduction to anticipatory classifier systems. In: Lanzi PL, Stolzmann W and Wilson SW (eds) Learning Classifier Systems: From Foundations to Applications, LNAI 1813, pp. 175–194. Springer-Verlag, Berlin

    Google Scholar 

  • Stolzmann W and Butz MV (2000) Latent learning and action-planning in robots with anticipatory classifier systems. In: Lanzi PL, Stolzmann W and Wilson SW (eds) Learning Classifier Systems: From Foundations to Applications, LNAI 1813, pp. 301–317. Springer-Verlag, Berlin

    Google Scholar 

  • Stolzmann W, Butz MV, Hoffmann J and Goldberg DE (2000) First cognitive capabilities in the anticipatory classifier system. In: Meyer J-A, Berthoz A, Floreano D, Roitblat H and Wilson SW (eds) From Animals to Animats 6: Proceedings of the Sixth International Conference on Simulation of Adaptive Behavior, pp. 287–296. MIT press, Cambridge, MA

    Google Scholar 

  • Sutton RS (1991a) DYNA, an integrated architecture for learning, planning, and reacting. In: Working Notes of the 1991 AAAI Spring Symposium on Integrated Intelligent Architectures, pp. 151–155

  • Sutton RS (1991b) Reinforcement learning architectures for animats. In: Meyer J-A and Wilson SW (eds) From Animals to Animats: Proceedings of the First International Conference on Simulation of Adaptive Behavior, pp. 288–296. MIT Press, Cambridge, MA

    Google Scholar 

  • Sutton RS and Barto AG (1998) Reinforcement Learning: An Introduction. MIT Press, Cambridge, MA

    Google Scholar 

  • Thislethwaite D (1951) A critical review of latent learning and related experiments. Psychological Bulletin 48(2): 97–129

    Google Scholar 

  • Tolman EC (1932) Purposive Behavior in Animals and Men. Appleton, New York Tomlinson A and Bull L (2001) Symbiogenesis in learning classifier systems: Corporations, niches and persistence. Artificial Life. in press

  • Watkins CJCH and Dayan P (1992) Q-learning. Machine Learning 8(3): 272–292

    Google Scholar 

  • Whitehead SD and Ballard DH (1991) Learning to perceive and act. Machine Learning 7(1): 45–83

    Google Scholar 

  • Widrow B and Hoff M (1960) Adaptive switching circuits. Western Electronic Show and Convention 4: 96–104

    Google Scholar 

  • Wilson SW (1994) ZCS: A zeroth level classifier system. Evolutionary Computation 2(1): 1–18

    Google Scholar 

  • Wilson SW (1995) Classifier fitness based on accuracy. Evolutionary Computation 3(2): 149–175

    Google Scholar 

  • Wilson SW (1998) Generalization in the XCS classifier system. In: Koza JR, Banzhaf W, Chellapilla K, Deb K, DorigoM, Fogel DB, GarzonMH, Goldberg DE, Iba H and Riolo R (eds) Genetic Programming 1998: Proceedings of the Third Annual Conference, pp. 665–674. Morgan Kaufmann, San Francisco

    Google Scholar 

  • Wilson SW (2000) Get real! XCS with continuous-valued inputs. In: Lanzi PL, Stolzmann W and Wilson SW (eds) Learning Classifier Systems: From Foundations to Applications, LNAI 1813, pp. 209–219. Springer-Verlag, Berlin Heidelberg

    Google Scholar 

  • Wilson SW and Goldberg DE (1989) A critical review of classifier systems. In: Schaffer JD (ed) Proceedings of the Third International Conference on Genetic Algorithms, pp. 244–255. Morgan Kaufmann, San Mateo, CA

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Cognitive Psychology, University of Würzburg, Würzburg, Germany

    A. Martin V. Butz

  2. Illinois Genetic Algorithms Laboratory, University of Illinois, Urbana-Champaign, Illinois, USA

    B. David E. Goldberg

  3. DaimlerChrysler AG, Research and Technology, Berlin, Germany

    C. Wolfgang Stolzmann

Authors
  1. A. Martin V. Butz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. B. David E. Goldberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. C. Wolfgang Stolzmann
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Butz, A.M.V., Goldberg, B.D.E. & Stolzmann, C.W. The anticipatory classifier system and genetic generalization. Natural Computing 1, 427–467 (2002). https://doi.org/10.1023/A:1021330114221

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1021330114221

Search

Navigation