Summary
Regularities in the environment are accessible to an autonomous agents as reproducible relations between actions and perceptions and can be exploited by unsupervised learning. Our approach is based on the possibility to perform and to verify predictions about perceivable consequences of actions. It is implemented as a three-layer neural network that combines predictive perception, internal-state transitions and action selection into a loop which closes via the environment. In addition to minimizing prediction errors, the goal of network adaptation comprises also an optimization of the minimization rate such that new behaviors are favored over already learned ones, which would result in a vanishing improvement of predictability. Previously learned behaviors are reactivated or continued if triggering stimuli are available and an externally or otherwise given reward overcompensates the decay of the learning rate. In the model, behavior learning and learning behavior are brought about by the same mechanism, namely the drive to continuously experience learning success. Behavior learning comprises representation and storage of learned behaviors and finally their inhibition such that a further exploration of the environment is possible. Learning behavior, in contrast, detects the frontiers of the manifold of learned behaviors and provides estimates of the learnability of behaviors leading outwards the field of expertise. The network module has been implemented in a Khepera miniature robot. We also consider hierarchical architectures consisting of several modules in one agent as well as groups of several agents, which are controlled by such networks.
Similar content being viewed by others
References
Arkin, R. C. (1998) Behavior-based robotics. Cambridge (Mass.) MIT Press.
Burgess, N. e. a. (eds.) The hippocampal and parietal foundations of spatial cognition. Oxford University Press, 1999.
Der R., Steinmetz, U.; Pasemann, F. (1999) Homeokinesis — A new principle to back up evolution with learning. In: M. Mohammadian (ed.) Concurrent Systems Engineering Series, vol. 55, Computational Intelligence for Modelling, Control, and Automation. IOS Press, pp. 43–47.
Duckett, T.; Nehmzow, U. (1997) Knowing your place in the real world. ECAL-97 Fourth European Conference on Artificial Life University of Sussex, http://www.cogs.susx.ac.uk/ecal97/.
Franz, M. O.; Schölkopf, B.; Mallot, H. A.; Bülthoff, H. H. (1998) Learning view graphs for robot navigation. Autonomous Robots, 5: 111–125.
Glasersfeld, E. von (1987) The Construction of Knowledge. Seaside: Intersystems Publications.
Herrmann, J. M.; Pawelzik, K.; Geisel, T. (1999) Selflocalization of autonomous robots by hidden representations. Autonomous Robots 7 (1): 31–40.
Hertz, J. A.; Krogh, A.; Palmer, R. G. (1991) Introduction to the theory of neural computation. Redwood: Addison-Wesley.
Hertz, J.; Prügel-Bennett, A. (1996) Learning short synfire chains by self-organization. Network: Computation in Neural Systems 7: 357–364.
Hinton, G. E.; Dayan, P.; To, A.; Neal, R. (1995) The Helmholtz machine through time. Proceedings of the International Conference on Artificial Neural Networks, EC2∖Cie, Paris, vol. 1, 483–490.
Muller, R. (1996) A quarter of a century of place cells. Neuron 17: 813–822.
Nolfi, S.; Floreano, D. (2000) Evolutionary Robotics: The Biology, Intelligence, and Technology of Self-Organizing Machines. Cambridge, MA: MIT Press/Bradford Books.
Oore, S.; Hinton, G. E.; Dudek, G. (1997) A mobile robot that learns its place. Neural Computation, 9: 683–699.
Redlich, A. N. (1993) Redundancy reduction as a strategy for unsupervised learning. Neural Computation 5: 289–304.
Shatkay, H.; Kaelbling, L. P. (1997) Learning topological maps with weak local odometric information. In: Proceedings of the 15th International Joint Conference on Artificial Intelligence.
Sutton, R. S.; Barto, A. G. (1998) Reinforcement learning: An introduction. Cambridge (Mass.) MIT Press.
Thrun, S. (1997) Learning metric-topological maps for indoor mobile robot navigation. Artificial Intelligence, 99: 21–71.
Yamauchi, B.; Schultz, A.; Adams, W. (1998) Mobile robot exploration and map-building with continuous localization. Proceedings of the 1998 IEEE Conference on Robotics and Automation, Leuven, Belgium, pp. 3715–3720.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Herrmann, J.M. Dynamical systems for predictive control of autonomous robots. Theory Biosci. 120, 241–252 (2001). https://doi.org/10.1007/s12064-001-0021-0
Issue Date:
DOI: https://doi.org/10.1007/s12064-001-0021-0