Abstract
Most of the straight-forward learning approaches in cooperative robotics imply for each learning robot a state space growth exponential in the number of team members. To remedy the exponentially large state space, we propose to investigate a less demanding cooperation mechanism—i.e., various levels of awareness—instead of communication. We define awareness as the perception of other robots locations and actions. We recognize four different levels (or degrees) of awareness which imply different amounts of additional information and therefore have different impacts on the search space size (Θ(0), Θ(1), Θ(N), o(N),1 where N is the number of robots in the team). There are trivial arguments in favor of avoiding binding the increase of the search space size to the number of team members. We advocate that, by studying the maximum number of neighbor robots in the application context, it is possible to tune the parameters associated with a Θ(1) increase of the search space size and allow good learning performance. We use the cooperative multi-robot observation of multiple moving targets (CMOMMT) application to illustrate our method. We verify that awareness allows cooperation, that cooperation shows better performance than a purely collective behavior and that learned cooperation shows better results than learned collective behavior.
Similar content being viewed by others
References
Aha, D. (Ed.) 1997. Lazy Learning,Kluwer Academic Publishers.
Balch, T. and Arkin, R. 1994. Communication in reactive multiagent robotic systems. Autonomous Robots, 1:27–52.
Braitenberg,V. 1984.Vehicles: Experiments in Synthetic Psychology, MIT Press.
Brooks, R. 1991. Intelligence without reason. In IJCAI'91, Sydney.
Cao, Y.U., Fukuaga, A., and Kahng, A. 1997. Cooperative mobile robotics: Antecedent and directions. Autonomous Robots, 4:7–27.
Dorigo, M. (Guest Editor) 1996. Introduction to the special issue on learning autonomous robots.IEEE Trans.on Systems, Man and Cybernetics—Part B, 26(3):361–364.
Heemskerk, J. and Sharkey, N. 1996. Learning subsumptions for an utonomous robot. In IEE Seminar on Self-Learning Robot, Digest No: 96=026, Savoy Place, London, 12, England.
Kaelbling, L., Littman, M., and Moore, A. 1996. Reinforcement learning: A survey. Journal of Artificial Intelligence Research, 4:237–285.
Kretchmar, R.M. and Anderson, C.W. 1997. Comparison of CMACs and radial basis functions for local function approximators in reinforcement learning. In Proc.of ICNN'97, Houston, Texas, USA.
Kube, R. and Zhang, H. 1994. Collective robotics: From social insects to robots. Adaptive Behavior, 2:189–218.
Lin, L.J. 1992. Self-improving reactive agents based on reinforcement learning, planning and teaching. Machine Learning, 8:293–321.
Mataric, M.J. 1997a. Reinforcement learning in multi-robot domain. Autonomous Robots, 4:73–83.
Mataric, M.J. 1997b. Learning social behavior. Robotics and Autonomous Systems, 20:191–204.
Ono, N. and Fukumoto, K. 1997. A modular approach to multi-agent reinforcement learning. In Distributed Artificial Intelligence Meets Machine Learning: Learning in Multi-Agents Environments, G. Weiss (Ed.), Springer-Verlag, Lecture Notes in Artificial Intelligence, Vol. 1221, pp. 25–39.
Parker, L.E. 1995. The effect of action recognition and robot awareness in cooperative robotic teams. In Proc.IROS 95, Pittsburgh, PA.
Parker, L.E. 1997. Cooperative motion control for multi-target observation. Proc.IROS 97, Grenoble, France.
Premvuti, S. and Yuta, S. 1996. Consideration on conceptual design of inter robot communications network for mobile robot system. In Distributed Autonomous Robotic Systems 2 (DARS 96), H. Asama, T. Fukuda, T. Arai, and I. Endo (Eds.), Springer-Verlag.
Santos, J.M. and Touzet, C. 1999. Exploration tuned reinforcement function. Neurocomputing, 28(1–3):93–105.
Sehad, S. and Touzet, C. 1994. Reinforcement learning and neural reinforcement learning. In Proc.of ESANN 94, russels.
Sheppard, J.W. and Salzberg, S.L. 1997. A teaching strategy for memory-based control. In Lazy Learning, D. Aha (Ed.),Kluwer Academic Publishers, pp. 343–370.
Sutton, R. and Barto, A. 1998. Reinforcement Learning, MIT Press.
Touzet, C. 1997. Neural reinforcement learning for behaviour synthesis, Special issue on Learning Robot: The NewWave, N. Sharkey (guest Ed.), Robotics and Autonomous Systems, 22(3–4):251–281.
Touzet, C. 1998. Bias incorporation in robot learning, submitted for publication.
Watkins, C.J.C.H. 1989. Learning from delayed rewards, Ph.D. Thesis, King's College, Cambridge, England.
Whithehead, S., Karlsson, J., and Tenenberg, J. 1993. Learning multiple goal behavior via task decomposition and dynamic policy merging. In Robot Learning, J. Connell and S. Mahadevan (Eds.), Kluwer Academic Publishers.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Touzet, C.F. Robot Awareness in Cooperative Mobile Robot Learning. Autonomous Robots 8, 87–97 (2000). https://doi.org/10.1023/A:1008945119734
Issue Date:
DOI: https://doi.org/10.1023/A:1008945119734