Abstract
In this paper, the calibration of a framework based in Multi-agent Reinforcement Learning (RL) for generating motion simulations of pedestrian groups is presented. The framework sets a group of autonomous embodied agents that learn to control individually its instant velocity vector in scenarios with collisions and friction forces. The result of the process is a different learned motion controller for each agent. The calibration of both, the physical properties involved in the motion of our embodied agents and the corresponding dynamics, is an important issue for a realistic simulation. The physics engine used has been calibrated with values taken from real pedestrian dynamics. Two experiments have been carried out for testing this approach. The results of the experiments are compared with databases of real pedestrians in similar scenarios. As a comparison tool, the diagram of speed versus density, known as fundamental diagram in the literature, is used.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Agre, P., Chapman, D.: Pengi: An implementation of a theory of activity. In: 6th Nat. Conf. on Artificial Intelligence, pp. 268–272. Morgan Kaufmann (1987)
Bierlaire, M., Robin, T.: Pedestrians choices. In: Timmermans, H. (ed.) Pedestrian Behavior, pp. 1–26. Emerald (2009)
Claus, C., Boutilier, C.: The dynamics of reinforcement learning in cooperative multiagent systems. In: Proceedings of the Fifteenth National Conference on Artificial Intelligence, pp. 746–752. AAAI Press (1998)
Curtis, S., Lin, M., Manocha, D.: Walk This Way: A Lightweight, Data-Driven Walking Synthesis Algorithm. In: Allbeck, J.M., Faloutsos, P. (eds.) MIG 2011. LNCS, vol. 7060, pp. 400–411. Springer, Heidelberg (2011)
García, J., López-Bueno, I., Fernández, F., Borrajo, D.: A Comparative Study of Discretization Approaches for State Space Generalization in the Keepaway Soccer Task. In: Reinforcement Learning: Algorithms, Implementations and Aplications. Nova Science Publishers (2010)
Helbing, D., Farkas, I., Vicsek, T.: Simulating dynamical features of escape panic. Nature 407, 487 (2000)
Helbing, D., Johansson, A.: Pedestrian, Crowd and Evacuation Dynamics. In: Encyc. of Complex. and Systems Science, Part 16, pp. 6476–6495. Springer (2009)
Helbing, D., Johansson, A., Al-Abideen, H.Z.: Dynamics of crowd disasters: An empirical study. Phys. Rev. E 75, 046109 (2007)
Helbing, D., Molnár, P., Farkas, I., Bolay, K.: Self-organizing pedestrian movement. Environment and Planning. Part B: Planning and Design 28, 361–383 (2001)
Herman, I.P.: Physics of the Human Body. Springer (2007)
Hoogerndoorn, S.P., Daamen, W.: A novel calibration approach of microscopic pedestrian models. In: Pedestrian Behavior, pp. 195–214. Emerald (2009)
Littman, M.L.: Markov games as a framework for multi-agent reinforcement learning. In: Proceedings of the Eleventh International Conference on Machine Learning, New Brunswick, NJ, pp. 157–163 (2005)
Martinez-Gil, F., Lozano, M., Fernández, F.: Multi-agent Reinforcement Learning for Simulating Pedestrian Navigation. In: Vrancx, P., Knudson, M., Grześ, M. (eds.) ALA 2011. LNCS, vol. 7113, pp. 54–69. Springer, Heidelberg (2012)
Masen, M.: A systems based experimental approach to tactile friction. Journal of the Mechanical Behavior of Biomedical Materials, 1620–1626 (2011)
Mataric, M.J.: Learning to behave socially. In: From Animals to Animats: Int. Conf. on Simulation of Adaptive Behavior, pp. 453–462. MIT Press (1994)
Mori, M., Tsukaguchi, H.: A new method for evaluation of level of service in pedestrian facilities. Transportation Research Part A 21(3), 223–234 (1987)
Seyfried, A., Steffen, B., Klingsch, W., Lippert, T., Boltes, M.: Steps toward the fundamental diagram. Empirical results and modelling. In: Pedestrian and Evacuation Dynamics 2005, Part 3, pp. 377–390. Springer (2007)
Pelechano, N., Allbeck, J., Badler, N.: Controlling individual agents in high-density crowd simulation. In: Proc. ACM/SIGGRAPH/Eurographycs Symp. Computer Animation, pp. 99–108 (2007)
Robin, T., Antonioni, G., Bierlaire, M., Cruz, J.: Specification, estimation and validation of a pedestrian walking behavior model. Transportation Research 43, 36–56 (2009)
Schadschneider, A., Seyfried, A.: Validation of ca models of pedestrian dynamics with fundamental diagrams. Cybernetics and Systems 5(40), 367–389 (2009)
Sen, S., Sekaran, M.: Multiagent Coordination with Learning Classifier Systems. In: Weiss, G., Sen, S. (eds.) IJCAI-WS 1995. LNCS, vol. 1042, pp. 218–233. Springer, Heidelberg (1996)
Shao, W., Terzopoulos, D.: Autonomous pedestrians. In: Proceedings of the 2005 ACM SIGGRAPH Symposium on Computer Animation, pp. 19–28. ACM Press, New York (2005)
Steiner, A., Philipp, M., Schmid, A.: Parameter stimation for pedestrian simulation model. In: 7th Swiss Transport Research Conf., pp. 1–29 (2007)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
Szepesvári, C.: Algorithms for reinforcement learning. Morgan Claypool (2010)
Teknomo, K.: Microscopic Pedestrian Flow Characteristics: Development of an Image Processing Data Collection and Simulation Model. PhD thesis, Tohoku University, Japan (2002)
Weidmann, U.: Transporttechnik der fussgänger - transporttechnische eigenschaften des fussgängerverkehrs (literaturstudie). Literature Research 90, IVT an der ETH Zürich, ETH-Hönggerberg, CH-8093 Zürich (1993)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Martinez-Gil, F., Lozano, M., Fernández, F. (2012). Calibrating a Motion Model Based on Reinforcement Learning for Pedestrian Simulation. In: Kallmann, M., Bekris, K. (eds) Motion in Games. MIG 2012. Lecture Notes in Computer Science, vol 7660. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-34710-8_28
Download citation
DOI: https://doi.org/10.1007/978-3-642-34710-8_28
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-34709-2
Online ISBN: 978-3-642-34710-8
eBook Packages: Computer ScienceComputer Science (R0)