Abstract
Multi-agent teamwork is critical in a large number of agent applications, including training, education, virtual enterprises and collective robotics. The complex interactions of agents in a team as well as with other agents make it extremely difficult for human developers to understand and analyze agent-team behavior. It has thus become increasingly important to develop tools that can help humans analyze, evaluate, and understand team behaviors. However, the problem of automated team analysis is largely unaddressed in previous work. In this article, we identify several key constraints faced by team analysts. Most fundamentally, multiple types of models of team behavior are necessary to analyze different granularities of team events, including agent actions, interactions, and global performance. In addition, effective ways of presenting the analysis to humans is critical and the presentation techniques depend on the model being presented. Finally, analysis should be independent of underlying team architecture and implementation.
We also demonstrate an approach to addressing these constraints by building an automated team analyst called ISAAC for post-hoc, off-line agent-team analysis. ISAAC acquires multiple, heterogeneous team models via machine learning over teams' external behavior traces, where the specific learning techniques are tailored to the particular model learned. Additionally, ISAAC employs multiple presentation techniques that can aid human understanding of the analyses. ISAAC also provides feedback on team improvement in two novel ways: (i) It supports principled “what-if” reasoning about possible agent improvements; (ii) It allows the user to compare different teams based on their patterns of interactions. This paper presents ISAAC's general conceptual framework, motivating its design, as well as its concrete application in two domains: (i) RoboCup Soccer; (ii) software agent teams participating in a simulated evacuation scenario. In the RoboCup domain, ISAAC was used prior to and during the RoboCup '99 tournament, and was awarded the RoboCup Scientific Challenge Award. In the evacuation domain, ISAAC was used to analyze patterns of message exchanges among software agents, illustrating the generality of ISAAC's techniques. We present detailed algorithms and experimental results from ISAAC's application.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
A. Aho, J. Hopcroft, and J. Ullman, Data Structures and Algorithms, Addison-Wesley, 1983.
T. Andou, Personal Communication, 1998.
E. André, G. Herzog, and T. Rist, “Generating multimedia presentations for RoboCup soccer games,” RoboCup-97: Robot Soccer World Cup I, 1997.
T. Balch, “The impact of diversity on performance in multi-robot foraging,” Proceedings of the Third Annual Conference on Autonomous Agents, 1999.
I. Bhandari, E. Colet, J. Parker, Z. Pines, R. Pratap, and K. Ramanujam, “Advanced scout: Data mining and knowledge discovery in NBA data,” Data Mining and Knowledge Discovery, 1997.
M. Bordegoni, G. Faconti, M. T. Maybury, R. Rist, S. Ruggieri, P. Trahanias, and M. Wilson, “A standard reference model for intelligent multimedia presentation systems,” Journal of Computer Standards and Interfaces, pp. 477–496, 1997.
R. Caruana and D. Freitag, “Greedy attribute selection,” 11th Proceedings of the 11th International Conference on Machine Learning (ICML), 1994.
B. Chiu and G. Webb, “Using decision trees for agent modeling,” Improving Prediction Performance, User Modeling and User-Adapted Interaction, Kluwer Academic Publisher Group, Dordrecht, Netherlands, vol. 8, pp. 131–152, 1988.
J. P. Crutchfield and K. Young, “Inferring statistical complexity,” Physical Review Letters, vol. 63, no. 2, pp. 105–108, 1989.
G. Dorais, R. Bonasso, D. Kortenkamp, B. Pell, and D. Schreckenghost, “Adjustable autonomy for humancentered autonomous systems,” Working notes of the Sixteenth International Joint Conference on Artificial Intelligence Workshop on Adjustable Autonomy Systems, 1999.
K. Driessens, N. Jacobs, N. Cossement, P. Monsieur, and L. DeRaedt, “Inductive verification and validation of the KULRot RoboCup team,” Proceedings of the Second RoboCup Workshop, 1998.
D. Goldberg and M. Matari?, “Coordinating mobile robot group behavior using a model of interaction dynamics,” Proceedings of The Third International Conference on Autonomous Agents, 1999.
N. Jennings, “Controlling cooperative problem solving in industrial multi-agent system using joint intentions,” Artificial Intelligence, vol. 75, 1995.
3D images from RoboCup soccer. http://www.techfak.unibielefeld.de/techfak/ags/wbski/3Drobocup/ 3DrobocupPics.html, 1999.
W. L. Johnson, “Agents that learn to explain themselves,” Proceedings of AAAI-94, 1994.
G. Kaminka, “Execution monitoring in multi-agent environments,” Ph.D. Dissertation, University of Southern California, Computer Science Department, 2000.
H. Kitano, M. Tambe, P. Stone, M. Veloso, I. Noda, E. Osawa and M. Asada, “The RoboCup synthetic agent's challenge,” Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), 1997.
S. C. Marsella and W. L. Johnson, “An instructor's assistant for team training in dynamic multi-agent virtual worlds,” in B. Goettl, H. Halff, C. Redfield and V. Shute, (eds.), Intelligent Tutoring Systems, Proceedings of the 4th International Conference on Intelligent Tutoring Systems, Springer, pp. 464–473, 1998.
J. Modi, H. Jung, M. Tambe, W. Shen, and S. Kulkarni, “Dynamic distributed resource allocation: a distributed constraint satisfaction approach,” Intelligent Agents VIII Proceedings of the International workshop on Agents, Theories, Architectures and Languages (ATAL'01), 2001.
H. Myritz, Personal Communication, 1999.
D. Ndumu, H. Nwana, L. Lee, and H. Haynes, “Visualization and debugging of distributed multi-agent systems,” Applied Artificial Intelligence Journal, vol 13, no. 1, 1999.
I. Noda, H. Matsubara, K. Hiraki, and I. Frank, “Soccer server: A tool for research on multi-agent systems,” Applied Artificial Intelligence, vol. 12, no. 2-3, 1998.
P. Politakis and S. Weiss, “Using empirical analysis to refine expert system knowledge bases,” Artificial Intelligence, no. 22, pp. 23–48, 1984.
T. Raines, M. Tambe, and M. Marsella, “Automated agents that help humans understand agent team behaviors,” Proceedings of the International Conference on Autonomous Agents (Agents), 2000.
T. Raines, M. Tambe, and M. Marsella, “Agent assistants for team analysis,” AI Magazine, vol. 21, no. 3, pp. 27–31, Fall, 2000.
E. Reiter, “Has a consensus NL generation architecture appeared, and is it psycholinguistically plausible?'' Proceedings of the Seventh International Workshop on Natural Language Generation, 1994.
D. Ron, Y. Singer, and N. Tishby, “On the learnability and usage of acyclic probabilistic finite automata,” Journal of Computer and System Sciences, vol. 56, no. 2, 1998.
P. Sengers, “Designing comprehensible agents,” Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), 1999.
SoftSport Inc.: http://www.softsport.com
A. Stolcke and S. M. Omohundro, “Hidden Markov model induction by bayesian model merging,” in S. J. Hanson, J. D. Cowan and C. L. Giles, eds., Advances in Neural Information Processing Systems, vol. 5, Morgan Kaufman, pp. 11–18, 1992.
P. Stone and M. Veloso, “Using decision tree confidence factors for multi-agent control,” Proceedings of the International Conference on Autonomous Agents, 1998.
P. Stone, Personal Communication, 1999.
K. Sycara, K. Decker, A. Pannu, M. Williamson, and D. Zeng, “Distributed intelligent agents,” IEEE Expert, 1996.
M. Tambe, W. L. Johnson, R. Jones, F. Koss, J. E. Laird, P. S. Rosenbloom, and K. Schwamb, “Intelligent agents for interactive simulation environments,” AI Magazine, vol. 16, no. 1, Spring, 1995.
M. Tambe, “Towards flexible teamwork,” Journal of Artificial Intelligence Research, vol. 7, 1997.
M. Tambe and H. Jung, “The benefits of arguing in a team,” AI Magazine, vol. 20, no. 4, Winter, 1999.
M. Tambe, D. Pynadath, and N. Chauvat, “Building dynamic agent organizations in cyberspace,” IEEE Internet Computing, vol. 4, no. 2, 2000.
K. Tanaka-Ishii, I. Noda, I. Frank, H. Nakashima, K. Hasida, and H. Matsubara, “MIKE: An automatic commentary system for soccer'' International Conference on Multi-Agent Systems, 1998.
K. Ting, “Inducing cost-sensitive trees via instance weighting,” Principles of Data Mining and Knowledge Discovery (PKDD 98), 1998.
T. Tomoichi, “LogMonitor: from player's action analysis to collaboration analysis and advice on formation,” Proceedings of the Third RoboCup Workshop, 1999.
S. Toulmin, The Uses of Argument. Cambridge University Press, 1958.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Nair, R., Tambe, M., Marsella, S. et al. Automated Assistants for Analyzing Team Behaviors. Autonomous Agents and Multi-Agent Systems 8, 69–111 (2004). https://doi.org/10.1023/B:AGNT.0000009411.79208.f4
Issue Date:
DOI: https://doi.org/10.1023/B:AGNT.0000009411.79208.f4