Praveen Paruchuri
ABSTRACT
Security in multiagent systems is commonly defined as the ability of the system to deal with intentional threats from other agents. This paper focuses on domains where such intentional threats are caused by unseen adversaries whose actions or payoffs are unknown. In such domains, action randomization can effectively deteriorate an adversary's capability to predict and exploit an agent/agent team's actions. Unfortunately, little attention has been paid to intentional randomization of agents' policies in single-agent or decentralized (PO)MDPs without significantly sacrificing rewards or breaking down coordination. This paper provides two key contributions to remedy this situation. First, it provides three novel algorithms, one based on a non-linear program and two based on linear programs (LP), to randomize single-agent policies, while attaining a certain level of expected reward. Second, it provides Rolling Down Randomization (RDR), a new algorithm that efficiently generates randomized policies for decentralized POMDPs via the single-agent LP method.
- E. Altman. Constrained Markov Decision Process. Chapman and Hall, 1999.Google Scholar
- R. Beard and T. McLain. Multiple uav cooperative search under collision avoidance and limited range communication constraints. In IEEE CDC, 2003.Google ScholarCross Ref
- D. Bernstein, S. Zilberstein, and N. Immerman. The complexity of decentralized control of MDPs. In UAI, 2000. Google ScholarDigital Library
- D. Dolgov and E. Durfee. Constructing optimal policies for agents with constrained architectures. Technical report, Univ of Michigan, CSE-TR-476-03, 2003.Google Scholar
- R. Emery-Montemerlo, G. Gordon, J. Schneider, and S. Thrun. Approximate solutions for partially observable stochastic games with common payoffs. In AAMAS, 2004. Google ScholarDigital Library
- E. Hansen, D. Bernstein, and S. Zilberstein. Dynamic programming for partially observable stochastic games. In AAAI, 2004. Google ScholarDigital Library
- J. Hu and P. Wellman. Multiagent reinforcement learning: theoretical framework and an algorithm. In ICML, 1998. Google ScholarDigital Library
- D. A. Huffman. A method for the construction of minimum redundancy codes. In Proc. IRE 40, 1952.Google Scholar
- T. Jaakkola, S. Singh, and M. Jordan. Reinforcement learning algorithm for partially observable markov decision problems. Advances in NIPS, 7, 1994.Google Scholar
- M. Littman. Markov games as a framework for multi-agent reinforcement learning. In ML, 1994.Google Scholar
- R. Nair, D. Pynadath, M. Yokoo, M. Tambe, and S. Marsella. Taming decentralized POMDPs: Towards efficient policy computation for multiagent settings. In IJCAI, 2003. Google ScholarDigital Library
- S. Paquet, L. Tobin, and B. Chaib-draa. An online POMDP algorithm for complex multiagent environments. In AAMAS, 2005. Google ScholarDigital Library
- P. Paruchuri, M. Tambe, F. Ordonez, and S. Kraus. Towards a formalization of teamwork with resource constraints. In AAMAS, 2004. Google ScholarDigital Library
- D. V. Pynadath and M. Tambe. The communicative multiagent team decision problem: Analyzing teamwork theories and models. JAIR, 16:389--423, 2002. Google ScholarDigital Library
- C. Shannon. A mathematical theory of communication. The Bell Labs Technical Journal, pages 379--457, 623, 656, 1948.Google Scholar
- S. Vavasis. Nonlinear optimization: Complexity issues. In University Press, New York, 1991. Google ScholarDigital Library
Recommendations
Organizing Multiagent Systems
Despite all the research done in the last years on the development of methodologies for designing MAS, there is no methodology suitable for the specification and design of MAS in complex domains where both the agent view and the organizational view can ...
Computing effective communication policies in multiagent systems
AAMAS '07: Proceedings of the 6th international joint conference on Autonomous agents and multiagent systemsCommunication is a key tool for facilitating multiagent coordination in cooperative and uncertain domains. We focus on a class of multiagent problems modeled as Decentralized Markov Decision Processes with Communication (DEC-MDP-COM) with local ...
Norm Governed Multiagent Systems: The Delegation of Control to Autonomous Agents
IAT '03: Proceedings of the IEEE/WIC International Conference on Intelligent Agent TechnologyWhen agents make decisions, they have to deal with norms regulating the system. In this paper we therefore propose a rule-based qualitative decision and game theory combining ideas from multiagent systems and normative systems. Whereas normative systems are ...
Comments