The Effects of Locality and Asymmetry in Large-Scale Multiagent MDPs

Summary

As multiagent systems scale up, the complexity of interactions between agents (cooperative coordination in teams, or strategic reasoning in the case of self-interested agents) often increases exponentially. In particular, in multiagent MDPs, it is generally necessary to consider the joint state space of all agents, making the size of the problem and the solution exponential in the number of agents. However, often interactions between the agents are only local, which suggests a more compact problem representation. We consider a subclass of multiagent MDPs with local interactions where dependencies between agents are asymmetric, meaning that agents can affect others in a unidirectional manner. This asymmetry, which often occurs in large-scale domains with authority-driven relationships between agents, allows us to make better use of the locality of agents’ interactions. We discuss a graphical model that exploits this form of problem structure and use it to analyze the effects of locality and asymmetry on the complexity and structure of optimal policies. For problems where the solutions retain some of the compactness of problem representation, we present computationally-efficient algorithms for constructing optimal multiagent policies.