Parameterised verification for multi-agent systems

Article history: Received 28 October 2014 Received in revised form 5 January 2016 Accepted 13 January 2016 Available online 18 January 2016


Introduction
With the development and deployment of autonomous agents and multi-agent systems (MAS) in diverse applications such as robot-based search-and-rescue [1], web-services [2], personal negotiation assistants [3], a growing need has emerged to develop powerful and versatile methodologies for the validation and verification of MAS. Model checking [4] is a leading logic-based technique for the verification of systems that has emerged in the past twenty years. Model checking enables us to check whether a model M S representing a system S, satisfies a formula φ P encoding a specification P .
While plain reactive systems [5] are typically specified by means of reachability or purely temporal statements, autonomous agents are typically specified by means of high level properties inspired from AI. As a consequence, in the case of MAS the specification φ P is typically given in agent-based logics, such as epistemic logic [6], BDI [7], Desires-Goal-Intention [8], and ATL [9]. Over the past ten years a number of techniques have been put forward for the efficient model checking of MAS against agent-based specifications including binary decision diagrams [10,11], abstraction [12], partial order reduction [13], bounded model checking [14], parallel model checking [15], thereby making it possible to verify systems with large state spaces. Yet, since the number of states is exponential in the number of agents in the system, systems of many agents typically remain intractable.
A further difficulty consists on the fact that some agent-based protocols, such as auctions, do not specify how many agents may be present at runtime. By model checking we may be able to verify a system for a given number of agents. But this does not enable us to draw any conclusion as to whether the specification would still hold should more agents be present. Intuitively, additional agents may possibly interfere with the system in unpredicted ways resulting in the specification to be violated. Yet, our practical experience (e.g., networking and security protocols) tells us that some, albeit not all, protocols are correct irrespective of the number of components. Any technique that enables us to verify specifications independently of the number of agents present would clearly be beneficial in validating a wide range of MAS.
Cutoffs have been studied in the formal analysis of systems to try to address this, often in the context of networking protocols [16,17]. A cutoff for a specification is the number of components that need to be analysed to be able to draw general conclusions that hold irrespective of the number of components in a system. Since the problem in its generality is undecidable [18], sound but incomplete methods have been put forward [17,19,20] that impose restrictions on the systems and the properties to be studied. However, as we discuss below the current literature does not address the needs of MAS, or AI systems in general, as they are tailored to temporal specifications only and they often rely on specific semantics that abstract from the particular way in which agents may interact.
The aim of this paper is to present a technique for the automatic verification of MAS populated by arbitrarily many agents adhering to different roles. In particular we isolate three classes of MAS for which we show that cutoffs can be given when certain sufficient conditions are met. We illustrate the semantic classes correspond to different ways in which the agents may interact among themselves and with the environment. In addition to exploring the theoretical side of the problem we also present an implementation based on ideas here presented and discuss the experimental results obtained.

Parameterised model checking
The traditional model checking problem [4] concerns establishing whether a specification φ P representing a property P holds on a finite model M S built from a finite number of components implementing the system S, or M S |= φ P . In the traditional approach the behaviours of all the components are specified beforehand; the model M S resulting from their synchronisation is then constructed and the property φ P is then checked. While the traditional model checking problem establishes whether a particular system satisfies a given specification, the parameterised model checking problem (PMCP) is concerned with establishing whether any system composed of any number of agents following a certain behavioural template satisfies a given specification. Clearly any attempt to reduce the parameterised model checking problem to the standard model checking problem would entail checking an infinite number of models, i.e., all possible systems built from any number of agents. Given the number of agents is not bounded it would also imply checking models of unbounded size.
In traditional computer science the PMCP can potentially be used to verify specific networking protocols and a wide range of distributed algorithms. In MAS and AI in general, techniques for the PMCP could in principle be used to establish properties of a wide and diverse range of systems ranging from robotic swarms to e-commerce applications where the number of agents is not known at design time.
In the general setting the PMCP is undecidable [18]. However, given its importance, it is of interest to develop sound but incomplete techniques to solve it. The PMCP is typically formulated in a finitary, abstract way by giving a template for the agents in the system, a template for the environment, and the formula to be verified. By providing the parameter n specifying the actual number of agents in the system, we can then construct a concrete system upon which the standard model checking problem can be solved. A way to limit the generality of the problem is to restrict the systems considered. For example, we may consider a specific topology, e.g., rings, when analysing network protocols for an unbounded number of hosts. In this paper we follow a different approach. We do not impose many constraints in terms of how the agents may behave, but we are constrain their interaction.

Related work
In the past 10 years several methods have been put forward for verifying MAS by means of symbolic model checking. Most techniques support epistemic specifications [13,14,[21][22][23][24]; others target deontic specifications [25,26], or specifications expressing strategic abilities [27,28]. The resulting performance differs depending on a number of assumptions; symbolic checkers such as MCK [10], MCMAS [11] and VerICS [29] are all capable of handling state-spaces of the region of 10 15 and beyond.
While these techniques have received considerable attention, they all suffer from a key limitation in that they only deal with closed MAS where the number of components is known at design time. This makes it impossible to verify MAS where the number of agents is not known at design time.
Verification of systems with an arbitrarily large number of components has been investigated, however, in the context of reactive systems where the problem has been shown to be undecidable in general [18]. The techniques put forward typically assume a number of restrictions either on the systems or in the specifications considered so that either soundness or decidability can be retained. The approaches can be classified into abstraction techniques, network invariant techniques, regular model checking, and cutoff techniques.
Abstraction techniques [30][31][32][33][34][35][36][37] rely on the analysis of a single finite state abstract system encoding all possible concrete systems. Typically these methods require manual guidance for obtaining the abstract mapping. Further, they are often incomplete: if a certain specification is falsified in the abstract model, then it does not necessarily follow that there is a concrete system falsifying the specification. Among these techniques we identify counter abstraction and environment abstraction.

Scheme of the paper
The rest of the paper is organised as follows. In Section 2 we recall the semantics of interleaved interpreted systems, and we put forward PIIS as an abstract semantics for MAS on which the parameterised model checking problem can be defined. In particular we give the definitions for the agent templates for a given role, the environment template and we identify five types of interactions that the agents and the environment can engage in. We then proceed to define three special classes of PIIS, each representing different synchronisation patterns for the agents in the system. In Section 3 we introduce our specification language which consists of an indexed version of a temporal-epistemic logic. Here we adopt a limited form of quantification over the agents to account for the unbounded number of agents in the system, but we remove the next state temporal operator to avoid undecidability. The section continues with some key theoretical observations regarding the notions of stuttering simulations that can be defined on these systems and an exploration on the extent to which these preserve logical satisfaction. This enables us to formally define the PMCP and the notion of cutoffs on the semantics in Section 4.
Sections 5, 6 and 7 include the main theoretical results of the paper. We study each of the classes identified in Section 2 and give cutoff results for them. By means of these results the PMCP for a class of PIIS can be solved by model checking all systems up to the cutoff. Given the cutoffs are typically low natural numbers these results provide algorithms for the effective verification of various classes of MAS. Each respective class is exemplified via a concrete example showing the applicability of the results.
Section 8 reports an implementation that we built realising the techniques described in Sections 5 to 7. Specifically the section introduces MCMAS-P, a parameterised version of MCMAS, an open-source model checker for the verification of MAS. As we explain, MCMAS-P conducts an iterative check on the existence of certain simulations that guarantee, by the methods of Sections 5 to 7, that a cutoff exists. If this can be shown, the checker performs plain model checking on corresponding concrete systems in line with the requirements of the theory developed. We report the experimental results obtained.
We conclude in Section 9, where we discuss possible future work.

Parameterised systems with multiple roles
Interpreted systems are a standard semantics for describing multi-agent systems [6]. They provide a natural setup to interpret specifications in a variety of languages including temporal-epistemic logic and alternating temporal logic [6]. Interleaved Interpreted Systems (IIS) are a class of interpreted systems constraining the interleaved evolution of the agents' actions [13]. Here we extend IIS to reason about temporal-epistemic properties in an unbounded MAS setting. To do this, we define parameterised IIS to give a generic description of a MAS irrespective of the number of agents present. This will enable us to define three important classes of parameterised IIS that exhibit attractive properties towards verification. We then proceed to define an indexed temporal-epistemic logic to express properties in the unbounded system. This is followed by the formal definition of the PMCP and the notion of cutoff. We show that cutoffs do not exist in general, thereby paving the road to the subclasses' analysis in the following sections.

Interleaved interpreted systems
We begin by assuming a MAS composed of n agents A = {1, . . . ,n} acting in an environment E. The environment is treated as a special agent allowing us to consider a MAS as composed of the set A ∪ {E} of agents. Each agent i ∈ A ∪ {E} is described by a nonempty set of local states L i , a unique initial local state ι i ∈ L i , and a nonempty set of actions Act i . Actions are performed in compliance with a protocol P i : L i → Act i governing which actions can be performed at a given local state. The evolution of an agent i's local states is specified by a transition function t i : L i × Act i → L i returning the next local state given the agent's (current) local state and action.
A "null" action i is assumed to be a member of any set Act i . It is assumed that for every state l i ∈ L i we have that: (i) i ∈ P i (l i ) (i.e., the null action is enabled at every local state); (ii) t i (l i , i ) = l i (i.e., an agent stutters in its current local state whenever it performs the null action).

Definition 2.1 (Interleaved interpreted system). An interleaved interpreted system is a tuple
is a valuation function for a set AP of atomic propositions.
A global state g = (l 1 , . . . , l n , l E ) is a tuple of local states for all the agents in the system; it describes the system at a particular instant of time. Given a global state g = (l 1 , . . . , l n , l E ) and an agent i, we write ls i (g) to denote the local state ls i (g) = l i of agent i in g. The system's global states evolve over time in compliance with the agents' local protocols and local evolution functions, thereby inducing a global transition function. To define the transition function, given an action a ∈ i∈A∪{E} Act i , let Agent(a) = {i ∈ A ∪ {E} : a ∈ Act i } be the set of agents admitting the action in their repertoire.

Definition 2.2 (Global transition function).
The global transition function t : G × Act 1 × . . . Act n × Act E → G on a set G of global states is a partial function defined as follows: t(g, a 1 , . . . , a n , a E ) = g iff there is an action b ∈ i∈A∪{E} Act i such that for all i ∈ Agent(b), we have that a i = b, a i ∈ P i (ls i (g)), and t i (ls i (g), a i ) = ls i (g ); and for all i ∈ (A ∪ {E}) \ Agent(b), we have that a i = i and t i (ls i (g), a i ) = ls i (g ) = ls i (g). In short we write the above as g → a g .
Thus the global transition function is given in a similar fashion to blocking synchronisation in automata. At each round all agents participating in the global transition are required to perform the same local action; the agents not participating in the global transition are assumed to perform the null action. Every agent admitting said local action in its repertoire has to perform it at the round; if there is a local protocol not permitting this, then the local action cannot be performed in the system. A local action is said to be shared by two or more agents if said agents admit that action in their repertoire of actions. So, communication in IIS is by means of shared actions. We assume that the joint silent action is always enabled. Therefore t is serial.
Given a set of actions X ⊆ i∈A∪{E} Act i , we write g → X g to mean that g → a g for some a ∈ X . The reflexive and transitive closure of → X is denoted by → X * . A path π is either a finite or an infinite sequence π = g 1 a 1 g 2 a 2 g 3 . . . such that g i → a i g i+1 , for every i ≥ 1. Given a path π , we write π(i) (respectively π(i, Act)) for the i-th state (action respectively) in π . If π is finite, then we write π[] for the last state in π . By π[i], we denote the suffix g i a i g i+1 . . . of π , and by [i]π we denote its prefix g 1 a 1 . . . g i . The set of all paths originating from a state g is denoted by (g). A global state g is said to be reachable from a global state g 1 if there is a path π ∈ (g 1 ) such that π(i) = g, for some i ≥ 1. Since the global transition relation is deterministic we sometimes (uniquely) denote a path g 1 a 1 g 2 a 2 . . . by the sequence g 1 a 1 a 2 . . . . We associate temporal models to IIS that, as shown below, can be used to interpret temporal-epistemic formulae as follows.

Definition 2.3 (Model). Given an IIS
where G is the set of global states reachable via R from the initial global state ι = (ι 1 , . . . , ι n ), R ⊆ G × G is a global transition relation defined as (g, g ) ∈ R iff g → a g for some action a, and ∼ i = (g, g ) ∈ G × G : ls i (g) = ls i (g ) is the epistemic accessibility relation for agent i defined on local equalities for the agents' states. Example 2.4. Fig. 1 presents the interleaved interpreted system of the untimed version of the Train-Gate-Controller (TGC) as presented in [55] and adapted from [56]. The system of TGC is composed of a controller and two trains. Each train runs along a circular track and both tracks pass through a narrow tunnel. The tunnel can accommodate only one train to be in it at any time. Both sides of the tunnel are equipped with traffic lights, which can be either green or red. The controller operates the colour of the traffic lights to let the trains enter and exit the tunnel. In the figure, the initial states of the controller and the trains are GREEN and WAIT respectively. The transitions that are depicted with the same style of edges are synchronised. Null actions are omitted in the figure.

Parameterised interleaved interpreted systems
We introduce a semantics for parameterised MAS representing several types of agents. Agents of a type are said to be adhering to a role. Each role is associated with a generic agent template which specifies the behaviour of each agent of said role. So, the generic description of a parameterised system consists of the descriptions of a finite number of agent templates and the description of the environment template. A parameter for a parameterised system is a tuple of natural numbers, one for each role, whose sum determines the actual number of agents in the system. Given a parameter (n 1 , . . . , n k ) for the system, the concrete interleaved interpreted system corresponding to the composition of n i agents, for each role i, can be constructed. Hence, a parameterised system gives a finite description of an unbounded number of differently populated interleaved interpreted systems.
We now describe an agent template. The template is similarly defined to agents in IIS. However, to determine the agents' synchronisation patterns in a concrete system, an agent template distinguishes between five types of actions: (i) asynchronous actions; (ii) agent-environment actions; (iii) role-synchronous actions; (iv) global-synchronous actions; (v) multi-role actions. Each type of action model a different type of interaction.
i. An asynchronous action from a template i is uniquely instantiated for each concrete agent performing role i. Thus, whenever an asynchronous action is performed, exactly one agent is active in the global transition. ii. An agent-environment action, modelling agent-environment communication, is instantiated as an asynchronous action, but each instantiation is shared by the concrete environment. Hence, whenever an agent-environment action is performed, exactly one agent and the environment participate in the global transition. iii. A role-synchronous action, describing multi-agent-environment communication, is instantiated only once; the copy is shared by all the concrete agents performing role i and the concrete environment. Therefore, whenever a rolesynchronous action is performed, all the agents performing role i and the environment participate in the global transition. iv. Similarly, global-synchronous actions are instantiated only once, but the copy is shared by all the concrete agents and the concrete environment. Consequently, whenever a global-synchronous action is performed, all the agents and the environment participate in the global transition. v. Multi-role actions encode pairwise communication between the environment and agents performing different roles.
A multi-role action is always admitted in the repertoire of actions of exactly two agent templates. Similarly to disjunctive guards [46], one and only one of the two templates guards the action. A multi-role action is said to be guarded by an agent template if the transition function of the template returns the same template state at which the action is performed. A multi-role action shared by templates i and r and guarded by r is instantiated for each pair of concrete agents performing roles i and r; the instantiation is also admitted by the concrete environment. As a result, whenever a multi-role action is performed, the following agents are participating in the global transition: exactly one agent performing role i, exactly one agent performing role r, and the environment. The agent performing role i may update its state via the global transition, whereas the agent performing role r remains in its current local state. Intuitively, the agent from role r guards the action as it has to be in a local state where the action is enabled for the global transition to occur.

Definition 2.5 (Agent template). An agent template
The environment template E is similarly described as an agent, but for the synchronisation purposes described above, E's set of actions is the union of the agent templates' sets of agent-environment, role-synchronous, global-synchronous, and multi-role actions.

Definition 2.6 (Environment template). An environment template
A parameterised interleaved interpreted system consists of a finite collection of agent templates and a template environment.

Definition 2.7 (Parameterised interleaved interpreted system). A Parameterised Interleaved Interpreted System is a tuple
. . , T k } is a nonempty and finite set of agent templates, E is an environment template, and V = {V i : L i → P(AP i ) : 1 ≤ i ≤ k} is a set of valuation functions, one for each agent template. It is assumed that AP 1 , . . . , AP k are pairwise disjoint sets of atomic propositions.
Let PIIS = T , E, V be a parameterised system with k ≥ 1 roles. Let n ∈ N k be a value of the system's parameter where N = {i : i ≥ 1} denotes the set of natural numbers. Assume n(i) to denote the i-th component in n. We now describe the n-st concrete instantiation of a parameterised system. The concrete system PIIS(n) results from the parallel composition of n(i) instantiations (i, 1), . . . , (i, n(i)) of each agent template T i and an instantiation E(n) of the environment template. We write A(n) for the set A(n) = {(i, j) : 1 ≤ i ≤ k, 1 ≤ j ≤n(i)} of all concrete agents. Each concrete agent is instantiated by taking indexed copies of its agent template.
is defined as follows.
• L j i = L i × { j} is set of concrete local states; • ι j i ∈ L j i is the initial concrete state; • Act j i is the set of concrete local actions that is defined as the union of the following sets of actions.
is the set of multi-role actions shared between the concrete agents (i, j), (r, s) and guarded by (i, j).
where a τ (l τ , respectively) denotes the corresponding template action (state, respectively) from which a (l, respectively) has been instantiated; So, each local state of a concrete agent is made of the template local states indexed by the name of the agent in question and inherits from its template the actions, the protocols and the transition function. The concrete environment is similarly obtained by instantiating each action shared with the agent templates.
Finally, a parameterised system's instantiation, and the concrete semantics we consider, is the IIS composed of the concrete agents and the concrete environment. The concrete system's valuation function is defined on atomic propositions indexed by the agents' identities so that a proposition holds on a global state iff the proposition holds by the template valuation function on the template state that the agent indexing the proposition is in the global state. This will enable us in Section 3 to specify collective properties that range over all concrete agents irrespectively of the size of the system. Definition 2.10 (Concrete system). Given a PIIS = T , E, V of k roles and n ∈ N k , the concrete system PIIS(n), composed of 1≤i≤kn (i) concrete agents, is a tuple for p ∈ AP i and 1 ≤ j ≤n(i), where l is the template local state of agent (i, j) in g. In compliance with the interleaved semantics, we can distinguish five types of transitions on a concrete system. In particular, a global transition from a state g can only happen in the following cases (see Fig. 2): (i) a concrete asynchronous A j i action is enabled for agent (i, j) performing role i at g; (ii) a concrete agent-environment AE j i action is enabled for the environment and for agent (i, j) performing role i at g; (iii) a concrete role-synchronous RS i action is enabled for the environment and for all the agents performing role i at g; (iv) a concrete global-synchronous GS action is enabled for the environment and for all the agents at g; (v) a concrete multi-role MR j,q i,r action is enabled for the environment, for agent (i, j) performing role i, and for agent (r, q) performing role r at g.
To summarise, we introduced a notion of parameterised systems giving a concise description of an arbitrarily big set of IIS. Each system is built from n(i) identical agents for each role i ∈ {1, . . . ,k} and from the concrete environment corresponding to the n-th instantiation of the template environment. The concrete agents may evolve asynchronously, communicate with the environment via agent-environment actions, synchronise with the agents of the same role via role-synchronous actions, synchronise with all the agents in the system via global-synchronous actions, and communicate with an agent performing another role via multi-role actions. We refer to Appendix A for a summary of the notation used in the paper.

Examples
We exemplify the technical notions introduced above on three examples: a train-gate-controller model [55], a robot foraging scenario [57], an autonomous robot example [6]. The train-gate-controller illustrates the agent-environment and global-synchronous communication patterns. The robot-foraging scenario gives an intuitive example of multi-role synchronisations. We discuss role-synchronous communication in the context of the autonomous robot example. We here focus on the semantic modelling. We will later discuss specifications and verification methodologies.

Robot foraging scenario
Swarm robotics concerns the coordination and analysis of an unbounded collection of behaviourally simple robotic agents [58][59][60]. The interaction between the agents and their environment is meant to exhibit a collective, emergent behaviour often inspired by biological systems, e.g., ant colonies [61]. As argued in [59], despite the lack of centralised coordination, biological swarm-based systems can still be robust, scalable, and flexible. It is therefore of interest to design swarm robotic systems that can be shown to be in compliance with their specifications. To do this, we need to analyse the properties of a swarm irrespectively of the number of robots in the system.
In the following we describe an untimed version of the robot foraging scenario (RFS) from [57]. The RFS includes an arbitrary number of robots initially resting in a nest before undertaking a campaign in search for food by means of a random walk. Upon observing a food source, a robot tries to reach for it. If it succeeds, then (i) it collects and deposits the food in the nest; (ii) it makes the location of the food known so that all other robots can find it. Otherwise, if it fails to reach the food source, it then scans the area to locate the source again, or locate a new source. If the scan is successful, then the robot attempts to reach the food source. Otherwise, if the scan is not successful (under a timeout), then the robot returns to its nest.
We can encode the scenario as a PIIS S RFS composed of a template agent TR representing the robots and a template agent TFS representing the food sources. The template robot is depicted in Fig. 3a. TR is initially in state R representing that the robot is resting in its nest. The states RW , MF, SA represent that the robot is performing a random walk, the robot is moving to the food, and the robot is scanning the area, respectively. The template food source is given by Fig. 3b. TFS is initially in the state N_F representing that the food source has not been found, whereas the state F represents that the food source is found.
We now describe the global transitions induced by the templates. As discussed in the previous section, a multi-role action is always admitted in the repertoire of actions of two agent templates, and it is guarded by one of them. A multi-role action is instantiated for each pair of agents instantiated from the two templates. In a global transition induced by a multi-role action only the agents for which the action is instantiated and the environment are participating in the transition. The concrete agent from the template not guarding the action may update its state in the global transition, whereas the Fig. 3. The parameterised interleaved interpreted system for the robot foraging scenario. R stands for "Resting", RW stands for "Random Walk", MF for "Move to Food", SA for "Scan Area", N_F for "Not Found", and F for "Found". The actions search, fail are asynchronous actions, whereas the actions observe, deposit, scan, reached are multi-role actions.
concrete agent from the template guarding the action remains in its current local state. We describe the possible actions for the templates in the system.
• search. This an asynchronous action that is defined for the template robot. It is enabled at state R and it represents a robot moving out of its nest to search for food. A global transition by means of the search action results the robot performing the action to move to state RW.
• fail. This is also an asynchronous action that is enabled at states RW and SA of the template robot. The action represents a robot failing to locate a food source when performing a random walk and when scanning the area, respectively. A global transition via the fail action results the robot performing the action to move to state R.
• observe. This is a multi-role action that is guarded by TFS. Assume the instantiation (observe, i, j) of the action for robot i and food source j. A concrete transition via the (observe, (i, j)) action is only enabled if the robot i is either in state RW or in state SA, and the food source j is in state N_F . Intuitively the robot can observe the food source if the latter has not already been found. The action causes the robot i to change its state to MF.
• reached. This is also a multi-role action that is guarded by TR. Following the transition described above, a concrete transition via the (reached, ( j, i)) action is enabled. This transition causes the food source j to change its state to F thus modelling that robot i has succeeded in reaching the food source j.
• deposit. The above transition enables the multi-role action (deposit, (i, j)) that is guarded by TFS. A transition via this action causes the robot to move to state R.
• scan. Finally, scan is a multi-role action that is guarded by TFS. Intuitively, robot i may fail to reach the food source j (i.e., the (reached, ( j, i)) action is not performed). In this case the (scan, (i, j)) action is enabled. Upon this transition the robot updates its state to SA.

Train-gate-controller
In Section 2.1 we defined the IIS of the train-gate-controller (TGC) composed of a controller and two trains. We now give the PIIS model of a parameterised version of the TGC. We extend the original description to include an arbitrary number of two types of trains: prioritised trains and normal trains. A prioritised train can enter the tunnel at any given time, assuming there is no other train in the tunnel, whereas a normal train can only enter the tunnel when there is no other train waiting to enter the tunnel. To accomplish this, the traffic lights include two shades of the green colour: prioritised green and normal green. Prioritised green is used by the controller to serve prioritised trains, whereas normal green is used by the controller to serve normal trains.
The scenario can be encoded as a PIIS composed of an agent template representing prioritised trains (Fig. 4a), an agent template representing normal trains (Fig. 4c), and an environment template representing the controller (Fig. 4b). A prioritised train is initially in state WAIT, the controller is initially in state P _GREEN, and a normal train is initially in state TUNNEL_LOCKED. Therefore prioritised trains are initially waiting to enter the tunnel, normal trains are initially locked from entering the tunnel, and the controller initially serves only prioritised trains. The actions p_enter and p_exit are agent-environment actions modelling the prioritised trains entering and exiting the tunnel. Similarly, the actions n_enter and n_exit are agent-environment actions enabling the normal trains to enter and exit the tunnel. The action n_lock is a global-synchronous action and represents the normal trains taking the lock on the tunnel. Also, the action p_lock is a globalsynchronous action; it models the prioritised trains taking the lock on tunnel. Finally, the actions p_approach, n_appoach are asynchronous actions. The templates induce the following agent-environment and global-synchronous concrete transitions: • p_enter, n_enter. In addition to the agent performing the action entering the tunnel, the environment participates in the global transition. This causes the environment to change its state to RED, thereby disallowing other trains to enter the tunnel.
• p_exit, n_exit. The environment synchronises with the agent that is currently in the tunnel via the p_exit and n_exit actions. The synchronisation causes the environment to change its state to P _GREEN if the agent is a prioritised train or to N_GREEN if the agent is a normal train. Following this, other trains are allowed to enter the tunnel.
• n_lock. This action is only enabled if: (i) the environment is in state P _GREEN; (ii) there is no train in the tunnel; (iii) all prioritised trains are in state AWAY . A concrete global-transition via the n_lock action causes the environment to update its state to N_GREEN. Thus the transition frees the tunnel to serve normal trains whenever there are no prioritised trains waiting to be served.
• p_lock. This action is only enabled if: (i) the environment is in state N_GREEN; (ii) there is no train in the tunnel. Upon performing this action the environment moves to state P _GREEN. Therefore the transition locks the tunnel to serve prioritised trains and it can happen irrespective of whether there are normal trains waiting to be served.
The above transitions are depicted in Fig. 5 for a fragment of the concrete system with two prioritised trains and two normal trains.

Autonomous robot
We now consider a parameterised version of the autonomous robot (AR) scenario from [6]. The scenario includes an autonomous robot running along an endless straight track. The position of the robot is given in terms of locations numbered as 0, 1, 2, . . . . The robot can only move forward along the track starting at position 0 and its movement is controlled by the environment. A sensor is attached to the robot measuring its position. The sensor is faulty in the sense that a sensor reading at position q can be any of the values in {q − 1, q, q + 1}. The only action the robot can perform is to halt. If the robot halts, then the environment can no longer move the robot. Otherwise, the environment may move the robot one position forward at each time step. The goal of the robot is to halt in the goal region GR = {2, 3, 4}. A solution to the AR problem in the single robot case is for the robot to do nothing while the value of its sensor is less than 3 and to halt once the value of its sensor is greater than or equal to 3 [6]. We show in Section 7 that this solution applies to the arbitrary case with an unbounded number of robots.
We model a generalisation of the above description in which an arbitrary number of robots run synchronously along the track and in which the robots have access to a unique shared sensor. To illustrate the role-synchronous actions, we assume a second type of robots, identical to the description of the first type, but with no access to a sensor. We refer to the two types of robots as type 1 robots and type 2 robots, respectively. Type 2 robots halt after receiving a halting event from type 1 robots. The event is signaled after the type 1 robots have halted.
We encode the AR scenario as a PIIS S AR composed of a template agent TR1 representing robots with access to a sensor, a template agent TR2 representing robots with no access to a sensor, and a template environment E for synchronisation purposes. The encoding assumes a finite track with 8 distinct locations.
TR1 is given by Fig. 6a. A template state represents the position of the robot, its sensor reading, and whether it has halted or not, respectively. TR2 is depicted in Fig. 6b. A template state represents the position of the robot and whether it has halted or not, respectively. Finally, E is defined by Fig. 6c. A template state represents the position of the robots and whether or not the type 1 robots have halted. • move + , move = , move − . These are global-synchronous actions. A concrete transition via these actions causes all the robots to move one step forwards. Additionally, type 1 robots change their sensor reading to be either the correct reading (move = ), the correct reading plus 1 (move + ), or the correct reading minus 1 (move − ).
• halt. The role-synchronous action halt is enabled at any state in which the sensor reading of type 1 robots is greater than or equal to 3. Type 1 robots halt upon this transition and the environment stores in its state the fact that they have halted.
• signal. Following the above transition, a concrete transition via the global-synchronous action signal is enabled. The transition causes the type 2 robots to halt.

The systems SMR, SGS, SFE
Because of their importance with respect to their amenability to verification, we now identify three noteworthy classes of PIIS. The classes correspond to different combinations of template actions. They are defined as follows.
SMR is the class of PIIS generated from agent templates defined only on asynchronous, agent-environment, and multirole actions.
Decentralised systems may be encoded in SMR using the machinery of multi-role actions, whereas centralised systems can be represented in SMR using the communication primitive of agent-environment actions. As a result, the SMR class is particularly suitable for modelling swarm robotics, which are naturally decentralised systems, but interacting with their environment [58].
SGS is the class of PIIS generated from agent templates defined only on asynchronous, agent-environment, and globalsynchronous actions.

SGS
This class can represent broadcast protocols [37], cache coherence protocols, swarm aggregation algorithms in a grid environment, and several scenarios where synchronous handshaking is required.
SFE is the class of PIIS generated from agent templates defined only on asynchronous, role-synchronous, and globalsynchronous actions.
The absence of agent-environment actions implies that all the agents evolve in the same way following synchronisation with the environment. Differently from the SMR and SGS classes, the PMCP for this class is, as we will show, decidable. This gives clear advantages when protocols can be expressed by SFE. An example of an SMR system is the robot foraging scenario discussed in Section 2.3.1, an example of an SGS system is the train-gate-controller described in Section 2.3.2, an example of an SFE system is the autonomous robot example given in Section 2.3.3. We will study the SMR, SGS, and SFE classes in detail in Section 5, Section 6, and Section 7, respectively.

The parameterised specification language indexed ACTL * K \ X
We verify parameterised MAS against properties expressed in indexed ACTL * K \ X . This logic extends ACTL * K \ X [13] by introducing indexed atomic propositions and indexed epistemic modalities. As we describe below indices enable us to express properties irrespectively of the number of agents present. We first recall ACTL * K \ X ; we then introduce indexed ACTL * K \ X . This is followed by the definition of a notion of simulation between concrete systems and an analysis on the preservation of logical satisfaction between similar systems.
3.1. ACTL * K \ X ACTL * K \ X is a temporal-epistemic logic combining the epistemic logic S5 with the temporal logic ACTL * \ X , the universal fragment of CTL * without the next time operator X . Note that restrictions on the specification language are typically assumed in parameterised verification given the problem's general undecidability. It is known that if the language can express the number of agents in the system, then the parameterised verification problem is undecidable [51]. The next operator is therefore excluded to accommodate this [62,46,51]. We further restrict the language to universal path quantification to establish the behavioural equivalence results required, as presented in Section 5, Section 6, and Section 7.
Given sets of atomic propositions AP 1 , . . . , AP k for each agent template and a set A(n) of concrete agents, the state and path formulae of ACTL * K \ X are defined by the following BNF expressions: where φ and ψ are state and path formulae, (i, j) ∈ A(n) (1 ≤ i ≤ k, 1 ≤ j ≤n(i)), and p ∈ AP i . The knowledge modality K j i stands for "agent j of role i knows that"; the path quantifier A is read "for all paths"; the temporal operators U and R denote the "until" and "release" modalities. Formulae expressed in ACTL * K \ X are interpreted on a model S(n) as standard [6]: the temporal modalities are interpreted by means of the global transition relation, and the epistemic modalities are interpreted by the respective epistemic accessibility relations. We write (S(n), g) |= φ ((S(n), π) |= φ respectively) to mean that a state formula (path formula respectively) is true at a state g (path π respectively) in S(n). If S(n) is clear, then we simplify the notation to g |= φ (π |= φ respectively).

Definition 3.1 (Satisfaction).
Given a model S(n), the satisfaction relation |= is inductively defined as follows.
The customary abbreviations of truth and falsity are assumed: p ∨ ¬p, ⊥ p ∧ ¬p, for some p ∈ AP i and 1 ≤ i ≤ k. Further we define F φ U ( , φ) with the usual meaning of "Eventually φ", and Gφ R(⊥, φ) standing for "Always φ".
To establish the correctness of a system irrespectively of the number of agents present, we express properties that reflect its parameterised nature. In other words, we are interested in expressing collective behaviours for the system under consideration; this interest corresponds, for example, to emergent behaviours [63] in swarm-based systems. Such properties are expressible by introducing indexed atomic propositions and indexed epistemic modalities. In particular, the atomic propositions and epistemic modalities appearing in a formula are indexed with variables instead of the identities of the concrete agents. Then, given an arbitrary concrete system, the variables in a formula are quantified over the concrete agents in the system. This gives an ACTL * K \ X formula which can be evaluated on the concrete system by means of Definition 3.1.
Let VAR = VAR 1 ∪ . . . ∪ VAR k be the union of disjoint sets of variable symbols, where each VAR i is associated with role i, and recall that AP 1 , . . . , AP k are disjoint sets of atomic propositions, one for each template role. The state and path formulae of indexed ACTL * K \ X are defined as the state and path formulae of ACTL * K \ X , but built from template atomic propositions, and with each proposition p ∈ AP i and epistemic modality K i (note that only the template role is specified in K i ) indexed by a variable v ∈ VAR i . The domain of a variable v ∈ VAR i appearing in a formula φ is defined by the concrete system on which φ is evaluated: if φ is evaluated on S(n), then the potential set of values for v is {1, . . . ,n(i)}. We The properties we use in the verification of PIIS are constructed in compliance with the following formula schema: where ∀ is a universal quantifier over the variables. We denote such a formula by ∀v φ(v).
When evaluated on a concrete system, an m-indexed formula ∀v φ(v) denotes a specific formula in ACTL * K \ X corresponding to the conjunction of all ground instantiations of ∀v φ(v). Given a concrete system S(n) (n ≥m) 1 and an injective function Using the above schema we may now express properties independently of the number of agents as the following examples illustrate.
Example 3.2. In Section 2.3.1 we encoded the robot foraging scenario as a PIIS composed of a template agent TR representing the robots and a template agent TFS representing the food sources. TFS was built from two states expressing whether or not a food source has been found by a robot. Having modelled the scenario as a PIIS we are interested in checking the property: "whenever a food source is found, every robot knows that the source is found". This property can be expressed by the following (1, 1)-indexed formula: where u is variable of TR, x is a variable of TFS and the atomic proposition f holds in the template state in which the template food source is "Found". When evaluated on a concrete system with 2 robots and 2 food sources φ RFS is a shortcut for the ACTL * K \ X formula We show how to check φ RFS against all possible concrete systems in Section 5.
Example 3.3. In Section 2.3.2 we encoded the train-gate-controller as a PIIS composed of an agent template PT representing prioritised trains, an agent template representing NT representing normal trains, and a template environment representing the controller. A commonly used benchmark concerns assessing the correctness of the train-gate-controller against the property "whenever a train is in the tunnel, it knows that no other train is in the tunnel at the same time". This property can be expressed in indexed ACTL * K \ X by the following (2, 2)-indexed formula: where u, v are variables of PT, x, y are variables of NT, the atomic proposition pt holds in the template states in which the template prioritised train is in the tunnel and the atomic proposition nt holds in the template states in which the template normal train is in the tunnel. We show how to check φ TGC against all possible concrete systems in Section 6.
Example 3.4. In Section 2.3.3 we encoded the autonomous robot example as a PIIS composed of an agent template TR1 representing robots with access to a sensor, an agent template TR2 representing robots with no access to a sensor, and an environment template representing the environment moving the robots forward along the track. According to the scenario the goal of the robots is to halt in the goal region {2, 3, 4} of the track. A solution to the problem in the single-robot case is for the robot to do nothing while the value of its sensor is less than 3 and to halt once the value of its sensor is greater than or equal to 3. We are interested to show the correctness of the autonomous robot solution in the unbounded case, i.e., whenever the robots halt, they know that they are in the goal region. This is expressed by the following (1, 1)-indexed formula: where v is a variable of TR1, u is a variable of TR2, the atomic proposition gr_1 (gr_2, respectively) holds in the template states where the value of the position component of template robot 1 (template robot 2, respectively) is in {2, 3, 4}, and the atomic proposition h_1 (h_2, respectively) holds in the template states where the template robot 1 (template robot 2, respectively) has halted. We show how to check φ AR against all possible concrete systems in Section 7.

Symmetry reduction for ACTL * K \ X
Consider an m-indexed formula ∀v φ(v). As noted above, the evaluation of the formula on a concrete system S(n) corresponds to evaluating the conjunction of all its ground instantiations. But the conjuncts are identical up to re-indexing of the agents' indices. Thus the symmetric nature of ∀v φ(v) suggests that its evaluation on a concrete system is equivalent to checking only one of its ground instantiations. The following Lemma, adapted from [64], shows precisely this by taking (for simplicity) the aforementioned ground formula to be the trivial instantiation.
Example 3.5. The following formulae are the trivial instantiations of φ RFS , φ TGC and φ AR , respectively.
Lemma 3.6 (Symmetry reduction). Given a PIIS S composed of k roles, an m-indexed formula ∀v φ(v) and n ≥m, we have that For each 1 ≤ i ≤ k, consider ζ i to be either a bijective mapping from natural numbers to natural numbers, or a bijective mapping from global states to global states, or a bijective mapping from actions to actions, or a bijective mapping from  3), l E ) is mapped to the state ((t, 1), (u, 2), (s, 3), l E ).
i is an asynchronous action or an agent-environment action of agent (i, j), then ζ i ((a, j)) = (a, ζ i ( j)); • if a ∈ RS i ∪ GS is a role-synchronous or a global-synchronous action of agent (i, j), then ζ i (a) = a; • if (a, ( j, r)) ∈ MR i,q is a multi-role action of agent (i, j) that is shared with an agent (q, r), then ζ i ((a, ( j, r))) = a, ζ i ( j), ζ q (r) .
. But as all concrete agents share a unique template local state in ι(n), it follows that ζ(ι(n)) = ι(n). Consequently, (S(n), ι(n)) |= φ[ξ ]. As ξ was arbitrary, the latter concludes S(n) |= ∀v φ(v). 2 The above will enable us to reduce the size of the formulae to check when we introduce the parameterised model checking problem in Section 4.

Stuttering simulations
As noted above, cutoff techniques concern the identification of a cutoff, the number of agents in a concrete system that can be used to check whether a given property holds for all concrete systems. A notion of equivalence between the system instances is often used to show this result. Stuttering insensitive logics (i.e., logics without the next-time operator) are accompanied with the standard notion of stuttering simulation [65]. A system stuttering simulates another system if for every behaviour of the latter, there is a stuttering equivalent behaviour of the former. Informally, two behaviours are stuttering equivalent if the behaviours coincide when each sequence of stutter steps (i.e., steps that do not affect the labelling of the states) are collapsed onto a single step. Since we consider only universal path quantification, it follows that any ACTL * \ X formula satisfied by the simulating model is also satisfied by the simulated model. We here define an epistemic variant of the notion of stuttering simulation between concrete systems. We show that similar systems preserve ACTL * K \ X specifications. To do this, we fix throughout this section a PIIS S of k roles and two concrete instantiations is a stuttering simulation between S(n) and S(n ) if ι(n) ∼ ss ι(n ) and whenever g ∼ ss g we have the following.
(iii) For every π ∈ (g), there is a π ∈ (g ), a partition B 1 , B 2 , . . . of the states in π , and a partition B 1 , B 2 , . . . of the states in π such that for each j ≥ 1, B j and B j are nonempty and finite, and every state in B j is related by ∼ ss to every state in B j .
We say that a model S(n ) stuttering simulates a model S(n), denoted S(n) ≤ ss S(n ), if there is a stuttering simulation relation between S(n) and S(n ). If S(n) ≤ ss S(n ), then S(n ) |= φ implies that S(n) |= φ [13]. Following Lemma 3.6, since we will only be checking a concrete system against a formula's trivial instantiation, the above simulation conditions on the atomic propositions and epistemic modalities only need to refer to agents in A(m). Therefore we relax the notion of stuttering simulation to one of m-stuttering simulation.

Definition 3.8 (m-stuttering simulation).
Given m ∈ N k , an m-stuttering simulation is a relation ∼m ss ⊆ G(n) × G(n ) (n, n ≥m) between S(n) and S(n ) if it is defined as in Definition 3.7, but replacing conditions (i) and (ii) with the following.
, then g ∼ j i g 1 for some g 1 such that g 1 ∼m ss g 1 .
We say that a concrete system S(n )m-stuttering simulates a concrete system S(n), denoted S(n) ≤m ss S(n ), if there is an m-stuttering-simulation relation between S(n) and S(n ). The following shows that m-indexed formulae are preserved under m-stuttering simulation. Theorem 3.9. Let S(n) ≤m ss S(n ) for a given m ∈ N k such that n ≥m, n ≥m. If S(n ) |= ∀v φ(v), then S(n) |= ∀v φ(v), for any m-indexed formula ∀v φ(v).
Proof. Fix an m-indexed formula ∀v φ(v). By Lemma 3.6 it suffices to show that S(n ) |= φ[trivial] implies S(n) |= φ [trivial]. Thus note that ACTL * K \ X formulae are known to be preserved under stuttering simulation [13]. Then note that atomic propositions and epistemic modalities in φ[trivial] refer only to agents in A(m). The latter two observations entail S(n) |= φ([trivial]). 2 The result above will enable us to access the truth of a specification on an arbitrarily big system by checking it on a smaller system. To do this, however, only conditions (i) and (iii) of Definition 3.8 have to be considered. Indeed, as the following shows, in the context of fully symmetric parameterised systems, the notion of stuttering simulation preserving temporal formulae [65] and the notion of stuttering simulation preserving temporal-epistemic formulae [13] coincide. Proposition 3.10. Fix m ∈ N k and n ≥m, n ≥m. If ∼ x ⊆ G(n) × G(n ) is a relation between S(n) and S(n ) that satisfies conditions (i) and (iii) of Definition 3.8, then ∼ x is an m-stuttering simulation between S(n) and S(n ).
Proof. Suppose the LHS of the thesis. We show that ∼ x is an m-stuttering simulation between S(n) and S(n ). By assumption on ∼ x we only have to show simulation requirement ii. Let g ∼ x g and suppose that g ∼ j i g 1 for some (i, j) ∈ A(m). We show that there is a g 1 ∈ G(n ) with g ∼ j i g 1 and g 1 ∼ x g 1 . To show this, consider a path π ∈ (ι(n)) such that π(z) = g 1 for some z ≥ 1. As ι(n) ∼ x ι(n ), there is a path π ∈ (ι(n )) satisfying condition (iii) of m-stuttering simulation. Thus π(z) ∼ x π (z ) for some z ≥ 1. Therefore ls j i (π (z)) = ls j i (π (z )) as ∼ x satisfies condition (i) of m-stuttering simulation. Consequently g ∼ j i π (z ). So g 1 = π (z ) is as required. Since (g, g ) was arbitrary we obtain S(n) ≤m ss S(n ). 2

Parameterised model checking problem and MAS cutoffs
We now proceed to state the parameterised model checking problem.
If the above holds, then ∀v φ(v) is said to be satisfied by S and this is denoted by S |= ∀v φ(v).
In other words, differently from the standard model checking problem, the PMCP involves establishing whether a specification is satisfied on an unbounded number of systems resulting from the instantiations of the agent templates. The problem is known to be undecidable in general [18]. Following this, restrictions are typically imposed on the systems or the properties to be checked thereby obtaining decidable fragments [16,31,46,51].
In this paper we study the SMR, SGS, and SFE classes of systems and for each of these we put forward a parameterised model checking procedure. The common theoretical notion underlying all three procedures is the concept of MAS cutoffs. This reflects the key observation that to solve the PMCP it is at times sufficient to consider a finite number (the cutoff) of concrete systems. While theoretically cutoff identification procedures seldomly provide complete approaches to the parameterised model checking problem, there is empirical evidence that errors often manifest themselves on small systems with a small number of participants [46,47,19,16,17,49]. We now give a formal definition of MAS cutoffs, and we show that no cutoff-based model checking procedure exists in general. We will then focus our analysis on the SMR, SGS and SFE classes of systems in Section 5, Section 6 and Section 7, respectively.
A cutoff for a system is the number of components that is sufficient to consider when evaluating a given specification. for any m-indexed formula ∀v φ(v). We say that S admits a cutoff for m-indexed formulae and we call S(c) the cutoff system.
By definition, if a cutoff exists, then the PMCP can be reduced to model checking all concrete systems up to the cutoff system. However the following shows that cutoffs do not generally exist. Proof. We show that the PIIS S specified in Fig. 7 does not admit a cutoff. S is composed of one agent template that is defined on the agent-environment actions a, b, d and the asynchronous action e. For contradiction suppose that S admits a cutoff. Then there is a c ≥ 1 such that S(c) We inductively construct an 1-indexed γ c such that S(c) |= γ c and S(c + 1) |= γ c thereby contradicting our assumption.
To do this let p_ι, p_s, p_u and p_t be atomic propositions that are only true in templates states ι, s, u and t, respectively. Now inductively define a string δ c as follows: Finally define γ c as the following 1-indexed formula.
where ι(c + 1) is the initial global state in S(c + 1) and each (x, i) is a concrete action of agent i. In the path, for 2 ≤ i ≤ c, each sequence (a, i)(d, 1)(e, 1) of actions corresponds to one loop of agent 1 through the cycle (ι, u, ι); the suffix (a, c + 1)(b, 1) results in agent 1 moving to state t.
We have thus shown that S does not admit a cutoff for 1-indexed formulae. 2 Theorem 4.3 applies for the SGS and SMR classes of systems to give that cutoffs for these classes do not generally exist. Having established the negative result above, we now proceed to exploit some properties of the SMR and SGS classes. The properties follow the existence of a notion of simulation between the agent and environment templates. This assumption will enable us to devise a simple and efficient test to check whether cutoff exists for a given system and specification. We do this for the SMR class in Section 5 and for the SGS class in Section 6. In Section 7 we show that cutoffs always exist for SFE systems.

Verifying SMR systems
Following the negative result on the existence of cutoffs in the general case, we now analyse each of the SMR, SGS and SFE classes. For each of the subclasses we put forward a parameterised model checking procedure. We use the procedures to verify examples from the MAS literature in a parametric setting, including the robot foraging scenario (Section 2.3.1), the trains-gate-controller (Section 2.3.2) and the autonomous robot example (Section 2.3.3).
We begin with the SMR class of systems defined on asynchronous, agent-environment and multi-role actions. Given Theorem 4.3 cutoffs do not generally exist for this class. However, we identify a notion of simulation, and show that the existence of this simulation between the agent and environment templates guarantees the existence of a cutoff, which we show how to calculate given a PIIS and a specification. This will enable us to define a model checking procedure which we will exemplify on the robot foraging scenario following a discussion on its applicability.

Agent-environment simulation
We introduce a notion of agent-environment simulation between the agent templates and the environment template.
We fix a PIIS S = T , E, V composed of k agent templates T = {T i = L i , ι i , Act i , P i , t i : 1 ≤ i ≤ k} and an environment template E. For a template T i we write l a l to mean that t i (l, a) = l . Given a set of template actions X ⊆ Act i we let l X l to denote that l a l for some a ∈ X . The reflexive and transitive closure of X is denoted by X * . In a similar manner, for a concrete system S( , V(n) we write g → a g to denote that (g, g ) ∈ R(n) by means of action a. For a set of actions X ⊆ (i, j)∈A(n) Act j i , g → X g expresses that g → a g for an action a ∈ X . The reflexive and transitive closure of → X is denoted by → X * . To simplify our analysis in the following we assume without loss of generality that for each action a ∈ Act E we have that |{l ∈ L E : a ∈ P E (l)}| = 1; i.e., an environment's action is enabled at exactly one template state. Note that any PIIS can be translated into a PIIS for which each action of the environment is enabled at exactly one template state. The translation is given in two steps. It is easy to see that a PIIS S and its translation S given above are equivalent in that every concrete system S(n) can simulate S (n) and vice versa. S is bigger than S by a polynomial factor in the number of transitions of the environment template.  Intuitively, the existence of an agent-environment simulation between the agent and environment templates restricts the environment to one of shared resources, where each resource is accessible by exactly one agent at a given time. Formally, there is an agent-environment simulation between agent template T i and E if E can simulate T i only by means of the template states in which an action shared with the environment is enabled (see Fig. 8). 2

Definition 5.1 (Agent-environment simulation). A relation ∼ aes ⊆ L i × L E is an agent-environment simulation between T i and E
if ι i ∼ aes ι E and whenever l i ∼ aes l E the following condition holds: if l i We write T i ≤ aes E to denote that there is an agent-environment simulation between T i and E. We write T ≤ aes E if T i ≤ aes E for all T i ∈ T . Intuitively, if T ≤ aes E, then an agent can always take the lock on a resource by synchronising with the environment via an agent-environment or a multi-role action. As we show below, following this synchronisation the agent is the only agent that may synchronise with the environment. The agent releases the resources whenever the environment performs a loop in which case other agents can synchronise with the environment.

Definition 5.2 (Environment loop).
A subsequence g i a i . . . g j , j > i > 0, of a path g 1 a 1 . . . in S(n) is an environment loop if ls E (g i ) = ls E (g j ).
So if T ≤ aes E, then the concrete environment conforms to a certain looping behaviour whenever it synchronises between different agents. As we show below, an environment loop occurs whenever the environment synchronises between two different agents in successive synchronisations. If g is a global state occurring in a path from the first synchronisation to the latter, then we say that g has the environment loop condition.

Definition 5.3 (Environment loop condition).
A global state π(d) in a path π in S(n) has the environment loop condition, denoted ELC(π (d)), if there is a pair of agents (i, r) and ( j, q) with (i, r) = ( j, q) such that the following hold: In other words a global state g in a path π has the environment loop condition if: (i) there is an agent who lastly synchronised with the environment earlier in the path; (ii) there is a different agent who firstly synchronised with the environment later in the path (see Fig. 9).
Example 5.4. Consider the following path of the concrete Train-Gate-Controller with two prioritised trains and two normal trains (Fig. 5).
The states (A 1 , W 2 , PG, L 1 , L 2 ) and (A 1 , A 2 , PG, L 1 , L 2 ) have the environment loop condition, whereas the other states do not.
We now show that the environment loop condition is a sufficient condition for the occurrence of an environment loop. Specifically, we show that whenever ELC(g) holds, the environment's local state in g is equal to its initial local state. Thus, intuitively, whenever ELC(g) holds, an agent releases the lock on a shared resource to a different agent.
Lemma 5.5. Consider a PIIS S = T , E, V ∈ SMR with T ≤ aes E and a path π in S(n). If ELC(π (d)) for some d > 1, then [d]π is an environment loop.
Proof. Assume that ELC(π (d)) for some d > 1. This means that there is a pair of agents (i, i ), ( j, j ) with (i, i ) = ( j, j ) such that agent (i, i ) is the last agent to synchronise with the environment in the prefix [d]π of π , and agent ( j, j ) is the first agent to synchronise with the environment in the suffix π[d] of π . We show that ls E (π (1)) = ls E (π (d)) by induction on d.
Suppose d = 2. Consider the template action a from which the concrete action π(1, Act) has been instantiated. Observe that a ∈ AE i ∪ MR i . Therefore tls j j (π (1)) A j * tls j j (π (d − 1)) a tls j j (π (d )) for some d ≥ 2. As T ≤ aes E, the latter gives ls E (π (1)) a l E for a template state l E with tls j j (π (d )) ∼ aes l E . Since the set {l ∈ L E : a ∈ P E (l)} is singleton, it follows that ls E (π (1)) = ls E (π (d − 1)). Additionally, ls E (π (2)) = ls E (π (d − 1)) as the subsequence π(d), . . . , π(d − 1) of π does not contain agent-environment or multi-role actions. Therefore ls E (π (1)) = ls E (π (2)). So the claim is true of paths with two states. Suppose that it is true for all paths with at most x − 1 states for x ≥ 3. Let π be a path with x states. We show that ls E (π (1)) = ls E (π (x)) in terms of two cases.
Case 1: [x]π does not contain any AE j j action or any MR j j action. In this case we can proceed with the same argument used in the base step and conclude that ls E (π (1)) = ls E (π (x)). have l E = ls E (π (q)) since π(q − 1, Act) is enabled by the environment's protocol at exactly one template state. Therefore tls j j (π (q)) ∼ aes ls E (π (q)). From the latter observation and the observation that tls j j (π (q)) A j * tls j j (π (d − 1)) for some d ≥ x, the base step's argument applies to conclude that ls E (π (q)) = ls E (π (x)). It is left to show that [q]π is an environment loop thereby showing that [x]π is an environment loop. To show this, note that there is an agent (r, r ) = ( j, j ) which either performs an agent-environment or a multi-role action in the path sequence π(q), . . . , π(x). Therefore ELC(π (q)). So ls E (π (1)) = ls E (π (q)) holds by the inductive hypothesis. 2 The above lemma reports a key consequence of the agent-environment simulation assumption. This will be central to the behavioural equivalence results shown in the next section. Lemma 5.5 can be interpreted as stating that the environment implements a mutual exclusion controller governing the access to shared resources.

Model checking procedure for SMR systems
The model checking procedure Check_SMR for SMR systems is defined by Algorithm 1. Given a PIIS S ∈ SMR and an m-indexed formula ∀v φ(v), the procedure first establishes whether or not T ≤ aes E. Upon a successful simulation test the procedure calculates the cutoff for the given system and specification.  The cutoff function cutoff _SMR maps a PIIS S of k roles and a k-tuple m of natural numbers into a k-tuple c of natural numbers that corresponds to the cutoff for m-indexed formulae. Intuitively, to calculate a cutoff, we require that each agent in the cutoff system, referred to by the atomic propositions and epistemic modalities in the trivial instantiation of an m-indexed formula, is able to make any transition that the agent can make in any bigger system. This will enable us to establish that the cutoff system m-stuttering simulates every bigger system. By Theorem 3.9 we can then conclude that if the cutoff system satisfies a formula then the formula is satisfied by every bigger system.
Consider an agent (i, j) indexing an atomic proposition or an epistemic modality in the trivial instantiation of an m-indexed formula and a transition by means of a multi-role action of agent (i, j) that is guarded by agent (r, q). As discussed in Section 2.2, the enabling of the action depends on whether or not the action is enabled by the protocols at the current local states of the agents and the environment. If the action is enabled, then the agent (i, j) may update its state upon performing the action, whereas the agent (r, q) remains in its current local state. Obviously, agent (r, q) may not be present in cutoff system, as the index q can be arbitrarily large. Still, we can simulate the transition in the cutoff system by insisting on the presence of an agent, say (r, q ), in the local state of agent (r, q). Then, the instantiation of the multi-role action that is shared between the agents (i, j) and (r, q ) is enabled, and so the agent (i, j) is able to simulate the transition of the bigger system by performing said instantiation. To simulate any multi-role action performed by agent (i, j), we insist on the presence of an agent in every local state enabling a multi-role action. In other words, for each agent template, in addition to the number of agents that need to be simulated, we require that the cutoff system composes as many agents as the cardinality of the set of template states that enable a multi-role action guarded by the template. We call this set the action dependency set of the template. Definition 5.6 (Action dependency set). Given a PIIS S = T , E, V ∈ SMR, the action dependency set D i ⊆ L i of an agent template T i is a subset of template T i 's states that is defined as follows: So, each action dependency set D i reflects the states of template T i that enable at least one multi-role action shared with another template. The cutoff function is defined in terms of m and the action dependency sets for the agent templates. for any S ∈ SMR with k ≥ 1 roles and any m ≥ N k .
Following the cutoff calculation, Check_SMR checks the set {S(x) :m ≤x ≤c} of concrete systems against the trivial instantiation φ[trivial] of ∀v φ(v). If φ[trivial] is not satisfied by at least one system, then the procedure returns false, otherwise it returns true. We assess the soundness of the Check_SMR procedure in Section 5.4. First, we exemplify it by means of the robot foraging scenario.

Verifying the robot foraging scenario
In Section 2.3.1 we represented the robot foraging scenario as a PIIS S RFS composed of an agent template TR encoding the robots and an agent template TFS encoding the food sources. In Section 3.2 we expressed the property "whenever a food source is found, every robot knows that the source is found" in the following (1, 1)-indexed formula: where u is variable of TR, x is a variable of TFS and the atomic proposition f holds in the template state in which the template food source is "Found".
Observe that model checking S RFS against φ RFS amounts to checking each concrete system S RFS (n) for any n ≥ (1, 1). Since the number of systems involved is unbounded the problem cannot be solved by traditional model checking techniques. In contrast, we now show how S RFS can be verified using the Check_SMR procedure. Clearly S RFS satisfies the agent-environment simulation assumption as the system does not specify an environment template. Therefore we may proceed to compute the cutoff function cutoff _SMR(S RFS , (1, 1)). To do this we first calculate the action dependency sets. We have that D TR = {MF} and D TFS = {N_F , F } for TR and TFS, respectively. Hence, c = cutoff _SMR(S RFS , (1, 1)) = (1 + 1, 1 + 2) = (2, 3) Thus, we need to check whether or not S RFS (x) |= φ RFS [trivial], for (1, 1) ≤x ≤ (2, 3). These checks can be performed on a standard model checker; the result is true thereby establishing the correctness of the protocol irrespectively of the number of robots and the number of food sources.

Soundness of the Check_SMR procedure
We can now assess the soundness of the Check_SMR procedure. To prove this result we need to establish some intermediate steps. Firstly observe that by the definition of the PMCP and Theorem 3.9, it suffices to show that if T ≤ aes E, then: (i) the cutoff system S(c)m-stuttering simulates every bigger system; (ii) every bigger system m-stuttering simulates S(c). As related states in each of the simulations need only to agree on the atomic propositions indexed by the agents in A(m), we only have to simulate transitions taken by the agents in A(m). We first show the former.

Part A: the cutoff system m-stuttering simulates every bigger system
Consider an arbitrarily big system S(n) with n ≥c, where S(c) is  ((i, j)). Thus, S(c) can mimic the multi-role transitions of the agents in A(m) in S(n) as follows: for each multi-role action (b, (r, q)) ∈ MR r,q i, j shared by the agents (i, r) ∈ A(m), ( j, q) ∈ A(n) that is performed at a global state g in S(n), the multi-role action (b, (r, q )) shared by the agents (i, r) and ( j, q ), where q = λ j tls q j (g) is performed in S(c).
We now give a formal definition of the action dependency path described above. The path is inductively defined so that at each step an agent (i, j) ∈ A(c) \ A(m) moves to its associated state λ −1 i ((i, j)). Following Lemma 5.5, however, the definition insists on the occurrence of an environment loop at each step. If an agent cannot move to its associated state while the environment performs an environment loop, then said agent remains in its initial state. This is because the environment is locked on synchronising with exactly one agent unless an environment loop occurs.
Definition 5.9 (Action dependency path). Let S = T , E, V ∈ SMR be a PIIS of k roles. Consider D = D 1 ∪ . . . ∪ D k to be the union of the action-dependency sets. For a global state g, assume D(g) = l ∈ D : there is an agent (i, r) with tls r i (g) = l to be the set of template states in D that appear in g. Then, for m ∈ N k and c = cutoff _SMR(S, m), the action dependency path p in S(c) is inductively defined as follows.
be a set of finite paths such that π ∈ X if and only if π satisfies the following conditions. i. π is an environment loop; ii.
iii. Every action occurring in π is in A q q) is the agent associated with l.
If X = ∅, then p i+1 = p i . Otherwise, p i+1 = p i • π for an arbitrary π ∈ X . Example 5.10. Consider the robot forging scenario discussed in Section 5.3, where the action dependency sets were calculated to be D TR = {MF} for the template robot, and D TFS = {N_F , F } for the template food source. The cutoff was shown to be c = (2, 3) for (1, 1)-indexed formulae. We now construct the action dependency path. To do this, we first associate the concrete agents with the action dependency sets by the mappings: So, the concrete robot 2 is associated with the state MF, the concrete food source 2 is associated with the state N_F , Clearly the state of the environment in S(n) may be updated after the environment synchronises on the a 2 , . . . , a x−2 actions. Even so, the environment in S(c) may still synchronise on the a x−1 action after performing the null actions. This is because both g 2 and g x−1 satisfy the environment loop condition. Therefore, the state of the environment in g 2 is the same to its state in g x−1 . Intuitively, whenever an agent not in A(m) in S(n) takes the lock on a resource via synchronising with the environment, it has to release the resource via the occurrence of an environment loop before an agent in A(m) can take the lock on a resource. Proof. Let n ≥c be arbitrary. We show that S(n) ≤m ss S(c). The simulation relation we define follows Fig. 10. This ensures that whenever an action is performed by an agent in S(n) that is required to satisfy condition (i) of m-stuttering simulation, the agent can also perform the action in S(c). In line with this, the state of the environment is defined to allow the actions to be performed. The simulation relation ∼m ss = ( (g, g ) ∈ R 1 iff ls r i (g) = ls r i (g ), for (i, r) ∈ A(m) The above ensures that the local states of the agents in A(m) are the same in the related global states of S(n) and S(c). The following ensures that the agents in A(c) \ A(m) in S(c) are in their associated local states as per the action dependency path.
-related states must also agree on the state of the environment. Assume (g, g ) ∈ G(n) × G(c). Suppose that there is a state g 1 reachable from g at which an agent-environment or a multi-role action of the agents in A(m) is enabled.
Clearly for the action to be enabled, the environment is in a local state at which the environment's protocol enables the action. Further assume that g 1 can be reached by asynchronous actions and by actions of agents not in A(m). As previously discussed, S(c) simulates these actions by performing asynchronous and null actions. These do not change the state of the environment in S(c). So, we define R 3 to insist on the equality of the environment's states in g and g 1 as S(c) simulates the agent-environment or the multi-role action enabled at g 1 by performing the same action.
(g, g ) ∈ R 3 iff g → X * g 1 → a g 2 implies that ls E (g ) Act r i Finally, to simulate the multi-role action in S(n), S(c) first executes the action dependency path. R 4 suggests exactly this.
(ι(n), g) ∈ R 4 iff g appears in p We show that ∼m ss is an m-stuttering simulation between S(n) and S(c). As ι(c) = p(1), we have that (ι(n), ι(c)) ∈ R 4 , therefore ι(n) ∼m ss ι(c). Now assume that g ∼m ss g for an arbitrary pair of global states in G(n) × G(c). We show the simulation requirements (i) and (iii). We have that (g, g ) ∈ R 1 ∩ R 2 ∩ R 3 or (g, g ) ∈ R 4 . The first simulation requirement follows trivially by the definition of R 1 and by observing that each agent in A(m) remains in its initial local state in p.
To show the simulation requirement (iii), let π ∈ (g). We inductively construct a path π ∈ (g ) as required by the m-stuttering-simulation.
For the base step, we have two cases: • (g, g ) ∈ R 4 . Then, there is a q ≥ 1 such that g = p(q). Consider π = p[q]. Define the first blocks B 1 , B 1 by B 1 = g and In this case, define π = g , B 1 = g, and B 1 = g .
For the inductive step, assume that we have already constructed a prefix [x]π , a prefix [x ]π , and a partition of the states in [x]π and [x ]π into corresponding blocks. We now define [x + 1]π and the next blocks B x and B x by considering cases on the type of the action π(x, Act). , (r, q )), where q = λ j (tls q j (π (x))); • Otherwise, π (x , Act) = .
Let B x = π(x + 1) and B x = π (x + 1). We have to show that π (x ) → π (x ,Act) π (x + 1) is a valid transition. The case follows trivially when π (x , Act) is an asynchronous action or when π (x , Act) is the joint null action. If π (x , Act) is an agent-environment action, then we have to show that the state of the environment enables the action in π (x ). If π (x , Act) is a multi-role action, say π (x , Act) = (a, (r, q )) ∈ MR r,q i, j , then we additionally have to show that the state of the agent ( j, q ) enables the action as well. So, it suffices to show the case for multi-role actions. That is, we need to show: (i) a ∈ P E (l E ), where l E = ls E (π (x )); (ii) a ∈ P j (l), where l = tls q j (π (x )). We show the former by means of two cases.
2. For all y with 1 ≤ y ≤ x − 1 we have that π(y, Act) / ∈ (i,r)∈A(m) AE r i ∪ MR r i . We have two cases.
For the second case assume that (g, g ) / ∈ R 3 . Then it must be the case that (g, g ) ∈ R 4 . It follows that ls E (g) = ι E . By construction of π , ls E (π (x )) = ι E . We show that ls E (π (x)) = ι E is also the case. If there is no AE or MR action in [x]π , then the case follows trivially. Otherwise, we deduce that ELC(π (x)), hence [x]π is an environment loop. Therefore, ls E (π (x)) = ι E . Thus, ls E (π (x)) = ls E (π (x )). Hence, a ∈ P G (l E ).
To show the latter, let ρ in S(n) be a finite path of length len such that ρ(len) = g and consider the concatenation γ = ρ • π of ρ and π . If there is a y with 1 ≤ y ≤ x + len such that ELC(γ (y)), then let z be the greatest integer such that ELC(γ (z)); otherwise, let z = 1. As there is no state in the path γ (z + 1), . . . , γ (len + x) that satisfies the environment loop condition, and as γ (len + x, Act) = (a, (r, q)), a ∈ MR r,q i, j for some ( j, q) ∈ A(n), it follows that only agent (i, r) Fig. 11. Them-stuttering simulation of the cutoff system S(c) by a bigger system S(n).
It follows that simulation requirement (iii) is satisfied. As (g, g ) was arbitrary, we have that S(n) ≤m ss S(c). 2 Lemma 5.11 concludes part A of the proof for Theorem 5.8: the cutoff system m-stuttering simulates every bigger system. The following constitutes part B of the proof: every bigger system m-stuttering simulates the cutoff system. Fig. 11 shows the m-stuttering simulation between S(c) and S(n). S(n) performs precisely every action performed by S(c) while it lets the agents not in A(c) stutter at their initial states. Proof. Choose an arbitrary n ≥c. We show that S(c) ≤m ss S(n). We do this by letting each additional agent in S(n) stutter at its initial state. Then, S(n) can simulate S(c) by performing the same actions performed in S(c).

Part B: every bigger system m-stuttering simulates the cutoff system
Define R 1 , R 2 , R 3 ⊆ G(c) × G(n) to be the following relations.
(g, g ) ∈ R 1 iff ls r i (g ) = ls r i (g), for (i, r) ∈ A(c) (g, g ) ∈ R 2 iff tls r the information encoded in a state in S(c) is present in a related state in S(n) by R 1 ; the additional agents in S(n) are left stuttering at their initial states by R 2 ; the environment's states are equal in every pair of related states by R 3 .
We show that ∼m ss is an m-stuttering simulation between S(c) and S(n). It should be clear that ι(c) ∼m ss ι(n).
To show the simulation requirements (i) and (iii), assume that g ∼m ss g for an arbitrary pair of global states in G(c) × G(n). Requirement (i) follows by definition of R 1 . To show the requirement (iii), let π ∈ (g). Define a path π by g π(1, Act), π(2, Act), . . .. Clearly, π ∈ (g ) and π( j) ∼m ss π ( j), for all j ≥ 1. Thus, define B 1 , B 2 , . . . and B 1 , B 2 , . . . to be a partition of π and π into singleton blocks. It follows that S(c) ≤m ss S(n). 2 Proof. Let n ≥c be arbitrary. By Theorem 3.9 it suffices to show that S(n) ≤m ss S(c) and S(c) ≤m ss S(n). The former is given by Lemma 5.11, and the latter is given by Lemma 5.12. 2 This concludes the proof of Theorem 5.8. Corollary 5.13 provides a constructive methodology for solving the PMCP by giving the conditions under which the problem can be solved by checking each concrete system up to the cutoff system.

Verifying SGS systems
We now investigate the SGS class of systems. The class is generated from agent templates defined on asynchronous, agent-environment and global-synchronous actions. By Theorem 4.3, cutoffs do not generally exist for this class. However, we show that the existence of an agent-environment simulation between the agent and environment templates guarantees a cutoff. We show how to calculate the cutoff under these conditions.
We begin with an analysis of the agent-environment simulation for SGS systems. We then define the model checking procedure, and we exemplify it on the train-gate-controller. Finally, we prove its soundness. To simplify our analysis in the following we assume without loss of generality that for each action a ∈ Act E we have that |{l ∈ L E : a ∈ P E (l)}| = 1; i.e., an environment's action is enabled at exactly one template state.

Agent-environment simulation
Similarly to the SMR class, the concrete environment for the SGS class implements a mutual exclusion controller governing the access to shared resources. This is modelled by means of the environment's looping behaviour as per Lemma 5.5.
As we show below, an analogous lemma holds for the SGS class. However, an environment loop is guaranteed to occur only in path sections that do not contain global-synchrnous actions. A subsequence of a path π that does not contain GS actions is said to be a GS-free section of π . A GS-free section π(i), . . . , π( j) is said to be maximal if π(i − 1, Act) ∈ GS whenever i > 1, and π( j, Act) ∈ GS. Lemma 6.1. Consider a PIIS S = T , E, V ∈ SGS with T ≤ aes E and a maximal GS-free section π in S(n). If ELC(π (d)) for some Proof. Assume that ELC(π (d)) for some d > 1. Given π is a maximal GS-free section, it follows that: (i) π does not contain GS actions; (ii) tls j i (π (1)) ∼ aes ls E (π (1)) for each (i, j) ∈ A(n). Therefore, Lemma 5.5 applies to π , and therefore [d]π is an environment loop. 2 Intuitively, a global-synchronous action determines a subclass of the shared resources the agents can access. Upon performing a global-synchronous action the system enters a GS-free section where the agents can access the shared resources associated with the action in the same way described for SMR systems.

Model checking procedure for SGS systems
The model checking procedure Check_SGS for SGS systems is defined by Algorithm 2. Given a PIIS S ∈ SGS and an m-indexed formula ∀v φ(v), the procedure first establishes whether or not T ≤ aes E. Upon a successful simulation test the procedure calculates the cutoff for the given system and specification.  The cutoff function cutoff _SGS maps a PIIS S and tuple m of natural numbers into a tuple c of natural numbers that corresponds to the cutoff for m-indexed formulae. The function is defined in terms of m.  1,m(1)), . . . , max(1,m(k))) for any S ∈ SGS with k ≥ 1 roles and any m ∈ N k . Note that differently from cutoff _SMR, cutoff _SGS depends on the given system only in terms of the number of roles specified for the system. Following the cutoff calculation, the model checking procedure checks the concrete system S(c) against the trivial instantiation φ[trivial] of ∀v (φ(v)). If φ[trivial] is satisfied by S(c), then the procedure returns true, otherwise it returns false. We assess the soundness of the Check_SGS procedure in Section 6.4. First, we exemplify it by means of the train-gate-controller.

Verifying the train-gate-controller
In Section 2.3.2 we represented the train-gate-controller as a PIIS S TGC composed of an agent template PT encoding prioritised trains, an agent template NT encoding normal trains, and an environment template E encoding the controller. In Section 3.2 we expressed the property "whenever a train is in the tunnel, it knows that no other train is in the tunnel at the same time" in the following (2, 2)-indexed formula: where u, v are variables of PT, x, y are variables of NT, the atomic proposition pt holds in the template states in which the template prioritised train is in the tunnel and the atomic proposition nt holds in the template states in which the template normal train is in the tunnel. We now use the procedure Check_SGS to establish whether or not the train-gate-controller meets the above specification. To do this, we first observe that PT ≤ aes E and NT ≤ aes E. Thus, we can use the cutoff function to calculate a cutoff: Therefore, we need to check whether or not S TGC ((2, 2)) |= φ TGC [trivial]. This check, if mechanised on a standard checker such as MCMAS, would return true thereby establishing the correctness of the protocol irrespectively of the number of agents present.

Soundness of the Check_SGS procedure
We now assess the soundness of the Check_SGS procedure. To prove this result we firstly observe that by the definition of the PMCP and Theorem 3.9, it suffices to show that if T ≤ aes E, then : (i) the cutoff system S(c)m-stuttering simulates every bigger system; (ii) every bigger system m-stuttering simulates S(c). We first show the former.

Part A: the cutoff system m-stuttering simulates every bigger system
Consider an arbitrarily big system S(n) with n ≥c, where S(c) is the cutoff system. To simulate an action performed by the agents in A(c) in S(n), S(c) performs the same action. To simulate an action performed by an agent not in A(c) in S(n), S(c) performs the null action. Similarly to the simulation defined in Lemma 5.11 for SMR systems, the latter does not block potential synchronisations of the environment with the agents in A(c) at a later a point. To see this, note that a sequence of actions that are not admitted in the repertoire of the agents in A(c) happens only in a GS-free section. Thus, if, for example, g 1 a 1 g 2 . . . a x−1 g x is a GS-free section in S(n), where a 1 , a x−1 ∈ (i, j)∈A(c) AE j i and a 2 , . . . , a x−2 / ∈ (i, j)∈A(c) Act j i , then both g 2 and g x−1 satisfy the environment loop condition. Therefore, the state of the environment in g 2 is the same to its state in g x−1 . Intuitively, whenever an agent not in A(m) in S(n) takes the lock on a resource via synchronising with the environment, it has to release the resource via the occurrence of an environment loop before an agent in A(m) can take the lock on a resource. Proof. Let n ≥c be arbitrary. We show that S(n) ≤m ss S(c). Define ∼m ss = R 1 ∩ R 2 ⊆ G(n) × G(c) as follows: We show that ∼m ss is an m-stuttering simulation relation between S(n) and S(c).
Clearly, (ι(n), ι(c)) ∈ R 1 . To show that (ι(n), ι(c)) ∈ R 2 , suppose that ι(n) → X * g 1 → a g 2 , where a ∈ (i,r)∈A(c) AE r i and X = (i,r)∈A(c) A r i ∪ (i,r)/ ∈A(c) Act r i . We have to show that ls E (g 1 ) = ls E (ι(c)). If the path ι(n), . . . , g 1 consists only of asynchronous actions, then the case follows trivially. Otherwise, there must be an agent-environment action in ι(n), . . . , g 1 . In this case, it follows that ELC(g 1 ). By Lemma 6.1, ι(n), . . . , g 1 is an environment loop. So, as ls E (ι(n)) = ls E (ι(c)), ls E (g 1 ) = ls E (ι(c)). Therefore, (ι(n), ι(c)) ∈ R 2 , and therefore, ι(n) ∼m ss ι(c). Now assume that g ∼m ss g for an arbitrary pair of global states in G(n) × G(c). We show the simulation requirements (i) and (iii). Requirement (i) follows by the definition of R 1 . To show requirement (iii), let π ∈ (g). Define a path π ∈ (g ) as π = π (1)π (1, Act)π (2,Act) . . ., where π (x, Act) is defined for each x ≥ 1 as follows: Partition π and π into singleton blocks. We use induction on the length of π to show that: (i) π is a valid path; (ii) π(y) ∼m ss π (y) for each y ≥ 1. The base step for π of length 1 follows trivially. For the inductive step, assume that [x − 1]π ∈ (g ) and π(y) ∼m ss π (y) for each y with 1 ≤ y ≤ x − 1. We show that [x]π ∈ (g ) and π(x) ∼m ss π (x). The latter follows by the same argument used to show earlier that ι(n) ∼m ss ι(c). To establish the former, we consider cases on the type of the action π (x − 1, Act). The case where π (x − 1, Act) is the action, or an asynchronous action, or a global-synchronous action follows trivially. So, assume that π (x − 1, Act) ∈ (i,r)∈A(c) AE r i . We have to show that the action is enabled by the protocol of the agent and the protocol of the environment. The former is clear by the definition of R 0 . In the following we show the latter.

Define an integer d is follows. If there is no global-synchronous action in
GS-free section. We have two cases.
Consequently, π is a valid path, and π(y) ∼m ss π (y) for each y ≥ 1. Therefore, ∼m ss satisfies simulation requirement (iii). As (g, g ) was arbitrary, S(n) ≤m ss S(c) is entailed as required. 2 We have thus shown that the cutoff system m-stuttering simulates every bigger system. We conclude the proof for Theorem 6.3 by showing that every bigger system m-stuttering simulates the cutoff system. Fig. 13 shows the m-stuttering simulation between the cutoff system S(c) and a bigger system S(n). To simulate an asynchronous or an agent-environment action of S(c), S(n) performs the same action. Consider a global-synchronous action performed in S(c). Clearly, every agent in S(c) is at a state enabling the global-synchronous action. S(n) simulates a global-synchronous action as follows. One by one, every agent (i, j) ∈ A(n) \A(c) in S(n) mimicks agent (i, 1) by performing the same sequence of actions, thereby reaching the local state of agent (i, 1) in which the global-synchronous action is enabled. Then, S(n) performs the global-synchronous action. The following lemma formally defines this simulation and shows it to be an m-stuttering simulation. Proof. Let n ≥c. Assume an integer i with 1 ≤ i ≤ k. Define n to be a k-tuple of integers such that: n (i) =n(i) + 1; n ( j) =n( j) for all j with 1 ≤ j ≤ k and j = i. We show that S(n) ≤m ss S(n ). That is, we prove that the system obtained by adding one concrete agent (of an arbitrary role) to S(n)m-stuttering simulates S(n). The inductive application of the latter entails the main claim of the lemma. Let r =n (i). The simulation relation ∼m ss = R 1 ∩ R 2 ∩ R 3 ⊆ G(n) × G(n ) is defined by the following:

Part B: every bigger system m-stuttering simulates the cutoff system
The above ensures that the local state of each agent in S(n) is equal to its local state in a related global state in S(n ). The relations R 2 and R 3 specify the local state of the environment and the local state of agent (i, r) in related global states. Specifically, R 2 and R 3 are defined to induce a mimicking behaviour on agent (i, r) in S(n ) w.r.t. agent (i, 1) in S(n): and ls E (g 1 ) = ls E (g 1 ) If (g, g ) ∈ R 2 and a state g 1 is reachable from g in which a GS action is enabled, then agent (i, r) in S(n ) is able to change its local state to the local state of agent (i, 1) in S(n) via asynchronous and agent-environment transitions. Thus, whenever a GS-transition is taken in S(n), agent (i, r) is able to move to a local state in which the GS action is enabled. Note that (R 1 ∩ R 2 )-related states may disagree on the environment's local state. Because of this, given an arbitrary pair of (R 1 ∩ R 2 )-related states, S(n ) may not be able to simulate S(n). To circumvent this, we define R 3 to ensure that whenever (R 1 ∩ R 2 )-related states disagree on the environment's state, agent (i, r) can cause the environment to appropriately change its state.
, and ls E (g 1 ) = ls E (g) We show that ∼m ss is an m-stuttering simulation relation between S(n) and S(n ). To do this, we first show that (ι(n), ι(n )) ∈∼m ss . It is obvious that (ι(n), ι(n )) ∈ R 1 ∩ R 3 . To show that (ι(n), ι(n )) ∈ R 2 , suppose that ι(n) → X * g 1 → GS g 2 . We have to show that ι(n ) → Y * g 1 , where tls r i (g 1 ) = tls 1 i (g 1 ) and ls E (g 1 ) = ls E (g 1 ). Let θ be the sequence of actions of agent (i, 1) in the path π = ι(n), . . . , g 1 . Define θ to be the sequence θ , but with each action indexed with r instead of 1. Let π = ι(n ) • θ be a path in S(n ). It should be clear that π is a valid path if it consists only of asynchronous actions. If this is not the case, then observe that for every two successive synchronisations, say in states π(d) and π(d ), of agent (i, 1) with the environment in π , if there is a different agent that synchronises with the environment in the path from π(d) to π(d ), then the local state of the environment is the same in π(d) and π(d ). This follows by Lemma 6.1 which gives ELC(π (d)) and ELC(π (d )). So, the environment allows for the actions in θ to be performed. It follows that ι(n ) where tls r i (π []) = tls 1 i (g 1 ) and ls E (π []) = ls E (g 1 ). Now assume an arbitrary pair (g, g ) ∈ G(n) × G(n ) of global states such that g ∼m ss g . We show the simulation requirements (i) and (iii). Requirement (i) follows by the definition of R 1 . Consider requirement (iii) and a path π ∈ (g). We construct a path π ∈ (g ) as required by the m-stuttering-simulation. The construction is inductive on the length of π .
Having shown simulation requirements (i) and (iii) for an arbitrary pair of global states, it follows that S(n) ≤m ss S(n ). By the inductive application of the latter and by transitivity of ≤m ss , S(c) ≤m ss S(n). 2 Proof. Let n ≥c be arbitrary. By Theorem 3.9 it suffices to show that S(n) ≤m ss S(c) and S(c) ≤m ss S(n). The former is given by Lemma 6.4. The latter is given by Lemma 6.5. 2 This concludes the proof of Theorem 6.3. In contrast with the SMR class where cutoffs depend on the cardinality of the action dependency sets, Corollary 6.6 provides a methodology for solving the PMCP by giving the conditions under which the problem can be solved by checking only the cutoff system. This clearly has considerable advantages in applications as Section 6.3 demonstrates.

Verifying SFE systems
We finally investigate the SFE class of systems defined on asynchronous, role-synchronous and global-synchronous actions. In contrast with SMR and SGS, the existence of cutoffs in the SFE class does not depend on the existence of an agent-environment simulation between the agent and environment templates. In particular, we show that a cutoff exists for any given system and any given specification. This enables us to define a sound and complete model checking procedure. We begin with the definition of the model checking procedure and we exemplify it on the autonomous robot scenario. We then show it to be sound.  The model checking procedure for SFE systems is defined by Algorithm 3. Given a PIIS S ∈ SFE and an m-indexed formula ∀v φ(v), the procedure calculates the cutoff for the given system and specification. The cutoff function cutoff _SFE maps a PIIS S and a tuple m of natural numbers into a tuple c of natural numbers that corresponds to the cutoff for m-indexed formulae. Identically to cutoff _SGS and differently from cutoff _SMR, cutoff _SFE depends on the system under analysis only in terms of m.  1,m(1)), . . . , max(1,m(k))) for any S ∈ SFE with k ≥ 1 roles and any m ∈ N k . Following the cutoff calculation, the model checking procedure checks the concrete system S(c) against the trivial instantiation φ[trivial] of ∀v (φ(v)). If φ[trivial] is satisfied by S(c), then the procedure returns true, otherwise it returns false. Note that, differently from the model checking procedures for SMR and SGS systems, the model checking procedure for SFE systems always produces an output. We show that this output is always correct in Section 7.3. First, we compute its value for the autonomous robot example.

Verifying the autonomous robot example
In Section 2.3.3 we encoded the autonomous robot example as a PIIS S AR composed of an agent template TR1 representing robots with access to a sensor, an agent template TR2 representing robots with no access to a sensor, and a template environment E representing the environment. In Section 3.2 we expressed the property "whenever a robot halts, it knows that it is in the goal region" in the following (1, 1)-indexed formula: where v is a variable of TR1, u is a variable of TR2, the atomic proposition gr_1 (gr_2, respectively) holds in the template states where the value of the position component of template robot 1 (template robot 2, respectively) is in {2, 3, 4}, and the atomic proposition h_1 (h_2, respectively) holds in the template states where the template robot 1 (template robot 2, respectively) has halted. We now use the procedure Check_SFE to establish whether or not the autonomous robot meets the above specification. We have that c = cutoff _SFE(S AR , (1, 1)) = (1, 1) Thus, we need to check whether or not S AR ((1, 1)) |= φ AR [trivial]. The latter query can be tested with a standard model checker; this will return true thereby establishing the correctness of the protocol irrespectively of the number of robots present.
We prove this result by showing that: (i) the cutoff system S(c)m-stuttering simulates every bigger system; (ii) every bigger system m-stuttering simulates S(c). Differently from the corresponding results for the SMR and SGS classes, the absence of agent-environment and multi-role actions in the SFE class removes the necessity to simulate synchronisations of the environment on actions that are not admitted by the agents in A(m). Indeed, whenever an agent not in A(m) synchronises with the environment, all agents of the same role (or all agents in the system depending on the type of the action) synchronise the environment as well.

Part A: the cutoff system m-stuttering simulates every bigger system
Consider an arbitrarily big system S(n) with n ≥c, where S(c) is the cutoff system. Fig. 14 shows the m-stuttering simulation between S(n) and S(c). To simulate an action performed by the agents in A(c) in S(n), S(c) performs the same action. Any action performed in S(n) that is not admitted by an agent in A(c) is bound to be an asynchronous action. As only the state of the agent performing the action is updated, S(c) simulates these actions by performing the null action. Proof. Choose an arbitrary n ≥c. We show that S(n) ≤m ss S(c). Define the simulation relation ∼m ss ⊆ G(n) × G(c) as follows: Fig. 15. Them-stuttering simulation of the cutoff system S(c) by a bigger system S(n).
We show that ∼m ss is an m-stuttering simulation between S(n) and S(c). It is clear that ι(n) ∼m ss ι(c). Let g ∼m ss g for an arbitrary pair of global states in G(n) × G(c). We show simulation requirements (i) and (iii). Requirement (i) follows by the definition of ∼m ss . For simulation requirement (iii), let π ∈ (g). Construct a path π ∈ (g ) as follows: for each j ≥ 1, π ( j, Act) = π( j, Act) if π( j, Act) ∈ (i,r)∈A(c) Act r i ; otherwise, π ( j, Act) = . It can be checked that π is a valid path, and that π( j) ∼m ss π ( j), for each j ≥ 1. Thus, partition π and π into singleton blocks. It follows that S(n) ≤m ss S(c). 2 7.3.2. Part B: every bigger system m-stuttering simulates the cutoff system Fig. 15 shows the m-stuttering simulation between S(c) and S(n). An asynchronous action a defined for template i is simulated by S(c) by means of two cases. If a is not admitted in the repertoire of agent (i, 1), then S(n) simply performs a. If a is admitted in the repertoire of agent (i, 1), then, one by one, the agents (i, 1), (i, c(i) + 1), . . . , (i, n(i)) perform a. Thus, for each role i, every agent in {c(i) + 1, . . . ,n(i)} mimics agent (i, 1). S(n) may then simulate a role-synchronous action or a global-synchronous action by performing the action in question. Proof. Let n ≥c. Assume an integer i with 1 ≤ i ≤ k. Define n to be a k-tuple of integers such that: n (i) =n(i) + 1; n ( j) =n( j) for all j with 1 ≤ j ≤ k and j = i. We show that S(n) ≤m ss S(n ). That is, we prove that the system obtained by adding one concrete agent (of an arbitrary role) to S(n)m-stuttering simulates S(n). The inductive application of the latter entails the main claim of the lemma.
Assume r =n (i). Define the simulation relation ∼m ss ⊆ G(n) × G(n ) as follows: If g ∼m ss g , then the local states of the agents in A(n) and the local states of the environment are the same in the two global states. Additionally, the agent (i, r) in S(n ) is able to change its local state to the local state of agent (i, 1) in S(n ) via asynchronous transitions. We show that ∼m ss is an m-stuttering simulation between S(n) and S(n ).
The case of ι(n) ∼m ss ι(n ) follows trivially. Suppose that g ∼m ss g for an arbitrary pair of global states in G(n) × G(n ).
We show simulation requirements (i) and (iii). The former requirement follows by the definition of ∼m ss . For the latter requirement, let π ∈ (g). By induction on the length of π , we construct a path π ∈ (g ) such that: (i) π is as required by the m-stuttering simulation; (ii) tls r i (π []) = tls 1 i (π []). For the base step, g ∼m ss g gives g → A r i * g 1 , where tls r i (g 1 ) = tls 1 i (g ). Let π be the path from g to g 1 . The first blocks B 1 , B 1 are defined as follows: B 1 = g and B 1 is the sequence of states in π . Clearly, g is ∼m ss -related to every state in π .
Simulation requirement (iii) is therefore satisfied. It follows that S(n) ≤m ss S(n ). By the inductive application of the latter and by transitivity of ≤m ss , we obtain that S(c) ≤m ss S(n). 2 Proof. Let n ≥c be arbitrary. By Theorem 3.9 it suffices to show that S(n) ≤m ss S(c) and S(c) ≤m ss S(n). The former is given by Lemma 7.3. The latter is given by Lemma 7.4. 2 This concludes the proof of Theorem 7.2. Corollary 7.5 gives a methodology for solving the PMCP by model checking the cutoff system. Differently from Corollary 5.13 and Corollary 6.6, where certain conditions are assumed, by means of Corollary 7.5 we always solve the PMCP for any given system and specification. In other words, while Check_SMR and Check_SGS do not provide an answer to the PMCP in the absence of an agent-environment simulation, Check_SFE always provides a solution. Therefore, the PMCP for the SFE class of PIIS is decidable.
Corollary 7.6. The PMCP for the SFE class of PIIS is decidable.

The model checker MCMAS-P
In this section we introduce MCMAS-P [66], an experimental model checking toolkit that implements the techniques presented in Section 5, Section 6, and Section 7. MCMAS-P is built on top of the open source model checker MCMAS [11]. In its current version, MCMAS-P supports indexed ACTLK \ X formulae. Fig. 16 presents the key steps carried out by the checker. We extended ISPL, MCMAS's input language, to allow for the definition of the semantic structures and the parametric specifications considered here. As exemplified in Fig. 17, a PIIS is described in the extended language, called PISPL, by declaring the agent templates and the environment template. These extend ISPL's semantics by considering, among other concepts, the different kind of actions that PIIS are defined on. Given a PIIS S = T , E, V specified in PISPL and a set of m-indexed formulae , the steps 2-6 of Fig. 16 are performed automatically by the checker. In the following, we describe these steps.

MCMAS-P architecture
• In step 2, the PISPL input file is parsed. The declarations of the agent templates and the environment template are stored in temporary structures to be used in the following steps. • In step 3, the simulation test is performed to check if there is an agent-environment simulation between the agent templates and the environment template. This step is only performed if the specified PIIS is either a member of the SMR class or a member of the SGS class. Instead of explicitly traversing the template transition relations, the test is more efficiently performed by utilising the OBDD representation of the templates. In particular, the test for an agentenvironment simulation between an agent template i and the environment template is performed by checking the system composed of the two templates against a set of formulae expressing that whenever an AE i action, or a GS action, or an MR i action is enabled for the agent, the action is also enabled for the environment. We discuss this in more detail in Section 8.2 where we present the underlying theoretical background.
• Following a successful simulation test, in step 4, MCMAS-P calculates the cutoff as in Definition 5.7, Definition 6.2, and Definition 7.1. If S ∈ SMR, then the cutoff is computed from the action dependency sets and the cardinality of the sets of variables in the specifications to check; if S / ∈ SMR, then the cutoff is only computed in terms of the latter. • In step 5, the concrete system S(m) is built and encoded symbolically using the structures obtained in step 2. In step 6, the specification formulae are reduced to their trivial instantiations [trivial] as in Lemma 3.6. MCMAS is then called to verify S(m) against [trivial]. These steps are repeated for each concrete system up to the cutoff system S(c).
If all checks up to the cutoff system satisfy the corresponding specification, then MCMAS-P returns true; otherwise, MCMAS-P returns false.

Simulation test
We describe the procedure utilised by MCMAS-P for the existence of an agent-environment simulation. The procedure is based on the assumption that the agent-environment actions, global-synchronous actions, and multi-role actions are enabled at exactly one state for the environment template. This allows us to check for an agent-environment simulation between an agent template T i = L i , ι i , Act i , P i , t i and the environment template E = L E , ι E , Act E , P E , t E by model checking the interleaved interpreted system S i against the set of formulae i , where S i and i are defined as follows.
• S i = T i , E, V i is the IIS composed of the agent template T i and the environment template E. The global states G i ⊆ L i × L E in S i are assigned atomic propositions by the valuation function V i : L i × L E → P(AP), where AP = {a T , a E | a ∈ AE i ∪ GS ∪ MR i }, defined as a T ∈ V i ((l, l E )) iff a ∈ P i (l) and a E ∈ V i ((l, l E )) iff a ∈ P E (l E ). In other words, a global state g ∈ G i is labelled with a T (respectively, a E ) if the action a is enabled for the agent template (respectively, the environment template) at g.
S i |= δ is entailed. Therefore, ∀δ ∈ i . S i |= δ. ⇐ Suppose that ∀δ ∈ i . S i |= δ. We show that T i ≤ aes E. Let ∼ aes = G i . We show that ∼ aes is an agent-environment simulation between T i and E. Clearly, ι i ∼ aes ι E . Let g = (l, l E ) ∈∼ aes be arbitrary and suppose that l A i * l 1 a l 2 for some a ∈ AE i ∪ GS ∪ MR i . We need to show that l E a l 1 E for some l 1 E with (l 2 , l 1 E ) ∈∼ aes . As l 1 is reachable from l through asynchronous actions, there is a global state g 1 reachable from g such that ls T i (g 1 ) = l 1 and ls E (g 1 ) = l E . As a ∈ P (l 1 ) and S i |= AG(a T → a E ), it follows that a ∈ P E (l E ). As the action a is enabled at g 1 , we obtain (l 2 , l 1 E ) ∈∼ aes for l 1 a). Therefore, T i ≤ aes E. 2 So, we have that T ∼ aes E iff S i satisfies the formulae in i for all i with 1 ≤ i ≤ k. This has considerable advantages in applications in terms of efficiency as the following demonstrates.

Experimental results
We tested the cutoff techniques on the robot foraging scenario (RFS), the train-gate-controller (TGC) model, and the autonomous robot (AR) example against the specifications φ RFS , φ TGC , and φ AR , respectively. Table 1 shows the intractability of these problems in traditional model checking: the time and space requirements grow exponentially in the number of agents to consider. In our case the base model checker MCMAS could not verify the RFS system S RFS ( (30,30)) and the TGC system S TGC ( (40,40)) with a timeout of one hour. In comparison, MCMAS-P established the simulations as above and verified the cutoff instances S RFS ((2, 3)), S TGC ((2, 2)), and S AR ((1, 1)) in under 0.1 seconds for each of the three scenarios. The MAS cutoffs MCMAS-P found correspond to MAS with 5 agents, 4 agents, and 2 agents respectively. The formulae checked, and found to be true, were φ RFS [trivial], φ TGC [trivial] and φ AR [trivial]. This establishes the correctness of the RFS, TGS and AR scenarios with any number of agents. The above exemplifies the efficiency of the model checking procedures. Indeed, recall that the complexity of the CTL labelling algorithm is O ( f · (V + E)), where f is the number of connectives in the formula, V is the number of states, and E is the number of transitions [4]. Given the cutoffs are typically small, the procedures provide an effective verification for unbounded MAS. To see this in terms of the simulation test, observe that each S i has at most as many states and transitions as T i . Thus, checking a template agent against δ ∈ i requires O (3 · |T i |) time, where |T i | is the sum of the number of states and the number of transitions in T i . Therefore, establishing an agent-environment simulation between T i and E requires O (|AE i ∪ MR i ∪ GS| · |T i |) time. If x = max (|AE i ∪ MR i ∪ GS| : 1 ≤ i ≤ k) and y = max(|T i | : 1 ≤ i ≤ k), then the simulation test for a PIIS of k roles requires O (k · x · y) time.

Conclusions
With the deployment of systems based on MAS-architectures there has been a growing interest in their verification. Considerable progress has been made in model checking MAS against specifications based on temporal, epistemic, deontic and strategic properties; open-source implementations based on efficient symbolic approaches have been put forward and compared.
While this work has proven to be valuable, it is limited to scenarios where the number of components is known at design time. This is not a realistic assumption in certain MAS where the number of components cannot be known before deployment. A typical case is robotic swarms, whereby the properties of the swarm need to hold irrespective of how many robots are present in the system and should hold even if some of the robots develop a fault and leave the system. In this paper we put forward a methodology to solve the parameterised model checking problem for MAS in a number of noteworthy classes of PIIS. As we showed, this enabled us to give a methodology for the cutoff identification of a given parametric system. When a cutoff can be determined, the parameterised model checking problem can be solved via standard model checking by verifying all system instances up to the cutoff. We showed the procedure to be sound. Table 2 summarises the theoretical results obtained. Since the PMCP is in general undecidable, no complete results can be established in general. In Section 5 we presented an incomplete technique for the SMR class; in Section 6 we analysed an incomplete technique for the SGS class; in Section 7 we studied a complete technique for the SFE class. Incompleteness for the SMR and the SGS classes follows in the absence of an agent-environment simulation between the templates since in this case the techniques cannot assess the correctness of a given specification. In contrast, we can always assess the correctness of a specification on SFE systems. This level of confidence, which follows from the decidability result of Corollary 7.6, comes with considerable limitations on the range of systems that the technique can be applied to. For instance, the result cannot be applied to any scenario where the agents evolve in any other way other than lock-step evolution. Other systems may be modelled as SMR or SGS systems. SMR is suitable for scenarios requiring refined interactions between agents of different roles, whereas SGS is suitable for simulating synchronous semantics. The technique for the former class generally returns cutoffs that are larger than those for the latter class. Both techniques are limited by the requirement of an agent-environment simulation between the agent and environment templates. This makes it difficult to model certain applications of interest; including cache coherence protocols [67].
In future work we plan to explore other methodologies, including counter-abstraction [40] for obtaining cutoffs for even wider classes of systems. Furthermore, we believe the technique here put forward can serve as an ideal stepping stone to verify open MAS where the agents can dynamically join or leave the system at runtime. We are not aware of any formal methodologies addressing this aspect. Table 3 summarises the notation used in the paper. Table 3 Summary of the notation.

S(n)
Then-st instantiation of the PIIS S. T ≤ aes E Agent-environment simulation between the agent template T and the environment template E. S(n) ≤m ss S(n )m-stuttering simulation between S(n) and S(n ).

SMR
The class of PIIS generated from agent templates defined on asynchronous, agent-environment, and multi-role actions.

SGS
The class of PIIS generated from agent templates defined on asynchronous, agent-environment, and global-synchronous actions.

SFE
The class of PIIS generated from agent templates defined on asynchronous, role-synchronous, and actions.

A r i
The set of asynchronous actions of the agent (i, r).

AE r i
The set of agent-environment actions of the agent (i, r).

RS i
The set of role-synchronous actions of every agent performing role i.

GS
The set of global-synchronous actions of every agent in the system. MR r i The set of multi-role actions of the agent (i, r).

MR rq ij
The set of multi-role actions of the agent (i, r) that are shared with agent ( j, q).
The local state of agent (i, r) in global state g.
The template local state of agent (i, r) in global state g.

(g)
The set of all paths originating from global state g.

π (i)
the ith state in the path π . π (i, Act) the ith action in the path π .