Exact and heuristic methods for solving Boolean games

Abstract

Boolean games are a framework for reasoning about the rational behavior of agents whose goals are formalized using propositional formulas. Compared to normal form games, a well-studied and related game framework, Boolean games allow for an intuitive and more compact representation of the agents’ goals. So far, Boolean games have been mainly studied in the literature from the Knowledge Representation perspective, and less attention has been paid on the algorithmic issues underlying the computation of solution concepts. Although some suggestions for solving specific classes of Boolean games have been made in the literature, there is currently no work available on the practical performance. In this paper, we propose the first technique to solve general Boolean games that does not require an exponential translation to normal-form games. Our method is based on disjunctive answer set programming and computes solutions (equilibria) of arbitrary Boolean games. It can be applied to a wide variety of solution concepts, and can naturally deal with extensions of Boolean games such as constraints and costs. We present detailed experimental results in which we compare the proposed method against a number of existing methods for solving specific classes of Boolean games, as well as adaptations of methods that were initially designed for normal-form games. We found that the heuristic methods that do not require all payoff matrix entries performed well for smaller Boolean games, while our ASP based technique is faster when the problem instances have a higher number of agents or action variables.

Appendix: Proofs

Proposition 1

To improve readability, we will first briefly sketch the intuition behind the proof. Every strategy profile of the game G corresponds to an answer set of the ASP program $\mathcal {P}_1$. Program part $\mathcal {P}_2$ encodes the possible alternative strategies of the agents. On the one hand, if I is an answer set of $\mathcal {P}$, the minimal model of the reduct must contain $\textit{sat}$. On the other hand, if there exists an agent whose alternative strategy (encoded by $\mathcal {P}_2$) yields a better response to the actions of the other agents (encoded by $\mathcal {P}_1$) than the strategy profile encoded by $\mathcal {P}_1$, the literal $\textit{sat}$ in $\mathcal {P}_3$ cannot be derived. Hence every strategy profile $S_I$ that corresponds to an answer set I of $\mathcal {P}$ must be a PNE. Conversely, given a PNE S, it is straightforward to write down the corresponding answer set of $\mathcal {P}$ and prove that no smaller model of the reduct exists, since $\textit{sat}$ must be in every minimal model as a consequence of the assumption that S is a PNE.

Let I be an arbitrary answer set of $\mathcal {P}$. Since for all $p\in V$, either the corresponding literal $\textit{act}(p)$ or its negation $\lnot \textit{act}(p)$ is contained in I due to rule (1), it holds that $S_I$ is a well-defined strategy profile for G. We denote $S_i = \pi _i \cap S_I$ and $S_{-i}=S_I \setminus \pi _i$, for every agent $i \in N$. It remains to prove that for every agent i: $u_i(S_I) \ge u_i(S_{-i},s'_i)$, $\forall s'_i \subseteq \pi _i$. Let i be an arbitrary agent in N. If $u_i(S_I)=1$, then this condition is fulfilled since $u_i$ only takes values from $\{0,1\}$. If $u_i(S_I)=0$, then the condition is fulfilled iff $u_i(S_{-i},s'_i)=0$, $\forall s'_i \subseteq \pi _i$. Suppose by contradiction that there exists an alternative set of actions $s'_i \subseteq \pi _i$ such that $u_i(S_I) < u_i(S_{-i},s'_i)$, i.e. $u_i(S_I)=0$ and $u_i(S_{-i},s'_i)=1$. We can now construct a model J of the reduct $\mathcal {P}^I$, which is strictly contained in I. To this end, we make the following observations:

• 1. For every agent $j\in N$: $\textit{goal}(j) \in I$ iff $u_j(S_I)=1$: this follows from rule (2) and the fact that $\varphi _j$ is true iff $u_j(S_I)=1$.

• 2. Since I is an answer set of $\mathcal {P}$, rule (7) implies that $\textit{sat} \in I$. This rule is not contained in the reduct $\mathcal {P}^I$ and (8) implies that I contains all variables of the form $\textit{act}\,'(p)$ and $\textit{nact}\,'(p)$ for all $p\in V$ and $\textit{ngoal}\,'(j)$ for every agent $j \in N$. In turn rule (8) implies that I also contains all atoms of $X'$ defined in $\mathcal {P}_2$ as components of formulas. The previous observation together with rules (5) imply that $\textit{pleased}(j)$ is in I for every $j \in N$, since $\textit{ngoal}\,'(j)$ is also in I. Obviously all the facts $\textit{action}(p)$ and $\textit{agent}(j)$ of $\mathcal {P}$ also belong to I for every $p\in V$ and $j\in N$.

• 3. If we denote all literals occurring in program $\mathcal {P}_1$ as $\mathcal {B}_1$ and we define $J_1 = I \cap \mathcal {B}_1$, then clearly $J_1$ satisfies all the rules of $\mathcal {P}_1$ since I is an answer set of $\mathcal {P}$. Moreover, $J_1$ is a model of the reduct $\mathcal {P}_1^I$, since $\mathcal {P}_1^I=\mathcal {P}_1^{J_1}$. Note that $J_1$ also contains all the facts $\textit{action}(p)$ and $\textit{agent}(j)$ for every $p\in V$ and $j\in N$.

• 4. Define the interpretation $J_2=A \cup \textit{NA} \cup X'_2 \cup \textit{NG}$ with

  $$\begin{aligned} A&=\{\textit{act}\,'(p)\,|\, p \in (S_{-i},s'_i)\},\\ \textit{NA}&=\{\textit{nact}\,'(p) \,|\, p \notin (S_{-i},s'_i)\},\\ X'_2&= \{a \in X' \,|\, \exists r: a \leftarrow B \in \mathcal {P}: \forall b \in B: b \in A \cup \textit{NA} \cup X'_2 \},\\ \textit{NG}&= \{\textit{ngoal}\,'(j)\,|\, j \in N\setminus \{i\}, \textit{goal}(j) \notin I\}. \end{aligned}$$

  The interpretation $J_2$ represents a valid strategy profile $S_{J_2}=\{p\,|\, \textit{act}(p)\in J_2\}$ and the actions of all agents but i coincide with their actions in $S_I$. Hence rule (4) will also be satisfied by $J_2$ for every agent $j\ne i$. Agent i’s strategy in $J_2$ corresponds to $s'_i$, hence the assumption $u_i(S_{-i},s'_i)=1$ implies that $\sim \varphi _i(\textit{act}(\pi _{-i}),\textit{nact}\,'(\lnot \pi _i),\textit{act}\,'(\pi _i))$ is false and $\textit{ngoal}\,'(i)$ cannot be derived, hence $J_2$ satisfies rule (4) for i as well. Since $\mathcal {P}_2^I=\mathcal {P}_2$, $J_2$ is a model of $\mathcal {P}_2^I$.

• 5. Define $J_3=\{\textit{pleased}(j)\,|\, j \in N\setminus \{i\}\}$ and $J=J_1\cup J_2 \cup J_3$. Our assumption $u_i(S_I)=0$ and the first observation imply that $\textit{goal}(i) \notin I$. By definition, we also have $\textit{goal}(i) \notin J_1$ for $J_1 \subset J$. By definition $\textit{ngoal}\,'(i) \notin J_2$ for $J_2 \subset J$. Hence the bodies of the rules (5) are false, which means that $\textit{pleased(i)}$ cannot be derived for any i. Hence, J satisfies these rules for agent i. For every other agent $j \ne i$ such that $\textit{goal}(j) \in I$, it follows that $\textit{goal}(j) \in J_1$ for $J_1 \subset J$, and by definition $\textit{ngoal}\,'(j) \in J_2$ for $J_2 \subset J$ for every $j \ne i$ such that $\textit{goal}(j) \notin I$. In any case the rules (5) are fulfilled by J. Since $\textit{pleased}(i)$ and sat are not in J, rule (6) is also fulfilled by J. Since the bodies of the rules in (8) are not satisfied, these rules are all satisfied by J.

Combining these subresults, we see that J is a model of $\mathcal {P}^I$. Moreover, we have $J \subset I$ and $\textit{sat} \in I \setminus J$, which implies that I is not a minimal model of $\mathcal {P}^I$, contradicting the assumption that I is an answer set of $\mathcal {P}$.

Suppose $S=(s_1,\ldots ,s_n)$ is a PNE. Let the sets of atoms X and $X'$ be defined as in the construction of the induced PNE program. We define the sets A, $\textit{NA}$ and Z as follows:

$$\begin{aligned} A&= \{\textit{act}(p) \,|\, p \in S\},\\ \textit{NA}&= \{\lnot \textit{act}(p) \,|\, p \notin S\}, \\ Z&= \{a \in X \,|\, \exists r: a \leftarrow B \in \mathcal {P}: \forall b \in B: b \in A \cup \textit{NA} \cup Z \}. \end{aligned}$$

We define an interpretation $I_S$ and prove that it is a minimal model of $\mathcal {P}^{I_S}$:

$$\begin{aligned} I_S&= A \cup \textit{NA} \cup Z \cup \{\textit{goal}(i) \,|\, i\in N,u_i(S)=1\} \cup \{\textit{sat}\} \\&\cup \{\textit{act}\,'(p) \,|\, p \in V\} \cup \{\textit{nact}\,'(p) \,|\, p \in V\} \cup X' \cup \{\textit{ngoal}\,'(i)\,|\, i \in N\}\\&\cup \{\textit{pleased}(i) \,|\, i \in N\} \\&\cup \{\textit{agent}(i) \,|\, i \in N\} \cup \{\textit{action}(p) \,|\, p \in V\} \end{aligned}$$

Any literal $\textit{act}(p)$ in A cannot be omitted without requiring that $\lnot \textit{act}(p)$ is in $\textit{NA}$, due to rule (1). Hence it suffices to prove that $I_S$ is minimal among the models of $\mathcal {P}^{I_S}$ containing $A \cup \textit{NA}$. Clearly the rules of the form (2) uniquely determine which variables of X should be in a minimal model of $\mathcal {P}^{I_S}$ that already contains A and $\textit{NA}$. The same reasoning holds for the variables of the form $\textit{goal}(i)$ for every agent i, i.e. the variables of the form $\textit{goal}(i)$ with $u_i(S)=1$ should be in every model of $\mathcal {P}^{I_S}$ already containing A and $\textit{NA}$. Since S is a PNE, the actions in S are best responses for every agent i to $s_{-i}$. We check whether it is possible that there exists a model J of $\mathcal {P}^{I_S}$ that contains $A \cup \textit{NA}$ but does not contain sat. If we define $S'=(s'_1,\ldots ,s'_n)$ with $s'_i= \{p\,|\, \textit{act}\,'(p) \in J\}$, rules (3) imply that $S'$ is a valid strategy profile of the game: sat is not derived hence $\{p \,|\, \textit{act}\,'(p) \in J\}$ and $\{p\,|\, \textit{nact}\,'(p) \in J\}$ form a partition of V. Moreover, for every agent i, it holds that $\textit{ngoal}\,'(i)$ belongs to J iff $u_i(S_{-i},s'_i)=0$ due to rule (4). But then the fact that S is a PNE implies that $\textit{pleased}(i)$ is derived by the rules (5) for every agent i, and in turn rule (6) derives $\textit{sat}$. Hence $\textit{sat}$ should be in any model of $\mathcal {P}^{I_S}$ which contains A and $\textit{NA}$. But rules (8) imply that such a model of $\mathcal {P}^{I_S}$ should contain $\{\textit{act}\,'(p) \,|\, p \in V\} \cup \{\textit{nact}\,'(p) \,|\, p \in V\}$. Moreover, the rules of the form (4) imply that $X' \cup \{\textit{ngoal}\,'(i)\,|\, i \in N\}$ should be contained as well in the presence of $\{\textit{act}\,'(p) \,|\, p \in V\} \cup \{\textit{nact}\,'(p) \,|\, p \in V\}$. The rules (5) imply that $\textit{pleased}(i)$ should be in such a minimal model as well for every agent i, since for every agent i we proved that it contains $\textit{ngoal}\,'(i)$. Finally note that the facts $\textit{agent}(1..n)$ and $\textit{action}(p)$ ($p\in V$) should be in every model of $\mathcal {P}^{I_S}$. We have proved that every variable of $I_S$ is part of every minimal model of $\mathcal {P}^{I_S}$ containing $A \cup \textit{NA}$. Moreover, $I_S$ satisfies all rules of $\mathcal {P}^{I_S}$, hence $I_S$ is an answer set of $\mathcal {P}$. $\square$

Proposition 2

This proof is analogous to the previous proof, except that the induced core-program has some additional literals: $\textit{control}(.,.)$, $\textit{coalition}(.)$ and $\textit{ncoalition}(.)$. The intuition, however, is very similar, except that program part $\mathcal {P}_2$ encodes a coalition and an alternative strategy profile such that non-coalition members play the same strategy as in $\mathcal {P}_1$.

Let I be an arbitrary answer set of $\mathcal {P}$. Since for all $p\in V$, either the corresponding literal $\textit{act}(p)$ or its negation $\lnot \textit{act}(p)$ is contained in I due to rule (1), it holds that $S_I$ is a well-defined strategy profile for G. We write $\pi _C = \bigcup _{i \in C} \pi _i$, $S_C = \pi _C \cap S_I$ and $S_{-C}=S_I \setminus \pi _C$, for every coalition $C \subseteq N$. It remains to prove that for every coalition $C \subseteq N$ and for all $s'_C \subseteq \pi _C$: $\exists i \in C$: $u_i(S_I) \ge u_i(S_{-C},s'_C)$. Let C be an arbitrary coalition in N. If there exists a coalition player $i \in C$ such that $u_i(S_I)=1$ then the condition is fulfilled since $u_i$ takes values in $\{0,1\}$. If $u_i(S_I)=0$ for all $i \in C$ then the condition is fulfilled iff $\forall s'_C \subseteq \pi _C: \exists i \in C$: $u_i(S_{-C},s'_C)=0$. Suppose by contradiction that there exists an alternative set of actions $s'_C \subseteq \pi _C$ such that $\forall i \in C$: $u_i(S_I) < u_i(S_{-C},s'_C)$ i.e. $u_i(S_I)=0$ and $u_i(S_{-C},s'_C)=1$ for every $i \in C$. We can then construct a model J of the reduct $\mathcal {P}^I$, which is strictly contained in I. To this end, we make the following observations:

• 1. For every agent $j\in N$: $\textit{goal}(j) \in I$ iff $u_j(S_I)=1$: this follows from rule (2) and the fact that $\varphi _j$ is true iff $u_j(S_I)=1$.

• 2. Since I is an answer set of $\mathcal {P}$, rule (7) implies that $\textit{sat} \in I$. This rule is not contained in the reduct $\mathcal {P}^I$ and (8) implies that I contains all variables of the form $\textit{act}\,'(p)$, $\textit{nact}\,'(p)$ (for all $p\in V$), $\textit{coalition}(i)$, $\textit{ncoalition}(i)$ and $\textit{ngoal}\,'(i)$ (for all $i \in N$). In turn rule (8) implies that I also contains all atoms of $X'$ defined in $\mathcal {P}_2$ as components of formulas. The previous observation together with rules (5) imply that $\textit{pleased}(j)$ is in I for every $j \in N$, since $\textit{ngoal}\,'(j)$ is also in I. Obviously all the facts $\textit{action}(p)$, $\textit{agent}(i)$ and $\textit{control}(i,p_i)$ of $\mathcal {P}$ also belong to I for every $p\in V$, $i\in N$ and $p_i \in \pi _i$.

• 3. If we denote all literals occurring in program $\mathcal {P}_1$ as $\mathcal {B}_1$ and we define $J_1 = I \cap \mathcal {B}_1$, then clearly $J_1$ satisfies all the rules of $\mathcal {P}_1$ since I is an answer set of $\mathcal {P}$. Moreover, $J_1$ is a model of the reduct $\mathcal {P}_1^I$, since $\mathcal {P}_1^I=\mathcal {P}_1^{J_1}$. Note that $J_1$ also contains all the facts $\textit{action}(p)$, $\textit{agent}(i)$ and $\textit{control}(i,p_i)$ of $\mathcal {P}$ for every $p\in V$, $i\in N$ and $p_i \in \pi _i$.

• 4. Define the interpretation $J_2=A \cup \textit{NA} \cup X'_2 \cup \textit{CL} \cup \textit{NCL}$ $\cup \ \textit{NG}$ with

  $$\begin{aligned} A&= \{\textit{act}\,'(p)\,|\, p \in (S_{-C},s'_C)\},\\ \textit{NA}&= \{\textit{nact}\,'(p) \,|\, p \notin (S_{-C},s'_C)\},\\ X'_2&= \{a \in X' \,|\, \exists r: a \leftarrow B \in \mathcal {P}: \forall b \in B: b \in A \cup \textit{NA} \cup X'_2 \},\\ \textit{CL}&= \{\textit{coalition}(i)\,|\,i\in C\},\\ \textit{NCL}&= \{\textit{ncoalition}(i)\,|\,i\notin C\},\\ \textit{NG}&= \{\textit{ngoal}\,'(i)\,|\, i\notin C, u_i(S_{-C},s'_C)=0\}. \end{aligned}$$

  Since $J_2$ represents a valid strategy profile $S_{J_2}=\{p\,|\, \textit{act}(p)\in J_2\}$, sat cannot be derived through rule (3). Similarly, rule (10) cannot derive sat since $J_2$ corresponds to a valid coalition of N (every agent is either a member or not and the coalition is non-empty). Moreover, the actions in $J_2$ of all non-coalition players coincide with their actions in $S_I$, hence rules (11) will be satisfied. The actions of coalition members in $J_2$ corresponds to $s'_C$, hence $\sim \varphi _i(\textit{nact}\,'(\lnot V),\textit{act}\,'(V))$ is true for $i \notin C$ iff $u_i(S_{-C},s'_C)=0$, implying that $J_2$ satisfies rule (12) for non-coalition members. Furthermore, the assumption $u_i(S_{-C},s'_C)=1$ for every $i\in C$ implies that $\sim \varphi _i(\textit{nact}\,'(\lnot V),\textit{act}\,'(V))$ is false and $\textit{ngoal}\,'(i)$ cannot be derived, hence $J_2$ satisfies rule (12) for coalition members as well. Since $\mathcal {P}_2^I=\mathcal {P}_2$, $J_2$ is a model of $\mathcal {P}_2^I$.

• 5. Define $J_3=\{\textit{pleased}(i) \,|\, i\notin C, u_i(S_I)\ge u_i(S_{-C},s'_C)\}$ and $J=J_1 \cup J_2 \cup J_3$. In case of a non-coalition member i, the rules (5) will be satisfied by $J_3$ since one of the bodies is true iff $u_i(S_I)\ge u_i(S_{-C},s'_C)$. Our assumption $u_i(S_I)=0, \forall i \in C$ and the first step imply that $\textit{goal}(i) \notin I$ for every $i \in C$. By definition the same holds for $J_1\subset J$. By definition it holds that $\textit{ngoal}\,'(i) \notin J_2$ with $J_2 \subset J$ for every $i \in C$. Therefore, the bodies of (5) are not satisfied for coalition members i and $\textit{pleased}(i)$ cannot be derived, hence J satisfies these rules for every agent $i \in C$. If $i \notin C$, then either $u_i(S_I)\ge u_i(S_{-C},s'_C)$ or $u_i(S_I) < u_i(S_{-C},s'_C)$. In the first case, $\textit{pleased}(i) \in J$ holds, hence rules (5) are satisfied. In the second case, $u_i(S_{-C},s'_C) \ne 0$ and $u_i(S_I)=0$, hence the first step and the definition of J imply that $\textit{goal}(j) \notin J$ and $\textit{ngoal}\,'(i) \notin J$. Therefore, rules (5) are satisfied. Since $\textit{pleased}(i)$ is not in J for every $i \in C$, rule (13) is also satisfied by J. The bodies of rules (8) and (15) are falsified by J, hence the rules are satisfied by J.

Combining these subresults, we see the J is a model of $\mathcal {P}^I$. Moreover, $J \subset I$ and $\textit{sat} \in I \setminus J$. This implies that I is not a minimal model of $\mathcal {P}^I$, contradicting the assumption that I is an answer set of $\mathcal {P}$.

Suppose $S=(s_1,\ldots ,s_n)$ is an element of $\textit{Core}(G)$. Let the sets of atoms X and $X'$ be defined as in the construction of the induced core-program. Define the following sets of literals:

$$\begin{aligned} A&= \{\textit{act}(p) \,|\, p \in S\},\\ \textit{NA}&= \{\lnot \textit{act}(p) \,|\, p \notin S\},\\ Z&= \{a \in X \,|\, \exists r: a \leftarrow B \in \mathcal {P}: \forall b \in B: b \in A \cup \textit{NA} \cup Z \}. \end{aligned}$$

and an interpretation $I_S$:

Any atom $\textit{act}(p)$ in A cannot be omitted without requiring that $\lnot \textit{act}(p)$ is then in $\textit{NA}$, due to rule (1). Hence we have to prove that $I_S$ is minimal among the models of $\mathcal {P}^{I_S}$ containing $A \cup \textit{NA}$. Clearly the rules of the form (2) uniquely determine which atoms of X should be in a minimal model of $\mathcal {P}^{I_S}$ which already contains A and $\textit{NA}$. The same reasoning holds for the literals of the form $\textit{goal}(i)$ for every agent i, i.e. the literals of the form $\textit{goal}(i)$ with $u_i(S)=1$ should be in every model of $\mathcal {P}^{I_S}$. This is fulfilled by definition of $I_S$. Since S is a core element, S is not blocked by any coalition. We show that there does not exist a minimal model J of $\mathcal {P}^{I_S}$ which contains $A\cup \textit{NA}$ but does not contain sat. If we define $S'=(s'_1,\ldots ,s'_n)$ with $s'_i= \{p\,|\, \textit{act}\,'(p) \in J\}$, rules (3) imply that $S'$ is a valid strategy profile of the game: sat is not derived hence $\{p \,|\, \textit{act}\,'(p) \in J\}$ and $\{p\,|\, \textit{nact}\,'(p) \in J\}$ form a partition of V. Moreover for every agent i it holds that $\textit{ngoal}\,'(i)$ belongs to J iff $u_i(S_{-C},s'_C)=0$ due to rule (12). But then S being a core element implies that $\textit{pleased}(i)$ will be derived for a coalition player i by the rules (5) for at least one agent i, and in turn rule (13) derives $\textit{sat}$. So $\textit{sat}$ should be in any minimal model of $\mathcal {P}^{I_S}$ which contains A and $\textit{NA}$. But rules (8) and (15) imply that such a model of $\mathcal {P}^{I_S}$ should contain $\{\textit{act}\,'(p) \,|\, p \in V\} \cup \{\textit{nact}\,'(p) \,|\, p \in V\} \cup \{\textit{coalition}(i) \,|\, i \in N\} \cup \{\textit{ncoalition}(i) \,|\, i \in N\}$. Moreover, the rules of the form (12) imply that $X' \cup \{\textit{ngoal}\,'(i)\,|\, i \in N\}$ should be contained as well in the presence of $\{\textit{act}\,'(p) \,|\, p \in V\} \cup \{\textit{nact}\,'(p) \,|\, p \in V\}$. The rules (5) imply that $\textit{pleased}(i)$ should be in such a minimal model as well for every agent i, since for every agent i we proved that $\textit{ngoal}\,'(i)$ is in any minimal model. Finally note that the facts $\textit{agent}(1..n)$, $\textit{action}(p)$ ($p\in V$) and $\textit{control}(i,p)$ ($i \in N, p\in \pi _i$) should be in every minimal model of $\mathcal {P}^{I_S}$. We have proved that every variable of $I_S$ is part of every minimal model of $\mathcal {P}^{I_S}$ containing $A \cup \textit{NA}$. Moreover, $I_S$ satisfies all rules of $\mathcal {P}^{I_S}$, hence $I_S$ is an answer set of $\mathcal {P}$. $\square$

Proposition 3

First note that S is an absorbing state iff S is a PNE.

Convergence of WSLpS applied to G boils down to the fact that the induced Markov chain ends up in an absorbing state. In particular, there must exist an absorbing state S. Due to the observation above, it holds that $\exists S: S$ is a PNE.

Assume that there exists a PNE $S^g=(s_1^g,\ldots ,s_n^g)$ of G. Then $S^g$ is an absorbing state of the induced Markov chain. For the chain to be absorbing, it is sufficient to prove that for each non-absorbing state there exists an accessible absorbing state. We now prove that $S^g$ is accessible from every non-absorbing state.

Let $S^1=(s_1^1,\ldots ,s_n^1)$ be an arbitrary non-absorbing state, then $S^1$ is no PNE. In particular there must exist at least one agent $i_1$ which does not play a best response and hence does not reach its goal, since reaching your goal corresponds to a best response by definition. We now show that a strategy profile $S^2$ exists such that

• 1. $s^2_j = s^g_j, \forall j \in RA(i_1)$ and $s^2_j = s^1_j, \forall j \in N \setminus RA(i_1)$,

• 2. the probability of a transition from $S^1$ to $S^2$ is non-zero,

For every $j \in RA(i_1)$ there exists a subset of neighbors $r_j$ with $i_1 \in r_j$ and ${\textit{success}}(S^1,r_j)=0$. We define $s^2_j=s^g_j$. For every $j \in N \setminus RA(i_1)$ we define $s^2_j = s^1_j$. Obviously, this $S^2$ fulfills the first condition. Note that this implies that $i_1$ plays a best response in $S^2$, since it plays a best response in $S^g$ and $s^2_j=s^g_j, \forall j \in RA(i_1)$. The fact that the transition probability from $S^1$ to $S^2$ is non-zero is due to the following reasons. First, all agents with positive feedback from $S^1$ stick to their current strategy. Second, all agents with negative feedback can change to any strategy with a non-zero probability; in particular, the probability that all agents j with negative feedback from $S^1$ ‘switch’ to $s^1_j$ is non-zero. To see this, note that the probability of agent j flipping a variable is $\beta _j(S^1,r_j)=\max (\alpha - \frac{|\{j' \in r_j\,|\, u_{j}\,'(S^1) = 1\}|}{k_j},0)$. If agent j got negative feedback from $S^1$, then ${\textit{success}}(S^1,r_j)=0$ and there must be at least one neighbor $j'$ of j with $u_{j}\,'(S^1) \ne 1$. Considering the fact that $\alpha > \frac{k-1}{k}$ and $k_j=\min (k,|{\textit{Neigh}}(j)|)$, it follows that $\alpha > \frac{k_j-1}{k_j}$. Therefore it holds that $\beta _j(S^1,r_j)$ is in ]0, 1[ if agent j got negative feedback from $S^1$. Hence, irrespective of whether the transition from $s^1_j$ to $s^2_j$ requires flipping variables or not, the transition probability is non-zero.

Either every agent i in N plays a best response in $S^2$, hence $S^2$ is an accessible absorbing state, or there exists an agent $i_2$ in N that does not play a best response. In the latter case it holds that $\exists i_2 \in N: u_{i_2}(S^2)=0$. As long as there exists an $i_l \in N$ such that $i_l$ does not play a best response in $S^l$ we can find a state $S^{l+1}=(s_1^{l+1},\ldots ,s_n^{l+1})$ such that the transition probability from $S^{l}$ to $S^{l+1}$ is non-zero, $s^{l+1}_j = s^g_j, \forall j \in RA(i_l)$, and $s^{l+1}_j = s^l_j, \forall j \in N \setminus RA(i_l)$. Moreover, for every $i_{m}$ with $1 \le m \le l$ it holds that $i_m$ plays a best response in $S^{l+1}$ because $\forall j \in RA(i_m):$ $s^{l+1}_j=s^{g}_j$. To see this, note that we assumed that every $j \in RA(i_m)$ switched to $s^g_j$ in the transition from $S^m$ to $S^{m+1}$. In all the next transitions, j kept the previous strategy, hence $s^{l+1}_j=s^g_j$. Clearly $l=n$ is the maximum value for which $S^{l+1}$ will be an absorbing state accessible from $S^1$, as a strategy profile for which every agent plays a best response is a PNE by definition. $\square$