A Compositional Treatment of Iterated Open Games

Compositional Game Theory is a new, recently introduced model of economic games based upon the computer science idea of compositionality. In it, complex and irregular games can be built up from smaller and simpler games, and the equilibria of these complex games can be defined recursively from the equilibria of their simpler subgames. This paper extends the model by providing a final coalgebra semantics for infinite games. In the course of this, we introduce a new operator on games to model the economic concept of subgame perfection.


Introduction
Compositionality, where one sees complex systems as being built from smaller subsystems, is widely regarded within computer science as best practice. As the subsystems are smaller, they are easier to reason about, and compositionality also promotes modularity and reuse; a particular system can be a subsystem of many different supersystems. Can compositionality be applied also to economic games? In general, not all reasoning is compositional, especially if significant emergent behaviour is present in a large system but not in its subsystems. This is unfortunately the case for economic games. For example, if σ is an optimal strategy for G, then is σ part of an optimal strategy for G * H, where G * H is a super-game built from G and H? Clearly not, e.g. the Iterated Prisoners' Dilemma has equilibria -such as cooperative equilibria -that do not arise from repeatedly playing the Nash equilibrium from the Prisoners' Dilemma (Axelrod and Dion, 1988).
However, Ghani et al. (2016) produced a compositional model of game theory which included a limited set of operators for building new games from old. There was no operator to compositionally build the infinite prisoners' dilemma from the one-shot prisoners dilemma and, more generally, to compositionally build infinite iterations of games. This paper addresses that problem. Within programming language theory, these sort of issues are tackled by final coalgebra semantics (Rutten and Turi, 1994) and we follow this practice, with the added benefit of bringing related bisimulation techniques to the game theory community. In doing this, we deal with a number of issues: • Each round of an infinite game produces utility. Traditionally, this infinite sequence of staged utilities is combined into a single utility in one of a number of ad hoc manners. We take the bolder approach of not requiring the choice of a single mechanism for combining utilities.
• The general approach of Compositional Game Theory deals with a new concept of coutility: if utility is gained by one agent, it must come from another agent. However, this produces problems for modelling infinite games, and so we make a simplifying assumption with respect to coutility. This is not a limitation in practice, as standard treatments do not consider coutility.
• The coalgebraic approach we advocate dovetails well with the economic concept of subgame perfection where a strategy must be an optimal response in all subgames of the supergame (Shubik, 1984).

Related
Work. An introduction to the economic treatment of iterated games can be found in Mailath and Samuelson (2006). The fundamental concept of game theory is that of Nash equilibrium (Nash, 1951), which has been adapted for the study of repeated and dynamic games to the concept of subgame perfect equilibrium first introduced by Selten (1965). Significantly influential work on using logical methods and coalgebraic reasoning in economics include Lescanne (2012) and Abramsky and Winschel (2017). Open games are also closely related to the 'partially defined games' of Oliva and Powell (2015).
Structure of the paper. Section 2 consists of preliminaries and a summary of previous work on open games; Section 3 introduces a modality for dealing with subgame perfection; Section 4 introduces morphisms between games, and Section 5 consists of our final coalgebra semantics for infinite open games. Finally Section 6 contains concluding remarks and discussions of further work.

Preliminaries
The key concept of Ghani et al. (2016) is the following: Definition 1 (Open Game). Let X, Y , R and S be sets. An open game • a set Σ G of strategy profiles, • a play function P G : We sometimes write G : (X, S) Σ − → (Y, R) to make the set of strategies explicit. Intuitively, the set X contains the states of the game, Y the moves, R the utilities and S the coutilities. The set Σ G contains the strategies we are trying to pick an optimal one from. The play function P G selects a move given a strategy and a state, while the coutility function C G computes the coutility extruded from the game, given a strategy, state and utility. Finally, if σ ∈ E G x k, then σ is an optimal strategy in state x and with utility given by k : Y → R. The main result of Ghani et al. (2016) can be stated as follows: Proof. The composition of G and H is given by the game with strategies Σ H•G = Σ G × Σ H , play function the composition of the respective play functions from H and G, and coutility function the composition in reverse of the coutility functions from H and G, using the play function of G to produce a state for H.
The monoidal product is given by Cartesian product in the category of sets, with componentwise action on the strategies, play functions and coutility functions of open games, and (σ 1 , ). The unit of this monoidal structure is (1, 1), while the symmetry is inherited from the Cartesian product in Set.

Subgame-Perfection and Conditioning
Intuitively, we play two rounds of a game by composing the game with itself. However, this is not quite right: in the composite game Σ H•G = Σ H × Σ G , and thus the second game H cannot react to the moves played by the first game G. This clearly does not match practice. Rather than introduce a new form of composition, we introduce a modality which allows us to condition a game to react to every possibility in some set A.
Note how a strategy in A → H is a set of strategies, one for each element of A, and that for a strategy f to be optimal in A → H, each of its components must be optimal in H. This captures the notion of subgame-perfection. Clearly we have:

2-Cells and Coutility Free Games
Fundamentally, if we have a game G : (X, S) Σ − → (Y, R), its infinite iteration G ω will be constructed compositionally as the final coalgebra of the functor F G defined by H → (Y → H) • G. However, this means that games will acquire universal properties and thus we need a notion of morphism between games. Further, G ω will satisfy G ω ∼ = (Y → G ω ) • G, and hence the equation relating coutility of G ω and coutility of G must hold. Here, the strategy σ for G ω decomposes into σ 0 for the first round and σ 1 for later rounds, and x ′ is the state after the first round is completed. This equation does not always have a unique solution -for instance if C G x σ r = r. Hence, to recover uniqueness, we restrict to games G where C G x σ r = r in this paper. This is not a great restriction as in standard game theory there is no coutility. For the sake of presentation, we will also only consider state free games. Next, for F G to type check, the type of utility and coutility of G must be the same, and thus we fix some set R and only consider games whose utility and coutility is R. To summarise, in this paper we consider open games G : (1, R) Σ − → (Y, R) with state 1, utility and coutility the set R, and coutility function C σ r = r. We define morphisms between such games as follows: Definition 5. Let R be a set. Given two games G : (1, R) Σ − → (Y, R) and We trust the reader will not be confused by the fact that games are morphisms in Open but also have morphisms between them -this simply reflects inherent 2-categorical structure. The category whose objects are open games G : (1, R) Σ − → (Y, R) for some Σ, Y (and a fixed R), and whose morphisms are the morphisms between such open games is denoted 2Open R . We are now in position to define the functor F G : 2Open R → 2Open R whose final coalgebra will be the infinite iteration of the game G.
Theorem 6. Let R be a set and G : (1, R) The play function and equilibrium preservation conditions are easily checked.

The iterated game as a final coalgebra
From now on, let R be an arbitrary set, used as utility and coutility for all our games, and write 2Open for 2Open R .

Definition of the iterated game
Let us fix an arbitrary open game G : (1, R) Σ − → (Y, R) that we want to iterate infinitely often via the final coalgebra of the functor F G : 2Open → 2Open from the previous section, mapping H : (1, R) . We first describe F G -coalgebras, then our candidate G ω for the final F G -coalgebra, and conclude with a proof that G ω really is final. As a first step we need to recall two endofunctors on the category of sets and their final coalgebras.
The above final coalgebras are fundamental for our representation of iterated games: The final S(Y )-coalgebra consists of all infinite sequences of moves of the one-round game, while the final D(Y, Σ)-coalgebra represents the set of strategies that map lists of moves -representing moves chosen in previous rounds -to a strategy for the next round. As notation, for σ : Y * → Σ we abbreviate now(σ) to σ 0 , ltr(σ) to σ ′ , and use (::) : Y × Y ω → Y ω to denote the cons-operator on lists. Let us now define the ω-iteration of G.
Notice that the above approach means we do not have to fix a particular utility function Y ω → R in advance by some arbitrary form of discounting, but rather work with all possible utility functions, allowing the user maximum flexibility.
Lemma 9. Let σ ∈ Σ Gω . Then Here ≤ denotes the order on Proof. The first item follows since E Gω is a fixpoint of Φ, the second because it is the greatest such, thus also the greatest post-fixpoint wrt the order ≤.

Proof of finality
In this section we are going to show that G ω is a final coalgebra of the functor F G = (Y → ) • G : 2Open → 2Open. We have two things to show: The first item is formulated in the following proposition -its straightforward proof can be found in the appendix.
We are now ready to prove that G ω indeed is the final F G -coalgebra. To this end we consider an arbitrary F G -coalgebra H with coalgebra map ( now H , ltr H , hd H , tl H ). We have to prove that there is a morphism unf Σ , unf Y : H → G ω such that the following diagram commutes: It is easy to see that such a F G -coalgebra morphism -if it exists -must be unique because commutativity of the above diagram implies commutativity of the following two diagrams in the category of sets: In other words unf Σ and unf Y have to be D(Y, Σ G )-and S(Y )-coalgebra morphisms, respectively, and these are uniquely determined by the fact that their codomains are the respective final coalgebras.
This means that to show that G ω is a final F G -coalgebra, we have to prove that the pair of functions unf Σ , unf Y defined via the diagrams in (1) is a F G -coalgebra morphism. We need several lemmas.
Proof. To see this we define a relation and we prove that Q is a S(Y )-bisimulation, i.e., that for each (τ 1 , τ 2 ) ∈ Q we have hd(τ 1 ) = hd(τ 2 ) and (tl(τ 1 ), tl(τ 2 )) ∈ Q. From the coinduction principle it follows that any two streams related by Q are equal which implies the lemma. The proof that Q is a bisimulation is contained in the appendix.
We now turn to the verification of the equilibrium condition for (unf Σ , unf Y ). First we use (unf Σ , unf Y ) to define an indexed predicate on Σ Gω (which can be thought of as the image of E H under (unf Σ , unf Y )). This predicate will be a post-fixpoint of Φ which will then imply the desired equilibrium condition.
H ) and -as hd H , tl H is a morphism of open games -we obtain now H , ltr H (σ ′ ) ∈ E F G H (k * ). The lemma now follows by spelling out the definition of E F G H (k * ).
We are now ready to prove the key fact thatÊ H is a post-fixpoint of Φ.
We are now ready to prove the main theorem of this section.
Theorem 16. Let G : (1, R) → (Y, R) be an open game and let G ω be its ω-iteration. Then G ω is a final F G -coalgebra.
Proof. By our discussion at the beginning of this subsection it suffices to show that for an arbitrary F G -coalgebra (H, ( now H , ltr H , hd H , tl H )) the map (unf Σ , unf Y ) consisting of the coalgebra morphisms in (1) is a morphism of open games. Lemma 11 shows that (unf Σ , unf Y ) satisfies the morphism condition wrt play functions. For checking the equilibrium condition consider an arbitrary σ ′ ∈ Σ H and a k : Y ω → R such that σ ′ ∈ E H (k • unf Y ). Then clearly we have unf Σ (σ ′ ) ∈Ê H (k). AsÊ H is a post-fixpoint of Φ by Lemma 15, we haveÊ H (k) ⊆ E Gω (k), and thus unf Σ (σ ′ ) ∈ E Gω (k) as required.

Conclusions and Future Work
The main contributions of this paper are on the one hand a notion of morphism between open games and -based on this notion -the representation of the infinite iteration of a given game as a final coalgebra. This provides a first extension of the compositionality results from Ghani et al. (2016) to infinitely repeated games. Nevertheless a number of challenges remain: firstly, we need to extend our construction to state-full games and to games with non-trivial coutility function. The former seems straightforward, at least if we confine ourselves to games that share the same state space X. Secondly, we need to make the link of our work to subgame-perfect equilibria more explicit. Finally, after having represented infinitely repeated games as final coalgebra, we will be able to provide new reasoning tools for such games based on coinduction and coalgebraic logics.

Appendix A. Omitted Proofs
Proof of Prop. 10. The type of α is ok, we need to show that α is a morphism of open games. Firstly we need to check that α interacts well with the play functions, i.e., we need to check that for all σ ∈ Σ Gω we have hd, tl (P Gω σ) = P (Y →Gω)•G ( now, ltr (σ)). This is routine.