Linear-Quadratic Mean Field Games

Abstract

We provide a comprehensive study of a general class of linear-quadratic mean field games. We adopt the adjoint equation approach to investigate the unique existence of their equilibrium strategies. Due to the linearity of the adjoint equations, the optimal mean field term satisfies a forward–backward ordinary differential equation. For the one-dimensional case, we establish the unique existence of the equilibrium strategy. For a dimension greater than one, by applying the Banach fixed point theorem under a suitable norm, a sufficient condition for the unique existence of the equilibrium strategy is provided, which is independent of the coefficients of controls in the underlying dynamics and is always satisfied whenever the coefficients of the mean field term are vanished, and hence, our theories include the classical linear-quadratic stochastic control problems as special cases. As a by-product, we also establish a neat and instructive sufficient condition, which is apparently absent in the literature and only depends on coefficients, for the unique existence of the solution for a class of nonsymmetric Riccati equations. Numerical examples of nonexistence of the equilibrium strategy will also be illustrated. Finally, a similar approach has been adopted to study the linear-quadratic mean field type stochastic control problems and their comparisons with mean field games.

Appendix: Comparison of the Approaches of [44] and the Present Paper

We consider the single-agent problem studied by [44] (HCM), that is, the distribution F(a) in HCM is now a Dirac distribution, to compare the approaches of HCM and the present paper (BSYY) on the same problem. More precisely, we want to find the optimal control u which minimizes the cost functional , for $\gamma \in {\mathbb {R}}^{n\times n}$ and $\eta \in {\mathbb {R}}^{n}$,

$$\begin{aligned} J(u) = {\mathbb {E}}\,\left[ \int ^T_0|z_t - \gamma (\bar{z}_t+\eta )|^2 + ru_t^2\,\hbox {d}t\right] , \end{aligned}$$

(13)

where

$$\begin{aligned} \hbox {d}z_t = (az_t + bu_t)\,\hbox {d}t + \alpha \bar{z}_t\,\hbox {d}t + \sigma \,\hbox {d}w_t, \quad z(0) = z_0, \end{aligned}$$

(14)

$\bar{z}_t$ is fixed, deterministic, and $z_0$ is a random variable with zero mean, independent of the Wiener process. At the second stage, we consider a fixed point problem

$$\begin{aligned} \bar{z}_t = {\mathbb {E}}\,[z_t], \end{aligned}$$

(15)

where $z_t$ is the optimal state of the control problem (13), (14). In the sequel, we shall describe both approaches in detail to make the comparison explicit. As in HCM, we use the notation $z_t^* := \gamma (\bar{z}_t+\eta )$.

1.1 HCM Approach

For given $\bar{z}$, one solves the stochastic control problem (13), (14) by the Riccati differential equation approach. The optimal control is given by

$$\begin{aligned} u_t = -\frac{b}{r}({\varPi }_t z_t + s_t), \end{aligned}$$

(16)

where ${\varPi }_t$ is the positive solution of the Riccati equation

$$\begin{aligned} \frac{\hbox {d}{\varPi }_t}{\hbox {d}t} + 2a{\varPi }_t - \frac{b^2}{r}{\varPi }_t^2 + 1 = 0, \quad {\varPi }_T = 0, \end{aligned}$$

(17)

and $s_t$ solves the linear differential equation

$$\begin{aligned} \frac{\hbox {d}s_t}{\hbox {d}t} + \left( a - \frac{b^2}{r}{\varPi }_t\right) s_t + \alpha {\varPi }_t \bar{z}_t - z_t^* = 0, \quad s_T = 0. \end{aligned}$$

(18)

Therefore, from (14), the optimal trajectory satisfies

$$\begin{aligned} \hbox {d}z_t = \left( a - \frac{b^2}{r}{\varPi }_t\right) z_t\,\hbox {d}t + \left( -\frac{b^2}{r}s_t + \alpha \bar{z}_t\right) \,\hbox {d}t + \sigma \,\hbox {d}w_t, \quad z(0) = z_0. \end{aligned}$$

Furthermore, to satisfy (15), we must have

$$\begin{aligned} \frac{\hbox {d}\bar{z}_t}{\hbox {d}t} = \left( a - \frac{b^2}{r}{\varPi }_t\right) \bar{z}_t - \frac{b^2}{r}s_t + \alpha \bar{z}_t, \quad \bar{z}(0) = 0, \end{aligned}$$

(19)

and it suffices to find a solution of the deterministic system (18), (19). We now state the sufficient condition in HCM that guarantees the unique solution for the system (18), (19).

1.2 HCM Proof

Introduce

$$\begin{aligned} {\varPhi }(t, \tau ) = \exp \left( -\int ^t_\tau \left( a - \frac{b^2}{r}{\varPi }_{\sigma }\right) \hbox {d}\sigma \right) , \end{aligned}$$

and note that t is not necessarily greater than $\tau$. Solving (18) by the formula

$$\begin{aligned} s_t = \int ^T_t{\varPhi }(t,\tau )\left( \left( \alpha {\varPi }_{\tau } - \gamma \right) \bar{z}_{\tau } - \gamma \eta \right) \,\hbox {d}\tau , \end{aligned}$$

and substituting in (19) yields

$$\begin{aligned} \bar{z}_t = \int ^t_0 {\varPhi }(\sigma ,t)\left\{ \alpha \bar{z}_{\sigma } - \frac{b^2}{r}\int ^T_{\sigma }{\varPhi }(\sigma ,\tau )\big ((\alpha {\varPi }_{\tau } - \gamma )\bar{z}_{\tau } - \gamma \eta \big )\,\hbox {d}\tau \right\} \,\hbox {d}\sigma . \end{aligned}$$

(20)

The problem is reduced to find a fixed point of Eq. (20), and HCM uses the contraction principle to solve for it in the space C(0, T).

For simplicity, let $\eta =0$. Consider a continuous function $\varphi$ and the linear map

$$\begin{aligned} {\varGamma } \varphi (t) = \int ^t_0 {\varPhi }(\sigma , t)\left\{ \alpha \varphi _{\sigma } - \frac{b^2}{r} \int ^T_{\sigma }{\varPhi }(\sigma ,\tau )(\alpha {\varPi }_{\tau } - \gamma ) \varphi _{\tau }\,\hbox {d}\tau \right\} \,\hbox {d}\sigma , \end{aligned}$$

then the norm $\Vert {\varGamma }\Vert$ must be required to be strictly less than 1. Since ${\varPhi }$ and ${\varPi }$ are positive, we have

$$\begin{aligned} |{\varGamma }\varphi (t)| \le \Vert \varphi \Vert \int ^t_0 {\varPhi }(\sigma , t)\left\{ |\alpha | + \frac{b^2}{r} \int ^T_{\sigma }{\varPhi }(\sigma ,\tau )(|\alpha | {\varPi }_{\tau } + |\gamma |)\,\hbox {d}\tau \right\} \,\hbox {d}\sigma , \end{aligned}$$

and thus, the assumption

$$\begin{aligned} \Vert {\varGamma }\Vert \le \sup _{0 < t < T}\int ^t_0 {\varPhi }(\sigma , t)\left\{ |\alpha | + \frac{b^2}{r} \int ^T_{\sigma }{\varPhi }(\sigma ,\tau )(|\alpha | {\varPi }_{\tau } + |\gamma |)\,\hbox {d}\tau \right\} \,\hbox {d}\sigma < 1,\nonumber \\ \end{aligned}$$

(21)

guarantees the contraction property.

Remark 7.1

There are three typos in the statement of the condition in HCM. On Page 170 (although the previous one is corrected), $\frac{b^2}{r}$ appears as a multiplicative factor for the whole right-hand side of (21), which should not be. Also the integral sign $\int ^t_0$ is written as $\int ^T_0$. On Page 171, the integral sign $\int ^T_{\sigma }$ is written as $\int ^T_0$.

Correcting these typos, this is the HCM result in the present framework.

1.3 BSYY Approach

The present paper uses the stochastic maximum principle to solve (13), (14). The optimal control is

$$\begin{aligned} u_t = -\frac{b}{r}\,p_t, \end{aligned}$$

(22)

where $p_t = {\mathbb {E}}\,[\omega _t|{\mathcal {F}}_t]$, ${\mathcal {F}}_t$ is the filtration generated by $z_0$, and the Wiener process up to time t and $\omega _t$ solves the adjoint equation

$$\begin{aligned} -\frac{\hbox {d}\omega _t}{\hbox {d}t} = a \omega _t + z_t - z^*_t, \quad \omega _T = 0. \end{aligned}$$

Combining the following system of necessary conditions holds:

$$\begin{aligned} \begin{aligned} \hbox {d}z_t&= \left( az_t - \frac{b^2}{r}\,p_t + \alpha \bar{z}_t\right) \,\hbox {d}t + \sigma \,\hbox {d}w_t,\\ z(0)&= z_0,\\ -\frac{d\omega _t}{\hbox {d}t}&= a\omega _t + z_t - z^*_t,\\ \omega _T&= 0, \end{aligned} \end{aligned}$$

(23)

where $p_t = {\mathbb {E}}\,[\omega _t | {\mathcal {F}}_t]$. It is well known that $p_t = {\varPi }_tz_t + s_t$ and (16), (22) coincide. The difference concerns the fixed point argument.

In our approach, we define $\bar{z}_t = {\mathbb {E}}\,[z_t]$ and $\bar{p}_t = {\mathbb {E}}\,[p_t] = {\mathbb {E}}[\omega _t]$. Therefore, (23) becomes

$$\begin{aligned} \begin{aligned} \frac{\hbox {d}\bar{z}_t}{\hbox {d}t}&= (a + \alpha )\bar{z}_t - \frac{b^2}{r}\,\bar{p}_t ,\\ \bar{z}(0)&= 0,\\ -\frac{\hbox {d}\bar{p}_t}{\hbox {d}t}&= a\bar{p}_t + (1 - \gamma )\bar{z}_t - \gamma \eta ,\\ \bar{p}_T&= 0, \end{aligned} \end{aligned}$$

(24)

which we shall solve. Again, $\bar{p}_t = {\varPi }_t\bar{z}_t + s_t$, and the systems (24) and (18), (19) are equivalent.

1.4 BSYY Proof and Nonsymmetric Riccati Equation

The trouble with the relation $\bar{p}_t = {\varPi }_t \bar{z}_t + s_t$ is that it does not express the adjoint variable $\bar{p}_t$ as an affine function of $\bar{z}_t$ alone. In fact, we need both $\bar{z}_t$ and $s_t$, which is a coupled system, to be solved first, as done in HCM. Our approach is to express $\bar{p}_t$ as an affine function on $\bar{z}_t$ only. We write

$$\begin{aligned} \bar{p}_t = P_t\bar{z}_t + \rho _t, \end{aligned}$$

(25)

and by identification, we obtain:

$$\begin{aligned} \frac{\hbox {d}P_t}{\hbox {d}t}&= -(2a + \alpha )P_t + \frac{b^2}{r}P^2_t - 1 + \gamma , \quad P_T = 0, \nonumber \\ \frac{\hbox {d}\rho _t}{\hbox {d}t}&= -\left( a - \frac{b^2}{r}{P_t}\right) \rho _t + \gamma \eta , \quad \rho _T = 0. \end{aligned}$$

(26)

The Riccati Eq. (26) is different from (17), and it is called a nonsymmetric Riccati equation because in dimension larger than 1, it leads to (nonstandard) nonsymmetric Riccati equations. Solving (24) amounts to solve the Riccati equation (26), since using (25) in (24), $\bar{z}_t$ is a solution of linear equation.

If we assume that

$$\begin{aligned} \gamma \le 1 \end{aligned}$$

(27)

(which is independent of the choice of b), then the second-order equation

$$\begin{aligned} -\frac{b^2}{r}\varsigma ^2 + (2a + \alpha )\varsigma + 1 - \gamma = 0 \end{aligned}$$

has two roots $\varsigma _1 \ge 0$, $\varsigma _2 \le 0$, and the solution of the Riccati equation is

$$\begin{aligned} P_t = \frac{(1-\gamma )r}{b^2}\frac{\exp \left( (\varsigma _1 - \varsigma _2)\frac{b^2}{r}(T-t)\right) - 1}{\varsigma _1-\varsigma _2\exp \left( (\varsigma _1 - \varsigma _2)\frac{b^2}{r}(T-t)\right) }. \end{aligned}$$

We can compare assumption (27) with respect to assumption (21), in obtaining the fixed point property. For instance, take $a = 0$, $\alpha = 0$ and $r = 1$, Condition (21) means that

$$\begin{aligned} \sup _{0 < t < T}b^2|\gamma |\int ^t_0{\varPhi }(\sigma ,t)\left( \int ^T_{\sigma }{\varPhi }(\sigma ,\tau )\,\hbox {d}\tau \right) \,\hbox {d}\sigma < 1 \end{aligned}$$

(28)

where

$$\begin{aligned} {\varPhi }(t, \tau ) = \exp \left( b^2\int ^t_{\tau }{\varPi }_{\sigma }\,\hbox {d}\sigma \right) \end{aligned}$$

and

$$\begin{aligned} \frac{\hbox {d}{\varPi }_t}{\hbox {d}t} - b^2{\varPi }_t^2 + 1= 0, \quad {\varPi }_T = 0. \end{aligned}$$

Thus,

$$\begin{aligned} b{\varPi }_t + 1 = \frac{2}{1 + \hbox {e}^{-2b(T-t)}}, \end{aligned}$$

and from (28),

$$\begin{aligned} \sup _{0 < t < T}b^2|\gamma |\int ^t_0\exp \left( \int ^{\sigma }_tb^2 {\varPi }_{\lambda }\,\hbox {d}\lambda \right) \left\{ \int ^T_{\sigma }\exp \left( \int ^{\sigma }_{\tau }b^2{\varPi }_{\mu }\,\hbox {d}\mu \right) \,\hbox {d}\tau \right\} \,d\sigma < 1, \end{aligned}$$

which means that

$$\begin{aligned} \sup _{0 < t < T}b^2|\gamma |\int ^t_0\exp \left( -\int ^t_{\sigma }b^2 {\varPi }_{\lambda }\,\hbox {d}\lambda \right) \left\{ \int ^T_{\sigma }\exp \left( -\int _{\sigma }^{\tau }b^2{\varPi }_{\mu }\,\hbox {d}\mu \right) \,\hbox {d}\tau \right\} \,\hbox {d}\sigma < 1.\nonumber \\ \end{aligned}$$

(29)

Since

$$\begin{aligned} b{\varPi }_t = \frac{1 - \hbox {e}^{-2b(T-t)}}{1 + \hbox {e}^{-2b(T-t)}} < 1, \end{aligned}$$

from (29), we obtain

$$\begin{aligned} 1&> \sup _{0 < t < T}b^2|\gamma |\int ^t_0\hbox {e}^{-b(t-\sigma )}\left\{ \int ^T_{\sigma }\hbox {e}^{-b(\tau - \sigma )}\,\hbox {d}\tau \right\} \,\hbox {d}\sigma \\&= \sup _{0 < t < T}b|\gamma |\int ^t_0\hbox {e}^{-b(t-\sigma )}(1 - \hbox {e}^{-b(T-\sigma )})\,\hbox {d}\sigma \\&= |\gamma |\sup _{0 < t < T}\left( 1 - \hbox {e}^{-bt}-\frac{1}{2}\hbox {e}^{-b(T-t)}+\frac{1}{2}\hbox {e}^{-b(T+t)}\right) , \end{aligned}$$

which amounts to

$$\begin{aligned} |\gamma |\,(1 - \hbox {e}^{-bT}) < 1. \end{aligned}$$

(30)

Obviously, (27) and (30) are not equivalent. We remark that in order to guarantee the existence uniformly for arbitrarily choice of b, (30) only holds when $|\gamma | \le 1$.

1.5 Generalization to High Dimensions

Both approaches can be considered in n dimension. However, the condition for the existence and uniqueness of the fixed point in the HCM approach becomes extremely difficult to be checked, since it involves the solution of Riccati equations. Our approach leads to conditions which are much easier to be verified and also introduces an interesting and new direction for solvable nonsymmetric Riccati equations that do not correspond to any usual control problems at all. The contribution of BSYY provides a complement to HCM theory, with insights which deserve to be known.