A mean-field game price model with noise

In this paper, we propose a mean-field game model for the price formation of a commodity whose production is subjected to random fluctuations. The model generalizes existing deterministic price formation models. Agents seek to minimize their average cost by choosing their trading rates with a price that is characterized by a balance between supply and demand. The supply and the price processes are assumed to follow stochastic differential equations. Here, we show that, for linear dynamics and quadratic costs, the optimal trading rates are determined in feedback form. Hence, the price arises as the solution to a stochastic differential equation, whose coefficients depend on the solution of a system of ordinary differential equations.


Introduction
Mean-field games (MFG) is a tool to study the Nash equilibrium of infinite populations of rational agents. These agents select their actions based on their state and the statistical information about the population. Here, we study a price formation model for a commodity traded in a market under uncertain supply, which is a common noise shared by the agents. These agents are rational and aim to minimize the average trading cost by selecting their trading rate. The distribution of the agents solves a stochastic partial differential equation. Finally, a market-clearing condition characterizes the price.
We consider a commodity whose supply process is described by a stochastic differential equation; that is, we are given a drift b S : [0, T ]×R 2 → R and volatility σ S : [0, T ]×R 2 → R + 0 , which are smooth functions, and the supply Q s is determined by the stochastic differential equation At time t, an agent who holds x and observes q and w chooses a progressively measurable control process v to minimize the expected cost functional J(x, q, w, t; v) = E T t L(X s , v(s)) + s v(s)ds + Ψ (X T , Q T , T ) , (1.4) subject to the dynamics (1.1), (1.2), and (1.3) with initial condition X t = x, and the expectation is taken w.r.t. the standard filtration generated by the Brownian motion. The Lagrangian, L, takes into account costs such as market impact or storage, and the terminal cost Ψ stands for the terminal preferences of the agent. This control problem determines a Hamilton-Jacobi equation addressed in Section 2.1. In turn, each agent selects an optimal control and uses it to adjust its holdings. Because the source of noise in Q t is common to all agents, the evolution of the probability distribution of agents is not deterministic. Instead, it is given by a stochastic transport equation derived in Section 2.2. Finally, the price is determined by a market-clearing condition that ensures that supply meets demand. We study this condition in Section 2.3.
Mathematically, the price model corresponds to the following problem.
Given a solution to the preceding problem, we construct the supply and price processes which also solve and satisfy the market-clearing condition In [10], the authors presented a model where the supply for the commodity was a given deterministic function, and the balance condition between supply and demand gave rise to the price as a Lagrange multiplier. Price formation models were also studied by Markowich et al. [18], Caffarelli et al. [2], and Burger et al. [1]. The behavior of rational agents that control an electric load was considered in [17,16]. For example, turning on or off space heaters controls the electric load as was discussed in [14,15,13]. Previous authors addressed price formation when the demand is a given function of the price [12] or that the price is a function of the demand, see, for example [6], [5], [7], [8], and [11]. An N -player version of an economic growth model was presented in [9].
Noise in the supply together with a balance condition is a central issue in price formation that could not be handled directly with the techniques in previous papers. A probabilistic approach of the common noise is discussed in Carmona et al. in [4]. Another approach is through the master equation, involving derivatives with respect to measures, which can be found in [3]. None of these references, however, addresses problems with integral constraints such as (1.7).
Our model corresponds to the one in [10] for the deterministic setting when we take the volatility for the supply to be 0. Here, we study the linear-quadratic case, that is, when the cost functional is quadratic, and the dynamics (1.1) and (1.2) are linear. In Section 3.2, we provide a constructive approach to get semi-explicit solutions of price models for linear dynamics and quadratic cost. This approach avoids the use of the master equation. The paper ends with a brief presentation of simulation results in Section 4.

The Model
In this section, we derive Problem 1 from the price model. We begin with standard tools of optimal control theory. Then, we derive the stochastic transport equation, and we end by introducing the market-clearing (balance) condition.

Hamilton-Jacobi equation and verification theorem.
The value function for an agent who at time t holds an amount x of the commodity, whose instantaneous supply and price are q and w, is where J is given by (1.4) and the infimum is taken over the set A ((t, T ]) of all progressively measurable functions v : [t, T ] → R. Consider the Hamiltonian, H, which is the Legendre transform of L; that is, for p ∈ R, Then, from standard stochastic optimal control theory, whenever L is strictly convex, if u is C 2 , it solves the Hamilton-Jacobi equation in with the terminal condition u(x, q, w, T ) = Ψ (x, q, w) .
Moreover, as the next verification theorem establishes, any C 2 solution of (2.3) is the value function.

2.2.
Stochastic transport equation. Theorem 2.1 provides an optimal feedback strategy. As usual in MFG, we assume that the agents are rational and, hence, choose to follow this optimal strategy. This behavior gives rise to a flow that transports the agents and induces a random measure that encodes their distribution. Here, we derive a stochastic PDE solved by this random measure. To this end, let u solve (2.3) and consider the random flow associated with the diffusion That is, for a given realization ω of the common noise, the flow maps the initial conditions (x,q,w) to the solution of (2.5) at time t, which we denote by (X ω Definition 2.2. Let ω denote a realization of the common noise W on 0 s T . Given a measurem ∈ P(R) and initial conditionsq,w ∈ R takeμ ∈ P(R 3 ) byμ =m × δq × δw and define a random measure µ t by the mapping ω → µ ω t ∈ P(R 3 ), where µ ω t is characterized as follows: for any bounded and continuous function ψ : Moreover, due to the structure of (2.5), σ(q, w, s) = (0, σ S (q, w, s), σ P (q, w, s)).
A measure-valued stochastic process µ = µ(·, t) = µ t (·) is a weak solution of the stochastic PDE with initial conditionμ if for any bounded smooth test function ψ :

9)
where the arguments for b, σ and ψ are (x, q, w, s) and the differential operators D and D 2 are taken w.r.t. the spatial variables x, q, w.
Theorem 2.5. Letm ∈ P(R) andq,w ∈ R. The random measure from Definition 2.2 is a weak solution of the stochastic partial differential equation (2.6) with initial condition µ =m × δq × δw.
Proof. Let ψ : R 3 × [0, T ] → R be a bounded smooth test function. Consider the stochastic process s → R 3 ψ(x, q, w, s)µ ω s (dx × dq × dw). Let (X t (x,q,w), Q t (q,w), t (q,w)) be the flow induced by (2.5). By the definition of µ ω t , Then, applying Ito's formula to the stochastic process the preceding expression becomes where arguments of b, σ and the partial derivatives of ψ in the integral with respect tom(dx) are (X s (x,q,w), Q s (q,w), s (q,w), s), and in the integral with respect to µ t (dx × dq × dw) are (x, q, w, t). Therefore, Hence, (2.7) holds.

Balance condition.
The balance condition requires the average trading rate to be equal to the supply. Because agents are rational and, thus, use their optimal strategy, this condition takes the form where µ ω t is given by Definition 2.2. Because Q t satisfies a stochastic differential equation, the previous can also be read in differential form as The former condition determines b P and σ P . In general, b P and σ P are only progressively measurable and not in feedback form. In this case, the Hamilton-Jacobi (2.3) must be replaced by either a stochastic partial differential equation or the problem must be modeled by the master equation. However, as we discuss next, in the linear-quadratic case, we can find b P and σ P in feedback form.

Potential-free Linear-quadratic price model
Here, we consider a price model for linear dynamics and quadratic cost. The Hamilton-Jacobi equation admits quadratic solutions. Then, the balance equation determines the dynamics of the price, and the model is reduced to a first-order system of ODE. Suppose that L(x, v) = c 2 v 2 and, thus, H(x, p) = 1 2c p 2 . Accordingly, the corresponding MFG model is Assume further that Ψ is quadratic; that is, The balance condition is Q t = − 1 c ( t + Π t ). Furthermore, Definition 2.2 provides the identity where the arguments for the partial derivatives of u are (X * t (x,q,w), Q t (q,w), t (q,w), t).
Proof. By Itô's formula, the process t → u x (X * t , Q t , t , t) solves with v * (t) = − 1 c ( t + u x (X * t , Q t , t , t)). By differentiating the Hamilton-Jacobi equation, we get The preceding identity simplifies to R u xq σ S + u xw σ P m(dx)dW t .
Using Lemma 3.1, we have Thus, .

Quadratic solutions to the Hamilton Jacobi equation.
If u is a second-degree polynomial with time-dependent coefficients, then are deterministic functions of time. Accordingly, b P and σ P are given in feedback form by (3.4), thus, consistent with the original assumption. Here, we investigate the linearquadratic case that admits solutions of this form. Now, we assume that the dynamics are affine; that is, (3.5) Then, (3.4) gives .
Because all the terms in the Hamilton Jacobi equation are at most quadratic, we seek for solutions of the form u(t, x, q, w) =a 0 (t) + a 1 1 (t)x + a 2 1 (t)q + a 3 1 (t)w + a 1 2 (t)x 2 + a 2 2 (t)xq + a 3 2 (t)xw + a 4 2 (t)q 2 + a 5 2 (t)qw + a 6 2 (t)w 2 , where a j i : [0, T ] → R. Therefore, the previous identities reduce to (3.6) Using (3.6) and grouping coefficients in the Hamilton Jacobi PDE, we obtain the following ODE systeṁ Therefore, Replacing the above in the balance condition at the initial time, that isw = −cq − Π 0 , we obtain the initial condition for the pricē where a 1 1 can be obtained after solving for a 1 2 , a 2 2 and a 3 2 . Now, we proceed with the price dynamics using the balance condition. Under linear dynamics, we have a 2 2 (r) σ S 0 (r) + Q r σ S 1 (r) + r σ S 2 (r) + a 3 2 (r) σ P 0 (r) + Q r σ P 1 (r) + r σ P 2 (r) dW r .
Thus, replacing the price coefficients for (3.6), we obtain which determines the dynamics for the price.

Simulation results
In this section, we consider the running cost corresponding to c = 1; that is, 2 v 2 and terminal cost at time T = 1 Ψ(x) = (x − α) 2 . We takem to be a normal standard distribution; that is, with zero-mean and unit variance. We assume the dynamics for the normalized supply is mean-reverting with initial conditionq = 1. Therefore, the dynamics for the price becomes with initial conditionw given by (3.7), and a 2 2 and a 3 2 solvė a 2 2 = − a 3 2 + a 2 2 (1 + 2a 1 2 ) a 3 2 =2a 1 2 (1 + a 3 2 ), with terminal conditions a 2 2 (1) = 0 and a 3 2 (1) = 0. We observe that the coefficient multiplying Q t in the volatility of the price is now time-dependent. For a fixed simulation of the supply, we compute the price for different values of α. Agents begin with zero energy average. The results are displayed in Figure 1. As expected, the price is negatively correlated with the supply. Moreover, as the storage target increases, prices increase, which reflects the competition between agents who, on average, want to increase their storage.