Importance Sampling for the Empirical Measure of Weakly Interacting Diffusions

Abstract

We construct an importance sampling method for computing statistics related to rare events for weakly interacting diffusions. Standard Monte Carlo methods behave exponentially poorly with the number of particles in the system for such problems. Our scheme is based on subsolutions of a Hamilton–Jacobi–Bellman (HJB) equation on Wasserstein space which arises in the theory of mean-field (McKean–Vlasov) control. We identify conditions under which such a scheme is asymptotically optimal. In the process, we make connections between the large deviations principle for the empirical measure of weakly interacting diffusions, mean-field control, and the HJB equation on Wasserstein space. We also provide evidence, both analytical and numerical, that with sufficient regularity of the HJB equation, our scheme can have vanishingly small relative error in the many particle limit.

Appendix A. Differentiation on Spaces of Measures

Definition A.1

Given a function $u:\mathcal { P}_2(\mathbb {R}^d)\rightarrow \mathbb {R}$, we may define a lifting of u to $\tilde{u}:L^2({\tilde{\Omega }},{\tilde{\mathcal {F}}},{\tilde{\mathbb {P}}};\mathbb {R}^d)\rightarrow \mathbb {R}$ via ${\tilde{u}} (X) = u(\mathcal { L}(X))$ for $X\in L^2({\tilde{\Omega }},{\tilde{\mathcal {F}}},{\tilde{\mathbb {P}}};\mathbb {R}^d)$. Here we assume ${\tilde{\Omega }}$ is a Polish space, ${\tilde{\mathcal {F}}}$ its Borel $\sigma$-field, and ${\tilde{\mathbb {P}}}$ is an atomless probability measure (since ${\tilde{\Omega }}$ is Polish, this is equivalent to every singleton having zero measure).

Here:

$$\begin{aligned} \mathcal { P}_2(\mathbb {R}^d) {:}{=}\lbrace \mu \in \mathcal { P}(\mathbb {R}^d): \int _{\mathbb {R}^d}|x|^2 \mu (dx)<\infty \rbrace . \end{aligned}$$

$\mathcal { P}_2(\mathbb {R}^d)$ is a Polish space under the $L^2$-Wasserstein distance

$$\begin{aligned} \mathbb {W}_2 (\mu _1,\mu _2){:}{=}\inf _{\pi \in \mathcal { C}_{\mu _1,\mu _2}} \biggl [\int _{\mathbb {R}^d\times \mathbb {R}^d} |x-y|^2 \pi (dx,dy)\biggr ]^{1/2}, \end{aligned}$$

where $\mathcal { C}_{\mu _1,\mu _2}$ denotes the set of all probability measures on $\mathbb {R}^{d}\times \mathbb {R}^d$ with first marginal $\mu _1$ and second marginal $\mu _2$.

We say u is L-differentiable or Lions-differentiable at $\mu _0\in \mathcal { P}_2(\mathbb {R}^d)$ if there exists a random variable $X_0$ on some $({\tilde{\Omega }},{\tilde{\mathcal {F}}},{\tilde{\mathbb {P}}})$ satisfying the above assumptions such that $\mathcal { L}(X_0)=\mu _0$ and ${\tilde{u}}$ is Fréchet differentiable at $X_0$.

The Fréchet derivative of ${\tilde{u}}$ can be viewed as an element of $L^2({\tilde{\Omega }},{\tilde{\mathcal {F}}},{\tilde{\mathbb {P}}};\mathbb {R}^d)$ by identifying $L^2({\tilde{\Omega }},{\tilde{\mathcal {F}}},{\tilde{\mathbb {P}}};\mathbb {R}^d)$ and its dual. From this, one can find that if u is L-differentiable at $\mu _0\in \mathcal { P}_2(\mathbb {R}^d)$, there is a deterministic measurable function $\xi : \mathbb {R}^d\rightarrow \mathbb {R}^d$ such that $D\tilde{u}(X_0)=\xi (X_0)$, and that $\xi$ is uniquely defined $\mu _0$-almost everywhere on $\mathbb {R}^d$. We denote this equivalence class of $\xi \in L^2(\mathbb {R}^d,\mu _0;\mathbb {R}^d)$ by $\partial _\mu u(\mu _0)$ and call $\partial _\mu u(\mu _0)[\cdot ]:\mathbb {R}^d\rightarrow \mathbb {R}^d$ the Lions derivative of u at $\mu _0$. Note that this definition is independent of the choice of $X_0$ and $({\tilde{\Omega }},{\tilde{\mathcal {F}}},{\tilde{\mathbb {P}}})$. See [19] Section 5.2.

To avoid confusion when u depends on more variables than just $\mu$, if $\partial _\mu u(\mu _0)$ is differentiable at $z_0\in \mathbb {R}^d$, we denote its derivative at $z_0$ by $\partial _z\partial _\mu u(\mu _0)[z_0]$.

Definition A.2

([19] Definition 5.83) We say $u:\mathcal { P}_2(\mathbb {R}^d)\rightarrow \mathbb {R}$ is Fully $\mathbf {C^2}$ if the following conditions are satisfied:

• (1) u is $C^1$ in the sense of L-differentiation, and its first derivative has a jointly continuous version $\mathcal { P}_2(\mathbb {R}^d)\times \mathbb {R}^d\ni (\mu ,z)\mapsto \partial _\mu u(\mu )[z]\in \mathbb {R}^d$.

• (2) For each fixed $\mu \in \mathcal { P}_2(\mathbb {R}^d)$, the version of $\mathbb {R}^d\ni z\mapsto \partial _\mu u(\mu )[z]\in \mathbb {R}^d$ from the first condition is differentiable on $\mathbb {R}^d$ in the classical sense and its derivative is given by a jointly continuous function $\mathcal { P}_2(\mathbb {R}^d)\times \mathbb {R}^d\ni (\mu ,z)\mapsto \partial _z\partial _\mu u(\mu )[z]\in \mathbb {R}^{d\times d}$.

• (3) For each fixed $z\in \mathbb {R}^d$, the version of $\mathcal { P}_2(\mathbb {R}^d)\ni \mu \mapsto \partial _\mu u(\mu )[z]\in \mathbb {R}^d$ in the first condition is continuously L-differentiable component-by-component, with a derivative given by a function $\mathcal { P}_2(\mathbb {R}^d)\times \mathbb {R}^d\times \mathbb {R}^d\ni (\mu ,z,z')\mapsto \partial ^2_\mu u(\mu )[z][z']\in \mathbb {R}^{d\times d}$ such that for any $\mu \in \mathcal { P}_2(\mathbb {R}^d)$ and $X\in L^2({\tilde{\Omega }},{\tilde{\mathcal {F}}},{\tilde{\mathbb {P}}};\mathbb {R}^d)$ with $\mathcal { L}(X)=\mu$, $\partial ^2_{\mu }u(\mu )[z][X]$ gives the Fréchet derivative at X of $L^2({\tilde{\Omega }},{\tilde{\mathcal {F}}},{\tilde{\mathbb {P}}};\mathbb {R}^d)\ni X'\mapsto \partial _\mu u(\mathcal { L}(X'))[z]$ for every $z\in \mathbb {R}^d$. Denoting $\partial ^2_\mu u(\mu )[z][z']$ by $\partial ^2_\mu u(\mu )[z,z']$, the map $\mathcal { P}_2(\mathbb {R}^d)\times \mathbb {R}^d\times \mathbb {R}^d\ni (\mu ,z,z')\mapsto \partial ^2_\mu u(\mu )[z,z']$ is also assumed to be continuous in the product topology.

We recall now a useful connection between the Lions derivative as defined in A.1 and the empirical measure.

Proposition A.3

For $g:\mathcal { P}_2(\mathbb {R}^d)\rightarrow \mathbb {R}$ which is fully $C^2$ in the sense of definition A.2, we can define the empirical projection of g, as $g^N: (\mathbb {R}^d)^N\rightarrow \mathbb {R}$ given by

$$\begin{aligned} g^N(x_1,...,x_N){:}{=}g\left( \frac{1}{N}\sum _{i=1}^N \delta _{x_i}\right) . \end{aligned}$$

Then $g^N$ is twice differentiable on $(\mathbb {R}^d)^N$, and for each $x_1,..,x_N\in \mathbb {R}^d$, $(i,j)\in \lbrace 1,...,N \rbrace ^2$:

$$\begin{aligned} \partial _{x_i} g^N(x_1,...,x_N)= \frac{1}{N} \partial _\mu g\left( \frac{1}{N}\sum _{i=1}^N \delta _{x_i}\right) [x_i] \end{aligned}$$

(64)

and

$$\begin{aligned} \partial _{x_i} \partial _{x_j} g^N(x_1,...,x_N)&= \frac{1}{N} \partial _z \partial _\mu g\left( \frac{1}{N}\sum _{i=1}^N \delta _{x_i}\right) [x_i] \mathbbm {1}_{i=j} \nonumber \\&\quad + \frac{1}{N^2} \partial ^2_\mu g\left( \frac{1}{N}\sum _{i=1}^N \delta _{x_i}\right) [x_i,x_j]. \end{aligned}$$

(65)

Proof

This follows from Propositions 5.35 and 5.91 of [19]. $\square$