A Large Deviation Principle and an Expression of the Rate Function for a Discrete Stationary Gaussian Process

Abstract

We prove a large deviation principle for a stationary Gaussian process over ℝ^b indexed by ℤ^d (for some positive integers d and b), with positive definite spectral density, and provide an expression of the corresponding rate function in terms of the mean of the process and its spectral density. This result is useful in applications where such an expression is needed.

1. Introduction

In this paper, we prove a large deviation principle (LDP) for a spatially-stationary, indexed by ℤ^d, Gaussian process over ℝ^b and obtain an expression for the rate function. Our work in mathematical neuroscience involves the search for asymptotic descriptions of large ensembles of neurons [1–3]. Since there are many sources of noise in the brains of mammals [4], the mathematician interested in modeling certain aspects of brain functioning is often led to consider spatial Gaussian processes that (s)he uses to model these noise sources. This motivates us to use large deviation techniques. Being also interested in formulating predictions that can be experimentally tested in neuroscience laboratories, we strive to obtain analytical results, i.e., effective results from which, for example, numerical simulations can be developed. This is why we determine a more tractable expression for the rate function in this article.

Our result concerns the large deviations of ergodic phenomena, the literature of which we now briefly survey. Donsker and Varadhan obtained a large deviation estimate for the law governing the empirical process generated by a Markov process [5]. They then determined a large deviation principle for a ℤ-indexed stationary Gaussian process, obtaining a particularly elegant expression for the rate function using spectral theory. Chiyonobu et al. [6–8] obtain a large deviation estimate for the empirical measure generated by ergodic processes satisfying certain mixing conditions. Baxter et al. [9] obtain a variety of results for the large deviations of ergodic phenomena, including one for the large deviations of ℤ-indexed ℝ^b-valued stationary Gaussian processes. Steinberg et al. [10] have proven an LDP for a stationary ℤ^d-indexed Gaussian process over ℝ, and [11] obtain an LDP for an ℝ-indexed, ℝ-valued stationary Gaussian process.

In the work we are developing [12], we need large deviation results for spatially-ergodic Ornstein–Uhlenbeck processes. This requires Theorem 1 of this paper.

In the first section, we make some preliminary definitions and state the chief theorem, Theorem 1, for zero mean processes. In the second section, we prove the theorem. In Appendix A, we state and prove several identities involving the relative entropy, which are necessary for the proof of Theorem 1. In Appendix B, we prove a general result for the large deviations of exponential approximations. We prove Corollary 2, which extends the result in Theorem 1 to non-zero mean processes in the Appendix C.

2. Preliminary Definitions

For some topological space Ω equipped with its Borelian σ-algebra B(Ω), we denote the set of all probability measures by $ℳ$(Ω). We equip $ℳ$(Ω) with the topology of weak convergence. Our process is indexed by ℤ^d. For j ∈ ℤ^d, we write j = (j(1),…,j(d)). For some positive integer n > 0, we let V_n = {j ∈ ℤ^d: |j(δ)| ≤ n for all 1 ≤ δ ≤ d}. Let $\mathcal{T}={ℝ}_b$, for some positive integer b. We equip $\mathcal{T}$ with the Euclidean topology and ${\mathcal{T}}^{{\mathbb{Z}}^d}$ with the cylindrical topology, and we denote the Borelian σ-algebra generated by this topology by $B({\mathcal{T}}^{{\mathbb{Z}}^d})$. For some $\mu \in ℳ({\mathcal{T}}^{{\mathbb{Z}}^d})$ governing a process ${(X^j)}_{j\in {\mathbb{Z}}^d}$, we let ${\mu }^{V_n}\in ℳ({\mathcal{T}}^{V_n})$ denote the marginal governing ${(X^j)}_{j\in V_n}$. For some j ∈ ℤ^d, let the shift operator $S^j:{\mathcal{T}}^{{\mathbb{Z}}^d}\to {\mathcal{T}}^{{\mathbb{Z}}^d}$ be S(ω)^k = ω^j+k. We let ${ℳ}_s({\mathcal{T}}^{{\mathbb{Z}}^d})$ be the set of all stationary probability measures μ on $({\mathcal{T}}^{{\mathbb{Z}}^d},B({\mathcal{T}}^{{\mathbb{Z}}^d}))$, such that, for all j ∈ ℤ^d, μ ○ (S^j)⁻¹ = μ. We use Herglotz’s theorem to characterize the law $Q\in {ℳ}_s({\mathcal{T}}^{{\mathbb{Z}}^d})$ governing a stationary process (X^j) in the following manner. We assume that E[X^j] = 0 and:

$$E[X_0{}^{†}{X}_k]=\frac{1}{{(2\pi )}^d}{\displaystyle {\int }_{{[-\pi ,\pi ]}^d}\exp (i\langle k,\theta \rangle )}\tilde{G}(\theta )d\theta .$$

(1)

Our convention throughout this paper is to denote the transpose of X as ^†X and the spectral density with a tilde. 〈⋅,⋅〉 is the standard inner product on ℝ^b. Here, $\tilde{G}(\theta )$ is a continuous function [−π, π]^d → C^b×b, where we consider [−π, π]^d to have the topology of a torus. In addition, $\tilde{G}(-\theta ){=}^{†}\tilde{G}(\theta )=\overline{\tilde{G}}$ ( $\overline{x}$ indicates the complex conjugate of x). We assume that for all θ ∈ [−π, π]^d, det $\tilde{G}(\theta )>{\tilde{G}}_{min}$ for some ${\tilde{G}}_{min}>0$ from which it follows that for each θ, $\tilde{G}(\theta )$ is Hermitian positive definite. If x ∈ C^b, then ^†xX^j is a stationary sequence with spectral density ${}^{†}x\tilde{G}\overline{x}$. We employ the operator norm over C^b×b. Let $p_n:{\mathcal{T}}^{{\mathbb{Z}}^d}\to {\mathcal{T}}^{{\mathbb{Z}}^d}$ be such that p_n (ω)^k = ω^k. Here, and throughout the paper, we take k mod V_n to be the element l ∈ V_n, such that, for all 1 ≤ γ ≤ d, l(γ) = k(γ) mod (2n + 1). Define the process-level empirical measure ${\widehat{\mu }}_n:{\mathcal{T}}^{{\mathbb{Z}}^d}\to {ℳ}_s({\mathcal{T}}^{{\mathbb{Z}}^d})$ as:

$${\widehat{\mu }}_n(\omega )=\frac{1}{{(2n+1)}^d}{\displaystyle \sum _{k\in V_n}{\delta }_{S^k{p}_n(\omega )}.}$$

(2)

Let ${\prod }^n\in ℳ\left({ℳ}_s({\mathcal{T}}^{{\mathbb{Z}}^d})\right)$ be the image law of Q under ${\widehat{\mu }}_n$. We note that in this definition, we need not have chosen V_n to have an odd side length (2n + 1): this choice is for notational convenience, and these results could easily be reproduced in the case that V_n has side length n.

In the context of mathematical neuroscience, (X^j) could correspond to a model of interacting neurons on a lattice (d = 1, 2 or 3), as in [12,13]. We note that the large deviation principle of this paper may be used to obtain an LDP for other processes using standard methods, such as the contraction principle or Lemma 13.

Definition 1. Let (Ω, H) be a measurable space and μ, ν probability measures.

$$I^{(2)}(\mu \Vert v)=\underset{f\in B}{\sup }\{{\mathbb{E}}^{\mu }[f]-\log {\mathbb{E}}^v[\exp (f)]\},$$

where B is the set of all bounded measurable functions. If Ω is Polishand H = B(Ω), then we only need to take the supremum over the set of all continuous bounded functions.

Let (Y^j) be a stationary Gaussian process on $\mathcal{T}$, such that E[Y ^j] = 0, E[Y ^j†Y^k] = 0 and E[Y^{j †}Y^j] = Id _b. Each Y^j is governed by a law P, and we write the governing law in ${ℳ}_s({\mathcal{T}}^{{\mathbb{Z}}^d})$ as $P^{{\mathbb{Z}}^d}$. It is clear that the governing law over V_n may be written as $P^{\otimes V_n}$ (that is the product measure of P, indexed over V_n).

Definition 2. Let ε₂ be the subset of ${ℳ}_s({\mathcal{T}}^{{\mathbb{Z}}^d})$ defined by:

$${\epsilon }_2=\{\mu \in {ℳ}_s({\mathcal{T}}^{{\mathbb{Z}}^d})|{\mathbb{E}}^{\mu }[{\Vert {\omega }^0\Vert }^2]<\infty \}.$$

We define the process-level entropy to be, for $\mu \in {ℳ}_s({\mathcal{T}}^{{\mathbb{Z}}^d})$:

$$I^{(3)}(\mu )=\underset{n\to \infty }{\lim }\frac{1}{{(2n+1)}^d}{I}^{(2)}({\mu }^{V_n}\Vert P^{\otimes V_n}).$$

It is a consequence of Lemma 11 that if μ ∉ ε₂, then I⁽³⁾(μ) = ∞. For further discussion of this rate function and a proof that I⁽³⁾ is well-defined, see [8].

Definition 3. A sequence of probability laws (Γⁿ) on some topological space Ω equipped with its Borelian σ-algebra is said to satisfy a strong large deviation principle (LDP) with rate function I: Ω → ℝ if I is lower semicontinuous, for all open sets O,

$$\underset{n\to \infty }{\underset{¯}{\lim }}\frac{1}{{(2n+1)}^d}\log {\mathrm{\Gamma }}^n(O)\ge -\underset{x\in O}{\inf }I(x)$$

and for all closed sets F:

$$\underset{n\to \infty }{\overline{\lim }}\frac{1}{{(2n+1)}^d}\log {\mathrm{\Gamma }}^n(F)\le -\underset{x\in F}{\inf }I(x).$$

If, furthermore, the set {x: I(x) ≤ α} is compact for all α ≥ 0, we say that I is a good rate function.

Definition 4. For μ ∈ ε₂, we denote the C^b × C^b-valued spectral measure on ([−π, π]^d, B([−π, π]^d)) (which exists due to Herglotz’s theorem) by $\tilde{\mu }$. We have:

$$\frac{1}{{(2\pi )}^d}{\displaystyle {\int }_{{[-\pi ,\pi ]}^d}\exp (i\langle k,\theta \rangle )d\tilde{\mu }}(\theta )={\mathrm{E}}^{\mu }[{\omega }^{0†}{\omega }^k].$$

For For θ ∈ [−π, π]^d let $\tilde{H}(\theta )=\tilde{G}{(\theta )}^{-\frac{1}{2}}$ be the Hermitian positive definite square root of $\tilde{G}{(\theta )}^{-1}$ and:

$$\tilde{H}(\theta )={\displaystyle \sum _{j\in Z^d}{H}^j\exp (-i\langle j,\theta \rangle ).}$$

(3)

The b × b matrices H^j are the coefficients of the absolutely convergent Fourier series (due to Wiener’s theorem) of ${\tilde{G}}^{-1/2}$. Define $\beta :{\mathcal{T}}^{Z^d}\to {\mathcal{T}}^{Z^d}$ as follows:

$${(\beta (\omega ))}^k={{\displaystyle \sum _{j\in Z^d}{}^{†}H}}^j{\omega }^{k-j}.$$

The theorem below is the chief result of this paper.

Theorem 1. (∏ⁿ) satisfies a strong LDP with good rate function I⁽³⁾ (μ ○ β⁻¹). Here:

$$I^{(3)}(\mu \circ {\beta }^{-1})=\{\begin{array}{ll}{I}^{(3)}(\mu )-\mathrm{\Gamma }(\mu )\hfill & if\mu \in {\epsilon }_2\hfill \\ \infty \hfill & o\mathit{\text{the}}rwise,\hfill \end{array}$$

(4)

where:

$$\mathrm{\Gamma }(\mu )=\{\begin{array}{ll}{\mathrm{\Gamma }}_1+{\mathrm{\Gamma }}_2(\mu )\hfill & if\mu \in {\epsilon }_2\hfill \\ 0\hfill & o\mathit{\text{the}}rwise.\hfill \end{array}$$

Here:

$$\begin{array}{c}{\mathrm{\Gamma }}_1=-\frac{1}{2{(2\pi )}^d}{\displaystyle {\int }_{{[-\pi ,\pi ]}^d}\log \det \tilde{G}(\theta )d\theta }\\ {\mathrm{\Gamma }}_2(\mu )=\frac{1}{2{(2\pi )}^d}{\displaystyle {\int }_{{[-\pi ,\pi ]}^d}\text{tr}\left(\left({\text{Id}}_{\mathfrak{b}}-\tilde{\mathrm{G}}{\left(\theta \right)}^{-1}\right)\mathrm{d}\tilde{\mu }(\theta )\right)}.\end{array}$$

Finally, the rate function uniquely vanishes at μ = Q.

Corollary 2. Suppose that the underlying process Q is defined as previously, except with mean E^Q[⍵^j]= c for all j ∈ ℤ^d and some constant c ∈ ℝ^b. If we denote the image law of the empirical measure by${\prod }_c^n$ then$({\prod }_c^n)$ satisfies a strong LDP with a good rate function (for μ ∈ ε₂):

$$I^{(3)}(\mu )-\mathrm{\Gamma }(\mu )+\frac{1_{†}}{2}c\tilde{G}{(0)}^{-1}c{-}^{†}c\tilde{G}{(0)}^{-1}{m}^{\mu }.$$

(5)

Here, m^μ = E^μ(ω)[ω^j] for all j ∈ ℤ^d. If μ ∉ ε₂, then the rate function is infinite. The rate function has a unique minimum, i.e., $I^{(3)}(\mu )-\mathrm{\Gamma }(\mu )+\frac{1}{2}†c\tilde{G}{(0)}^{-1}c{-}^{†}c\tilde{G}{(0)}^{-1}-m^{\mu }=0$ if and only if μ = Q.

We prove this in Appendix C.

3. Proof of Theorem 1

In this proof, we essentially adapt the methods of [9,14]. We introduce the following metric over ${\mathcal{T}}^{{\mathbb{Z}}^d}$ For j ∈ ℤ^d, let ${\lambda }^j=3^{-d}{\prod }_{\delta =1}^d{2}^{-|j(\delta )|}$ Define the metric d^λ as follows,

$$d^{\lambda }(x,y)={\displaystyle \sum _{j\in {\mathbb{Z}}^d}{\lambda }^j}min\left(\Vert x^j-y^j\Vert ,1\right),$$

where the above is the Euclidean norm. Let d^λ$ℳ$ be the induced Prokhorov metric over $ℳ({\mathcal{T}}^{{\mathbb{Z}}^d})$. These metrics are compatible with their respective topologies.

For θ ∈ [−π, π]^d, $\tilde{F}(\theta )=\tilde{G}{(\theta )}^{1/2}$ be the Hermitian positive square root of $\tilde{G}$ and:

$$\tilde{F}(\theta )={\displaystyle \sum _{j\in {\mathbb{Z}}^d}{F}^j\exp \left(-i\langle j,\theta \rangle \right).}$$

(6)

The b × b matrices F^j are the coefficients of the absolutely convergent Fourier series of the positive square root. Define $\tau :{\mathcal{T}}^{{\mathbb{Z}}^d}\to {\mathcal{T}}^{{\mathbb{Z}}^d}$ and ${\tau }_{(\mathcal{M})}:{\mathcal{T}}^{{\mathbb{Z}}^d}\to {\mathcal{T}}^{{\mathbb{Z}}^d}$ as follows,

$$\begin{array}{l}{\left(\tau (\omega )\right)}^k={{\displaystyle \sum _{j\in {\mathbb{Z}}^{\mathrm{d}}}{}^{†}F}}^j{\omega }^{k-j},\\ {\left({\tau }_{(n)}(\omega )\right)}^k={{\displaystyle \sum _{j\in V_n}{}^{†}F}}^j{\omega }^{k-j}{\displaystyle \prod _{\delta =1}^d\left(1-\frac{|j(\delta )|}{2n+1}\right)}.\end{array}$$

(7)

We note that τ = β⁻¹ (on a suitable domain, where the series are convergent) and that τ_(n) is a continuous map, but τ is not continuous (in general).

We note (using Lemma 6) that $P^{{\mathbb{Z}}^d}\circ {\tau }^{-1}$ has spectral density $\tilde{G}$. We define:

$${\tilde{F}}_{(n)}(\theta )={\displaystyle \sum _{j\in V_n}{F}^j\exp \left(-i\langle j,\theta \rangle \right)}{\displaystyle \prod _{\delta =1}^d\left(1-\frac{|j(\delta )|}{2n+1}\right).}$$

(8)

We write $\epsilon (n)={\sup }_{\theta \in {[-\pi ,\pi ]}^d}{\Vert \tilde{F}(\theta )-{\tilde{F}}_{(n)}(\theta )\Vert }^2$. By Fejer’s theorem, ε(n) → 0 as n → ∞. We define ${\tilde{G}}_{(n)}(\theta )={\tilde{F}}_{(n)}{(\theta )}^2$, noting that this is the spectral density of $P^{{\mathbb{Z}}^d}\circ {\tau }_{(n)}^{-1}$. Let Γ_(n) (μ) = Γ_1,(n) + Γ_2,(n) (μ), where for μ ∈ ε₂,

$${\mathrm{\Gamma }}_{1,(n)}=-\frac{1}{2{(2\pi )}^d}{\displaystyle {\int }_{[-\pi ,\pi }\log \det {\tilde{G}}_{(n)}(\theta )d\theta ,}$$

(9)

$${\mathrm{\Gamma }}_{2,(n)}(\mu )=\frac{1}{2{(2\pi )}^d}{\displaystyle {\int }_{[-\pi ,\pi }\text{tr}\left(\left({\text{Id}}_{\mathfrak{b}}-{\tilde{\mathrm{G}}}_{(n)}^{-1}(\theta )\right)d\tilde{\mu }(\theta )\right).}$$

(10)

If μ ∉ ε₂, let Γ_1,(n) = Γ_2,(n)(μ) = 0. Let $R^n\in \mathcal{M}({\mathcal{M}}_s({\mathcal{T}}^{z^d}))$ be the law governing ${\widehat{\mu }}_n(Y)$, where we recall that the stationary process (Y^j) is defined just below Definition 1. Let ${\Pi }_{(m)}^n\in \mathcal{M}({\mathcal{M}}_s({\mathcal{T}}^{Z^d}))$ be the law governing ${\widehat{\mu }}_n({\tau }_{(m)}(Y))$.

Lemma 3. (Rⁿ) satisfies a large deviation principle with good rate function I⁽³⁾(μ). If μ ∉ ε₂, then I⁽³⁾(μ) = ∞.

Proof. The first statement is proven in [8]. The last statement follows from Lemma 11 below.

Lemma 4. $({\prod }_{(m)}^n)$ satisfies a strong LDP with good rate function given by, for μ ∈ ε₂:

$$I_{(m)}^{(3)}(\mu )=\underset{v\in {\epsilon }_2:\mu =v\circ {\tau }_{(m)}^{-1}}{\inf }{I}^{(3)}(v)=I^{(3)}(\mu )-{\mathrm{\Gamma }}_{(m)}(\mu ).$$

(11)

If μ ∉ ε₂, then$I_{(m)}^{(3)}(\mu )=\infty$.

Proof. The sequence of laws governing ${\widehat{\mu }}_n(Y)\circ {\tau }_{(m)}^{-1}$ (as n → ∞, with m fixed) satisfies a strong LDP with good rate function as a consequence of an application of the contraction principle to Lemma 3 (since τ_(m) is continuous). Now, through the same reasoning as in Lemma 2.1, Theorem 2.3 in [14], it follows from this that $({\prod }_{(m)}^n)$ satisfies a strong LDP with the same rate function as that of ${\widehat{\mu }}_n(Y)\circ {\tau }_{(m)}^{-1}$. The last identification in (11) follows from Lemmas 6 and 9 in Appendix A. We only need to take the infimum over ε₂, because by Lemma 3, I⁽³⁾ (ν) is infinite for ν ∉ ε₂.

Lemma 5. If$0<\mathfrak{b}<\frac{1}{2\epsilon (m)}$, then for all n > 0:

$$\frac{1}{{(2n+1)}^d}\log {\mathrm{E}}^{P^{Z^d}}\left[\exp \left(\mathfrak{b}{{\displaystyle \sum _{k\in V_n}\Vert {\tau }_{(m)}{(\omega )}^k-\tau {(\omega )}^k\Vert }}^2\right)\right]\le -\frac{1}{2}\log (1-2\mathfrak{b}\epsilon (m)).$$

The proof is almost identical to that in Lemma 2.4 in [14]. We are now ready to prove Theorem 1.

Proof. We apply Lemma 13 in the Appendix to the above result. We substitute Y_(m) = τ_(m)(ω) and W = τ(ω). Taking m → ∞ in the equation in Lemma 5, we find that (38) is satisfied if we stipulate that κ = 0. After noting the LDP governing $({\prod }_{(m)}^n)$ in (11), we may thus conclude that (Πⁿ) satisfies a strong LDP with good rate function:

$$\underset{\delta \to 0}{\lim }\underset{m\to \infty }{\underset{¯}{\lim }}\underset{\gamma \in B^{\delta }(\mu )}{\inf }{I}_{(m)}^{(3)}(\gamma ),$$

(12)

where B^δ(μ) = {γ: d^λ,M(μ,γ) ≤ δ}.

It remains for us to identify (12) with the rate function in (4). We claim that for each δ > 0,

$$\underset{m\to \infty }{\underset{¯}{\lim }}\underset{\gamma \in B^{\delta }(\mu )}{\inf }\left(I_{(m)}^{(3)}(\gamma )\right)=\underset{\gamma \in B^{\delta }(\mu )}{\inf }(I^{(3)}(\gamma )-\mathrm{\Gamma }(\gamma )).$$

(13)

To see this, we have from Lemma 11 that for all m and all γ and constants α₁ < 1, α₃ > 1 and α₂, α₄ ∈ ℝ, (note that if γ ∉ ε₂ the inequalities below are immediate from the definitions):

$$I^{(3)}(\gamma )-\mathrm{\Gamma }(\gamma )\ge (1-{\alpha }_1)I^{(3)}(\gamma )-{\alpha }_2$$

(14)

$$\begin{array}{r}{I}_{(m)}^{(3)}(\gamma )\ge (1-{\alpha }_1)I^{(3)}(\gamma )-{\alpha }_2\\ I^{(3)}(\gamma )-\mathrm{\Gamma }(\gamma )\le (1+{\alpha }_3)I^{(3)}(\gamma )-{\alpha }_4\\ I_{(m)}^{(3)}(\gamma )\le (1+{\alpha }_3)I^{(3)}(\gamma )-{\alpha }_4\end{array}$$

(15)

We thus see that if I⁽³⁾(γ) = ∞ for all γ ∈ B^δ(μ), then (13) is identically infinite on both sides. Otherwise, it may be seen from (14) and (15) that it suffices to establish (13) in the case that $B^{\delta }(\mu )=B_l^{\delta }(\mu ):=\{\gamma :d^{\lambda ,\mathcal{M}}(\mu ,\gamma )\le \delta ,I^{(3)}(\gamma )\le l\}$ for some l < ∞. However, it follows from (29) and (34) that for all $\gamma \in B_l^{\delta }(\mu )$, there exist constants $({\alpha }_5^m)$, which converge to zero as m → ∞ and such that $|I_{(m)}^{(3)}(\gamma )-I^{(3)}(\gamma )+\mathrm{\Gamma }(\gamma )|\le {\alpha }_5^m$. We may thus conclude (13). The expression for the rate function in (4) now follows, since I⁽³⁾(γ) – Γ(γ) is lower semicontinuous, by Lemma 12.

For the second statement in the Theorem, if I⁽³⁾ (μ ○ β⁻¹) = 0, then $I^{(2)}({(\mu \circ {\beta }^{-1})}^{V_n}\Vert P^{\otimes V_n})=0$ for all n. However, since the relative entropy has a unique zero, this means that ${(\mu \circ {\beta }^{-1})}^{V_n}=P^{\otimes V_n}$ for all n. However, this means that $\mu \circ {\beta }^{-1}=P^{Z^{\mathrm{d}}}$, and therefore (using Lemma 6), μ = Q.