Parametric Estimation from Approximate Data: Non-Gaussian Diffusions

Abstract

We study the problem of parameters estimation in indirect observability contexts, where $X_t \in R^r$ is an unobservable stationary process parametrized by a vector of unknown parameters and all observable data are generated by an approximating process $Y^{\varepsilon }_t$ which is close to $X_t$ in $L^4$ norm.We construct consistent parameter estimators which are smooth functions of the sub-sampled empirical mean and empirical lagged covariance matrices computed from the observable data. We derive explicit optimal sub-sampling schemes specifying the best paired choices of sub-sampling time-step and number of observations. We show that these choices ensure that our parameter estimators reach optimized asymptotic $L^2$-convergence rates, which are constant multiples of the $L^4$ norm $|| Y^{\varepsilon }_t - X_t ||$.

Appendix: \(L^2\)-consistency for Unobservable Estimators of Lagged 2nd Order Moments

In this appendix we present a detailed Proof of Theorem 2, which addresses the $L^2$- consistency results for the unobservable sub-sampled empirical estimators $\bar{X}^\varepsilon$ and $\hat{K}_X^\varepsilon (u)$ of means and lagged covariances. The hypotheses and notations are those of Theorem 2. Replacing $X_t$ by the centered process $X_t - \mu$ and setting $\mu =0$ is a trivial change in the proof , so we only need to consider the case where all $X_t$ are centered and $\mu = 0$.

Step 1 Sums of decorrelation values   For all $D>0$ and $j \ge 1$ one has $D f(jD) \le \int _{(j-1)D}^{j D} f(T) dT$ since the decorrelation rate f(T) is decreasing. This implies

$$\begin{aligned} \sum _{j=1}^{\infty } f(j D) \le \frac{1}{D}\sum _{j=1}^{\infty } \int _{(j-1)D}^{j D} f(T) dT = I(f) / D. \end{aligned}$$

(37)

Define the function g(q, D) for all integers $q \ge 2$ and all $D>0$ by

$$\begin{aligned} g(q, D) = \sum _{1 < m < n \le 1+q} f( (n - m) D) = \sum _{j=1}^{q-1} j f(j D) . \end{aligned}$$

(38)

Due to (37), the following inequality holds for all $D>0$ and $q \ge 2$

$$\begin{aligned} g(q, D) \le (q-1) \sum _{j=1}^{q-1} f(j D) \le ( q-1) I(f) /D. \end{aligned}$$

(39)

Step 2 Sub-sampled empirical means converge in ${\varvec{L}^2}$   Fix an integer $j \in [ 1 \ldots r]$. Denote the j-th coordinates of $X_{n \Delta }$ and of the empirical mean estimator $\bar{X}^\varepsilon$ by

$$\begin{aligned} U_n = X_{n \Delta }(j) \quad \text {and} \;\; \bar{X}^\varepsilon (j) = \frac{1}{N} (U_1 + \cdots + U_N). \end{aligned}$$

With the notation $s_j ^2 = {\mathbb {E}}(U_n^2)$, this implies

$$\begin{aligned} N^2 {\mathbb {E}}\left[ (\bar{X}^\varepsilon (j))^2 \right] = N s_j ^2 + 2 \sum _{1 \le m < n \le N } {\mathbb {E}}[U_m U_n] . \end{aligned}$$

(40)

Applying the decorrelation hypothesis (22) and the relations (38), (39), we obtain

$$\begin{aligned} \left| \sum _{1 \le m < n \le N } E(U_m U_n) \right| \le \sum _{1 \le m < n \le N } f((n - m) \Delta ) = g(N-1, \Delta ) \le I(f) \frac{N}{\Delta }. \end{aligned}$$

The definition of the $L^q$-norm also implies $s_j \le \Vert X_t \Vert _2 \le \Vert X_t \Vert _4 =\nu$. Hence, (40) implies

$$\begin{aligned} \left( \Vert \bar{X}^\varepsilon (j) \Vert _2 \right) ^2 \le \frac{\nu ^2 }{N} + \frac{ 2 I(f) }{N \Delta }. \end{aligned}$$

For any $( r_1 \times r_2)$ random matrix M, and any $q \ge 1$ our definition of the norm $\Vert M\Vert ^q$ implies

$$\begin{aligned} \Vert M \Vert _q \le (r_1 r_2)^{1/q} \max _{i, j} \Vert M_{i, j} \Vert _q. \end{aligned}$$

(41)

The inequality $\Vert \bar{X}^\varepsilon \Vert _2 \le \sqrt{r} \max _{j} \Vert \bar{X}^\varepsilon (j) \Vert _2$, then yields, due to (41),

$$\begin{aligned} \Vert \bar{X}^\varepsilon \Vert _2 \le \sqrt{r} \left( \frac{\nu ^2}{N} + \frac{ 2 I(f) }{N \Delta } \right) ^{1/2} \le \frac{1}{\sqrt{N \Delta } } \left( \nu (r \Delta )^{1/2} + (2 r I(f) )^{1/2} \right) . \end{aligned}$$

Since $\Delta (\varepsilon ) \rightarrow 0$ this proves the $L^2$-bound in (24) when $X_t$ is centered and hence in general.

Step 3 Sub-sampled empirical means converge in ${\varvec{L}^4}$   Basic algebra yields the identities

$$\begin{aligned} N^4 \left( \bar{X}^\varepsilon (j) \right) ^4 = \sum _{m, n, a, b \in [ 1 \ldots N ]} U_a U_b U_m U_n = S_1 + S_2 + 2 S_3 +24 S_4, \end{aligned}$$

(42)

where the sums $S_1$, $S_2$, $S_3$, and $S_4$ are defined by

$$\begin{aligned} S_1= & {} \sum _{1 \le m \le N } U_m^4 ,\\ S_2= & {} \sum _{1 \le m < n \le N } \left[ 2 U_m^2 U_n^2 + U_m^3 U_n + U_m U_n^3 \right] ,\\ S_3= & {} \sum _{1 \le a < m < n \le N } \left[ U_a^2 U_m U_n + U_a U_m^2 U_n + U_a U_m U_n^2 \right] ,\\ S_4= & {} \sum _{1 \le a < b < m < n \le N } U_a U_b U_m U_n . \end{aligned}$$

Due to the assumption that the $L^4$ norm of $X_t$ is bounded uniformly by $\nu$, one clearly has $|{\mathbb {E}}( S_1) | \le N \nu ^4$ and $| {\mathbb {E}}(S_2) | \le 4 N^2 \nu ^4$.

Since we are considering the centered process $X_t$, $E(U_n) \equiv 0$, and for $a < m < n$ the decorrelation hypothesis implies

$$\begin{aligned} \left| {\mathbb {E}}(U_a^2 U_m U_n) \right| = \left| {\mathbb {E}}(U_a^2 U_m U_n) - {\mathbb {E}}(U_a^2 U_m) E (U_n) \right| \le f( (n-m) \Delta ). \end{aligned}$$

Similarly, one shows that

$$\begin{aligned} \left| {\mathbb {E}}(U_a U_m^2 U_n) \right| \le f( (n-m) \Delta ) \text {~~and~~} \left| {\mathbb {E}}(U_a U_m U_n^2) \right| \le f( (m-a) \Delta ). \end{aligned}$$

These bounds and definition (38) yield (for $N \ge 3$)

$$\begin{aligned} \left| {\mathbb {E}}(S_3) \right| \le \sum _{1 \le a < m < n \le N ]} \; \left[ 2 f( (n-m) \Delta ) + f( (m-a) \Delta ) \right] = 3 \sum _{2 \le m \le N-1 ]} g(m, \Delta ) \end{aligned}$$

which implies, due to the bound (39),

$$\begin{aligned} \left| {\mathbb {E}}(S_3) \right| \le 3 I(f) \sum _{2 \le m \le N-1 ]}(m-1) \le \frac{3 I(f)}{2} \frac{N^2 }{\Delta }. \end{aligned}$$

As above, one also has $| {\mathbb {E}}(U_a U_b U_m U_n) | \le f( (b-a) \Delta )$ for $a < b < m < n$. The expressions of $S_4$ and $g(m, \Delta )$ then yield (for $N \ge 4$)

$$\begin{aligned} \left| {\mathbb {E}}(S_4) \right| \le \sum _{1 \le a <b <m <n \le N ]} f( (b-a) \Delta ) = \sum _{3 \le m <n \le N} g(m, \Delta ) = \sum _{3 \le m \le N-1} (N-m) g(m, \Delta ). \end{aligned}$$

Therefore, due to (39) we obtain for $N \ge 4$

$$\begin{aligned} \left| {\mathbb {E}}(S_4) \right| \le \sum _{3 \le m \le N-1} (N-m) I(f) \frac{m}{\Delta } \le \frac{I(f)}{4} \frac{N^3}{\Delta }. \end{aligned}$$

Finally, the bounds on $| {\mathbb {E}}(S_k) |$, and Eq. (42) entail

$$\begin{aligned} {\mathbb {E}}\left[ \left( \bar{X}^\varepsilon (j) \right) ^4 \right] \le \frac{5 \nu ^4 }{N^2} + \frac{3 I(f)}{N^2 \Delta } + \frac{ 6 I(f)}{N \Delta } \le \frac{C}{N \Delta } \end{aligned}$$

(43)

for some explicit constant C, since $N(\varepsilon ) \rightarrow \infty$ and $\Delta (\varepsilon ) \rightarrow 0$ with $N(\varepsilon ) \Delta (\varepsilon ) \rightarrow \infty$. In particular for $\varepsilon$ small enough, one can clearly take $C = 7 I(f)$. Therefore, Eqs. (41) and (43) imply

$$\begin{aligned} \Vert \bar{X}^\varepsilon \Vert _4 \le r^{1/4} \max _{i, j} \Vert \bar{X}^\varepsilon \Vert _4 \le \frac{(r C)^{1/4}}{(N \Delta )^{1/4}} \end{aligned}$$

which proves the $L^4$-bound in (24).

Step 4 Convergence of empirical lagged covariance matrices estimators   Introduce the short-hand notations $V_n = X_{n \Delta }$ and

$$\begin{aligned} \bar{V}_N= & {} \bar{X}^\varepsilon = \frac{1}{N} \sum _{n=1}^N \, V_n , \end{aligned}$$

(44)

$$\begin{aligned} \tau \bar{V}_N= & {} \tau \bar{X}^\varepsilon =\frac{1}{N} \sum _{n=1}^N \, V_{n+ \kappa }, \end{aligned}$$

(45)

$$\begin{aligned} W_N= & {} \frac{1}{N} \sum _{n=1}^N \, V_n V_{n+ \kappa }^* . \end{aligned}$$

(46)

From the Definition (12), the covariance matrix estimators $\hat{K}^\varepsilon _X(u)$ can be rewritten as

$$\begin{aligned} \hat{K}^\varepsilon _X(u) = W_N - \bar{V}_N (\tau \bar{V}_N)^* . \end{aligned}$$

(47)

First, we evaluate the term $\bar{V}_N (\tau \bar{V}_N)^*$ in the equation above. Impose $0 \le u \le A$ for some fixed A. Thus, by construction

$$\begin{aligned} \Vert \bar{V}_N - \tau \bar{V}_N \Vert _4 \le 2 \kappa \nu / N \le 2 (u+ \Delta ) \nu /N \le 2 (1+A) \nu \frac{1}{N \Delta } \end{aligned}$$

and applying the inequality (18) one arrives at the following relation

$$\begin{aligned} \Vert \bar{V}_N (\tau \bar{V}_N)^* - \bar{V}_N (\bar{V}_N)^* \Vert _2 \le 4 (1+A) \nu ^2 \frac{1}{N \Delta }. \end{aligned}$$

(48)

Since $\mu = 0$, we also have

$$\begin{aligned} \Vert \bar{V}_N \Vert _4 \le \frac{C}{(N \Delta )^{1/4}} , \end{aligned}$$

as proven in Step 3. This implies, by inequality (18),

$$\begin{aligned} \Vert \bar{V}_N (\bar{V}_N)^* \Vert _2 \le 2 \left[ \frac{C}{(N \Delta )^{1/4}} \right] ^2 = \frac{2 C^2}{(N \Delta )^{1/2}} \end{aligned}$$

which yields, due to Eq. (48),

$$\begin{aligned} \Vert \bar{V}_N (\tau \bar{V}_N)^* \Vert _2 \le \frac{2 C^2}{\sqrt{N \Delta }} + \frac{4 (1+A) \nu ^2 }{N \Delta }. \end{aligned}$$

(49)

By the construction of $\kappa (u,\varepsilon )$, the “discrete” lag $\kappa \Delta$ is close to continuous lag u and $| \kappa \Delta - u | \le \Delta$. Since the true lagged covariance matrices K(u) are locally Lipschitz, there is a constant $\lambda = \lambda (A)$ such that for all $0 \le u \le A$ and all $\varepsilon >0$ the following deterministic inequality holds

$$\begin{aligned} \Vert K(u) - K(\kappa \Delta ) \Vert \le \lambda | u - \kappa \Delta | \le \lambda \Delta . \end{aligned}$$

(50)

Next, we compare the term $W_N$ in the expression of the covariance estimator (47), with the true covariance matrix $K(\kappa \Delta )$ evaluated at the “discretized ” time lag $\kappa \Delta$. Since $X_t$ is stationary, we have $K(\kappa \Delta ) = {\mathbb {E}}( V_n V_{n+ \kappa }^*)$ for all n, and formula (46) implies that

$$\begin{aligned} W_N - K(\kappa \Delta ) = \frac{1}{N} \sum _{n=1}^N \; \left( \, V_n V_{n+ \kappa }^* - {\mathbb {E}}[ V_n V_{n+ \kappa }^*] \, \right) . \end{aligned}$$

For any two coordinates $i, j \in [1 \ldots r]$ denote $T_n= V_n(i)$ and $U_n= V_n(j)$ as the i-th and j-th coordinates of $V_n$, respectively. In addition, we also define

$$\begin{aligned} H_n = T_n U_{n+ \kappa } - {\mathbb {E}}[ T_n U_{n+ \kappa }]. \end{aligned}$$

Clearly ${\mathbb {E}}[H_n] = 0$ and the (i, j) coefficient of the matrix $M = W_N - K(\kappa \Delta )$ is then

$$\begin{aligned} M_{i, j} = \frac{1}{N} \sum _{n=1}^N \, H_n \end{aligned}$$

and

$$\begin{aligned} N^2 {\mathbb {E}}[M_{i, j}^2] = \sum _{(m, n) \in [1 \ldots N] } {\mathbb {E}}[H_m H_n]. \end{aligned}$$

(51)

Next, we partition the summation interval in the expression above into two complementary sets, $Q^+$ and $Q^-$, as follows

• $(m,n) \in Q^+$ whenever $| n - m | > \kappa$ and $m, n \in [1 \ldots N]$,

• $(m,n) \in Q^-$ whenever $| n - m | \le \kappa$ and $m, n \in [1 \ldots N]$.

Due to bounded fourth moments of the process $X_t$ we have $\left| {\mathbb {E}}[H_m H_n] \right| \le 2 \nu ^4$. Moreover,

$$\begin{aligned} \text {cardinal} \, (Q^-) = N + \kappa (2N- \kappa -1) \le 3 N \kappa , \end{aligned}$$

and, therefore,

$$\begin{aligned} \left| \sum _{(m,n) \in Q^-} {\mathbb {E}}(H_m H_n) \right| \le 6 \nu ^4 N \kappa . \end{aligned}$$

(52)

For $(m,n) \in Q^+$, the decorrelation rate of the 2nd order moments yields

$$\begin{aligned} | {\mathbb {E}}[H_m H_n] | \le f( | n-m | \Delta ) , \end{aligned}$$

so that

$$\begin{aligned} \left| \sum _{(m,n) \in Q^+} {\mathbb {E}}[H_m H_n] \right| \le \sum _{(m,n) \in Q^+} f( | n-m | \Delta ) . \end{aligned}$$

(53)

Thus, relation (51) and inequalities (52), (53) imply

$$\begin{aligned} N^2 {\mathbb {E}}[M_{i, j}^2] \le \sum _{(m,n) \in Q^+} f( | n-m | \Delta ) + 6 \nu ^4 N \kappa . \end{aligned}$$

(54)

Easy algebra transforms the double sum above into

$$\begin{aligned} \sum _{(m,n) \in Q^+} f( | n-m | \Delta ) = 2 \sum _{\kappa +1 \le s \le N-1} ( N - s ) f( s \Delta ) \le 2 N \sum _{1 \le s \le N-1}f( s \Delta ) \le 2 I(f) \frac{N}{\Delta }, \end{aligned}$$

where Eq. (37) were used in the last inequality. Recall that for $0 \le u \le A$ and due to the construction of $\kappa$, one also has

$$\begin{aligned} \kappa \le \frac{u}{\Delta } + \Delta \le \frac{A+1}{\Delta } . \end{aligned}$$

Substituting the last two expressions into (54) we obtain

$$\begin{aligned} {\mathbb {E}}[M_{i, j}^2] \le \left( 2 I(f) + 6 (A+1) \nu ^4 \right) \frac{1}{N \Delta }. \end{aligned}$$

By Eq. (41) we further obtain

$$\begin{aligned} \Vert W_N - K(\kappa \Delta ) \Vert _2 = \Vert M \Vert _2 \le r \left( 2 I(f) + 6 (A+1) \nu ^4 \right) ^{1/2} \frac{1}{\sqrt{N \Delta } }. \end{aligned}$$

(55)

Using the expression for $\hat{K}^\varepsilon _X(u)$ in (47) and the triangle inequality we can write

$$\begin{aligned} \Vert \hat{K}^\varepsilon _X(u) - K(u) \Vert _2 \le \Vert W_N - K(\kappa \Delta ) \Vert _2 + \Vert K(\kappa \Delta ) -K(u) \Vert _2 + \Vert \bar{V}_N (\tau \bar{V}_N)^* \Vert _2 . \end{aligned}$$

Combining the three bounds in (49), (50), and (55), we obtain, for all $\varepsilon > 0$ and $0 \le u \le A$,

$$\begin{aligned} \Vert \hat{K}^\varepsilon _X(u) - K(u) \Vert _2 \le \frac{\Gamma }{\sqrt{N \Delta } } + \lambda \Delta , \end{aligned}$$

(56)

where

$$\begin{aligned} \Gamma = 2 C^2 + r \left( 2 I(f) + 6 (A+1) \nu ^4 \right) ^{1/2} + \frac{ 4 (1+A) \nu ^2}{\sqrt{N \Delta }}. \end{aligned}$$

Moreover, for $\varepsilon$ small enough, we can take $C^2 = \sqrt{7r I(f)}$ as discussed in Step 2, and ${4 (1+A) \nu ^2}{(N \Delta )^{-1/2}}$ will become much smaller than $\sqrt{r I(f)}$. Therefore, for $\varepsilon$ small enough, one has (using $\sqrt{a+b} \le \sqrt{a} + \sqrt{b}$) a simplified expression for the constant $\Gamma$

$$\begin{aligned} \Gamma \le \gamma = 8 \sqrt{r I(f)} + 2.5 \nu ^2 \sqrt{A+1}. \end{aligned}$$

This concludes the Proof of Theorem 2.