On Parameter Estimation of Hidden Ergodic Ornstein-Uhlenbeck Process

We consider the problem of parameter estimation for the partially observed linear stochastic differential equation. We assume that the unobserved Ornstein-Uhlenbeck process depends on some unknown parameter and estimate the unobserved process and the unknown parameter simultaneously. We construct the two-step MLE-process for the estimator of the parameter and describe its large sample asymptotic properties, including consistency and asymptotic normality. Using the Kalman-Bucy filtering equations we construct recurrent estimators of the state and the parameter.


Introduction
We are given a partially observed linear system, defined by the equations where a = 0, σ = 0, b = 0 and f > 0 are constants, W T = (W t , 0 ≤ t ≤ T ) and V T = (V t , 0 ≤ t ≤ T ) are two independent Wiener processes. The random variable ξ ∼ N (0, d 2 ) is independent of W T and V T .
The system (1)- (2) is defined by the four parameters a, f, b, σ 2 . Recall that the parameter σ 2 can be estimated without error by continuous time observations X T as follows. By the Itô formula we can write Hence, for any t ∈ (0, T ], we have the estimator and this estimator equals the true value. Therefore we consider only the estimation of the three other parameters f, b and a. Note that the consistent estimation of the three-dimensional parameter ϑ = (a, b, f ) is impossible because the observed process can be written as follows This means that the parameters a and b appear as product ab. We can have consistent estimation of two-dimensional parameters (f, a) and (f, b). This possibility we discuss in the last section.
The observations are X T = (X t , 0 ≤ t ≤ T ) and the Ornstein-Uhlenbeck process Y T is unobservable (hidden), i.e., we have partially observed linear model of observations. We consider estimation of the one-dimensional parameters f , b and a separately given the continuous time observations X T . The unknown parameter will be denoted by ϑ and we will assume that ϑ ∈ Θ = (α, β) for some constants α < β. In all the cases the set Θ does not contain 0. Thus we are faced with three different problems: ϑ = f , ϑ = b and ϑ = a. In each problem we propose a two-step construction of asymptotically efficient estimator-process of recurrent nature. First we propose a preliminary consistent estimator ϑ T δ based on the observations X T δ = X t , 0 ≤ t ≤ T δ with δ ∈ (1/2, 1). Then this estimator is used for construction of One-step MLE-process, which has recurrent structure. In the last section we discuss the possibilities of the joint estimation of two dimensional parameters ϑ = (f, b) and ϑ = (f, a).
Equations (1)-(2) is a prototypical model in the Kalman-Bucy filtering theory, which provides a closed form system of equations for the conditional expectation [9], [18]). The statistical problems for discretely observed hidden Markov processes were studied by many authors (see [2], [3], [6], [7] and the references therein). However, the literature on continuous time models is limited. For the results in continuous time setup, we refer the interested reader to [13] (linear and non linear partially observed systems with small noise), [6] (continuous-time hidden Markov models estimation), [4] and [11] (hidden telegraph process observed in the white Gaussian noise).
In the present paper we are particularly interested in the asymptotic behavior of the maximum likelihood estimator (MLE)θ T in the large sample asymptotic regime, i.e., when T → ∞. The statistical problems for such observation models have been widely studied, motivated by the importance of the Kalman-Bucy filtering in engineering applications.
Let us now recall the definitions of the MLE in the case ϑ = f , when the other two parameters a and b are known. As the parameters of the model take finite values and σ 2 > 0, the measures P Then the MLEθ T is defined by the equation This means that to calculateθ T we need the values of the family of stochastic processes (m (ϑ, t) , 0 ≤ t ≤ T ) , ϑ ∈ Θ. The random process m (ϑ, ·) is solution of the Kalman-Bucy filtering equations (see [1], [9], [18]) is the solution of the Ricatti equation Due to importance of this model in many applied problems, much engineering literature is concerned with identification of this model.
• Small noise in observation only, σ → 0, (T and b are fixed) [16] In all three cases I (ϑ) the Fisher information is different. It was also shown that the polynomial moments of the scaled estimation error converge and the MLE is asymptotically efficient.
It is evident that the numerical calculation of the MLEθ T according to (3)-(6) is quite a difficult problem. The goal of this work is to suggest the new estimator, called One-step MLE-process ϑ ⋆ t , τ ≤ t ≤ T , which has two advantages. First, its numerical calculation is much more simple than that of the MLE and, second, this estimator has a recurrent structure and can be used for the joint estimation of the hidden process Y t and the parameter ϑ. Similar One-step MLE's and Multi-step MLE-processes, introduced in [15], have been applied in the problem of parameter estimation of the hidden telegraph process [11], parameter estimation in diffusion processes by the discrete time observations [10], in the problem of frequency estimation [8], intensity parameter estimation for inhomogeneous Poisson processes [5], parameter estimation for the Markov sequences [17].

Preliminary estimator.
Following [11] One-step MLE process will be constructed in two steps. First we introduce a consistent and asymptotically normal preliminary estimator and then this estimator is used to define One-step MLE-process. Preliminary estimator is constructed using an asymptotically negligible amount of the Suppose that ϑ = f and introduce the statistic S K and the function Φ (ϑ) , ϑ ∈ Θ: In the cases ϑ = b and ϑ = a the counterparts of the latter function are respectively.
In this section we consider the case ϑ = f only. Therefore Note that the function Φ (ϑ) , α < ϑ < β is strictly decreasing. Define the preliminary estimatorθ K , base the observations X K : The asymptotic behavior ofθ K as K → ∞ is described in the following proposition.
Proof. We have For the probabilities we have the estimates Therefore we have to study the asymptotics of the statistic S K as K → ∞: We have The process Y T can be written as Using similar calculations we obtain the estimate which allows us to prove the law of large numbers: for K → ∞ we have convergence in mean square Hence sup α<ϑ 0 ≤β The function Φ (ϑ) , α < ϑ < β is strictly decreasing. If we denote its inverse function as Ψ (φ) = Φ −1 (φ) , Φ (β) < φ < Φ (α), then we have We can write If we put K = T δ , then 3 One-Step MLE-process. Case ϑ = f .
Suppose that the unknown parameter is ϑ = f and we have the model (7)- (8), where the process X T is observable and the Ornstein-Uhlenbeck process Y T is "hidden". We realize the asymptotically efficient estimation of the parameter ϑ ∈ Θ in two steps. First we calculate the preliminary estimator ϑ T δ and then using this estimator we construct the One-step MLE-process.
Recall that the equation (6) has explicit solution .
Therefore we have exponential convergence of γ (ϑ, t) to the stationary solution γ (ϑ) To simplify the exposition we suppose that d 2 = γ (ϑ); then we have γ (ϑ, t) = γ (ϑ). The case with an arbitrary d 2 requires cumbersome calculations, but the main results remain intact.
The equation for m (ϑ, t) in this case is Denote m t = m (ϑ 0 , t) and γ * (ϑ 0 ) = γ * , where ϑ 0 is the true value. Then for the process m t , 0 ≤ t ≤ T we obtain the equation Here we used the innovation theorem (see [18], ??) The innovation Wiener processW t is defined by this equation and m 0 is independent onW t , 0 ≤ t ≤ T . With probability 1, the random process m (ϑ, t) has continuous derivatives w.r.t. ϑ and derivative processesṁ (ϑ, t) ,m (ϑ, t) satisfy the equations The Fisher information for this model of observations is .
Note that I (ϑ) has continuous bounded derivatives and is uniformly in ϑ ∈ Θ separated from zero.
Theorem 1 One-step MLE-process ϑ ⋆ T (τ ) , T δ−1 < τ ≤ 1 with δ ∈ (1/2, 1) is consistent: for any ν > 0 and any τ ∈ (0, 1] and asymptotically normal Note that as it follows from the equations (12)- (13), the Gaussian processeṡ m (ϑ, t) andm (ϑ 0 , t) have bounded variances and therefore for any p > 1 we have where the constants do not depend on t. We can writė Further, for the Fisher information we have This allows us to write By the law of large numbers a στ T τ T T δṁ (ϑ 0 , s) 2 ds −→ I (ϑ 0 ) and therefore by the central limit theorem The similar arguments allow us to write Recall that as we have stationary regime E ϑ 0ṁ (ϑ 0 , s) 2 = σ 2 a −2 I (ϑ 0 ). Therefore where the integral (see, e.g., Proposition 1.23 in [14]) Hence we obtained the representation
Note that the process ϑ ⋆ t , T δ < t ≤ T can be written in recurrent form and we can introduce the adaptive filtering equations as follows with the initial value m T δ = m θ T δ , T δ . Here It will be interesting to see the behavior of the system (16)-(18) using numerical simulations.
Recall that if we put τ = 1, then ϑ ⋆ T is One-step MLE with √ T (ϑ ⋆ T − ϑ 0 ) =⇒ N 0, I (ϑ 0 ) −1 studied for ergodic diffusion processes in the Section 2.5 [14]. Therefore the estimator ϑ ⋆ T is asymptotically equivalent to the asymptotically efficient MLÊ ϑ T defined by the equation (4). There is essential computational difference between these two estimators. The calculation ofθ T using (3)-(6) requires solving the differential equations (5)- (6) for numerous values of ϑ ∈ Θ, which is computationally inefficient. To construct One-step MLE-process ϑ ⋆ T we have to calculate a simple preliminary estimatorθ T δ and then to solve the system (5)-(6) for just one value ϑ =θ T δ . The difference between these two approaches becomes even more significant in the case of multidimensional ϑ.
Suppose that the volatility b = ϑ is the unknown parameter and we have the equations As before all parameters a, σ, ϑ do not vanish and f > 0. The volatility ϑ ∈ (α, β) with α > 0 and the function is strictly increasing.
The statistic S K , with the new notations, converges to this function Therefore we have the explicit expression for the preliminary estimator Here the sets B ± are defined by the similar relations As before, we have the consistencȳ We need the equation forṁ (ϑ, t) and expression for Fisher information in this case. The filtering equations in the stationary regime are Therefore the Fisher information is Now we can write the One-step MLE-process ϑ ⋆ t , T δ < t ≤ T as follows If we change the variables t = τ T and denote ϑ ⋆ τ T = ϑ ⋆ T (τ ) , T δ−1 < τ ≤ 1, then we obtain the same assertions as in the Theorem 1: and asymptotically normal Proof. Similarly to (16), we have exactly the same representation for the estimator ϑ ⋆ t as in (14), with the only difference in the forms ofṁ (ϑ, t) and I (ϑ). Thus the previous proof works in this case as well.
It is possible to write the system of recurrent equations as in (16)-(18).
It is clear that the suggested estimation approach also works for the partially observed system where the unknown parameter is the drift ϑ = a.
The function is strictly increasing and the corresponding preliminary estimatorθ K admits the same asymptotic properties as in the preceding section.

MLE-process introduced in
The Two-step MLE-process is and the estimate where the constant C > 0 does not depend on T . The approach applied in the present work allows us the direct verification these two conditions.