Minimax prediction of random processes with stationary increments from observations with stationary noise

Abstract: We deal with the problem of mean square optimal estimation of linear functionals which depend on the unknown values of a random process with stationary increments based on observations of the process with noise, where the noise process is a stationary process. Formulas for calculating values of the mean square errors and the spectral characteristics of the optimal linear estimates of the functionals are derived under the condition of spectral certainty, where the spectral densities of the processes are exactly known. In the case of spectral uncertainty, where the spectral densities of the processes are not exactly known while a class of admissible spectral densities is given, relations that determine the least favorable spectral densities and the minimax robust spectral characteristics are proposed.


PUBLIC INTEREST STATEMENT
The crucial assumption of most of papers dedicated to the problem of estimating the unobserved values of random processes is that spectral densities of the involved processes are exactly known. However, the established results cannot be directly applied to practical estimation problems, because complete information of the spectral densities is impossible in most cases. This is a reason to derive the minimax estimates since they minimize the maximum of the mean-square errors for all spectral densities from a given set of admissible spectral densities simultaneously. In this article we deal with the problem of optimal estimation of functionals depending on unknown values of random processes from observations of the process with noise in the case of spectral uncertainty where spectral densities are not known exactly while a class of admissible spectral densities is given. Formulas that determine least favourable spectral densities and minimax (robust) spectra characteristics are derived.

Introduction
Traditional methods of finding solutions to problems of estimation of unobserved values of a random process based on a set of available observations of this process, or observations of the process with a noise process, are developed under the condition of spectral certainty, where the spectral densities of the processes are exactly known. Methods of solution of these problems, which are known as interpolation, extrapolation, and filtering of stochastic processes, were developed for stationary stochastic processes by A.N. Kolmogorov, N. Wiener, and A.M. Yaglom (see selected works by Kolmogorov (1992), books by Wiener (1966), Yaglom (1987aYaglom ( , 1987b, Rozanov (1967). Stationary stochastic processes and sequences admit some generalizations, which are properly described in books by Yaglom (1987aYaglom ( , 1987b. Random processes with stationary nth increments are among such generalizations. These processes were introduced in papers by Pinsker and Yaglom (1954), Yaglom (1955Yaglom ( , 1957, and Pinsker (1955). In the indicated papers, the authors described the spectral representation of the stationary increment process and the canonical factorization of the spectral density, solved the extrapolation problem, and proposed some examples.
Traditional methods of finding solutions to extrapolation, interpolation, and filtering problems may be employed under the basic assumption that the spectral densities of the considered random processes are exactly known. In practice, however, the developed methods are not applicable since the complete information on the spectral structure of the processes is not available in most cases. To solve the problem, the parametric or nonparametric estimates of the unknown spectral densities are found or these densities are selected by other reasoning. Then, the classical estimation method is applied, provided that the estimated or selected densities are the true ones. However, as was shown by Vastola and Poor (1983) with the help of concrete examples, this method can result in significant increase of the value of the error of estimate. This is a reason to search estimates which are optimal for all densities from a certain class of the admissible spectral densities. The introduced estimates are called minimax robust since they minimize the maximum of the mean square errors for all spectral densities from a set of admissible spectral densities simultaneously. The paper by Grenander (1957) should be marked as the first one where the minimax approach to extrapolation problem for stationary processes was proposed. Franke and Poor (1984) and Franke (1985) investigated the minimax extrapolation and filtering problems for stationary sequences with the help of convex optimization methods. This approach makes it possible to find equations that determine the least favorable spectral densities for various classes of admissible densities. A survey of results in minimax (robust) methods of data processing can be found in the paper by Kassam and Poor (1985). A wide range of results in minimax robust extrapolation, interpolation, and filtering of random processes and sequences belong to Moklyachuk (2000Moklyachuk ( , 2001Moklyachuk ( , 2008a . Later, Masyutka (2011 -2012) developed the minimax technique of estimation for vector-valued stationary processes and sequences. Dubovets'ka, Masyutka, and  investigated the problem of minimax robust interpolation for another generalization of stationary processes-periodically correlated sequences. In the further papers, Dubovets'ka and  investigated the minimax robust extrapolation, interpolation, and filtering problems for periodically correlated processes and sequences.
The minimax robust extrapolation, interpolation, and filtering problems for stochastic sequences with nth stationary increments were investigated by Luz and Moklyachuk (2012, 2015a, 2015b, 2015c; Moklyachuk and Luz (2013). In particular, the minimax robust extrapolation problem based on observations with and without noise for such sequences is investigated in papers by Luz and Moklyachuk (2015b), Moklyachuk and Luz (2013). Same estimation problems for random processes with stationary increments with continuous time are investigated in articles by Luz and Moklyachuk (2014a, 2015a, 2015b. In this article, we deal with the problem of the mean square optimal estimation of the linear functionals A = ∫ ∞ 0 a(t) (t)dt and A T = ∫ T 0 a(t) (t)dt which depend on the unknown values of a random process (t) with stationary nth increments from observations of the process (t) + (t) at points t < 0, where (t) is an uncorrelated with (t) stationary process. The case of spectral certainty as well as the case of spectral uncertainty are considered. Formulas for calculating values of the mean square errors and the spectral characteristics of the optimal linear estimates of the functionals are derived under the condition of spectral certainty, where the spectral densities of the processes are exactly known. In the case of spectral uncertainty, where the spectral densities of the processes are not exactly known while a class of admissible spectral densities is given, relations that determine the least favorable spectral densities and the minimax spectral characteristics are derived for some classes of spectral densities.

Stationary random increment process. Spectral representation
In this section, we present basic definitions and spectral properties of random processes with stationary increment. For more details, see the book by Yaglom (1987aYaglom ( , 1987b. Definition 2.1 For a given random process (t), t ∈ ℝ, the process where B is a backward shift operator with a step ∈ ℝ, such that B (t) = (t − ) is called the random nth increment with step ∈ ℝ generated by the random process (t).
Definition 2.2 The random nth increment process (n) (t, ) generated by a random process (t), t ∈ ℝ, is in wide sense stationary, if the mathematical expectations exist for all t 0 , , t, 1 , 2 and do not depend on t 0 . The function c (n) ( ) is called the mean value of the nth increment and the function D (n) (t, 1 , 2 ) is called the structural function of the stationary nth increment (or the structural function of nth order of the random process (t), t ∈ ℝ).
The random process (t), t ∈ ℝ, which determines the stationary nth increment process (n) (t, ) by formula (1) is called the process with stationary nth increments.
The following theorem describes representations of the mean value and the structural function of the random stationary nth increment process (n) (t, ).
Theorem 2.1 The mean value c (n) ( ) and the structural function D (n) (t, 1 , 2 ) of the random stationary nth increment process (n) (t, ) can be represented in the following forms: where c is a constant and F( ) is a left-continuous nondecreasing bounded function, such that F(−∞) = 0. The constant c and the function F( ) are determined uniquely by the increment process (n) (t, ).
The representation (3) of the structural function D (n) (t, 1 , 2 ) and the Karhunen theorem (see Karhunen, 1947) allow us to write the following spectral representation of the stationary nth increment process (n) (t, ): where Z (n) ( ) is a random process with uncorrelated increments on ℝ connected with the spectral function F( ) from representation (3) by the relation

The Hilbert space projection method of extrapolation
Consider a random process (t), t ∈ ℝ, which generates a stationary random increment process (n) (t, ) with the absolutely continuous spectral function F( ) and the spectral density function f ( ).
Let (t), t ∈ ℝ, be another random process which is stationary and uncorrelated with (t). Suppose that the process (t) has absolutely continuous spectral function G( ) and the spectral density g( ).
The main purpose of this paper is to find optimal, in the mean square sense, linear estimates of the functionals which depend on the unknown values of the random process (t) at time t ≥ 0 based on observations of the process (t) = (t) + (t) at time t < 0.
For further analysis, we need to make the following assumptions. Let the function a(t), t ≥ 0, which determines the functionals A , A T , and the linear transformation D , being defined below, satisfy the conditions and Suppose also that the spectral densities f ( ) and g( ) satisfy the minimality condition for some function ( ) of the form = ∫ ∞ 0 (t)e i t dt. Assumption (8) guarantees that the mean square errors of estimates of the considered functionals are greater than 0.
Following the classical estimation theory developed for stationary processes, it is reasonable to apply the method proposed by Kolmogorov (see selected works by Kolmogorov (1992)), where the estimate is a projection of an element of the Hilbert space H = L 2 (Ω, , ) of the random variables with zero mean value, = 0, and finite variance, | | 2 < ∞ on a subspace of the space H = L 2 (Ω, , ). The inner product in the space H = L 2 (Ω, , ) is defined as ( 1 ; 2 ) = 1 2 . Since we have no observations of the process (t) to take as initial values, the issue is that both functionals A and A T have infinite variance. Thus, we need to derive other objects from the space H = L 2 (Ω, , ) to proceed with the Hilbert space projection method.
Consider a representation of the functional A in the form where (5) Under the condition (6), the functional A has finite variance and, hence, it belongs to the space H = L 2 (Ω, , ). A representation of the functional A is described in the following lemma.  (7), the functional B from Lemma 3.1 belongs to the space H = L 2 (Ω, , ), while the functional V is observed and can be considered as an initial value. Thus, Lemma 3.1 implies the following representation of the functional A : where the functional H : = B − A belongs to the space H = L 2 (Ω, , ) and the Hilbert space projection method can be applied. Since the functional V depends on the observations (t), − n ≤ t < 0, the following relations hold true for the estimates Â , Ĥ and the mean square errors Δ(f , g;Â ), Δ(f , g;Ĥ ): Therefore, the problem is reduced to finding the optimal mean square estimate Ĥ of the functional H .
The next step is to describe the spectral structure of the functional H . The stationary random process (t) admits the spectral representation (see Gikhman & Skorokhod, 2004).
where Z ( ) is a random process with uncorrelated increments on ℝ which correspond to the spectral function G( ). Taking into account (4), the spectral representation of the random process (n) (t, ) can be described by the formulas where One can easily conclude that the spectral density p( ) of the random process (t) is the following: The functional H admits the spectral representation where Denote by H 0− ( (n) + (n) ) the closed linear subspace of the space H = L 2 (Ω, , ), which is generated by observations { (n) (t, ) + (n) (t, ):t < 0}, > 0. Denote by L 0− 2 (p) the closed linear subspace of the Hilbert space L 2 (p) defined by the set of functions It follows from the equality that the operator which maps the vector (1 + 2 ) n g( ).
(n) (t, ) + (n) (t, ) = ∫ ∞ −∞ e i t (1 − e −i ) n (1 + i ) n (i ) n dZ (n) + (n) ( ) of the space L 0− 2 (p) to the vector (n) (t, ) + (n) (t, ) of the space H 0− ( (n) + (n) ) may be extended to a linear isometry between the above spaces. The following relation holds true: Every linear estimate Â of the functional A admits the representation where h ( ) is the spectral characteristic of the estimate Ĥ . We can find the estimate Ĥ as a projection of the element H of the space H on the subspace H 0− ( (n) + (n) ). This projection is characterized by two conditions: (1) Ĥ ∈ H 0− ( (n) + (n) ); (2) (H −Ĥ ) ⟂ H 0− ( (n) + (n) ). Condition (2) and property (15) imply the following relations which hold true for every t < 0: Let us define for ∈ ℝ the function and its Fourier transform We have (t) = 0 for t < 0, hence which allows us to construct the representation of the spectral characteristic It follows from the condition 1) that the spectral characteristic h ( ) admits the representation which leads to the following relations holding true for every s ≥ 0: Relation (17) (20) The obtained results can be summarized in the following theorem.
Theorem 3.1 Let (t), t ∈ ℝ, be a random process with stationary nth increment process (n) (t, ) and let (t), t ∈ ℝ, be an uncorrelated with (t) stationary random process. Suppose that the spectral densities f ( ) and g( ) of the random processes (t) and (t) satisfy the minimality condition (8) and the function a(t), t ≥ 0, satisfies conditions (6) and (7). Suppose also that the linear operator is invertible. The optimal estimate Â of the functional A based on observations (t) + (t) at time t < 0 is calculated by formula (16). The spectral characteristic h ( ) and the value of mean square error Δ(f , g;Â ) of the estimate Â can be calculated by formulas (19) and (20), respectively.
Remark 3.1 The spectral characteristic h ( ) determined by formula (19) can be presented in the The functions h 1 ( ) and h 2 ( ) are the spectral characteristics of the mean square optimal estimates B and Â of the functionals B and A , respectively, based on observations (t) + (t) at time t < 0.
In the case of observations without noise, we have the following corollary.
Corollary 3.2 Let (t), t ∈ ℝ, be a random process with stationary nth increment process (n) (t, ). Suppose that the spectral density f ( ) of the random processes (t) satisfies the minimality condition (8) with g( ) = 0 and the function a(t), t ≥ 0, satisfies conditions (6) and (7). Suppose also that the linear operator defined below is invertible. The optimal linear estimate Â of the functional A which depends on the unknown values (t), t ≥ 0, of the random process (t), based on observations of the process (t), t < 0, is calculated by the formula The spectral characteristic h ( ) and the mean square error Δ(f ;Â ) of the optimal estimate Â of the functional A are calculated by the formulas where is the linear operator in the space L 2 [0; ∞) determined by the formula Remark 3.2 In Corollary 3.2, we provide formulas for calculating the optimal linear estimate Â of the functional A and the value of the mean square error Δ(f ;Â ) of the estimate Â based on observations of the process (t) at time t < 0 using the Fourier transform of the function 2n |1 − e i | 2n (1 + 2 ) n f ( ) . In the article by Luz and Moklyachuk (2014a), the same problem is considered. However, a solution is derived in terms of the function (t), t ≥ 0, which is determined by the canonical factorization of the function Theorem (3.1) can be used to obtain the optimal estimate Â T of the functional A T which depends on the unknown values (t), 0 ≤ t ≤ T, of the random process (t), based on observations of the process (t) + (t) at time t < 0. To derive the corresponding formulas, let us put (t) = 0 if t > T. We get that the spectral characteristic h ,T ( ) of the optimal estimate is calculated by the formula where the linear operator T in .the space L 2 [0; ∞) is determined by the formula the function ,T (t), t ∈ [0; T + n], is calculated by formula The mean square error of the optimal estimate Â T is calculated by the formula where the linear operator T in the space L 2 [0; ∞) is determined by the formula and the function T (t), t ∈ [0;T], is determined as T (t) = a(t).
The described results can be summarized in the following theorem.
Theorem 3.2 Let (t), t ∈ ℝ, be a random process with stationary nth increment process (n) (t, ) and let (t), t ∈ ℝ, be an uncorrelated with (t) stationary random process. Suppose that the spectral densities f ( ) and g( ) of the random processes (t) and (t) satisfy the minimality condition (8) and the function a(t), 0 ≤ t ≤ T, satisfies conditions (6) and (7). Suppose also that the linear operator is invertible. The optimal linear estimate Â T of the functional A T based on observations of the process (t) + (t) at time t < 0 is calculated by formula (24). The spectral characteristic h ,T ( ) and the value of mean square error Δ(f , g;Â T ) of the optimal estimate Â T are calculated by formulas (25) and (26), respectively.

Minimax robust method of extrapolation
The values of the mean square errors and the spectral characteristics of the optimal estimates of the functionals A and A T based on observations of the process (t) + (t) or observations of the process (t) without noise can be calculated by formulas (20), (23), (26) and (19), (22), and (25), respectively, in the case where the spectral densities f ( ) and g( ) of the random processes (t) and (t) are exactly known. In the case where the spectral densities are not exactly known while sets  =  f ×  g or  =  f of admissible spectral densities are given, the minimax robust method of estimation of the functionals which depend on the unknown values of the random process with stationary increments can be applied. The method consists in determining an estimate which minimizes the value of the mean square error for all spectral densities from the given class  =  f ×  g or  =  f simultaneously. The following definitions formalize the proposed method.
Definition 4.1 For a given class of spectral densities,  =  f ×  g spectral densities f 0 ( ) ∈  f , g 0 ( ) ∈  g are called the least favorable in the class  for the optimal linear extrapolation of the functional A if the following relation holds true Definition 4.2 For a given class of spectral densities  =  f ×  g , the spectral characteristic h 0 ( ) of the optimal linear estimate of the functional A is called minimax robust if there are satisfied conditions: Let us now formulate lemmas which follow from the introduced definitions and formulas (20) and (23) derived in the previous section.

,
(1 + 2 ) n f 0 ( )g 0 ( ) (1 + 2 ) n f 0 ( ) + 2n g 0 ( ) In the case of estimating the functional A based on the observations of the process (t) at time t < 0 without noise, we can formulate the following theorem.
Theorem 6.4 Suppose that the spectral density f 0 ( ) ∈  u v satisfies condition (28). The spectral density is the least favorable in the class  =  u v for the optimal linear extrapolations of the functional A based on observations of the process (t) at time t < 0 if it determines a solution of the optimization problem (29). The function h (f 0 ) calculated by formula (22) is the minimax robust spectral characteristic of the optimal estimate of the functional A .

Conclusions
In this paper, we present results of investigating of the problem of optimal linear estimation of the functionals A = ∫ ∞ 0 a(t) (t)dt and A T = ∫ T 0 a(t) (t)dt which depend on the unknown values of a random process (t) with nth stationary increments based on observations of the process (t) + (t) at time t < 0. In the case where the spectral densities of the processes are known, we found formulas for calculating the values of the mean square errors and the spectral characteristics of the estimates of the functionals A and A T . In the case where the spectral densities are not exactly known, but a set of admissible spectral densities was available, we applied the minimax robust method to derive relations which determine the least favorable spectral densities from the given set and the minimax robust spectral characteristics.