On convergence of general wavelet decompositions of nonstationary stochastic processes

The paper investigates uniform convergence of wavelet expansions of Gaussian random processes. The convergence is obtained under simple general conditions on processes and wavelets which can be easily verified. Applications of the developed technique are shown for several classes of stochastic processes. In particular, the main theorem is adjusted to the fractional Brownian motion case. New results on the rate of convergence of the wavelet expansions in the space $C([0,T])$ are also presented.


Introduction
In the book [11] wavelet expansions of non-random functions bounded on R were studied in different spaces. However, developed deterministic methods may not be appropriate to investigate wavelet expansions of stochastic processes. For example, in the majority of cases, which are interesting from theoretical and practical application points of view, stochastic processes have almost surely unbounded sample paths on R. It indicates the necessity of elaborating special stochastic techniques.
Recently, a considerable attention was given to the properties of the wavelet orthonormal series representation of random processes. More information on convergence of wavelet expansions of random processes in various spaces, references and numerous applications can be found in [3,7,14,15,16,17,18,21,20,24]. Most known stochastic results concern the mean-square or almost sure convergence, but for various practical applications one needs to require uniform convergence. To give an adequate description of the performance of wavelet approximations in both cases, for points where the processes are relatively smooth and points where spikes occur, we can use the uniform distance instead of global integral L p metrics. A more in depth discussion, further references and various applications in econometrics, simulations of stochastic processes and functional data analysis can be found in [6,9,10,20,23]. In his 2010 Szekeres Medal inauguration speech, an eminent leader in the field, Prof. P. Hall stated the development of uniform stochastic approximation methods as one of frontiers in modern functional data analysis. Figures 1 and 2 illustrate some features of wavelet expansions of stochastic processes. Figure 1 presents a simulated realization of the Wiener process and its wavelet reconstructions by two sums with different numbers of terms. The figure has been generated by the R package wmtsa [22]. Besides providing a realization of the Wiener process and its wavelet reconstructions, we also plot corresponding reconstruction errors. Figure 2 shows maximum absolute reconstruction errors for 100 simulated realizations. To reconstruct each realization of the Wiener process two approximation sums (as in Figure 1) were used. We clearly see that empirical probabilities of obtaining large reconstruction errors become smaller if the number of terms in the wavelet expansions increases. Although this effect is expected, it has to be established theoretically in a stringent way for different classes of stochastic processes and wavelet bases. It is also important to obtain theoretical estimations of the rate of convergence for such stochastic wavelet expansions.  In this paper we make an attempt to derive general results on stochastic uniform convergence which are valid for wavelet expansions of wide classes of stochastic processes. The 2 paper deals with the most general class of such wavelet expansions in comparison with particular cases considered by different authors, see, for example, [1,3,8,13,20]. Applications of the main theorem to special cases of practical importance (stationary processes, fractional Brownian motion, etc.) are demonstrated. We also prove the exponential rate of convergence of the wavelet expansions. Throughout the paper, we impose minimal assumptions on the wavelet bases. The results are obtained under simple conditions which can be easily verified. The conditions are weaker than those in the former literature.
These are novel results on stochastic uniform convergence of general finite wavelet expansions of nonstationary random processes. The specifications of established results are also new (for example, for the case of stationary stochastic processes, compare [15,16]).
Finally, it should be mentioned that the analysis of the rate of convergence gives a constructive algorithm for determining the number of terms in the wavelet expansions to ensure the uniform approximation of stochastic processes with given accuracy. It provides a practical way to obtain explicit bounds on the sharpness of finite wavelet series approximations.
The organization of the article is the following. In the second section we introduce the necessary background from wavelet theory and certain sufficient conditions for mean-square convergence of wavelet expansions in the space L 2 (Ω). In §3 we formulate and discuss the main theorem on uniform convergence in probability of the wavelet expansions of Gaussian random processes. The next section contains the proof of the main theorem. Two applications of the developed technique are shown in section 4. In §5 the main theorem is adjusted to the fractional Brownian motion case. Lastly, we obtain the rate of convergence of the wavelet expansions in the space C([0, T ]).
In what follows we use the symbol C to denote constants which are not important for our discussion. Moreover, the same symbol C may be used for different constants appearing in the same proof.

Wavelet representation of random processes
Let φ(x), x ∈ R, be a function from the space L 2 (R) such that φ(0) = 0 and φ(y) is continuous at 0, where Suppose that the following assumption holds true: There exists a function m 0 (x) ∈ L 2 ([0, 2π]), such that m 0 (x) has the period 2π and φ(y) = m 0 (y/2) φ (y/2) (a.e.) In this case the function φ(x) is called the f -wavelet.
Let ψ(x) be the inverse Fourier transform of the function Then the function where φ(x) and ψ(x) are defined as above.
Remark 1. f -wavelets and m-wavelets are scaling functions.
Proof. The wavelets φ and ψ admit the following representations, see [11], If k ≥ 0, then Similarly, for k ≤ 0 we get |h k | ≤ CΦ φ (|k|/3) . Thus, for all k ∈ Z, Note that the series in the right-hand side of (2) converges in the L 2 (R)-norm. Therefore, there exists a subsequence of partial sums which converges to ψ (x) a.e. on R. Thus, by (2) and (3) we obtain a.e. on R.
If u := 2x − 1, then for u > 0 Notice also that Therefore, for u > 0 We are to prove that, and, similarly, Thus, Thus, we conclude that for u > 0 Since for u < 0 for every u ∈ R. By (4), (5), (6), and (7), The desired result follows from (8) if we chose Motivated by Lemma 1, we will use the following assumption instead of two separate assumptions S ′ (γ) for the f -wavelet φ and the m-wavelet ψ.
If sample trajectories of this process are in the space L 2 (R) with probability one, then it is possible to obtain the representation (wavelet representation) where The majority of random processes does not possess the required property. For example, sample paths of stationary processes are not in the space L 2 (R) (a.s.). However, in many cases it is possible to construct a representation of type (9) for X(t).
Consider the approximants of X(t) defined by where k n := (k ′ 0 , k 0 , ..., k n−1 ). Theorem 1 below guarantees the mean-square convergence of X n,kn (t) to X(t) if k ′ 0 → ∞, k j → ∞, j ∈ N 0 , and n → ∞. The latter means that we increase the number n of multiresolution analysis subspaces which are used to approximate X(t). For each multiresolution analysis subspace j = 0 ′ , 0, 1, 2... the number k j of its basis vectors, which are used in the approximation, increases too, as n tends to infinity. Thus, for each fixed k and j there is n 0 ∈ N 0 that the terms ξ 0k φ 0k (t) and η jk ψ jk (t) are included in all X n,kn (t) for n ≥ n 0 (i.e., each ξ 0k φ 0k (t) and η jk ψ jk (t) can be absent only in the finite number of X n,kn (t)).

Uniform convergence of wavelet expansions for Gaussian random processes
In this section we show that, under suitable conditions, the sequence X n,kn (t) converges in probability in Banach space More details on the general theory of random processes in the space C([0, T ]) can be found in [2].
where σ(h) is a function, which is monotone increasing in a neighborhood of the origin and where σ (−1) (u) is the inverse function of σ(u). If the random variables X n (t) converge in probability to the random variable X(t) for all t ∈ [0, T ], then X n (t) converges to X(t) in the space C([0, T ]).
The following theorem is the main result of the paper.
Theorem 3. Let a Gaussian process X(t), t ∈ R, its covariance function, the f -wavelet φ, and the corresponding m-wavelet ψ satisfy the assumptions of Theorem 1.
Suppose that (i) assumption S(γ), γ ∈ (0, 1), holds true for φ and ψ; Then X n,kn (t) → X(t) uniformly in probability on each interval [0, T ] when n → ∞, k ′ 0 → ∞, and k j → ∞ for all j ∈ N 0 . Remark 4. If both wavelets φ and ψ have compact supports, then some assumptions of Theorem 3 are superfluous. In the following theorem we give an example by considering approximants of the form Theorem 4. Let X(t), t ∈ R, be a separable centered Gaussian random process such that its covariance function R(t, s) is continuous. Let the f -wavelet φ and the corresponding m-wavelet ψ be continuous functions with compact supports and the integrals R ln α (1 + |u|)| ψ(u)| du and R ln α (1 + |u|)| φ(u)| du converge for some α > 1/2(1 − γ), γ ∈ (0, 1). If there exist constants c j , j ∈ N 0 , such that E|η jk | 2 ≤ c j for all k ∈ Z, and assumption (13) is satisfied, then X n (t) → X(t) uniformly in probability on each interval [0, T ] when n → ∞.
Proof. The assumptions of Theorem 1 and S(γ), 0 < γ < 1, are satisfied because φ and ψ have compact supports. Therefore, the desired result follows from Theorem 3.

Proof of the main theorem
To prove Theorem 3 we need some auxiliary results.
Lemma 2. If δ(x) is a scaling function satisfying assumption S ′ (γ), then where Proof. The lemma is a simple generalization of a result from [11]. Since S γ (x) is a periodic function with period 1, it is sufficient to prove (14) for x ∈ [0, 1]. Notice, that for x ∈ [0, 1] and integer |k| ≥ 2 the inequality |x − k| ≥ |k|/2 holds true. Hence, Φ(|x − k|) ≤ Φ (|k|/2) and Lemma 3. Letδ(x) denote the Fourier transform of the scaling function δ(x), δ jk (x) := 2 then for all x, y ∈ R and k ∈ Z Note that for v = 0 the following inequality holds: By (15) and (16) we obtain The assertion of the lemma follows from this inequality.
Lemma 4. If a scaling function δ(x) satisfies the assumptions of Lemmata 2 and 3, then for γ ∈ (0, 1) and α > 0 : 10 Proof. By lemma 3 for j ≥ 1 we obtain We now make use of the inequality |a + b| α ≤ q α (|a| α + |b| α ) , where By lemma 2 we get Inequality (17) follows from this estimate. The proof of inequality (18) is similar. Now we are ready to prove Theorem 3.
Note that (11) implies We will only show how to handle S j . A similar approach can be used to deal with the remaining term S.

Examples
In this section we consider some examples of wavelets and stochastic processes which satisfy assumption (13) of Theorem 3.

Example 1. Let ψ be a Lipschitz function of order
Assume that for the covariance function R(t, s) Now we show that E|η jk | 2 ≤ c j for all k ∈ Z and find suitable upper bounds for c j . By Parseval's theorem, By properties of the m-wavelet ψ we have ψ(0) = 0. Therefore, using the Lipschitz conditions, we obtain This means that √ c j ≤ C/2 j/2(1+2κ) and assumption (13) holds.
In the following example we consider the case of stationary stochastic processes. This case was studied in detail by us in [15]. Note that assumptions in the example are much simpler than those used in [15].
Example 2. Let X(t) be a centered short-memory stationary stochastic process and ψ be a Lipschitz function of order κ > 0. Assume that the covariance function R(t − s) := EX(t)X(s) satisfies the following condition R R(z) · |z| 2κ dz < ∞.
By Parseval's theorem we deduce Thus, by the Lipschitz conditions, for all k, l ∈ Z : This means that √ c j ≤ C/2 j/2(1+2κ) and assumption (13) is satisfied.

Application to fractional Brownian motion
In this section we show how to adjust the main theorem to the fractional Brownian motion case.
Let W α (t), t ∈ R, be a separable centered Gaussian random process such that W α (−t) = W α (t) and its covariance function is Lemma 5. If assumption S(γ), 0 < γ < 1, holds true and for some α > 0 then for the coefficients of the process W α (t), defined by (10), for all k, l ∈ Z.
In some case, for example, for the fractional Brownian motion the assumption |Eξ 0k ξ 0l | ≤ b 0 of Theorem 3 doesn't hold true. The following theorem gives the uniform convergence of wavelet expansions without this assumption.
Theorem 5. Let a random process X(t), t ∈ R, the f -wavelet φ, and the corresponding mwavelet ψ satisfy the assumptions of Theorem 1 and assumptions (i) and (ii) of Theorem 3.
Suppose that there exist (iii') constants c j , j ∈ N 0 , such that E|η jk | 2 ≤ c j for all k ∈ Z and (13) holds true; (iv) some ε > 0 such that Then X n,kn (t) → X(t) uniformly in probability on each interval [0, T ] when n → ∞, k ′ 0 → ∞, and k j → ∞ for all j ∈ N 0 .
Then assumption (iv) of Theorem 5 holds true for W α (t).

Analogously to (22) and (23) we obtain
To estimate |φ 0k (t) − φ 0k (s)|, we use the representations Repeatedly using integration by parts and the assumptions of the lemma, we obtain that for k = 0 : By inequalities (8) and (12) given in [15] we get where c α,T andc α,T are constants which do not depend on t, s and z.
Applying these inequalities to (24) we obtain If k = 0, then Consequently, we can estimate S as follows Theorem 6. If the assumptions of Lemmata 5, 6, and assumptions (i) and (ii) of Theorem 3 are satisfied, then the wavelet expansions of the fractional Brownian motion uniformly converge to W α (t).

Convergence rate in the space C[0, T ]
Returning now to the general case introduced in Theorem 3, let us investigate what happens when the number of terms in the approximants (11) becomes large.
First we specify an estimate for the supremum of Gaussian processes.
where H(ε) is the metric entropy of the space where u > 8I(ε 0 ).

Proof.
Since is an even function. Then the assertion of the theorem follows from Now we formulate the main result of this section.
Theorem 7. Let a separable Gaussian random process X(t), t ∈ [0, T ], the f -wavelet φ, and the corresponding m-wavelet ψ satisfy the assumptions of Theorem 3. Then where u > 8δ(ε kn ) and the decreasing sequence ε kn is defined by (29) in the proof of the theorem.
Proof. Let us verify that Y(t) := X(t) − X n,kn (t) satisfies (26) with σ(ε) given by (27). First, we observe that We will only show how to handle S ′ j . A similar approach can be used to deal with the remaining terms S ′ and R ′ j . By Lemmata 2 and 3 we get where Similarly to (28) by lemma 8 we obtain

Remark 8.
In the theorem we only require that k ′ 0 , and all k j , j ≥ 0, approach infinity. If we narrow our general class of wavelet expansions X n,kn (t) by specifying rates of growth of the sequences k n we can enlarge classes of wavelets bases and random processes in the theorem and obtain explicit rates of convergence by specifying ε kn .
For instance, consider the examples in Section 5. It was shown that √ c j ≤ C/2 j/2(1+2κ) .
Remark 9. Lemma 8 and formula (29) provide simple expressions to computer ε kn and δ(ε kn ). It allows specifying Theorem 7 for various stochastic processes and wavelets.