Estimation of the shift parameter in regression models with unknown distribution of the observations

This paper is devoted to the estimation of the shift parameter in a semiparametric regression model when the distribution of the observation times is unknown. Hence, we propose to use a stochastic algorithm which takes into account the estimation of the distribution of the observation times. We establish the almost sure convergence of our estimator and the asymptotic normality. The main result of the paper is that, with little assumptions on the regularity of the regression function, the asymptotic variance obtained is the same as when the distribution is known. In that sense, we improve the recent work of Bercu and Fraysse.


INTRODUCTION
We propose to study the problem of the estimation of the shift parameter θ in the semi parametric regression model defined, for all n ≥ 0, by where (X n ) and (ε n ) are two independent sequences of independent and identically distributed random variables. Model (1.1) belongs to the family of shape invariant models introduced by Lawton et al. [9]. One can find studies of that kind of models in the papers of Dalalyan et al. [5], of Gamboa et al. [7] or Vimond [15], whereas Castillo and Loubès [4] and Trigano et al. [13] are interesting of such a model when the parameter θ is random. Recent advances on the subject have also provided by Bigot and Charlier [2] and Bigot and Gendre [3]. Contrary to all the papers quoted previously, we are dealing with random observation times (X n ) and we assume that their distribution is unknown. Our goal is the estimation of θ in that case. More precisely, we propose to generalize the work of Bercu and Fraysse [1] when the distribution of (X n ) is assumed to be known. We implement a stochastic algorithm in order to estimate the unknown parameter θ without any preliminary evaluation of the regression function f . When the distribution of (X n ) is known, Bercu and Fraysse propose to use the algorithm similar to that of Robbins-Monro [12], defined, for all n ≥ 0, by where (γ n ) is a positive sequence of real numbers decreasing towards zero and (T n ) is a sequence of random variables such that E[T n+1 |F n ] = φ( θ n ) where F n stands for the σ-algebra of the events occurring up to time n. References on algorithm (1.2) can be found in [1]. Nevertheless, the expression of T n+1 depends on the distribution of (X n ). To overcome this problem, we propose to replace the algorithm given by (1.2) by the one defined, for all n ≥ 0, by where T n+1 depends only on an estimator of the distribution of (X n ) which will be explicited in the sequel. In particular, we no longer have E[ T n+1 |F n ] = φ( θ n ). Algorithms of the form (1.3) have been studied by Pelletier [10], [11] where the author establishes convergence results under the hypothesis that T n+1 − T n+1 2 = o P (γ n ). Nevertheless, in our situation, such an hypothesis is not verified and we can not apply this kind of convergence results. The paper is organized as follows. Section 2 is devoted to the explanation of the estimation procedure of θ. We establish the almost sure convergence of θ n as well as the asymptotic normality under some little assumptions on the regularity of f . In particular, we establish that the asymptotic variance is the same as the one obtained in the paper [1], that is to say the estimation of the distribution of (X n ) does not disturb the asymptotic behaviour of θ n . The proofs of the results are given is Section 3.

ESTIMATION PROCEDURE AND MAIN RESULTS
We focus our attention on the estimation of the shift parameter θ in the semiparametric regression model given by (1.1). We assume that (ε n ) is a sequence of independent and identically distributed random variables with zero mean and unknown positive variance σ 2 . Moreover, we add the two several hypothesis similar to that of [1].
(H 1 ) The observation times (X n ) are independent and identically distributed with unknown probability density function g, positive on its support [−1/2, 1/2]. In addition, g is continuous, twice differentiable with bounded derivatives.
When the density g is known, Bercu and Fraysse [1] propose to use the algorithm defined, for all n ≥ 0, by where the initial value θ 0 ∈ C and the random variable T n+1 is defined by We recall that π C is the projection on the compact set C = [−1/4; 1/4] defined, for all x ∈ R, by Moreover, we denote by the first Fourier coefficient of f and we define the function φ, for all x ∈ R, by Finally, (γ n ) is a decreasing sequence of positive real numbers satisfying When the density g is unknown, it is not possible to use algorithm (2.1). The idea is to replace g in the expression of (2.1) by an estimator of g. More precisely, we study the algorithm defined, for all n ≥ 0, by where g n is the Parzen-Rosenblatt kernel estimator of g (see [14], [16] and [17] for references) defined, for all x ∈ [−1/2; 1/2] and for all n ≥ 0, by and where the kernel K is a symmetric function, positive, with compact support and with All the results which follow are based on the following lemma.
Proof. The proof is given in Section 3.
In the sequel, we choose h n = n −α with 0 < α < 1 and γ n = 1/n. Our first result concerns the almost sure convergence of the estimator θ n .
Theorem 2.1. Assume that (H 1 ) and (H 2 ) hold and that |θ| < 1/4. Then, if K is a Lipschitz function, for all 0 < α < 1, θ n converges almost surely to θ. In addition, the number of times that the random variable θ n +sign(f 1 )γ n+1 T n+1 goes outside of C is almost surely finite.
Proof. The proof is given in Section 3.
Before establishing the asymptotic normality of θ n , we need the following lemma on the mean square error of θ n .
In order to establish the asymptotic normality of θ n , it is necessary to introduce a second auxiliary function ϕ defined, for all t ∈ R, by As soon as 4π|f 1 | > 1, denote Moreover, we need to add the following hypothesis on the regularity of f .
The shape function f is twice differentiable with bounded derivatives.

3.2.
Proof of Theorem 2.1. Without loss of generality, we suppose that f 1 > 0. Denote by F n the sigma-algebra F n = σ (X 0 , Y 0 , . . . , X n , Y n ). We calculate the two first conditional moments of T n+1 . On the one hand, for all n ≥ 0, where the function φ is defined by (2.2). Hence, we deduce from (3.12) that On the other hand, for all n ≥ 0, Hence, it follows from (3.15) that, for all n ≥ 0, Moreover, for all n ≥ 0, denote V n = θ n − θ 2 . Using the fact that π K is Lipschitz with constant 1, we have, for all n ≥ 0, Hence, it follows from (3.14), (3.16) and the previous inequality (3.17), that However, for all n ≥ 0, Since g does not vanish on its support, f is bounded and ε n+1 is independent of F n with finite moment of order 2, we immediately deduce the existence of C 1 > 0 and C 2 > 0 such that In addition, on the one hand and on the other hand, Then, it follows from Lemma 2.1 and the two previous calculations (3.23) and (3.24) that, for all We immediately deduce from (3.21) and (3.22) that a.s.
Following exactly the same lines as proof of Theorem 2.1 of [1] from equation (5.6), we deduce that θ n converges almost surely to θ and that the number of times that the random variable θ n + γ n+1 T n+1 goes outside of C is almost surely finite.
Moreover, since (3.29), there exists a constant L > 0 such that In addition, we have for all By the continuity of the function v, one can find 0 < ε < 1/2 such that, if |x−θ| < ε, Hence, it follows from (3.34) that for all n ≥ 1, with 2q = 1 − 4πf 1 which means that q < 0. Then, it follows from (2.7) and (3.35) that Hence, we deduce from the conjunction of (3.36) and (3.37) that, Since {T m > n + 1} ⊂ {T m > n}, we obtain by taking the expectation on both sides of (3.38) that for all n ≥ m, As γ n = 1/n, it follows from straightforward calculations that β n = O(n q ) and Moreover, one have from (3.22) and (3.24) that there exists C 2 > 0 such that, However, for all x ∈ [−1/2; 1/2], we have Then, taking expectation on both sides of (3.41), it follows from the previous inequality that The quantity E [|g(x) − g n (x)|] corresponds to the mean error of the recursive Parzen-Rosenblatt estimator. Hence, it is well-known that for 0 < α < 1/2, Then, one deduce from (3.42) that, for 0 < α < 1/2, Thus, (3.40) together the previous equation implies that Hence, for all m ≥ 0, , that is to say, for 0 < α < 1/2,

3.4.
Proof of Theorem 2.2. Without loss of generality, we suppose that f 1 > 0.
Then, we infer from the strong law of large numbers for martingales given by Theorem 1.3.15 of [6] that for any γ > 0, In the three cases, as q < −1/2, one can conclude that s. From the same way, one deduce from (3.54) that Hence, it follows from Lemma 2.1 that a.s.
Thus, if 1 + q + 4α > 1 and 3 + q − 2β > 1, the sequence (R 2 n−1 ) converges a.s. whereas if 1 + q + 4α = 1 and 3 + q − 2β = 1, one obtain that In the case where 1 + q + 4α < 1 and 3 + q − 2β < 1, one deduce that Finally, in the three cases, one obtain that, if α > 1/8 and β < 3/4,  where we recall that, for all n ≥ 1, However, one deduce from (2.2) and (3.13) the decomposition, for n ≥ 1, The sequence (M 1 n−1 ) is a square integrable martingale whose increasing process is given, for all n ≥ 1, by (3.76) Moreover, since f is bounded, g does not vanish on its support and (ε k ) has a moment of order 2, one immediately obtain from (3.76) that In addition, since ( θ n ) converges almost surely to θ and n−1 s. In addition, since ε n admits a moment of order > 2, the sequence (M 1 n−1 ) checks a Lyapunov condition. Consequently, one can conclude from the central limit theorem for martingales given e.g. by Corollary 2.1.10 of [6] that (3.77) n 1/2+q M 1 n−1 = o P (1). Hence, it follows from (3.73) and (3.77) that (3.78) n 1/2+q where M 2 n−1 is given by (3.75). Furthermore, since ( θ n ) converges almost surely to θ, one immediately deduce that the asymptotic behaviour of M 2 n is the same as the one of the sequence M 3 n−1 , given for all n ≥ 1, by Finally, one obtain from (3.72) and (3.78) that, for n ≥ 1, where M 3 n−1 is given by (3.79). In addition, (3.52) and the symmetry of f leads to, for all n ≥ 1, where for all −1/2 ≤ x ≤ 1/2, Hence, we deduce from (2.6) and the change of variables u = X i −x h i , that Then, we deduce from the previous equality and (3.83) and (3.84) that (3.85) Moreover, as f and g are two times differentiable, one can write a Taylor expansion of the function a given by (3.82). More precisely, there exists 0 < ξ i < 1 such that a.s.
From now, denote for all 1 ≤ k ≤ n − 1, In the following, we note U a random variable independent of the sequence (U n ) and with the same law as U n . We obtain from (3.89) that, for 1/4 < α < 1/2, where the function ψ 1 and ψ 2 are such that, for all x = (x 1 , x 2 ) ∈ R 2 , In addition, Thus, it follows from (3.94) that for n ≥ 3, Moreover, since ψ 1 (U n ) and ψ 2 (U n ) are integrable, we have One finally deduce from (3.92) and (3.96) that for 1/4 < α < 1/2, √ n θ n − θ = n 1/2+q where for n ≥ 3, and for 1 ≤ k ≤ n − 2, Moreover, as f is symmetric and (ε n ) is of mean 0, it is not hard to see that, for Consequently, the sequence (M n ) is a vectorial martingale whose increasing process is the matrix defined for n ≥ 3, by where Moreover, for q < −1/2, In addition, for n ≥ 3, Hence, one deduce from the Toeplitz lemma that, as q < −1/2, Hence, it immediately follows from (3.102) that In order to apply the central limit theorem for vectorial martingales, it remains to check the Lindeberg condition, that is to say, for all ε > 0, . However, for δ > 0, one have Hence, as (ε n ) has a moment of order > 2, one immediately deduce from (3.105) and (3.106) that there exists κ ε > 0 such that as soon as δ < 1/2+q 1+q . Moreover, in this case, 1/2+q 1+q > 0. If −1 < q < −1/2 there exists a constant c > 0 such that which implies that In the case where (1 + q)(2 + δ) < 1 then, as soon as δ > 1/2+q 1+q , which is right because 1/2+q 1+q < 0. In the case where (1 + q)(2 + δ) ≥ 1, we clearly have Moreover, as (ε n ) is of mean 0 and variance σ 2 and independent of (X n ), straightforward but tedious calculations lead to and σ 1,2 = E [ψ 1 (U)ψ 2 (U)] = σ 2 2 .