Efficient Estimation in Heteroscedastic Varying Coefficient Models

Abstract

This paper considers statistical inference for the heteroscedastic varying coefficient model. We propose an efficient estimator for coefficient functions that is more efficient than the conventional local-linear estimator. We establish asymptotic normality for the proposed estimator and conduct some simulation to illustrate the performance of the proposed method.

1. Introduction

Recently, the varying coefficient model has attracted much attention among econometricians and statisticians. One attractive feature of this model is its ability to capture the nonlinearity of the data without suffering from the “curse of dimensionality”. In general, it is of the form

$$Y_i={\mathbf{X}}_i^{\mathrm{T}}\mathit{\alpha }\left(U_i\right)+{\epsilon }_i,i=1,2,\cdots ,n$$

(1.1)

where $Y_i^{,}s$ are responses; $X_i={(X_{i1},X_{i2},\cdots ,X_{ip})}^{\mathrm{T}}$ and $U_i$ are associated covariates; $\mathbf{\alpha }(·)={({\mathbf{\alpha }}_1(·),{\mathbf{\alpha }}_1(·),\cdots ,{\mathbf{\alpha }}_p(·))}^{\mathrm{T}}$ is a p-dimensional vector of unknown functions; ${\epsilon }_i^{,}s$ are independent and identically distributed random errors with $\mathrm{E}\left({\epsilon }_i\right|{\mathbf{X}}_i,U_i)=0$ and $\mathrm{Var}\left({\epsilon }_i\right|{\mathbf{X}}_i,U_i)={\sigma }^2({\mathbf{X}}_i,U_i)$.

Due to its flexibility, the varying coefficient model has been studied in many different contexts and has been successfully applied to nonlinear time series analysis, longitudinal and functional data analysis, panel data analysis, spatial data analysis, and time-varying models in finance. See, for example, the work of Cai et al. [1], Cai [2], Cai and Li [3], Cai et al. [4], Fan and Zhang [5], Fan et al. [6], Fotheringham et al. [7], Hoover et al. [8], Li et al. [9] and Xiao [10], among others.

In the above models, the varying coefficient model is generally estimated by the local-linear approach. Usually, the errors are assumed to be i.i.d. to start. However, in applications, heteroscedasticity is often found in residuals from both cross-sectional and time series modelling. In the context of the linear regression model, it is well known that if the errors are heteroscedastic, then the generalized least-squares (GLS) estimator is more efficient than ordinary least-squares (OLS) estimator. To the best of our knowledge, there has been no work on the problem of designing an efficient estimation method for varying coefficient models with heteroscedastic errors. In this paper, we propose an efficient estimator for varying coefficients based on the local linear approach.

The paper is structured as follows. We introduce an efficient estimator in Section 2, and their asymptotic properties are given in Section 3. We report the results of some Monte Carlo simulations in Section 4.

2. Efficient Estimation

Without considering heteroscedasticity, we apply a local linear regression technique to estimate the varying coefficient functions. For each given u, the local linear estimator $\widehat{\mathbf{\alpha }}\left(u\right)$ of $\mathbf{\alpha }\left(u\right)$ is the part corresponding to $\mathbf{a}$ of the minimizer of

$$\sum _{i=1}^n{\left[Y_i-{\mathbf{X}}_i^{\mathrm{T}}\mathbf{a}-{\mathbf{X}}_i^{\mathrm{T}}(U_i-u)\mathbf{b}\right]}^2{\mathrm{K}}_h(U_i-u_0),$$

(2.1)

where K is a kernel function, h is a bandwidth and $K_h(·)=K(·/h)/h$. Then we have

$$\widehat{\mathbf{\alpha }}\left(u\right)={[{\widehat{\alpha }}_1\left(u\right),\cdots ,{\widehat{\alpha }}_p\left(u\right)]}^{\mathrm{T}}=\left({\mathbf{I}}_p{\mathbf{0}}_p\right){\left\{{\mathbf{D}}_u^{\mathrm{T}}{\mathbf{W}}_u^{\delta }{\mathbf{D}}_u\right\}}^{-1}{\mathbf{D}}_u^{\mathrm{T}}{\mathbf{W}}_u^{\delta }\mathbf{Y}.$$

(2.2)

where

$$\mathbf{X}=\left[\begin{array}{c}{\mathbf{X}}_1^{\mathrm{T}}\\ {\mathbf{X}}_2^{\mathrm{T}}\\ ⋮\\ {\mathbf{X}}_n^{\mathrm{T}}\end{array}\right],\mathbf{Y}=\left[\begin{array}{c}{Y}_1\\ Y_2\\ ⋮\\ Y_n\end{array}\right],{\mathbf{D}}_u=\left[\begin{array}{cc}{\mathbf{X}}_1^{\mathrm{T}}& \frac{U_1-u}{h}{\mathbf{X}}_1^{\mathrm{T}}\\ {\mathbf{X}}_2^{\mathrm{T}}& \frac{U_2-u}{h}{\mathbf{X}}_2^{\mathrm{T}}\\ ⋮& ⋮\\ {\mathbf{X}}_n^{\mathrm{T}}& \frac{U_n-u}{h}{\mathbf{X}}_n^{\mathrm{T}}\end{array}\right],$$

and ${\mathbf{W}}_u=\mathrm{diag}\{K_h(U_1-u),K_h(U_2-u),\cdots ,K_h(U_n-u)\}$.

The estimator $\widehat{\mathbf{\alpha }}\left(u\right)$ ignores the information contained in the variance matrix and it is inefficient. To overcome this, we propose a class of efficient estimators in the following.

Denote ${\sigma }_i=\sqrt{{\sigma }^2({\mathbf{x}}_i,u_i)}$, for the moment where we assume that ${\sigma }_i$ is known. Multiply both sides of model (1.1) by $1/{\sigma }_i$, we have the following homoscedastic varying coefficient model

$$Y_i^{*}={\mathbf{Z}}_i^{\mathrm{T}}\mathbf{\alpha }\left(u_i\right)+e_i,i=1,2,\cdots ,n,$$

(2.3)

where $Y_i^{*}=y_i/{\sigma }_i,{\mathbf{Z}}_i={\mathbf{X}}_i/{\sigma }_i$ and $e_i={\epsilon }_i/{\sigma }_i$ with $\mathrm{E}\left(e_i\right|{\mathbf{x}}_i,u_i)=0,\mathrm{Var}\left(e_i\right|{\mathbf{x}}_i,u_i)=1$.

Applying the local linear approach to model (2.3), the efficient estimator of $\alpha \left(u\right)$ is given as follows

$$\tilde{\mathbf{\alpha }}\left(u\right)=\left({\mathbf{I}}_{\mathrm{p}}{\mathbf{0}}_{\mathrm{p}}\right){\left\{{\mathbf{B}}_u^{\mathrm{T}}{\mathbf{W}}_u{\mathbf{B}}_u\right\}}^{-1}{\mathbf{B}}_u^{\mathrm{T}}{\mathbf{W}}_u{\mathbf{Y}}^{*}.$$

(2.4)

where ${\mathbf{Y}}^{*}=(Y_1^{*},Y_2^{*},\cdots ,Y_n^{*})$,

$${\mathbf{B}}_{\mathbf{u}}=\left[\begin{array}{cc}{\mathbf{Z}}_1^{\mathrm{T}}& {\mathbf{Z}}_1^{\mathrm{T}}\frac{U_1-u}{h}\\ {\mathbf{Z}}_2^{\mathrm{T}}& {\mathbf{Z}}_2^{\mathrm{T}}\frac{U_2-u}{h}\\ ⋮& ⋮\\ {\mathbf{Z}}_n^{\mathrm{T}}& {\mathbf{Z}}_n^{\mathrm{T}}\frac{U_n-u}{h}\end{array}\right].$$

3. Asymptotic Property

First, we make the following assumptions. Let ${\mu }_i={\int }_0^{\infty }{t}^{i}K\left(t\right)dt,{\nu }_i={\int }_0^{\infty }{t}^i{K}^2\left(t\right)dt$.

Assumption 1. The errors ${\epsilon }_i(i=1,2,\cdots ,n)$ are independent and $0<\mathrm{E}\left({\epsilon }_i^2\right)<\infty$ and $\mathrm{Var}\left({\epsilon }_i^2\right)>0$.

Assumption 2. The random variable U has a bounded support Π. Its density function $f(·)$ is Lipschitz continuous and bounded away from 0 on its support.

Assumption 3. The $p\times p$ matrixes $E\left[\mathbf{X}{\mathbf{X}}^{\mathrm{T}}\right|U]$ and $E[\mathbf{X}{\mathbf{X}}^{\mathrm{T}}/{\sigma }^2(\mathbf{X},U)|U]$ are non-singular for each $U\in \Pi$.

Assumption 4. There is an $s>2$ such that $E\parallel \mathbf{X}{\parallel }^{2s}<\infty$ and for some $k<2-s^{-1}$ such that $n^{2k-1}h\to \infty$ as $n\to \infty$.

Assumption 5.$\{{\alpha }_j(·),j=1,\cdots ,p\}$ have continuous second derivatives in $U\in \Pi$.

Assumption 6. The function $K(·)$ is a symmetric density function with compact support and the bandwidth h satisfies $n{h}^8\to 0$ and $n{h}^2/{(logn)}^2\to \infty$ as $n\to \infty$.

For the estimator $\widehat{\alpha }\left(u\right)$, Cai et al. [1] proved the following result:

Theorem 1 Under the assumptions 1–6, the estimator $\widehat{\alpha }\left(u\right)$ is asymptotically normal, namely,

$$\sqrt{nh}\left(\widehat{\mathbf{\alpha }}\left(u\right)-\mathbf{\alpha }\left(u\right)-\frac{1}{2}{h}^2{\mu }_2{\mathbf{\alpha }}^{\prime \prime }\left(u\right)\right)\to N(\mathbf{0},{\nu }_0\mathbf{\Psi }/f\left(u\right)),$$

where $\mathbf{\Psi }=\mathbf{\Gamma }{\left(U\right)}^{-1}\mathrm{E}\left[\mathbf{X}{\mathbf{X}}^{\mathrm{T}}{\sigma }^2(\mathbf{X},U)\right|U]\mathbf{\Gamma }{\left(U\right)}^{-1}$, $\mathbf{\Gamma }\left(U\right)=\mathrm{E}\left[\mathbf{X}{\mathbf{X}}^{\mathrm{T}}\right|U]$.

For the estimator $\tilde{\mathbf{\alpha }}\left(u\right)$, we obtain the following result by the Theorem 1 directly.

Theorem 2 Under the assumptions 1–6, the estimator $\tilde{\mathbf{\alpha }}\left(u\right)$ is asymptotically normal, namely,

$$\sqrt{nh}\left(\tilde{\mathbf{\alpha }}\left(u\right)-\mathbf{\alpha }\left(u\right)-\frac{1}{2}{h}^2{\mu }_2{\mathbf{\alpha }}^{\prime \prime }\left(u\right)\right)\to N(\mathbf{0},{\nu }_0{\mathbf{\Phi }}^{-1}/f\left(u\right)),$$

where $\mathbf{\Phi }=\mathrm{E}\left(\mathbf{Z}{\mathbf{Z}}^{\mathrm{T}}\right|U)=\mathrm{E}(\mathbf{X}{\mathbf{X}}^{\mathrm{T}}/{\sigma }^2(\mathbf{X},U)|U)$.

Denote $\overline{\mathbf{X}}={({\overline{\mathbf{X}}}_1,\cdots ,{\overline{\mathbf{X}}}_n)}^{\mathrm{T}},{\overline{\mathbf{X}}}_i=\sqrt{K_h(U_i-u)}{\mathbf{X}}_i,\Sigma =\mathrm{diag}\left\{{\sigma }^2({\mathbf{X}}_1,U_1),\cdots ,{\sigma }^2({\mathbf{X}}_n,U_n)\right\}$. By the proof of Theorem 1 in Cai et al. [1], we have

$$\frac{1}{n}\sum _{i=1}^n{K}_h(U_i-u){\mathbf{X}}_i{\mathbf{X}}_i^{\mathrm{T}}={\overline{\mathbf{X}}}^{\mathrm{T}}\overline{\mathbf{X}}\stackrel{p}{⟶}\mathrm{E}\left(\mathbf{X}{\mathbf{X}}^{\mathrm{T}}\right|U)f\left(u\right),$$

$$\frac{1}{n}\sum _{i=1}^n{K}_h(U_i-u){\mathbf{X}}_i{\mathbf{X}}_i^{\mathrm{T}}{\sigma }^2(\mathbf{X},U)={\overline{\mathbf{X}}}^{\mathrm{T}}\Sigma \overline{\mathbf{X}}\stackrel{p}{⟶}\mathrm{E}\left(\mathbf{X}{\mathbf{X}}^{\mathrm{T}}{\sigma }^2(\mathbf{X},U)\right|U)f\left(u\right),$$

and

$$\frac{1}{n}\sum _{i=1}^n{K}_h(U_i-u){\mathbf{X}}_i{\mathbf{X}}_i^{\mathrm{T}}/{\sigma }^2(\mathbf{X},U)={\overline{\mathbf{X}}}^{\mathrm{T}}{\Sigma }^{-1}\overline{\mathbf{X}}\stackrel{p}{⟶}\mathrm{E}(\mathbf{X}{\mathbf{X}}^{\mathrm{T}}/{\sigma }^2(\mathbf{X},U)|U)f\left(u\right).$$

Since ${\left({\overline{\mathbf{X}}}^{\mathrm{T}}\overline{\mathbf{X}}\right)}^{-1}{\overline{\mathbf{X}}}^{\mathrm{T}}\Sigma \overline{\mathbf{X}}{\left({\overline{\mathbf{X}}}^{\mathrm{T}}\overline{\mathbf{X}}\right)}^{-1}\ge {\left({\overline{\mathbf{X}}}^{\mathrm{T}}{\Sigma }^{-1}\overline{\mathbf{X}}\right)}^{-1}$, then we have

$$\mathbf{\Psi }\ge {\mathbf{\Phi }}^{-1}.$$

This implies that $\tilde{\mathbf{\alpha }}\left(u\right)$ is asymptotically more efficient than $\widehat{\mathbf{\alpha }}\left(u\right)$ in terms of asymptotic covariance matrix.

Remark 1. Since $\tilde{\mathbf{\alpha }}\left(u\right)$ depends on the unknown parameters ${\sigma }^2(\mathbf{X},U)$, it is infeasible. To provide a feasible efficient estimator of $\mathbf{\alpha }\left(u\right)$ , we need to estimate ${\sigma }^2(\mathbf{X},U)$ consistently. It is not difficult to show that the resultant feasible estimator has the asymptotic property as $\tilde{\mathbf{\alpha }}\left(u\right)$.

Remark 2. To obtain the consistent estimator of the variance function ${\sigma }^2(\mathbf{z},u)$, it is important to model ${\sigma }^2(\mathbf{z},u)$. Several kinds of variance function have been proposed. Discussion on the parametric variance function can be found in Carroll and Ruppert [11]. Muller and Stadtmuller [12], Chiou and Muller [13] and Ruppert et al. [14] studied nonparametric variance estimation. Muller and Zhao [15] proposed a general semiparametric variance function model in a fixed design regression setting. Keilegom and Wang [16] considered a general class of mean-variance regression models, in which both the mean function and the variance function were semiparametrically modeled. Zhu et al. [17] consider a single-index structure to study heteroscedasticity in a single-index regression model with high-dimensional predictors.

4. Simulation Studies

In this section we compare the behavior of the conventional estimator $\widehat{\mathbf{\alpha }}\left(u\right)$ with that of the new estimator $\tilde{\mathbf{\alpha }}\left(u\right)$, given in (2.2) and (2.4), respectively, when the sample size is finite. The data are generated from the following varying coefficient model

$$y_i=x_i\alpha \left(u_i\right)+\sigma (x_i,u_i){\epsilon }_i,i=1,2,\cdots ,n,$$

(4.1)

where $x_i\sim N(0,1),u_i=i/n$, $\alpha \left(u_i\right)=u_i+sin\left(2\pi u_i\right)$. Firstly, we consider the following four known variance functions:

$$\left(A\right):\sigma (x_i,u_i)=e^{x_i};\left(B\right):\sigma (x_i,u_i)=e^{u_i};$$

$$\left(C\right):\sigma (x_i,u_i)=1+x_i;\left(D\right):\sigma (x_i,u_i)=1+u_i.$$

Secondly, we consider the case that the variance function is unknown. For simplicity, the variance function is assumed to have the following parametric structure,

$$\left(E\right):E\left({\epsilon }_i^2\right|x_i,u_i)={\sigma }^2(x_i,u_i)=exp({\gamma }_0+{\gamma }_1{u}_i),$$

with ${\gamma }_0=1,{\gamma }_1=2$. Obviously, we can build the following linear regression model

$$ln{\epsilon }_i^2={\gamma }_0+{\gamma }_1{u}_i+{\xi }_i,$$

(4.2)

with $E{\xi }_i=0$. In practice, ${\epsilon }_i$ is not available, but it may be estimated by ${\widehat{\epsilon }}_i=y_i-x_i^{\mathrm{T}}\widehat{\alpha }\left(u_i\right)$, where $\widehat{\alpha }\left(u_i\right)$ are the local linear estimates of model (4.1) without considering the heteroscedasticity structure. Applying the least squares approach to liner model (4.2) with ${\epsilon }_i$ was replaced by ${\widehat{\epsilon }}_i$, we can obtain the estimators of ${\gamma }_0,{\gamma }_1$, denoted by ${\widehat{\gamma }}_0$ and ${\widehat{\gamma }}_1$ respectively. Accordingly, we get the estimator of ${\sigma }^2({\mathbf{x}}_i,{\mathbf{z}}_i,u_i)$, as ${\widehat{\sigma }}^2({\mathbf{x}}_i,{\mathbf{z}}_i,u_i)=exp({\widehat{\gamma }}_0+{\widehat{\gamma }}_1{u}_i)$.

To study the effect of the distribution of the error for our method, we take the following three different types of the error distribution, (1) ${\epsilon }_i\sim N(0,0.5^2)$, (2) ${\epsilon }_i\sim U(-\sqrt{3}/2,\sqrt{3}/2)$, (3) ${\epsilon }_i\sim \frac{1}{8}{\chi }_8^2-1$. The Gauss kernel function and $h=n^{-1/5}$ are used in our simulation studies.

We compare the proposed efficient estimator $\tilde{\alpha }\left(u\right)$ with that of the ordinary local linear estimator $\widehat{\alpha }\left(u\right)$ by using the estimated mean average squared error (MASE),

$$\mathrm{MASE}\left\{\widehat{\alpha }(·)\right\}=\frac{1}{1000*n}\sum _{l=1}^N\sum _{i=1}^n{[{\widehat{\alpha }}_j^l\left(u_i\right)-{\alpha }_j\left(u_i\right)]}^2,$$

where ${\widehat{\alpha }}^l\left(u_i\right),l=1,2,\cdots ,N,$ are the estimate of the coefficient $\alpha \left(u_i\right)$ in $N=1000$ replications. The simulation results are presented in Table 1, and for all the scenarios we studied, the proposed efficient estimators outperform the ordinary local linear estimators.

Table 1. Mean average squared error (MASE) index for the estimators of varying coefficients.

Variance	Sample	$N(0,0.5^2)$	$U(-\sqrt{3}/2,\sqrt{3}/2)$	$\frac{1}{8}{\chi }_8^2-1$
Function	n	$\widehat{\alpha }(·)$    $\tilde{\alpha }(·)$	$\widehat{\alpha }(·)$    $\tilde{\alpha }(·)$	$\widehat{\alpha }(·)$    $\tilde{\alpha }(·)$
A	30	0.4873 0.1354	0.5125 0.1277	0.5109 0.1374
	50	0.4063 0.0805	0.3820 0.0804	0.3686 0.0805
	80	0.3146 0.0539	0.3054 0.0523	0.2910 0.0525
B	30	0.2018 0.1936	0.1872 0.1791	0.1835 0.1752
	50	0.1129 0.1077	0.1174 0.1126	0.1114 0.1072
	80	0.0728 0.0696	0.0734 0.0706	0.0733 0.0706
C	30	0.1469 0.1302	0.1502 0.1269	0.1571 0.1304
	50	0.1024 0.0920	0.1039 0.0893	0.1015 0.0884
	80	0.0670 0.0679	0.0731 0.0660	0.0734 0.0645
D	30	0.1446 0.1422	0.1390 0.1368	0.1487 0.1474
	50	0.0841 0.0831	0.0844 0.0832	0.0819 0.0807
	80	0.0612 0.0606	0.0556 0.0549	0.0595 0.0589
E	30	0.3950 0.3936	0.4227 0.4133	0.3996 0.3913
	50	0.3094 0.3039	0.3137 0.3068	0.3077 0.3015
	80	0.2549 0.2508	0.2513 0.2508	0.2544 0.2520

Table 1. Mean average squared error (MASE) index for the estimators of varying coefficients.

Variance	Sample	$N(0,0.5^2)$	$U(-\sqrt{3}/2,\sqrt{3}/2)$	$\frac{1}{8}{\chi }_8^2-1$
Function	n	$\widehat{\alpha }(·)$    $\tilde{\alpha }(·)$	$\widehat{\alpha }(·)$    $\tilde{\alpha }(·)$	$\widehat{\alpha }(·)$    $\tilde{\alpha }(·)$
A	30	0.4873 0.1354	0.5125 0.1277	0.5109 0.1374
	50	0.4063 0.0805	0.3820 0.0804	0.3686 0.0805
	80	0.3146 0.0539	0.3054 0.0523	0.2910 0.0525
B	30	0.2018 0.1936	0.1872 0.1791	0.1835 0.1752
	50	0.1129 0.1077	0.1174 0.1126	0.1114 0.1072
	80	0.0728 0.0696	0.0734 0.0706	0.0733 0.0706
C	30	0.1469 0.1302	0.1502 0.1269	0.1571 0.1304
	50	0.1024 0.0920	0.1039 0.0893	0.1015 0.0884
	80	0.0670 0.0679	0.0731 0.0660	0.0734 0.0645
D	30	0.1446 0.1422	0.1390 0.1368	0.1487 0.1474
	50	0.0841 0.0831	0.0844 0.0832	0.0819 0.0807
	80	0.0612 0.0606	0.0556 0.0549	0.0595 0.0589
E	30	0.3950 0.3936	0.4227 0.4133	0.3996 0.3913
	50	0.3094 0.3039	0.3137 0.3068	0.3077 0.3015
	80	0.2549 0.2508	0.2513 0.2508	0.2544 0.2520

5. Conclusions

In this paper, we focus on the estimation problem of the varying coefficient model with heteroscedastic errors. Based on the local linear method, we develop a simple approach to estimate the nonparametric coefficient functions by taking the estimated error heteroscedasticity into account. The resulting estimators are shown to have smaller asymptotic variances than the conventional local-linear estimators. The asymptotic normality of the proposed estimator is established. Furthermore, some simulation experiments are performed to evaluate the finite sample behaviors of the proposed estimators.