Principal Components Regression Estimation in Semiparametric Partially Linear Additive Models

Partially linear additive model is a popular multivariate nonparametric fitting technique. This paper considers estimation for the semiparametric model in the presence of multicollinearity. Combining the profile least-squares method and principal components regression technique, we propose a novel biased estimator for the regression coefficients, and provide the asymptotic bias and covariance matrix of the proposed estimator. A Monte Carlo simulation study is conducted to examine the performance of the proposed estimators and the results are satisfactory.


Introduction
In using the linear regression models, we often encounter the problem of multicollinearity, and its statistical consequences are very well known in econometrics and statistics. in the last decade, estimation of semiparametric regression model by the biased estimate approach to solve the problem of multicollinearity has received much attention in the literature.Hu (2005) and Akdeniz and Tabakan (2009) applied ridge estimator and restricted ridge estimator for the regression coefficients.The asymptotic properties of restricted ridge estimators were discussed by Roozbeh and Arashi (2013).Akdeniz and Akdeniz Duran (2010) introduced a Liu-type estimator for the parametric component, and the properties of the Liu-type estimator were studied by Akdeniz Durana et al. (2011).Akdeniz Duran and Akdeniz (2013) proposed a new Liu estimator based on the difference method.
As we all know, an important alternative method to overcome the problem of multicollinearity is principal component regression (PCR) approach which can be found in any textbook of the linear regression models.In this paper, we introduce a novel PCR estimating approach for partially linear additive model, which is a generalization of additive model and semiparametric regression model.In general, the semiparametric model can be written as where Y is response, X and Without loss of generality, we also assume that both the Y i and X i are centered.Obviously, if β = 0, model (1) reduces to the additive model of Friedman and Stuetzle(1981) and Hastie and Tibshirani (1990).when q = 1, model (1) becomes the partially linear model of Engle et al. (1986).
For model (1), many approaches have been proposed to estimate the unknown regression coefficients and nonparametric functions.a backfitting estimator for β was proposed by Ospomer and Ruppert (1999), and it is root-n consistent.Series method was applied by Li (2000) to estimate the model (1).Manzan and Zerom (2005) proposed a estimator for β based on the marginal integration method.Following Liang et al. (2008), Wei and Liu (2012) proposed a profile least-squares estimator for β, and also construed a restricted estimator when some additional linear restrictions on the parametric component are available.In the following, we will develop a novel PCR estimator for β based on the profile least-squares method.
The rest of this paper is organized as follows.The profile PCR estimator is proposed in Section 2. A simulation study is conducted in Section 3. The proofs of the main results are given in Section 4.

Profile PCR Estimation
For the need of constructing the PCR estimator, we first introduce the profile least-squares method of Liang et al. (2008) and Wei and Liu (2012).
Like Liang et al. (2008), we assume q = 2 in model ( 1) for notational simplicity.Let {Y i , X i , Z 1i , Z 2i } n i=1 be a random sample from model (1) with q = 2. Then we have Suppose β is known, then model ( 2) reduce to Obviously, model ( 3) is a bivariate additive model which has been studied by Opsomer and Ruppert (1997).
1,z1 and s T 2,z2 be the equivalent kernels for the local linear regression at z 1 and z 2 , respectively, that is, where , By Opsomer and Ruppert (1997), m 1 and m 2 can be estimated by the backfitting method, and we have with Replacing m 1 and m 2 of model ( 2) by m1 and m2 , respectively, we have where Apply least-square approach to linear regression model ( 5), we can get the profile least-squares estimator for β, In the following, we apply PCR technique to linear model (5 are the eigenvalues of XT X.Then, we have the following transformation for model with Z = XΦ and α = Φ T β. Assume that (λ k+1 , λ k+2 , • • • , λ p ) are near zero, then we can define the subdivision, Then, we have We can define the profile least-squares estimator of α 1 with Z 2 α 2 omitted as Finally, the PCR estimator of β can be defined as The following theorem gives the asymptotic properties of βPCR .
Theorem 1 Under Assumptions 1-4 in the Section 4, for the profile PCR estimator βPCR , we have ] T .

Simulation Studies
In this section, some simulations are conducted to examine the performance of the proposed procedure.Consider the following partially linear additive model Following McDonald and Galerneau (1975), the explanatory variables are generated by where w ji are independent standard normal pseudo-random numbers, ρ is specified so that the theoretical correlation between any two explanatory variables is given by ρ 2 .The true parameters are taken as (β 1 , β 2 , β 3 , β 4 , β 5 ) = (1, 2, 1.5, 0.5, 1), and n = 80, 100, 120 are considered.In this study, ρ = 0.9, 0.99 and 0.999 are considered so that the condition numbers indicate a weak to severe collinearity.We take (1)ε i ∼ N(0, 1), (2 ).The Epanechnikov kernel K(x) = 0.75(1 − x 2 )I |x|≤1 is used in our simulation.Furthermore, we use the CV technique to choose the bandwidth.
To compare the proposed estimators, a criterion for measuring the "goodness" of an estimator is needed.For this purpose, the mean squared error (MSE) criterion is used throughout our study.For each setting 1000 replications are generated.β is the profile least-squares estimator of β, β1 PCR and β2 PCR are the PCR estimators of β with the first two principal components was selected, and the first three principal components was selected, respectively.The estimated mean squared error (EMSE) is computed for each of the above three estimators, and results is presented in Table 1.The EMSEs for the different estimators are calculated as follows: where β * k j denotes the estimate of the jth parameter in kth replication and β j , j = 1, 2, 3, 4, 5 are the true parameter values.

Table 1 .
EMSEs of the estimators