Impact analysis of correlated and non-normal errors in nonparametric regression estimation: A simulation study

: In nonparametric regression, the correlation of errors can have important consequences on the statistical properties of the estimators, but the focus is identification of the effect on Average Mean Squared Error (AMSE). This is performed by a Monte Carlo experiment where we use two types of correlation structures and examined with different correlation points/levels and different error distributions with different sample sizes. We concluded that if errors are correlated then distribution of error is important with correlation structures but correlation points/levels have a less significant effect, comparatively. When errors are uniformly distributed, then AMSE are smallest then any other distribution and if errors follow the Laplace distribution then AMSE are largest then other distributions also Laplace have some alarming effect. More keenly, kernel estimator is robust in case of simple correlation structure, and AMSEs attains their minimum when errors are uncorrelated.


Introduction
In real life, various aspects of life, statistically known as variables; are interconnected.To examine the relationship among these variables most common tool is regression.In regression estimation, relationship between response and explanatory variables are determined while prediction is central issues.To estimate the response, nonparametric regression can be applied when the model is unknown and assumptions of a model are relaxed.For this purpose, large sample size is required then compared to parametric methods.Nonparametric regression estimation methods are based on kernels, wavelets and splines [1] .The kernel regression is a nonparametric technique used to estimate response variable by conditional expectation.The purpose is to derive the nonlinear relation between the response variable and co-variate [1] .Ullah and Vinod [2] discussed different nonparametric kernel methods, i.e., Nadaraya-Watson (NW) kernel estimator [3,4] , K-Nearest neighbor (NN) estimator [5,6] , Mack and Muller (MM) estimator [7], Ahmad Lin estimator [8] and Gasser-Muller (GM) estimator [9] .problem is present in data then the estimates are biased [11,12] .Similarly, if errors have unequal variance, then the estimates will still unbiased but less efficient [13] also it is difficult to calculate standard deviation of the forecast errors, usually confidence intervals lie on extreme points; those become too wide or too narrow [14] .
The main purpose of our study is to observe the effect of correlated errors when the nonparametric regression estimation is applied.Altman [15] showed that performance of nonparametric regression estimation is same for both cases, either errors are correlated or not.We are going to examine the effect of correlated errors with non-normality.There is vast literature which provides different estimators or methods to tackle correlated errors and their performance is proven good theoretically and by simulation study, i.e., Muller and Stadtmuller [16] focused only on a fixed design case.Their estimator was based on squared differences of various spans of the data and Smith et al. [17] used a Bayesian method through which transformation of the dependent variable can be performed.Park et al. [14] provided an estimator that is simpler to apply because it does not require any information about the error correlation.Su and Ullah [18] used a pre-whitening transformation of the dependent variable, which is estimated from the data using a technique of local polynomial.Their new established estimator's distribution had weak dependence conditions, and they showed that it is more efficient, then, the local polynomial estimator.Lee et al. [19] method is very efficient in many error structures because their proposed method is based on approximating average squared of errors.
Similarly, Chiu [20] , Hart [21] , Herrmann et al. [22] , Opsomer et al. [23] and De Brabanter et al. [10] provided the modifications in bandwidth selection methods in the presence of correlated errors and they proved that under some restrictions, the proposed methods provided strong consistent results.Methods related to our work are presented in section 2 and finite sample properties of the estimators with the results and its related discussions are summarized in section 3.

Consider a nonparametric regression model
where Y is a dependent variable, X is explanatory variable, () is completely amorphous and   is a normal and random error.To estimate nonparametric regression, Nadaraya-Watson kernel method is used.

Nadaraya-Watson kernel estimator
This method was proposed by Nadaraya [3] and Watson [4] to estimate the unknown function () as given in Equation (1).To do this, they proposed an estimator as given by: where, k is a kernel and h is bandwidth.
Different types of kernels, i.e., Epanechnikov, Gaussian, Tri-weight etc. are available and it is important to note that the choice of the kernel does not affect the Mean Squared Error [24] .
We are using Gaussian kernel in this study as given by Silverman [25] described that a kernel () is a weighting function and it is nonnegative integrable function.Kernels are used in kernel regression to estimate the conditional expectation of a random variable but must satisfy the following conditions; The smoothing parameter; window width or bandwidth denoted by h is used to manage the roughness of the curve.Different methods of bandwidth are available in literature which can be categorized classical or first-generation method and plug-in or second-generation method.Rule of thumb, least squared cross validation, biased cross validation, etc. are part of classical methods; similarly, direct plug-in (DPI) method and solve the equation consists in plug-in method [26,27] .
In our study, we have used plug in method for bandwidth selection.The basic idea behind the selection of bandwidth is to obtain that value of h which minimizes the mean integrated squared error.
Effect of correlated errors is examined by different researchers in which they proposed different methods to tackle this problem like; Kim et al. [28] and De Brabanter et al. [10] used a bimodal kernel technique, Su and Ullah [18] utilized pre-whitening transformation, Lee et al. [19] approximate the squared error and etc.In our work, we examined the effect of correlated errors with four different error distributions on two different correlation structures with different correlation points.We include symmetric (normal, uniform, Laplace and t) error distributions.
To evaluate the performance of different correlation points(), Average Mean Squared Error (AMSE) is used [1,29] , i.e., given by

Monte Carlo experiment
The main purpose of our study is to examine the effect of different correlated errors with normal and independent co-variate.To perform this, the Monte Carlo experiment is conducted.Initially, we generated the error via uniform, t, normal and Laplace distributions when the co-variate is normally and independently distributed.We also compare the behavior of two different correlation patterns and discussed their results.For this work, two kinds of error models are considered: (i) structure given by Park et al. [14] , i.e.,  +1 =   + (1 −  2 ) The performance of the estimator is evaluated over different sample sizes.We have used, n = 25, 50, 100, 200, 500, 1000.For each distribution with various sample sizes, we repeat the original experiment 5000 times.
The Monte Carlo study is outlined.
The   are generated from six distributions.The setup for the generation of   is given by:    ~N(0, 2.5) Normally distributed with  = 0 and  2 = 2.5.
In this experiment plugin bandwidth is used with Gaussian kernel.

Nonparametric estimation with correlated errors with different errors
To examine the effect of different correlated errors on AMSE, we have used four different distributions of errors and results are summarized in Tables 1 and 2.
�   , which is adopted from Park et al. [14] , AMSEs for all error distributions are decreasing as sample size increased and when there is no correlation, the behavior of AMSE is almost stable for small and large sample sizes.Also, it has been noted that there is no effect of the direction of the correlation.
When errors are uniformly distributed AMSEs are smaller then for any other distribution either there is a correlation or not.When there is no correlation, then AMSEs are smaller and decrease with sample size.It is interesting to note, that in our first case, increase in level of correlation () results in decrease of AMSEs.
When errors follow t-distribution, all AMSEs are very close to each other and it seems that our first correlation structure makes AMSEs robust against levels of correlation.Maybe it is due to large degrees of freedom.Like other distributions, the AMSEs have also decreasing trend.In case of Laplace errors, AMSEs are very large and bearing same trend.It can be seen from Table 2 that, in the correlation structure;  +1 =   , AMSEs for all error distributions are decreasing as sample size increases for both positive and negative correlation points and same is the case for zero correlation for all types of error distributions either that is normal or non-normal, moreover, there is no effect of the direction of the correlation.Whatever the distribution of errors, AMSEs decreased when the intensity of correlation of errors increased.When errors are uniformly distributed AMSE are smallest, then, any other AMSE.In case when errors follow the Laplace distribution, AMSE are larger and the interesting thing is that there whatever the distribution of errors, AMSEs decreased when intensity of correlation of errors increased.When errors are uniformly distributed AMSE are smallest, then, any other AMSE.In case when errors follow the Laplace distribution, AMSE are largest and the interesting thing is that there whatever the distribution of errors, AMSEs decreased when intensity of correlation of errors increased.When errors are uniformly distributed AMSE are smallest, then, any other AMSE.In case when errors follow the Laplace distribution, AMSE are largest and the interesting thing is that there exists increasing trend with increase in sample size.

Conclusions
From above discussion, it is concluded that, the higher values of correlation affect the performance of the smoother and as the correlation approaches to zero, the AMSEs are very low.Structure of correlation also matters a lot with distribution of error.Non-normality also effects on the performance of the estimator.When errors follow t and uniform distributions, the performance of smoother is very good for high correlations also and even supersedes the normal errors.This may lead to the utilization of t and normal errors in case of correlated errors.The case of Laplace is very drastic, as the performance is very poor and also the AMSEs are increasing with the increase in sample size and not recommended for use.

Table 1 .
AMSE for NW estimation varying sample sizes, correlation and distribution of errors-Case I.

Table 2 .
AMSE for NW estimation varying sample sizes, correlation and distribution of errors-Case II.