Square Root Extended Kernel Recursive Least Squares Algorithm for Nonlinear Channel Equalization

This study presents a square root version of extended kernel recursive least square algorithm. Basically main idea is to overcome the divergence phenomena arise in the computation of weights of the extended kernel recursive least squares algorithm. Numerically stable givens orthogonal transformations are used to obtain the next iteration of the algorithm. The usefulness of the proposed algorithm is illustrated by discussing its application on the nonlinear multipath fading channel equalization based on Rayleigh distribution. Experiments are performed on slow fading Rayleigh channel with scattered signals.


INTRODUCTION
Kernel methods became very popular in the past decade because of their firm mathematical foundation and broadly used glorious applications. Powerful nonlinear extensions to linear algorithm in RKHS have been made in the recent past which includes support vector machines, kernel principal component analysis, kernel least mean square, affine projection algorithms, kernel Adaline, normalized, leaky and Quantized KLMS (Haykin, 1998(Haykin, , 1996Frieb and Harrison, 1999). Successful applications for the kernel based algorithm have been reported continuously in the various fields of signal processing for instance time series prediction, protein and DNA analysis, equalization of nonlinear channels, pattern recognition, regression and many more. Recently recursive least squares (Engel et al., 2004(Engel et al., , 2008 in RKHS have been developed and extension to this algorithm have made through L2-norm regularization (Vaerenbergh et al., 2006) to improve generalization referred as Sliding Window Kernel Recursive Least Squares (SWKRLS) algorithm. Least but most important extended recursive least squares algorithm in RKHS has been developed recently by Liu et al. (2009). This development will be extremely useful due to the close relationship between Ex-RLS and Kalman filter. After development of Ex-KRLS in RKHS, a new research line has been formed which leads to the formulation of nonlinear kalman filter. Different nonlinear extensions towards kalman filter have been developed so far in the literature, which includes extended, unscented and cubature kalman filter. These filters use either linearization or sampling method at each iteration for state estimation. Where, in Ex-KRLS the weights of the kernel regressor is obtained by using kalman filter with state transition process.
Many of the fast recursive algorithms suffer numerical stability issues, which typically or in some sense inborn from the parent algorithms. For example, the recursive least squares algorithm provides solution of the normal equation by obtaining inverse of correlation matrix as a difference of two correlation matrices. This difference may lose its positive definite characteristic (Sayed, 2003). Same is the case with the kernel RLS algorithm. After computation of the inverse of the correlation matrix in the feature space, numerical difficulties may arises, which results in unstable behavior and finally approaches towards divergence (Vapnic, 1995). In kernel methods the issue of negative definiteness is almost vanished, because they use Gaussian and tangent hyperbolic function to transform the incoming data. But the problem arises when the state error correlation matrix Sሺ݅ሻ is computed; there is a possibility that the resultant matrix will violate the symmetry and nonnegative property. The hypothesized general nonlinear state space model given by Liu et al. (2010) formulate the base for the derivation of algorithm.
which is similar to the model Ex-RLS (Engel et al., 2004) except the input uሺ݅ሻ after transformation in the feature space becomes φሺ‫ݑ‬ ሻ ∈ ℱ. ‫ܠ‬ሺ݅ሻ Also now lies in the infinite dimensional feature space. Where the state transition operator has the form A = αI. The detail of the algorithm is. To resolve this issue we apply cholesky factorization which greatly helps us in removing the nonnegative definite character of the state error correlation matrix. Stable givens orthogonal transformations are then used to obtain the next iterations of the algorithm, which greatly helps us in achieving numerical stability and also to overcome the divergence.
Square root Ex-KRLS: we introduce here square root extended kernel recursive least squares algorithm. Riccatti equation obtained from the formulation of Ex-kRLS is written as: Start with cholesky factorization of riccatti equation the state error correlation matrix Sሺ݅ሻ propagated in its square root form, according to which: where, the term reserved for the lower triangular matrix is Sሺ݅ሻ ଵ/ଶ and the transposed term is ‫܁‬ሺ݅ሻ /ଶ is upper triangular matrix. The matrix product written in the above equation never became negative definite, because the product and its transpose is always nonnegative definite (Pokharel et al., 2009). And generally square root form of the correlation matrix is generally better than the original one in terms of numerical errors.
L-by-L matrix: correlation matrix of the predicted state error Sሺ݅ − 1ሻ L-by-1 vector: Transformed input vector φሺu ୧ ሻ Scaler term: variance of the innovation or state error process ߚ + φ ሺu ୧ ሻSሺ݅ − 1ሻφሺu ୧ ሻ By keeping in view the dimensionalities of the three terms we may order as: And can be explained in the factored form as: The matrix product stated on the right hand side of the equation 4 may be viewed as the matrix product ܺ and its transpose X . Now the stage is possibly set for invoking the matrix lemma stated earlier, according to which where, Θ an orthogonal matrix, operates on X. In such a way that resulting matrix Y bearing a lower triangular structure having all its elements above the main diagonal are zero. Invoking orthogonality of transformation matrix Θ. We may write the above equation as: ሺ݅ = 1,2,3,4, … … … … . . . , ‫ܮ‬ሻ ሺ‫ܮ‬ + 1ሻ × ሺ‫ܮ‬ + 1ሻ Identitiy matrix ‫ܫ‬ ାଵ the elements at the position ሺ݅, ݅ሻ, ሺ݅, ‫ܮ‬ + 1ሻ, ሺ‫ܮ‬ + 1, ݅ሻܽ݊݀ ሺ‫ܮ‬ + 1, ‫ܮ‬ + 1ሻ. The rotation angle ߆ is carefully chosen to annihilate the elements at the position ሺ‫ܮ‬ + 1, ݅ሻ of the matrix to which givens rotation (Golub et al., 1996) applied. By following this procedure φሺ݅ሻ S ଵ/ଶ ሺ݅ − 1ሻ will be eliminated term by term: We proceed by comparing terms and expanding products on both sides of the equalities to get the identities: The equations defined in 6, now distinguishes between two arrays of numbers, which deserves close resemblance.
Prearreay: the array of numbers on the left side of Eq.
(2) is operated on by the orthogonal transformation, that is designed to annihilate the vector term λ ି భ మ φሺ݅ሻ Sሺ݅ − 1ሻ element by element.

Postarray:
The second array of numbers, on the right hand side of Eq. (2) will be a lower triangular matrix that results from the annihilation performed by the orthogonal rotation in the prearray. The scalar term in the prearray induces the generation of two useful terms: Square root of the mean square error covariance term ‫ݎ‬ሺ݅ሻ ଵ/ଶ and the vector product gሺiሻ‫ݎ‬ሺ݅ሻ ଵ/ଶ which makes possible in computing the gain vector.
Detail summary of Square Root Extended Kernel Recursive Least Squares (sqrEx-kRLS) Algorithm. Initialize the algorithm by setting: As the computation of the matrix Sሺ݅ − 1ሻ will play an important role in the development of the innovation‫ݎ‬ሺ݅ሻ. By reproducing the lemmas given in Liu et al. (2010), greatly helps us solving these values, Lemma 1: The matrices Sሺ݅ሻ of the reccatti recursive equation are of the following form.

Proof: Start with the equation:
If the claim is valid for ݇ = 1, then by induction, assuming that it is true for ݇ = ݅ − 1, Substituting the above value of Sሺ1ሻ in the recursive equation of the algorithm, we have: The terms in the above equation are defined as: By applying cholesky decomposition we can split the matrix Sሺ݅ሻ in upper and lower triangular matrix Sሺ݅ሻ ଵ/ଶ and Sሺ݅ሻ /ଶ respectively. The term ‫ݎ‬ሺ݅ሻ will be computed as: The weight vector which lies in a high dimension feature space can be expressed as discussed (Liu et al., 2008) in the next lemma: Lemma 2: Start with initial weights ‫ݓ‬ሺ0ሻ = 0, ‫ݓ‬ሺ1ሻ = ‫ݓ‬ሺ0ሻ + gሺ1ሻ݁ሺ1ሻ Then by induction, assuming that it is true for ݅ − 1. We pursue as: By putting the value of Sሺ݅ − 1ሻ from lemma 1 we have: Weight vector described in Eq. (12) simplifies the computation of the weights of square root kRLS. The proposed algorithm is tested for the equalization of nonlinear channel discussed briefly in the simulation section.

SIMULATIONS AND EXPERIMENTAL RESULTS
We apply square root extended kernel recursive least square filter to the problem of tracking a nonlinear Rayleigh multipath channel and compare its performance with the Ex-KRLS filter. We assumed traditional nonlinear Rayleigh fading channel, 10 paths have been chosen for this experiment with 15 assumed numbers of scatters. 500 symbols are assigned to run the experiment with maximum Doppler's frequency of 50 Hz. Sampling rate of 6ߤ seconds is chosen for all scattered paths. Channel matrix of specified dimensions are created and passed through the noise (AWGN) with variance 0.001. The kernel type taken here in this experiment is Gaussian with ߜ ଶ = .05. 500 symbols are used for the experiment and all the coefficients of the channel matrix are obtained assumed to be real. Mean square error has been plotted to show the effectiveness of the proposed square root Ex-RLS. (Fig. 1) The experiment is tested on two algorithms. The first one is Ex-KRLS filter and the second algorithm is the proposed square root Ex-KRLS filter. The experimental result clearly shows that there is an improvement of approximately 3dB in terms of mean square error, will be observed while using square root Ex-RLS filter and it outer performs its Ex-KRLS and enjoys better performance in terms of mean square error. Scattered multipath channel input signal is demonstrated in the next result. Details of the parameters of the filter used in the experiment:  The performed experiment is tested on two algorithms. The first one is Ex-RLS filter and the second algorithm is the proposed square root Ex-KRLS filter. The experimental result clearly shows that there is an improvement of approximately 3dB in terms of mean square error, will be observed while using square root Ex-RLS filter and it outer performs its Ex-KRLS and enjoys better performance in terms of mean square error. Scattered multipath channel input signal is demonstrated in the next result. Figure 2 demonstrates the response of a slow fading nonlinear Rayleigh channel with number of multipath is chosen as M = 10. Only the real coefficients are used in this experiment. With Doppler's frequency of 50 Hz and the sampling rate of 6ߤ seconds. In Rayleigh fading model several objects disperse the radio signal before it reaches at the receiver. In our model we assume 15 no. of scatterers. Figure 3 and 4 demonstrates the weights of both the algorithms, Ex-kRLS and sq.root Ex-kRLS. Which clearly support our claim? Figure 4 shows the weights of the proposed algorithm. The weights of the square root algorithm achieve stable behavior in last 200 iterations.

CONCLUSION AND RECOMMENDATIONS
In this study, a new square root extended kernel recursive least square algorithm is proposed based on cholesky decomposition and givens rotation. In order to overcome the problem of divergence in the weights of Ex-kRLS, the concept of square root filtering is applied in this study. The proposed algorithm is tested on nonlinear Rayleigh fading channel equalization problem illustrates that proposed square root Ex-kRLS algorithm achieves better tracking performance in comparison with the Ex-kRLS. The learning curves of the weights of both the algorithm are observed, which clearly demonstrates that proposed algorithm achieve early after the few no. of iterations to a stable state while the other didn't achieve its stable state throughout the iterations. Kernel algorithm gives better tracking performance in comparison with their linear counterparts, but this superiority is availed at the cost of greater computation complexity. Square root kernel RLS should be tested in the future for nonlinear problems.