A Variable Step-size CLMS Algorithm and Its Analysis

In this paper, a hyperbolic tangent variable step-size convex combination of the least mean square (HTVSCLMS) algorithm is proposed and analyzed. The compromise between the convergence speed and the steady-state error for two filters in a convex combination of the least mean square (CLMS) algorithm is avoided by this study. In the proposed algorithm, the big step-size filter is replaced by a filter whose iteration step-size is a modified function based on hyperbolic tangent function. Thus, hyperbolic tangent nonlinear relationship between step-size and error is constructed. Simultaneously, the small step-size filter remains unchanged but fixed. Therefore, the slow convergence speed and the weak anti-interference ability of fixed step-size CLMS were conquered. Simulation results show that the HTVSCLMS algorithm, compared with CLMS algorithm and variable step-size CLMS (VSCLMS) algorithm, not only has superior capability of tracking in the presence of noise and in a stable and even non-stable environment but also can maintain a better convergence.


Introduction
The step-size of the filter plays a significant role in the convergence process of the least mean square (LMS) algorithm [1], [2]. It not only controls the convergence speed of the initial stage but also determines the steadystate performance of the convergence stage. The bigger the step-size, the faster is the convergence speed and the smaller is the steady-state error. The compromise between convergence speed and steady-state error would be usually made to choose the step-size of LMS filter in order to improve performance; however, it cannot take advantage of the best performance of the filter.
In order to avoid different requirements of the stepsize factor for satisfying convergence speed and steadystate error in a single filter [3], [4], CLMS adaptive algo-rithm based on parallel computing is proposed [5], [6]. CLMS algorithm parallels independent LMS filters with different step-size. The convergence speed could be improved by the big step-size filter; the steady-state performance could be insured by the small step-size filter. Thus, the CLMS algorithm can decrease steady-state error as a basis for improving the convergence speed. However, parallel computation with different step-size usually requests precise algorithm parameters [7], which is not suitable for a filter system to maintain good steady-state performance in a noisy environment. Meanwhile, the CLMS algorithm has an unavoidable deficiency because of the fixed step-size in a time-varying or non-stable environment.
Motivated by this, we propose to introduce variable step-size adjustment in the CLMS algorithm, and change an independent fixed step-size LMS filter to a variable step-size LMS filter that can rapidly converge based on mean square error (MSE). Thus, it can effectively accelerate convergence speed and improve tracking performance of the original algorithm. Simultaneously, the other filter is set to be a small step-size LMS filter in order to ensure steady-state performance and reduce system misalignment. Besides, this method can also coordinate the characteristics of the variable step-size filter that step-size factor is susceptible to noise interference in a steady-state environment, and improve algorithm performance comprehensively while maintaining the advantages of original CLMS algorithm.

CLMS Algorithm
Usually, combination approaches can improve performance of adaptive filter [8]. As shown in Fig. 1, the scheme of the CLMS algorithm includes two independent LMS filters, combined into a preferred approximator in parallel convex [5]. Weight vectors of two filters are adjusted by the joint coefficient λ(n); therefore, each filter can play its own advantages. where y 1 (n) and y 2 (n) (i.e., y i (n) = w i T (n)u(n), i =1, 2, w i (n) being weight vectors of LMS filters and u(n) being their common input vector) are outputs of two transversal LMS filters at iteration time n. When λ(n) is assigned the proper value between zero and one at each iteration, then CLMS obtains the best performance of two filters. It can be defined that λ(n) is adjusted by a sigmoid function with the variable a(n) as Actually, it needs to consider that both w 1 (n) and w 2 (n) are independently adapted with individual LMS rules. Equivalent weight vector of the scheme can be represented as In order to update joint coefficient λ(n), the steepest descent method should be used to optimize the equivalent error of combination filters, i.e., e 2 eq (n) = [d(n) -y eq (n)] 2 [9], where d(n) stands for the desired signal. Due to the introduction of an intermediate variable a(n) shown in (2), the updating equation for a(n) can be obtained by 2 eq ( ) ( 1) ( ) 2 ( ) a e n a n a n a n where μ a is a constant tending to be larger. Thus, the convex combination structure tends to have a faster convergence speed compared to the filter with faster convergence speed. Due to the good robustness of LMS algorithm, a(n) obtains the good robustness as well. Thus, CLMS algorithm has good stability.
According to the principle of CLMS, the working process includes two stages. At the initial filtering stage, the filter with big step-size functions to make w eq (n) =w 1 (n) (λ(n) = 1), and the algorithm rapidly converges. At the stable filtering stage, the filter with small step-size functions to make w eq (n) = w 2 (n) (λ(n) = 0), reducing the steady-state error of the algorithm.
Although the performance could be improved by using two fixed step-size LMS filters in the convex structure, the CLMS algorithm does not achieve the ideal effect of the signal filtering process in the strong noise environment. In order to improve the defects, we put forward an improved algorithm named as HTVSCLMS algorithm to reduce fixed step-size CLMS algorithm's dependence on joint coefficient.

Algorithm Description
Yu [10] proposed the VSCLMS algorithm that is a variable step-size algorithm. It takes variable function based on sigmoid function as the step-size of fast convergence LMS filter. The variable function is expressed as where  and  are constants. e(n) is an equivalent error of convex filters. Then, VSCLMS algorithm can ensure fast convergence and obtains a small steady-state error.
Under the idea of variable step-size, the HTVSCLMS algorithm is proposed in this paper. In this algorithm, the variable function of hyperbolic tangent function is taken as the step-size of fast convergence LMS filter. The variable function of HTVSCLMS algorithm is expressed as In (6),  controls the span of step-size and  controls the changing shape of step-size, while m controls the smoothness of step-size round minimum zero.
The relation curve of (5) and (6) with the same parameters is shown in Fig. 2. It is easy to see that the changing rule of step-size function in HTVSCLMS is similar to that in VSCLMS. μ(n) would adjust itself according to the variation of e(n). At the initial filtering stage, the error is often big and a big step-size is required. The step-size can be set to the maximum convergence value of the algorithm. At the stable filtering stage, the error gradually decreases, and the step-size can be set to a smaller value gradually.

The proposed HTVSCLMS algorithm
For each iteration time n 1. Outputs of two LMS filters,

Errors of two LMS filters,
4. Equivalent error of CLMS, eq eq ( ) ( ) ( ) e n d n y n   , 5. Updating parameter ( ) a n , 2 eq ( ) ( 1) ( ) 2 ( ) a e n a n a n a n 7. The step-size of fast convergence LMS filter, 8. The weight vectors Tab. 1. The proposed HTVSCLMS algorithm.
Therefore, the steady-state error of the algorithm can be maintained at a smaller value. μ(n) would have a large shock to affect the stability of the filtering system because the MSE of variable step-size algorithm is easy to be polluted. Thus, the other filter of convex combination structure needs to be set as a small step-size μ 2 to conquer the deficiency of a variable step-size algorithm.
From the above, the proposed HTVSCLMS algorithm is summarized in Tab. 1.

Analysis of Computational Complexity
Except of convergence speed and steady-state performance, computational complexity is also the key factor influencing the application of an algorithm. In this paper, the computational complexity of the HTVSCLMS algorithm is compared with that of CLMS and VSCLMS. Exponent calculation is equal to one time multiplication operation [11] in step 6 and 7 of Tab. 1, so HTVSCLMS algorithm requires two time addition, m time multiplication and one time division when updating step-size for each iteration time. Besides, it requires one time addition, one time multiplication, and one time division to update the joint coefficient. HTVSCLMS algorithm has the same computational process except for updating step-size compared with VSCLMS, which requires one time addition and two time multiplication. For the CLMS algorithm, it is not required to update step-size. We assume that the length of each filter is N in convex combination. According to the operation steps of different algorithms, the computational complexity of one iteration time is presented in Tab. 2. Compared with CLMS and VSCLMS, the computational complexity of the HTVSCLMS algorithm slightly increases. The extra computing time brought by the increase in computation can be eliminated by using parallel computing [12], resulting in improving the convergence speed. Besides, m must be small.

Steady-state Performance
In this subsection, we analyze the performance of the HTVSCLMS algorithm. We make the following assumptions and notations [6]: where ŵ is the weight vector of an unknown system with length N and e 0 (n) is independent modeling of observed noise with mean zero and variance σ 0 2 .
For the convenience of discussion, the component filters and their combination are defined as follows:  Weight error vectors:  A-priori errors:  A-posteriori errors: Generally, the excess mean-square error (EMSE) is used to measure the performance of filter, which is described as the excess error over the minimum meansquare error (σ 0 2 ) of filter [3]. Iteration time n goes to  when analyzing steady-state performance. It should be noted that e eq (n) and e i (n) have the relationship as In order to analyze the performance for a convex combination, the cross-EMSE of the two filters is defined as According to (13) and Cauchy-Schwartz inequality, there is a conclusion that the cross-EMSE is never higher than the EMSE of the component filters, as shown in (14)  .
Besides, using the energy conservation equation [13] and assuming that u(n) 2 is independent of e a,i (n) of individual filter at the steady-state stage, the EMSEs of the component filters are formulated as where there is a convergence condition that μ i < 2/Tr(R), and μ 1 actually stands for step-size function μ 1 (n).
In order to derive a formula about the cross-EMSE, an equation [13] relating weight error vectors, a-priori errors and a-posteriori errors introduced has been introduced, as illustrated in (16) , , Notice that there is a fact in steady-state ( n   ) Then, we get the following relation as  Similarly, in view of the independent of u(n) 2 with e a,i (n), the cross-EMSE is finally obtained as 2 12 0 ex, 12 12 Tr( ) There is a fact that μ 0  μ 12  μ 2 for HTVSCLMS algorithm as well as VSCLMS; therefore, it can be concluded that J ex,1 ()  J ex,12 ()  J ex,2 () in view of (15) and (24). But for CLMS, there is a conclusion that J' ex,1 ()  J' ex,12 ()  J' ex,2 () [5]. Because the proposed algorithm doesn't change the iteration rule of small stepsize LMS filter, so J ex,2 () = J' ex,2 (). Thus, we conclude that J ex,12 ()  J' ex,12 () , that is, HTVSCLMS has better performance than CLMS. Besides, HTVSCLMS with some optimum parameters would also have a better performance than VSCLMS.

Simulation Results and Discussion
In this section, we verify the performance of the proposed HTVSCLMS algorithm for system identification through the Monte Carlo simulation. The tracking and steady-state performances of HTVSCLMS are compared with that of CLMS and VSCLMS in 200 Monte Carlo simulations, and we make an average. In the following experiments, the unknown system and the component filters have the same length of N = 8. The parameter of unknown system is set to ŵ T =[0.8783, -0.5806, 0.6537, -0.3223, 0.6577, -0.0582, 0.2895, -0.2710]. When the iteration time reaches to 1000, ŵ is suddenly changed to ŵ T = [0.6537, -0.3223, 0.6577, -0.0582, 0.2895, -0.2710, 0.1278, -0.1508] to study the ability to react to changes. The input signal u(n) is zero-mean white Gaussian noise with variance σ u 2 = 1, building input vector u(n) as u T (n) = [u(n), u(n -1),…, u(n - 7)]. According to (7), e 0 (n) is zero-mean white Gaussian noise as well with variance σ 0 2 , independent of u(n). We consider two cases that σ 0 2 = 0.01 and σ 0 2 = 0.1, namely, SNR = 20 dB and 10 dB.

Simulation Analysis of HTVSCLMS with Different Parameters
In this section, we discuss how the parameters in (6) influence the MSE performance of HTVSCLMS. SNR is set to 20 dB and ŵ is assumed to be stable until a sudden change occurs after 1000 iteration time. μ 2 is set to 0.01 referring to [10].
The MSE performance of HTVSCLMS with  = 0.8 and m = 1 while  = 0.005, 0.04, 0.08, 0.12 is illustrated in Fig. 3. It is clear to see that the convergence speed of HTVSCLMS increases as  rises from 0.005 to 0.08; meanwhile, the steady-state error has some reduction. In fact, we find that the performance remains the same when  varies between 0.08 and 0.16 in the simulation; it cannot converge when  is bigger than 0.16. Therefore, the optimal  is around 0.08 in the simulation; 0.08 is selected.
In Fig. 4, the MSE performance of HTVSCLMS with  = 0.08 and m = 1 while  = 0.05, 0.4, 0.8, 1.2 is displayed. It can be seen that the convergence speed of HTVSCLMS increases when  rises from 0.05 to 0.8; meanwhile, the steady-state error has some reduction.
When  is bigger than 0.8, the convergence speed and the   steady-state error have no obvious variation. Thus, the optimal  is around 0.8 in the simulation; 0.8 is selected.
The MSE performance of HTVSCLMS with  = 0.08 and  = 0.8 while m = 8, 4, 2, 1 is shown in Fig. 5. It can be found that the algorithm has the best performance with the fastest convergence speed and smallest steady-state error when m is 1, verifying the computational complexity discussion in Sec. 3.2. Therefore, the optimal m is 1 in the simulation; 1 is selected.

Simulation Analysis of Different Algorithms in a Stable Environment
In a stable environment, ŵ would not change. Referring to parameters in [5] and [10] and discussion in Sec. 4.1 and making some optimization in our simulations, relative optimal parameters are shown in Tab. 3.
The MSE performance of different algorithms with two SNRs in a stable environment is illustrated in Fig. 6. It is clear to see that HTVSCLMS and VSCLMS have faster convergence speed compared to CLMS in two SNRs situations. This is because that CLMS cannot update the stepsize on the basis of error of convex combination filters and has a weak tracking updating ability. In the steady-state stage, HTVSCLMS inherits the good ability of small steady-state error of VSCLMS, better than that of CLMS. When the unknown system is suddenly changed after 1000 iteration, all algorithms can quickly go to steady-state again and HTVSCLMS still has the best performance. Besides, in comparison with the case of SNR = 20 dB and SNR = 10 dB, the latter case has faster convergence speed, while it has a bigger steady-state error.

Simulation Analysis of Different Algorithms in a Non-stable Environment
In a non-stable environment, ŵ would change when iterations advance. The model of a time-varying system is given as where c(n) is zero-mean white Gaussian noise with variance σ c 2 = 0.0001. Referring to parameters in a stable environment and making some optimization, relative optimal parameters are shown in Tab. 4. In fact, only the parameters of CLMS have little change, verifying that variable step-size algorithms have better adaptability in a non-stable environment.
The MSE performance of different algorithms with two SNRs in non-stable environment is shown in Fig. 7. It can be found that HTVSCLMS has better convergence speed than that of the other two algorithms in non-stable environment as well. To a certain extent, CLMS has a slower convergence process because it adopts a limitation on the function boundary of the joint coefficient, leading to shaking in the asymptotic steady-state process. In the steady-state stage, HTVSCLMS has obviously smaller steady-state error compared with VSCLMS and CLMS.   (a) SNR = 20 dB (b) SNR = 10 dB When the unknown system is suddenly changed, HTVSCLMS still has the best performance. Compared with Fig. 6, the steady-state errors are relatively bigger with the same case in a non-stable environment while it has no effect on the good ability of convex combination scheme.
All in all, HTVSCLMS gets the smaller steady-state error and the faster convergence speed than those of VSCLMS and CLMS at the same iteration. Therefore, HTVSCLMS has excellent tracking and steady-state performance, indicating its effectiveness.

Conclusion
In this paper, a new variable step-size CLMS algorithm named HTVSCLMS has been proposed; its hyper-bolic tangent function is the step-size function of fast convergence LMS filter in the convex combination scheme. The proposed algorithm has constructed hyperbolic tangent nonlinear relationship between step-size and steady-state error based on a variable step-size rule. It has obtained better convergence speed and steady-state performance by taking such advantages of CLMS filters itself. Besides, the MSE performance of different algorithms is compared in the stable and non-stable environment through simulation respectively. Results show that HTVSCLMS has better performance compared with VSCLMS and CLMS.