Steady State Analysis of Convex Combination of Affine Projection Adaptive Filters

The aim of the study is to propose an adaptive algorithm using convex combinational approach to have both fast convergence and less steady state error simultaneously. For this purpose, we have used two affine projection adaptive filters with complementary nature (both in step size and projection order) as the component filters. The first component filter has high projection order and large step size which makes it to have fast convergence at the cost of more steady state error. The second component filter has slow convergence and less steady state error due to the selection of small step size and projection order. Both are combined using convex combiner so as to have best final output with fast convergence and less steady state error. Each of the component filters are updated using their own error signals and stochastic gradient approach is used to update the convex combiner so as to have minimum overall error. By using energy conservation argument, analytical treatment of the combination stage is made in stationary environment. It is found that during initial stage the proposed scheme converges to the fast filter which has good convergence later it converges to either of the two (whichever has less steady state error) and towards the end, the final output converges to slow filter which is superior in lesser steady state error. Experimental results proved that the proposed algorithm has adopted the best features of the component filters.


INTRODUCTION
Adaptive filters were used for various applications ranging from noise cancellation, system identification, echo cancellation, time delay estimation, model order selection etc.The desirable characteristic of an adaptive filter includes high convergence, small steady state error and low computational complexity.Some of the popular adaptive filters were Least Mean Square (LMS), Normalized Least Mean Square (NLMS), Affine Projection Algorithm (APA), Recursive Least Square (RLS) algorithm (Haykin, 2002).However each has their own limitations.The one with small steady state error has small convergence (RLS).If the convergence is good, then the steady state error is more (LMS).If both are satisfied then the computational complexity is more (APA).If all the above are met, then they have very poor tracking capability especially with colored or speech input (NLMS) (Haykin, 2002;Radhika and Arumugam, 2012).Several variants with improved performance includes variable step size (Paleologu et al., 2008) sparse (Naylor et al., 2006), proportionate (Haiquan et al., 2014;Zhao and Yu, 2014) frequency domain (He et al., 2014) adaptive filters.But none satisfies all the requirement of an adaptive filter.Therefore combinational approach is found to have a promising solution for an overall improvement in adaptive filter performance (Zhang and Chambers, 2006;Martinez-Ramon et al., 2002).In a combinational approach, more than one adaptive filter is operated in parallel in two or more stages.The output of one stage acts as the input for the second stage which is done efficiently by a suitable combiner.One such combiner is the convex combiner.Convex combination is used to combine different adaptive filters for an improved overall performance (Zhang and Chambers, 2006;Martinez-Ramon et al., 2002;Das and Chakraborty, 2014;Choi et al., 2014;Arenas-Garcia et al., 2005a;Kozat et al., 2010).Convex combination of slow and fast LMS adaptive filter is discussed in the context of system identification with variation in tap length (Zhang and Chambers, 2006).The simulation results were performed for low SNR and it is proved that the combination approach has improved performance even in low signal to noise ratio conditions.A similar analysis for plant identification is made using LMS adaptive filter (Martinez-Ramon et al., 2002).Convex combination of zero attracting LMS and conventional LMS is discussed for system identification application  (Das and Chakraborty, 2014).From the results, it is proved that the combinational approach adapts dynamically to the level of sparseness with less complexity than any other algorithm.In Azpicueta-Ruiz et al. (2011) kernel LMS and kernel RLS were combined to obtain the best performance applicable for echo cancellation application.
A general theoretical analysis of convex combination two adaptive filters is discussed in detail in Arenas-Garcia et al. (2006) and Martinez-Ramon et al. (2002).Steady state analysis of convex combination of two LMS is considered in Arenas-Garcia et al. (2005b).The convex combiner is represented as a sigmoid function so that the convex combiner never exceeds one.Energy conservation argument is used to obtain the steady state mean square error.From the results it is found that the steady state error is at least similar to one of the filter and even lesser than both under certain conditions.The extension of Arenas-Garcia et al. (2005a) is carried out in Arenas-Garcia et al. (2006) where tracking in stationary and non stationary environment is discussed in detail.Two LMS with variable step size is also discussed.In Arenas-Garcia et al. (2005b) a general rule for convex combination of multiple LMS adaptive filter is derived.Softmax activation function is used for the mixing of multiple LMS filters.Two variants of LMS which are M-LMS and D-LMS were introduced to obtain improved tracking and speed than the CLMS approach (Arenas-Garcia et al., 2006).Normalized rule for the updation is proposed in Azpicueta-Ruiz et al. (2008).From the paper it is proved that normalization gives more stability and also simplified the selection of step size than the CLMS approach.
In Kim et al. (2008) convex combination of two APA with different regularization for each component filter is proposed.In Choi et al. (2014) convex combination of NLMS and APA is proposed.The optimum value for the convex combiner is derived based on mean square error deviation analysis.It is also found that the performance is much improved than the individual filter acting alone.Kwang-Hoon et al. (2014) discussed the convex combination of two APA with one having large projection order and the other with small projection order.The simulation results proved that they outperform the single long filter approach both in Mean Square Error (MSE) and in convergence rate.
Thus from the above literature survey it is found that combinational approach has improved overall performance than single filter approach.Also APA is faster and is more suitable for speech input (Haykin, 2002).Therefore we propose to combine two APA using convex combiner.A theoretical analysis based on energy conservation argument is made to prove that there is performance improvement.
Figure 1 specifies the system model.It consists of two stages.The first stage consists of two independent APA filters.Their input is x (n) and they produce their own error signal to produce the estimated output y 1 (n) and y 2 (n).The second stage consists of the combination stage where the convex combiner λ (n) is used to obtain the overall system output as y (n) given by: ˑ{ˌ{ = {ˌ{ˑ {ˌ{ + − {ˌ{ ˑ {ˌ{ (1) Similarly if w 1 (n) and w 2 (n) are the component weight vectors, then they can be represented as: Here {J{ is called convex combiner.A combiner is said to be convex if the values are non negative and are restricted to a maximum of one.As done in Arenas-Garcia et al. (2006) it is more convenient to represent {J{ in terms of the sigmoidal function: (3) The updation for the convex combiner is done using gradient descent algorithm as done in (Arenas-Garcia et al., 2006) given by: Here the error signal for the overall filter structure is given by e (n) = d (n) -y (n) where d (n) is the desired output.Substituting y (n) from ( 1) in e (n) we get the a (n) in recursive form as: Practically the range of a (n) is restricted to [-a + , +a + ] which in turn restricts {J{ to [1-λ + , λ + ].

MATERIALS AND METHODS
In this section we combine two APA with different step size and projection order using convex combiner.Let the first one has larger step size µ 1 and projection order L 1 and the second has smaller step size µ 2 and projection order L 2 .The selection is made such that L 2 <L 1 <M.where M is the order of the filter.
The desired output and the input are related using linear regression model is given in (6).˱ {J{ is the unknown weight vector of length M, v (n) is the independent and identically distributed noise which is independent of x (n) with variance σ $ .The following are the assumptions of APA based on the energy conservation argument (Shin and Sayed, 2004;Sayed, 2003): The weight error vector ( {J{{ is defined as the difference between the optimum weight and estimated weight vector.For the component filter: For the final output: The weight error vector in recursive form can be written as: The a priori error vector and the a posteriori error vector for the component and the combinational filter is defined as in Shin and Sayed (2004): The MSE and Excess Mean Square Error (EMSE) for the component and combinational filter is defined as: The cross excess Mean Square Error {EMSE #,$ { for the component filter is given by: Also from Cauchy Schwarz inequality ˗H˟˗ #,$ ≤ ˗H˟˗ # ˗H˟˗ $ which implies that the cross excess mean square error cannot be greater than the component filter EMSE.We also define the change in EMSE as: Steady state performance: The overall Steady state mean square error analysis in stationary environment is done in this section as in Arenas-Garcia et al. (2006).
Taking expectation on both sides of ( 5) and using the assumption that λ (n) is independent of a priori errors of the component filter we get: In order to analyze the steady state error performance of the combinational filter, we need to find (21) and ( 22) which in turn requires the evaluation of ˗H˟˗ #,$ .The EMSE equation for the component filter uses the assumption that the regularization is a small value (Shin and Sayed, 2004).The ˗H˟˗ # for the component filter with large value of step size # and projection order H # is given as: Similarly for the component filter with smaller value of step size $ and projection order H $ the excess mean square value ˗H˟˗ $ is given as: To obtain the cross EMSE #,$ we use energy conservation relation (Shin and Sayed, 2004) as follows.For the component filters the energy conservation relation is given by: Multiplying the complex conjugate of ( 26) for i = 1 with (26) for i = 2 and neglecting the dependency of {J{ on past noisesand if expectation is taken on both sides, we get: In order to simplify (36) we adopt the partitioning method.Let us partition A 2 (n) as [A 1 (n) ˓2{J{ ].After little mathematics we get: If the regularization parameter is small, then Since # is larger and $ is smaller step size we get S 1 = S 3 = I and S 2 = 11 T .Therefore (36) gets simplified as: where, #$ = $ .
Since # is larger and $ is smaller, L 1 is larger than unity, the step size #$ is obtained as # > #$ > $ .Therefore we can substitute S = I.Thus ˗H˟˗ #,$ becomes: From the above we find that the there is similarities between ( 25) and ( 38).So we can write ˗H˟˗ #,$ as: ˗H˟˗ # > ˗H˟˗ #,$ > ˗H˟˗ $ (39) which makes us to conclude that we are in the second case of discussion made in Arenas-Garcia et al. (2006).
The ∆˗H˟˗ # is positive and ∆˗H˟˗ $ is negative.This makes us to write ˗H˟˗ #,$ ≅ ˗H˟˗ $ .Thus we can say that the combinational affine projection filter has steady state error is more or less same as that of the component filter with step size $ and projection order L 2 .Thus we have obtained similar results of Arenas-Garcia et al. (2006) with an improved convergence and lower steady state error than obtained by combinational LMS.

RESULTS AND DISCUSSION
The simulation is performed in the context of acoustic echo cancellation.The input consists of colored noise which is obtained by passing Gaussian white noise through a system with impulse response H (z) = 1/ (1-0.7z-1.).The impulse response is obtained using image source method.The sampling frequency chosen is 8 kHz.The number of samples taken is 212.The signal to noise ratio is taken as 40 db.The noise is a white Gaussian noise with zero mean and unity variance.The length of the sequence is 10,000.The simulation is conducted with 50 independent trials.
In Fig. 2 the excess mean square error performance of the single and combination adaptive filter is plotted.The parameters set for this simulation are µ 1 = 0.8, µ 2 = 0.1, µ a = 100.The projection order for the filter are chosen to be L 1 = 10, L 2 = 1.The sigmoid function a (n) limiting value is chosen between -4 to +4.We can able to see that inititially the performance of C-APA matches with the faster convergence filter and later it matches with the smaller steady state error filter.As found from the theoretical results, we can see that the steady state error is same as that of the smaller step size filter µ 2 .
Figure 3 shows the variation of EMSE 1, 2 for different values of µ 1 and µ 2 .From the figure we can see that the EMSE 1, 2 increases as µ 1 is taken as larger value.Figure 4 shows the variation of the convex combiner throughout the experiment.As seen from the graph we find that initially the convex combiner is nearer to unity which makes the combiner to select component filter 1 which has more convergence.As the number of iterations reaches half, the convex combiner reaches nearer to zero which makes the combination filter to align to component filter 2 which has less steady state error.

CONCLUSION
We have seen that the combinational approach can improve the overall performance of the adaptive filter.One such combinational approach is presented in this study.The theoretical and the simulated results depicts that its performance is better than single filter acting alone.Our future approach will be to reduce the computational complexity by replacing the higher complex affine projection adaptive filter by a proportionate APA.

Fig. 2 :
Fig. 2: Steady state performance of combinational affine projection filter and single filter with colored noise as input and wth SNR = 40 db