Improved Fast Ica Algorithm Using Eighth-order Newton's Method

Independent Component Analysis (ICA) is a computational method to solve Blind Source Separation (BSS) problem. In this study, an improved Fast ICA based on eighth-order Newton's method is proposed to solve BSS problems. Eight-order Newton's method for finding the solution of nonlinear equations is much faster than ordinary Newton's iterative method. The improved FastICA algorithm is applied to separate sound signals. The simulation results show the method has fewer iterations and faster convergence speed.


INTRODUCTION
Independent Component Analysis (ICA) or Blind Source Separation (BSS) is a signal processing method that extracts statistically independent components from a set of measured signals (Hyvarinen, 1998;Lee et al., 1999;Ye et al., 2006). There are various ICA approaches including maximum likelihood estimation (Shia et al., 2004;Cardoso, 1998), mutual information minimization (Amari et al., 1996;Bell and Sejnowski, 1995;Pham, 2004) and negentropy maximization (Hevarinen et al., 2001). The FastICA algorithm was presented by Hyvarinen (1999). A kind of Newton-type algorithm was used in FastICA and the algorithm was based on fixed-point algorithm iteration to maximize nongaussianity as a measure of statistical independence. In FastICA algorithm, the Newton iteration method with two-order convergence was employed in a fixedpoint algorithm for estimating the separation matrix.
In this study, we first briefly introduce ICA algorithms with Kurtosis and negentropy contrast function. Then, an improved FastICA based on an eighth-order convergence Newton's Method (Li et al., 2011) is proposed. Using this new algorithm, the update equations inside the FastICA algorithm is developed for estimating separation matrix with less iterative. Then, the new algorithm is applied to separation sound signals. Finally, the performance of the newalgorithm is compared with FastICA algorithm in terms of evaluate performance rate and convergence rates.

INDEPENDENT COMPONENT ANALYSIS AND FASTICA
Independent Component Analysis (ICA) has become a powerful method for signal processing in last decade (Cichochi and Amari, 2004;Hevarinen et al., 2001;Chio et al., 2005). The basic linear mixture model (without noise) for ICA is as follows: where, S = (S 1 , … , S m ) T denotes the original independent sources and X = (X 1 , … , X n ) T denoted mixtures of original sources and A is the unknown m × n constant mixing matrix. ICA aims to recover or estimate the independent components (ICs) from their mixtures. An estimated IC is denoted by y and a separation vector is denoted byw, such that y = w T X = w T AS The signal mixtures tend to have Gaussian probability density functions (pdfs) and the source signals have nongaussianpdfs (Stone, 1999). In terms of central limit theorem (CLT), Gaussian signals often consist of a mixture of nongaussian signals (Stone, 1999). Given a set of such Gaussian mixtures signal X = ( 1 , … , ) . Therefore, the process of ICA is to find source signal by finding separation vector that extract signal , which has to be the most nongaussian. One of the classic measures for estimation of nongaussianity of random variable is Kurtosis. We denote the Kurtosis of y by ( ) and it is defined by: where, is a random variable with zero mean. If is normalized, variance of is equal to one i.e.
[ 2 ] = 1. Then, (3) is simplified to: Negentropy for Nongaussianity: An important measure of nongaussianity is negentropy. The concept of negentropy strongly depended on entropy that it is another fundamental concept in information theory. Given as a discrete value random variable, Entropy for random variable is defined by: In Eq. (5), the are the value of random variable and is density function of the random variable (Cover and Thomas, 1991;Papoulis, 1991).
The definition of entropy for continues random variable is obtained from generalizing (5). Given as a continue random variable with density function (.). The differential entropy for continues random variable is defined by: Let that as a vector random variable. Negentropy of is showed by ( ) and is defined by: where, is gaussian random variable with same covariance of the . Negentropy is another measure for nongaussianity. But, for the estimate of negentropy, we must first estimate the pdf of the vector random variable. Therefore, it is very hard to obtain it from the computation. One method to approximate negentropy by nonpolynomial function was proposed by Hyvarinen (1998). In related to this study, a method using expansion of pdf to approximate negentropy was given by Hevarinen et al. (2001). Let be a whitened random variable. Then, approximate of ( ) was given: where, G is a nonquadratic function and variable is a gaussian variable with unit variance and zero mean. Note that "proportion to" is the means of symbol ∝ . In order to achieve a very strongly estimator, choose G that grow slowly. Whereby: and 1 ≤ 1 ≤ 2 (Hevarinen et al., 2001).
A fixed-point algorithm using negentropy: A FastICA algorithm based on maximizing negentropy was introduced by Hyvarinen (1999). In this algorithm, fixed-point iteration and Newton's method with twoorder convergence is used to find separation vector as follows: where, is separation vector, is function as Eq. (9) and (10), is the whitening observation vector, = � � ��and corresponds to optimum . Equation (11) can be simplified by multiplying both sides by − � ‫׳‬ ( )�and: Let is a vector random variable. Assume that is given by the whitening of . the fixed-point algorithm using negentropy is given as follow (Hevarinen et al., 2001): Step 1 : Let = 0 and (0) is an initial random vector with unit norm Step2 : Let Step 3 : Step 4 : If the algorithm is not converged, then = + 1 and go back to step 2.
From this algorithm, we obtain one component. Therefore, in order to estimate several independent components, the algorithm needs to be run for several times.

EIGHT-ORDER NEWTON'S METHOD
The classic Newton's method for single nonlinear equation can be expressed as follows: This is an important and basic method, which converge quadratically.Pota and Petak (Potra and Ptak, 1984) proposed a modification Newton's method with third-order convergence defined by: King (1973) developed a one-parameter family of forth-order methods, which is written as follows: Such that ∈ R . Many methods have been proposed to improve the order of convergence such as (Kou, 2007;Chun, 2007;Parhi and Gupta, 2008;Bi et al., 2008). Li et al. (2011) proposed an eighth-order Newton's method convergence defined by: It is observed that this method required less number of iterations then traditional Newton's method:

SIMULATION RESULTS
We used three sound signals from MATLAB database as shown in Fig. 1. Each size of simple signals is 1000. These source signals were randomly mixed. The mixed signals are shown in Fig. 2. After whitening the mixed signals, we ran the proposed improved FastICA algorithm to separate the mixed signals .The results are shown in Fig. 3.   We can also get the same result when using the Fast ICA algorithm (Hevarinen et al., 2001). In order to measure the accuracy of separation, we calculated the performance index (Amari et al., 1996;Shi and Zhang, 2006).
where is a separation matrix, is a whitening matrix and isa mixing matrix. The large value is the poor statistical performance of the BSS algorithm (Amari et al., 1996;Shi and Zhang, 2006). These algorithms were run six times to evaluate their performances. The results are presented in Table 1. Whereby the separation performance of proposed algorithm is almost the same as the FastICA algorithm. Furthermore, we compared the convergence speed of these two algorithms. We randomly ran each of these two methods for 15 times executively. We then calculated the average iterative numbers and presented in Table 2.
As a result, the improved Fast ICA algorithm performances with less iteration than Fast ICA, while maintaining comparable separation performance. This is due to the fact that convergence of eighth-order Newton's method is much faster than the traditional Newton's method with second-order convergence to find the solution for non-linear equation. To calculate the CPU time, two algorithms 15 times was run by using the MATLAB version R2010 on the Intel(R) Core(TM) 2 DUE CPU T6400 2.00 GHZ processor with 2.00 GB RAM. Table 2 describes iteration numbers and the CPU times required for sound signal separation from three mixed sound signal, whereby the average CPU times are 0.2412 s and 0.1381s, respectively. This demonstrated that the proposed method exhibited faster convergence with less iteration than Fast ICA algorithm, with comparable separation performance.

CONCLUSION
In this study, improved Fast ICA using eight-order Newton's method was proposed for BSS. The separation performance of the proposed algorithm was almost the same as the Fast ICA algorithm. Besides that, the mixed sound signal was separated by new algorithm with fewer iteration and faster convergence than the Fast ICA.

ACKNOWLEDGMENT
The researchers would like to thanks from School of post graduate Studies (SPS) of Universiti Teknologi Malaysia (UTM) for getting Fellowship (IDF).