Improving Stability and Convergence for Adaptive Radial Basis Function Neural Networks Algorithm. (On-Line Harmonics Estimation Application)

In this paper, an adaptive Radial Basis Function Neural Networks (RBFNN) algorithm is used to estimate the fundamental and harmonic components of nonlinear load current. The performance of the adaptive RBFNN is evaluated based on the difference between the original signal and the constructed signal (the summation between fundamental and harmonic components). Also, an extensive investigation is carried out to propose a systematic and optimal selection of the Adaptive RBFNN parameters. These parameters will ensure fast and stable convergence and minimum estimation error. The results show an improving for fundamental and harmonics estimation comparing to the conventional RBFNN. Also, the results show how to control the computational steps and how they are related to the estimation error. The methodology used in this paper facilitates the development and design of signal processing and control systems. Keywords: Energy efficiency, Power quality, Radial basis function, neural networks, adaptive, harmonic.Article History: Received Dec 15, 2016; Received in revised form Feb 2nd 2017; Accepted 13rd 2017; Available onlineHow to Cite This Article: Almaita, E.K and Shawawreh J.Al (2017) Improving Stability and Convergence for Adaptive Radial Basis Function Neural Networks Algorithm (On-Line Harmonics Estimation Application). International Journal of Renewable Energy Develeopment, 6(1), 9-17.http://dx.doi.org/10.14710/ijred.6.1.9-17


Introduction
Recently, power quality has become a major concern for modern power systems. One of the main power quality issues is the harmonic distortion in power systems. The excessive presence in harmonics in power system can cause many problems such as: malfunctioning of circuit breakers and relays, overheating of conductors and motors, insulation degradation, and communication interference (Eyad Almaita, 2016;Izhar, Hadzer, Masri, & Idris, 2003;Rahmani, Hamadi, & Al-Haddad, 2009;Sumaryadi, Gumilang, & Susilo, 2009). Different approaches have been used for harmonics mitigation in power systems. Active Power filters (APFs) are considered an effective means to reduce harmonics to acceptable levels. Any APF is consist of two main parts: (i) Control part that measures the distorted signal (voltage or current) then decomposed the measured signal into fundamental component and other harmonic components based on on-line harmonic estimation algorithm. (ii) Power part consists of power electronics * Corresponding author: eyad.k.almaita@gmail.com module to compensate for all harmonic distortions (Akagi, Watanabe, & Aredes, 2007). On-line harmonic estimation algorithms have been extensively studied. They can be classified into three main classes; (i) time domain algorithms, (ii) frequency domain algorithms, and (iii) artificial intelligent algorithms (Akagi, 1996;Rahmani et al., 2009;Wang, Wong, & Member, 2015;Yasmeena & Das, 2016;Zhang, Li, & Wang, 2010). Time domain algorithms always have to compromise between the phase delay and the attenuation, also, oscillations is always can be caused by fast transition time (Zhang et al., 2010). On the hand, frequency domain algorithms need high computations and considered not real-time filters (Zhang et al., 2010). The artificial intelligent (neural networks) algorithms have been successfully tested to overcome the disadvantages of the time and frequency domain filters. The three main techniques used in neural networks algorithms are (i) adaptive linear neuron (ADALINE), (ii) back propagation (BP), and (iii) radial basis function neural networks (RBFNN). The ADALINE is used as online harmonics identifier and its performance depends on the number of harmonics included in its structure. The convergence of the ADALINE slows as the number of harmonics included increases because the ADALINE can only approximate linear functions (Chang, Chen, & Teng, 2010;Zouidi, Fnaiech, AL-Haddad, & Rahmani, 2008). On the other hand, The BPNN is more capable and can approximate linear and nonlinear function. It uses offline supervised training to identify selected harmonics. The long training time required in BPNN and the chance of falling into local minima is always present (Haykin, 1999;Kasabov, 1996). The RBFNN has several advantages over ADALINE and BPNN; capable of approximating highly nonlinear functions, its structural nature facilitate the training process because the training can be done in a sequential manner, and the use of local approximation can give better generalization capabilities (Haykin, 1999;Kasabov, 1996). Several schemes for RBFNN have been used; some of these schemes have large number of hidden neurons and still uses training algorithm similar to that of BPNN (Chang et al., 2010). Other schemes gets better results and with small number of hidden neurons using direct inversion matrix as training algorithm (E Almaita & Asumadu, 2011a, 2011b). An adaptive version of RBFNN has been introduced in ( Almaita, 2012). In this adaptive version the parameters of RBFNN can be modified after the deployment without the need of retraining the RBFNN. This gives an additional strength to the RBFNN. The main problem of this adaptive version is the learning rate, which needs to be selected very carefully to ensure the convergence and stability of the system.
In this paper, which is an extension to the works in (Almaita, 2012;Almaita & Asumadu, 2011a, 2011b, a systematic approach is introduced to select the parameters of the adaptive RBFNN algorithm. This approach will ensure the stability and will also minimize the estimation error. Also, the adaptive RBFNN will be used to estimate the fundamental and harmonic contents in current signal of non-linear load.

Structure of RBFNN
The RBFNN structure consists of three main different layers as shown in Fig. 1; one input layer (source nodes with inputs I1, I2,.., IN), one hidden layer has K neurons, and one output layer (with outputs y1, y2,.., ym). The input-output mapping consists of two different transformations; nonlinear transformation from the input layer to the hidden layer and linear transformation from hidden to the output layer. The connections between the input and hidden layers are called centers and the connections between the hidden and output layers are called weights (Almaita & Asumadu, 2011a, 2011b.
The most common radial basis function used in RBFNN is given by i=1,2,…,K (1) This is a Gaussian basis function with φi as the output of the i th hidden neoron , x is the input vector data sample (I1, I2,…,IN) (could be training, actual, or test data), ci is centers vector of the i th hidden neuron (ci1,ci2,.., ciN), σi is the normalization factor, and (x-ci) T (x-ci) is the square of the vector (x-ci) (Almaita & Asumadu, 2011a, 2011b. The i th output node yi is a linear weighted summation of the outputs of the hidden layer and is given by where wi is the weight vector of the output node and Ф(x) is the vector of the outputs from the hidden layer (augmented with an additional bias which assumes a value of 1).

Training Algorithm of RBFNN
The block diagram shown in Fig. 2 illustrates one of the RBFNN training processes called hybrid learning process (Moody & Darken, 1989;Yousef & Hindi, 2005). The hybrid learning process has two different stages; (i) finding suitable locations for the radial basis functions centers of the hidden neurons (Moody & Darken, 1989;Yousef & Hindi, 2005) and (ii) finding the weights between the hidden and output layers. In the first stage the K-means (Moody & Darken, 1989;Yousef & Hindi, 2005) clustering algorithm is used to locate the centers in the input data space regions where a significant data are present (shown as I in Fig. 2). In the second stage (shown as II in Fig. 2) the weights between the hidden and the output layers are found by linear matrix inversion algorithm based on the least-square solution, which minimizes the sum-squared error function (Yousef & Hindi, 2005).  The weights matrix w is given by (3) where D is the desired output vector for l training data samples set and given by (4) where d(xj) describes the output vector corresponding to the j th training data samples vector (xj).  is a matrix where each element φi(xj), is a scalar value and represents the output of the i th hidden neuron for the j th training data samples vector (xj).
The  matrix for l training data samples is given by (5) A -1 , the variance matrix and given by One of the advantages of this method compare to other training algorithms is that it does not need iterations in the training phase; what it needs is the matrix inversion shown in Eq (6), which needs negligible time to be calculated.

Adaptive RBFNN
One of the major disadvantages of the feed forward neural networks (BPNN and conventional RBFNN) techniques is that; the obtained parameters do not changed once the training process is completed. In the presence of the noise, these fixed parameters can degrade the performance of the neural networks. The main objective of the adaptive RBFNN algorithm is to enhance the reliability of the conventional RBFNN after embedding the network in the system. This can be achieved by introducing an adaptive algorithm for RBFNN structure that allows the change of the weights of RBFNN after the training process is completed. As shown in II, the RBFNN adjustable parameters that will affect the output are the centers and the weights. This algorithm assumes that the noise present in the system can be mitigated only by adjusting the weights between the hidden and the output layers, without the need of adjusting the values of the centers between the input and hidden layers. Figure 3 shows the general structure of the adaptive RBFNN algorithm. It has the same conventional RBFNN structure regarding input layer, hidden layer, and output layer. But it has two extra components; (i) Summation component, which is located after the outputs of the RBFNN. The goal of this component is to calculate the error signal between the estimated outputs y and the reference (actual) signal (R). (ii) Weights updating component. The goal of this component is to adjust the weights in order to reduce the error signal. In the absence of the noise δ(k) in the input side, the summation of the outputs of the RBFNN model is equal to the reference signal R(k). In this case the error E(k) equal to zero and no change in the RBFNN weights.
In the presence of noise in the input side, the jth output node of the RBFNN will be affected by this noise as where ( ) is the jth output node without noise and ( ) is the added noise error to the jth output node.  In this case the error E(k) is not equal to zero.
In order to mitigate the effect of the noise in the performance of the RBFNN, the error E(k) is used to update the weights vectors based on the least-meansquare-error algorithm (Moody & Darken, 1989) as: where µj is the regulation factor for the j th output node.
The weights updating will continue until the error E(k) become zero again. The above algorithm has several advantages including the following:  It has a fast convergence time because it adjusts only the weights between the hidden and output layers, which is a linear relationship. Therefore, fast convergence can be achieved.  The updating process could be initiated based on threshold value for E(k) (different from zero), which gives the flexibility to the algorithm and saves excessive computations. This algorithm has greater capabilities compare to the popular neural linear adaptive algorithm (ADALINE) because the RBFNN structure can be used to realize linear and nonlinear functions. Figure 4 shows the inputs and outputs signals for the adaptive RBFNN used in this paper. The input signal is sampled at constant rate and passed through firstinput-first-output (FIFO) buffer to create a delayed vector with length 64, represent a half cycle of the signal, which match the length of the input vector of RBFNN. The adaptive RBFNN used in this paper has two outputs; one of them is to estimate the fundamental component (yf) and the other is to estimate the harmonic components (yh), these outputs are calculated as follows:

Methodology
where Wf, Wh, is the weight vector of the fundamental and harmonic components output , respectively and Ф(x) is the vector of the outputs from the hidden layer. The value of  can be express in terms of max as follows: So to achieve the system stability η should be in the range: Adapting Wh and Wf can improve RBFNN performance, if the values of f and h are selected carefully, where some values of them can cause the system to diverge, so defining the stable margin of them is the first goal of this paper, after that it is desired to choose the optimal values of f and h to minimize the error between the measured and estimated outputs.
In this paper, the values of f and h will be changed to investigate the effect of their values on the system performance based on the mean square error MSE value. First the fundamental weights will be only adapted to find the stable range of h and to define also the optimal   value of f that give the minimum MSE, Then adapting the harmonics weights only to find the stable and optimal range of h. Then both of fundamental and harmonic weights will be update and comparing the range of stability and optimality of f and h in this case with the previous values. The effect of the noise is also investigated in this paper by defining the stable range and the optimal value of f and h in terms of signal to noise ratio SNR.

Results and Discussion
At first the input current for a three-phase controlled rectifier shown in Fig.5 is measured and sampled using digital oscilloscope. The measured signal represents the input current for different firing angle to ensure the robustness of the system and to cover all the range. RBFN now will be used to estimate the fundamental and the harmonic components of the measured signal. First the values of ηf and ηh is examined carefully to ensure system stability, then to determine the optimal values of ηf and ηh to minimize the Mean Square Error (MSE) between the measured and the estimated signals, after that the effect of the noise on the optimal values of η is examined. Fig. 6 shows the actual signal and the estimated signal form conventional RBFNN. It is clear from Fig.6 that the error between the actual and estimated signal is significant and cannot be tolerated for on-line harmonic estimation, where the MSE equals 2.2224. By adopting adaptive RBFNN this error can be minimized without the need to retrain the RBFNN.

Minimizing Error Using Regulation Factors
First, the weight vector of the fundamental component Wf is only updated by using eq.14, while the weight vector of the harmonic component is remained unchanged (ηh=0), updating Wf depend on the value ηf, the value of ηf is changed over the whole stability range for η (from 0 to 2), and the Mean Square Error (MSE= 1 ∑ ( ( ) 2

=1
) is calculated for each value of ηf, as shown in Fig,7. The value of MSE is decreased as ηf increased until ηf equals 0.9779 (approximately in the mid of stability range). As ηf value increases further, the MSE increases. This pattern continues till the value of ηf approaches the stability range limit (ηf ≈ 2) the MSE is increased rapidly and this causes the system to diverge. This result is a perfect match with theoretical stability range of ηf . Fig 8  shows the actual signal and the estimated signal from adaptive RBFNN for ηf equals to 0.9778 (optimal value). The MSE for this ηf value is minima and equals to 0.276978  Secondly, the weight vector of the harmonic component Wh is only updated, while the weight vector of fundamental component is remained unchanged (ηf=0), updating Wh depend on the value ηh. ηh value also is changed over the whole stability range for η (from 0 to 2), and the mean square error (MSE) is calculated for each value of ηh, as shown in Figure 4. Again, the minimum value of MSE is 0.276978 and it is obtained when ηh value is approximately in the middle of stability range (ηh =0.9779). As ηh value approaches the stability range limit (ηh ≈ 2) the MSE is increased rapidly and this causes the system to diverge. These are the same results obtained when ηf is only changed. Thirdly, both of fundamental and harmonic weight vectors are adapted. Fig.10 shows the variation in MSE as ηh is changed for different values of ηf. It can be noticed that the minimum value of MSE is still the same value that is obtained from changing only ηh or ηf alone (i.e. 0.276978), but this time it occurs with several combinations of ηf and ηh values. The stability region for changing both ηf and ηh is shown in Fig.11. The two solid lines represent the range of the stability, which means that any values of ηf and ηh in the area between the two solid lines maintain the stability of the system, otherwise any combination out of this area will cause system divergence. The equations of the two solid lines are: The combinations of ηf and ηh that give the minimum MSE are shown also in Fig.6 and they are represented by the dashed line. It is clear that the relation between them is linear and can be given by: which means that any combination of ηf and ηh satisfies this equation will produce an estimated signal with minimum MSE (MSE= 0.276978), which is the minimum MSE can be achieved of this system.   Fig.13 shows the difference of the FFT component between the measured signal and the estimated signal using the conventional filter (red bar) and the difference between the FFT components of the measured signal and the estimated signal using the adaptive filter (blue bar). This figure prove that adapting technique improves system capability of detecting all the harmonics component, so the improvement of MSE value does not come from improving one harmonic component, but it comes from improvement of all the harmonics components. Figure 13. The difference between the FFT components between measured signal and conventional filter (red bar) and between measured signal and the adapting filter ( blue bar)

Effect of the noise
The effect of the noise in the performance of the adaptive RBFNN is investigated in this section. An additive white Gaussian noise is added to the original signal. The performance of the RBFNN is investigated under three different cases: (i) update only the weight vector of the harmonic component using optimal value for ηh (ηh=0.9778, ηf=0), (ii) update only the weight vector of the fundamental component using optimal value for ηf (ηf=0.9778, ηh=0), and (iii) update both the weight vectors of the harmonic component and fundamental component using optimal combination for ηh and ηf (ηh+ ηf =0.9778). The parameter used in this paper to measure the noise level in this paper is Signal to Noise Ratio (SNR), where the SNR in dB is given by: Where Psignal and Pnoise are the signal and noise power respectively. In this paper the SNR level will be changed from 20 db to 100 db. This level is above the common SNR level in electrical power signals that had been adopted in the previous literature, which was above 30 (Đurić & Đurišić, 2010;Liu, Wang, Liu, & Cui, 2016). Figure 14 shows the MSE for several values of signal to noise ratio SNR, the value of h is zero and f is 0.9778 where these values give a minimum MSE without noise, when the SNR of the measured signal is above 20, the adaptation values still give minimum error, the same curve of Fig.14 is also obtained when the values of are chosen to lie on the line h +f =0.9778, which shows that the noise with a SNR greater than or equal to 30dB, didn't affect the performance of the filter. In Fig.15 the values of h and f are changed, then the value of MSE is calculated for a signal has a SNR equals to 30dB, the minimum value of each curve is denoted by '*', if we examine the "*' points, it is clear that the relation between them is linear since they lie on the same line and

Effect of threshold value
Usually the adapting is required when the error exceeds a certain limit to avoid extra calculations, while in previous sections the threshold value of the error (i.e e(k)=abs(yest(k)-ymeasured(k))) was zero, which means that at each k the value of either h,f or both of them are modified. In this sectionf are adapted only when e(k) exceeds a threshold value, the threshold value is changed from 0 to 3, and for each one of them f is changed, then the MSE is calculated, the results of minimum MSE are shown in Fig. 16 and Fig.17. In Figure  16 x-axis is the threshold value and Y-axis is the minimum MSE value, this value is obtained by changing f, while Fig.17 shows the value of f that gives the minimum MSE for each threshold value. The result in those figures shows that the minimum MSE can be achieved when the threshold value is zero and f = 0.9778. Even though the MSE is decreased when the threshold value is decreased, but decreasing the threshold value means extra calculations (i.e extra calculation due to adaptation process) as it is shown in Fig. 18, where it shows that the number of calculations is decreased as the threshold value is increased.

Conclusions
In this paper, the stability and performance enhancement of adaptive RBFNN is investigated. Adapting the weight vectors of the fundamental and harmonic components based on LMS algorithm improve RBFNN capability to estimate output signal from a power electronic circuit, which is totally different from the signal that the RBFNN is trained for, the stability range of the adapting parameters ηf and ηh are investigated and by carefully selecting these values, the MSE between the estimated and the measured signal is minimized. The adapting RBFNN shows the capability for disturbance rejection even when the signal has 30 to 100 dB SNR of Gaussian noise. Further investigating for the adaptive RBFNN is carried out by exploring the effect of the adapting trigger threshold to prevent excessive calculations.