A Novel Optimized Adaptive Learning Approach of RBF on Biomedical Data Sets

,


INTRODUCTION
RBF was proposed in the year 1988 by Broomhead and Lowe.It has been used in many applications from the inception such as speech recognition, function approximation, time series prediction etc.It has been extensively used for solving pattern recognition problems (Antonios and George, 2012;Xiao-Yuan et al., 2007) and also used as universal approximators in a significant number of applications (Er et al., 2005;Min and Jianhui, 2004).The RBFNN is an artificial network uses radial basis function as activation functions and it is a type of feed forward Neural Network (NN), has recently attracted extensive research interest because of its simple architecture, high approximation and regularization capability and good local specialization and global generalization ability.The basic architecture for a RBF is a three layer network; input, hidden and output layers.The input layer is simply a fan-out layer; pass the input vector to hidden layer.The hidden layer performs a non-linear mapping from the input space into a higher dimensional space in which the patterns become linearly separable.The most commonly used Gaussian function is the activation function of hidden nodes given in Eq. (1) (Fig. 1).
The output layer performs a simple weighted sum with a linear output: where, is the distance between input space X and centre C j of j th hidden node.The variable sigma,  j , defines the width or radius of RBF node.The final layer of Radial Basis network calculates weighted sum with a linear output of hidden layer.It is calculated as: where, Y k (X) = The output of the k th neuron W jk = The weight from the j th neuron of the hidden layer to the k th neuron of the output layer RBFNN structure initialization and parameter optimization.In the stage of structure initialization, the number of hidden neurons is to be identified.The better structure initialization leads to good accuracy of the networks.Generally, clustering algorithms such as Kmeans, FCM (Grisales et al., 2004) determines initial structure of RBF.These algorithms work in two modes; supervised learning and unsupervised learning.In case of unsupervised learning process, the algorithm considers only input space for clustering (Harun et al., 2012) but in supervised mode, considers both input and output space (Xin-Zheng et al., 2012).The traditional unsupervised clustering methods group data into clusters based on the predefined number of clusters.
The wrong number of clusters may reduce system accuracy.The following Fig. 2 shows the clustering result of standard algorithm without considering the output and Fig. 3 shows the result of clustering while considering both input and output.Figure 3 ensures for proper clustering.Supervised learning methods consider both input and output for better system initialization.The output function is written as O = (XWY) where X is the input vector, Y is the output vector and W is the weight.
In general, hybridization of algorithms is used extensively in many applications for better performance by exploiting good properties and applying those properties in standard algorithm to mitigate its weakness.
Optimization algorithms were employed for system initialization and parameter optimization of RBF.Genetic algorithm, PSO, Differential evolution is popular optimization methods hybridized with RBF for better performance.PSO is used to train RBF and to find optimal parameters for fuzzy clustering (George and Tsekouras, 2013).A hybrid approach combining PSO and RBF is proposed to solve classification problems (Leung et al., 2012).A new Optimum Steepest Decent (OSD); a combination of PSO and gradient decent proposed to initialize RBF more accurately and interesting outcomes are found (Vahid and Gholam Ali, 2013).
Clustering investigates data distribution of domain and forms similar data into different groups.Classification quantifies the relationships among the data and separates them into various classes.Generally Classification techniques are non-parametric methods.The integrated approach of clustering and classification are used for better performance.The output of hierarchical clustering is fed into classification methods such as decision trees and regression analysis to understand and characterize petroleum reservoirs (Denis et al., 2011).In some applications, learning based clustering approaches are used to discover the knowledge of domain.Hybrid nature of clustering and classification has been adopted to identify road accidents in Iran (Mahdi et al., 2013).Clustering algorithms are used often in Neural Networks (NN) to find the function centres.A novel learning strategy comprising fuzzy clustering and RBF is introduced to find compact and accurate clusters (Antonios and George, 2012).A new clustering approach with a delayed connection (Adibi et al., 2005) in NN architecture is proposed to obtain high precision clusters.RBF has been widely used in real time classification problem.For better training in less number of iterations, a PSO based approach is demonstrated and showed interesting outcome (Vahid and Gholam Ali, 2013).To train RBFNN, a novel Fuzzy C-Means clustering is described for effective outcome (Antonino et al., 2006).Learning ability of algorithm is improved by employing maximum entropy based RBF system to control chaotic system (Liu et al., 2006).In this study, we propose an approach to design an adaptive RBF to cluster and classify the data of biomedical data sets.The design ensures for less error and good accuracy in less computation time.

MATERIALS AND METHODS
Optimization in RBF design: Swarm intelligence algorithms also referred as nature-inspired algorithms are motivated by social behavior of animals and have proved to be very efficient in solving real world optimization problems.These algorithms include Particle Swarm Optimization (PSO), Ant Colony Optimization, Stochastic Diffusion Search and Bacteria Foraging and so on.Quantum Particle Swarm Optimization (QPSO) is a variant of PSO (Kennedy and Eberhart, 1995) and is proposed for better search ability and good convergence speed.Inception of Quantum theory into PSO brought a new algorithm called QPSO was proposed by Sun et al. (2004).QPSO guaranteed optimal solution, unlike PSO no velocity vectors are needed and takes less number of parameters to adjust.QPSO has been used in wide range of optimization problems (Leandro dos Santos, 2010; Debao et al., 2012;Bo and Jiulun, 2008;Jun et al., 2012).In this study we apply QPSO to optimize the root mean square error and function centres of the network and to tune the network for adaptive nature.In PSO, particles in the swarm fly around in D dimensional space and interact with each other for optimal solution.Each particle in the population iteratively discovers its own position according to velocity and its previous positions.The individual best positions of particles are stored in order to find global best positions in each iteration.The positions of i th particle in D dimensions is represented with position vector Xi = (x i1 , x i2 , x i3 , …, x in ) given in Eq. ( 4) and velocity vector Vi = (v i1 , v i2 , v i3 .. v in ) given in Eq. ( 3) The optimal solution of the problem is determined according to fitness function: Each particle maintains two positions:  Personal best position (P best) is the best position experienced so far  Global best position (G best) is the best position over entire swarm The Eq. ( 3) and ( 4) are updated iteratively based on P best and G best values where ω is the inertia weight controls the speed of algorithm.The local and social search of PSO influenced by two positive parameters c 1 and c 2 , generally set between the range (0, 2).As mentioned above QPSO does not require velocity vectors and the introduction of mean best position (mbest) increase search scope of particles, thus making the algorithm more efficient.The particles positions are calculated using Eq. ( 5): where, And Eq. ( 7) shows the calculation of M best position which is the average of P best positions: An elite learning optimization approach (Leung et al., 2012), RBFNN using ALPSO proposed for classification problem.Velocity vectors of PSO are updated with linearly decreasing inertia weight and the proposed structure optimizes weight matrix and other controlling parameters.A PSO based optimized learning method used to Vahid and Gholam Ali (2013) optimizes RBF unit centres.Different architectures using PSO such as RBF1, RBF2, RBF3 and RBF4 (George and Tsekouras, 2013) are discussed and result shows that the number of hidden node increases as the values of fitness function decreases.PSO-RBFNN structure proposed for Electro Cardiogram (ECG) beat classification (Mehmet and Berat, 2010).A method of FCM optimized using Ant Colony Optimization (ACO) and Genetic Algorithm (GA) proposed to provide optimal RBF centres to increase accuracy rate (Zhide et al., 2009;Zhen et al., 2008).Similarly many optimization algorithms are tested with RBFNN.But very fewer papers talk about optimization of RBF using quantum techniques.In this study, a novel structure FCM-RBF-QPSO is discussed to optimize RBFNN and simulation results exhibits its superiority.
Proposed work: Generally, RBFNN training depends on three parameters such as RBF unit centres, width of RBF unit and weight matrix.RBF neuron centre usually determined either by applying clustering algorithms in the input space or choosing centres randomly.Some of the clustering algorithms used are K-means, Fuzzy C Means, Kohonen-SOM and Orthogonal Least Squares (OLS).In this study we apply FCM optimized using QPSO to find the centres of RBF.

Calculate centres and identify the number of hidden nodes:
The performance of network largely depends on cluster centres.Thus, in this study much attention is given to choosing neuron centre; we apply optimization technique QPSO to minimize the objective function given in Eq. ( 11) to produce optimal centres.The cluster centre determines suitable number of hidden nodes.The covers theorem says the number of RBFNN units is to be bigger than the dimension of input space.Otherwise, over smoothing or over adaptation may reduce the performance.Here, we try to propose an optimal number of hidden nodes using QPSO: where, ∑ ∑ 2/ 1 (10) where, x k = The input vector m = The fuzziness parameter v i = The centre of cluster i µ ik = The membership grade The width of j th RBF unit given in Eq. ( 12) and is calculated using the method proposed by Moody and Darken (1989): where, d max = The maximum distance between clusters α = The positive factor and is set as 2 The output of the network is given in Eq. ( 2) and the standard error is calculated as: where, H = Y k (X) given in Eq. ( 2).The optimal weights are obtained using Eq. ( 14): Algorithm of proposed method: 1. Initialize the swarm with random positions.Set number of clusters (CN = 2) 2. Optimize the objective function of FCM using QPSO 2.1 Initialize the swarm and its positions 2.2 Compute Fitness and update the positions of particles as per Eq. ( 5), ( 6) and ( 7) 2.3 Set gbest positions of particles as cluster centres 3. Find hidden nodes and width of RBF network using Eq. ( 12) 4. Sort RBF nodes according to distance d 5. Find weight matrix and optimize Root Mean Square Error (RMSE) using QPSO 6.If output is satisfactory then halt Else CN = CN+1 Goto step 2

RESULTS AND DISCUSSION
The experiments were performed on the datasets from UCI machine learning repository.The characteristics of datasets are given in Table 1.
In the proposed RBFNN, the centres are determined by Optimized FCM (OFCM) and also through random selection.The fitness function or objective function given in Eq. ( 8) and ( 12) are to be minimized to improve the structure of RBF NN.The position of particles in a swarm such as Pbest and Gbest gives fitness values.OFCM produces fuzzy clusters and crisp clusters are found by random selection.Too many or very few clusters does not guarantee for good outcome.Hence, an adaptive approach is proposed in this study to choose right number of clusters based on input space.The execution results on the dataset liver disorder is shown in Table 2 while Table 3 depicts execution on Breast cancer.In this study QPSO is  used as a multi objective optimization method since it optimizes centres and cost function simultaneously.The Root Mean Square Error (RMSE) is calculated for both cases on applying OFCM and choosing random centres from the input.The analysis is also carried out on D max; the distance between the centres.From Table 2 and 3, the RMSE on training and testing decreases as the number of cluster increases.
Figure 4 to 6 shows the fitness of cost function.The fitness decreases when the number of hidden nodes increases.
Figure 4 to 6 shows the fitness evaluation of cost function on liver disorder and breast cancer datasets.The proposed method produces the fitness for cluster numbers CN = 10, 15 and 20 as 0.2468, 0.24413 and 0.2211 respectively at 100 th iteration.

CONCLUSION
In this study we propose a supervised self adaptive RBF neural network to cluster the data sets efficiently.The network suffers with slow training, wastage of memory and less accuracy if correct number of hidden neurons is not chosen.Our approach identifies the suitable number of clusters for the data sets with reduced root mean square error.The prominent features of network such as good convergence speed and high precision are obtained by the proposed method.The inclusion of optimization technique QPSO improves the performance of the network.This supervised self adaptive network produce lower error for training and testing data when OFCM is executed on input space and the suitable number of hidden nodes automatically set by network from previous learning.

Table 1 :
The descriptions of dataset