Premise Parameter Optimization on Adaptive Network Based Fuzzy Inference System Using Modification Hybrid Particle Swarm Optimization and Genetic Algorithm

DOI: 10.31284/j.iptek.2018 .v22i2.274 ANFIS is a combination of the Fuzzy Inference System (FIS) and Neural Network (NN), which has two training parameters, premise and consequent. In the traditional ANFIS, Least Square Estimator (LSE) and Gradient Descent (GD) are commonly used learning algorithms to train the two parameters. The combination of those two learning algorithms tends to produce the local optimal solution. Particle Swarm Optimization (PSO) can converge quickly but still allow for getting the local optimal solution because PSO is unable to find a new solution space. Meanwhile, Genetic Algorithm (GA) has been reported to be able to find a wider solution space. Hybrid PSOGA is expected to give a better solution. In this study, modification of hybrid PSOGA is used to train the premise parameter of ANFIS. In experiments, the accuracy of the proposed classification method, which is called ANFIS-PSOGA, is compared to ANFIS-GA and ANFIS-PSO on Iris flowers, Haberman, and Vertebral datasets. The experiment shows that ANFIS-PSOGA achieves the best result compared to the other methods, with an average of accuracy 99.85% on Iris flowers, 84.52% on Haberman, and 91.83% on Vertebral.


INTRODUCTION
Fuzzy Inference System (FIS) uses the If-Then fuzzy rules to model aspects of human knowledge and thinking without using appropriate quantitative analyzes.FIS has no standard model to transform human knowledge or thinking into rules.In order to develop an effective method for tuning the parameters of fuzzy rules, an Adaptive Network-based Fuzzy Inference System (ANFIS) is proposed by [1].ANFIS is a FIS that adopts architecture and learning of Neural Network (NN) method to obtain the optimal rules.In most cases, a combination of Least Square Estimator (LSE) and Gradient Descent (GD) are used as the learning algorithms.ANFIS has been widely used to solve classification problems in various fields with good results [2], [3].
In ANFIS, there are two types of parameters that require learning in order to obtain optimal values, the first is premise parameters, which are used to form the membership functions; the second is consequent parameters which are used to obtain the fuzzy rule output.The GD is a commonly used algorithm to learn the premise parameters, whereas the LSE is used to learn the consequent parameters.Some researchers have reported that the GD learning algorithm can be stuck into local optimal solution [4]- [6].In recent years, some researches on ANFIS are conducted by replacing the GD learning algorithm with evolution optimization algorithms or metaheuristic optimization algorithms [5], [7], [8].
Characteristics of PSO are robust and can quickly solve non-linear problems [9].But some other researchers reported that the rapid-characteristic of PSO can produce convergence on local optimal solution space or commonly called the premature convergence [10]- [15].The situation occurs because PSO is unable to find a new solution space when it is close to the final solution.
In order to find a new solution space, PSO with Genetic Algorithm (GA) are combined [16].Although GA is slow to converge, GA has the ability to find a wider solution space by using the crossover and mutation operators.Hybrid of PSO and GA (PSOGA) is able to find a better result than the original PSO.In PSOGA, first, an individual is evaluated using PSO operator and then is evaluated using the GA operator.Therefore, the individual value generated by PSO operator will be replaced by a value generated by the GA operator.Replacing the individual value produced by PSO is not always giving a better solution, but it can yield a wider solution space.
The proposed method of this study is to modify the hybrid PSOGA and attach the modification on ANFIS.The modification is carried out by adding a process to select the best individuals of PSO and GA.The modification of hybrid PSOGA is used as a learning algorithm on ANFIS or we called ANFIS-PSOGA to obtain optimal premise parameter.In the following sections, we present the literature review, methodology, result and discussion, and conclusion.

Adaptive-Network-Based Fuzzy Inference System (ANFIS)
Artificial Neural Network (ANN) is a commonly used learning-based classification model.On the other hand, Fuzzy Logic (FL) has an important role to model some aspects of knowledge and human thoughts without using precise quantitative mathematical analysis.ANFIS which has been proposed by [1], is a combination of Artificial Neural Network and Fuzzy Inference System.Figure 1 shows the architecture of ANFIS proposed by [1], which has two inputs, five network layers and one output.The ANFIS architecture is functionally same as the architecture of Rule Based Fuzzy.Supposed there are two inputs, x and y, and one output f, then there will be two rules on Sugeno model.The two rules are shown in Equation ( 1) and ( 2), where A i and B i are fuzzy sets; p, q, and r are consequent parameters.The five layers in ANFIS architecture are described as follows.
Layer 1: Each neuron in the first layer is computed using the activation function with some adaptive parameters as shown in Equation ( 3).
a, b, and c are parameters whose value is adjustable (adaptive).If the value of these parameters is modified, then the shape of the function will also change.Standard deviation and mean are used as the initial values of parameters a and c.While value 1 is used to initiate parameter b.All parameter in this layer is known as the premise parameter.
Layer 2: Each neuron in the second layer is fixed neuron represented in Equation ( 4).Each node represents the predicate w of the i-th rules.
2 =   =    ()   (),  = 1,2 . . . . .(4) Layer 3: Each neuron in the third layer is a fixed node calculated as the ratio between the predicate w of the i-th rule and the sum of all predicates w, as shown in Equation (5).The calculation result is known as normalized firing strength.
Layer 4: Output of neuron in the fourth layer is calculated using Equation (6), which is described further in Equation (7).  is normalized firing strength in the third layer and {p, q, r} are parameters of the neuron.These parameters are called as consequent parameters.
Layer 5: Output of neuron in the fifth layer, which is also an output layer, is calculated as the sum of all inputs, as shown in Equation (8).

Hybrid PSOGA
Combining the advantages of GA and PSO is to balance natural selection and share individual knowledge to make robust and efficient searches in search dimensions [16].The goal of combining these two algorithms is to find a global solution for the entire search space.The combination of PSO and GA is done by changing the velocity and position of the particle in the

METODOLOGY ANFIS with Modification of Hybrid PSOGA
The algorithm of ANFIS with modification of hybrid PSOGA proposed in this research is shown in Figure 3.The dataset characteristics used to test the algorithm is shown in Table 1.

c. Mutation
The process of mutation is the process of replacing the worst parent with a new parent formed with a random value.9.This part is the modification of the original hybrid PSOGA.The modification is done by sorting Ind_PSO and Ind_GA according to the value of cost function.Take individuals with the amount equal to the population number from the top rank.Those individuals called ind_PSOGA.10.If a stopping criterion is met, then the best individual is found.If not, then back to step 4.

Cost Function
The cost function used in this study is the value of MSE obtained from the output of the ANFIS method.Pseudocode to calculate the cost function is as follows.1. Transform individual features of PSOGA into premise parameters such as Figure 4. 2. Calculate the first layer of ANFIS using Equation (3). 3. Calculate the w using Equation (4).4. Calculate the normalization of w using Equation (5).The output of this layer is  ̅. 5. Calculate the fourth layer of ANFIS using Equation ( 6) and (7).The values of variable p, q and r are obtained using LSE method.6. Calculate the final output of ANFIS.7. Calculate the MSE value using Equation ( 13).This value is the output of the cost function.
N represents the amount of data and   represents output data at i-th.

Testing Parameters
The purpose of the experiment presented in this sub-section is to find some optimal parameters for each method.In all of the three methods, i.e.ANFIS-PSOGA, ANFIS-GA, and ANFIS-PSO, the LSE learning algorithm is applied to learn the consequence value of ANFIS.However, they apply different algorithms to learn the premise value of ANFIS.The ANFIS-GA uses the Genetic Algorithm, the ANFIS-PSO uses the Particle Swarm Optimization, and the ANFIS-PSOGA uses the modification of hybrid Particle Swarm Optimization and Genetic Algorithm.Since they apply different algorithms to find the optimal premise value on ANFIS, then each method has a different set of parameters.ANFIS-PSOGA has inertia, crossover, and mutation parameters; ANFIS-GA has crossover and mutation parameters, while ANFIS-PSO has inertia parameter.The range and the value of the parameters tested in the experiment are shown in Table 2.
In order to obtain optimal parameter combination, all possible combination of parameter values is tested on each method.On ANFIS-PSOGA, 72 combinations of inertia, crossover, and mutation parameters are tested and evaluated.ANFIS-GA is tested using 12 combinations of crossover and mutation parameters.When ANFIS-PSO is evaluated using 6 combinations of inertia parameters.From the experiments, each combination of parameter values gives slightly different accuracy on each method.Table 3 shows the optimal parameter values for each method.Table 2.The range and the value of parameters.0.4-0.90.4, 0.5, 0.6, 0.7 , 0.8 , 0.9 Crossover Ratio-GA (Cr) 0.6-0.90.6 , 0.7 , 0.8 , 0.9 Mutation Ratio-GA (Mr) 0.1-0.30.1 , 0.2 , 0.3 Table 3.The optimal parameters.

Accuracy Comparison
The experiment conducted in this sub-section is to compare the accuracy of the four ANFIS-based classification method, i.e. the traditional ANFIS, ANFIS-GA, ANFIS-PSO, and ANFIS-PSOGA.The accuracy of each method is calculated on the same testing environment on the three different datasets.Each method is executed 5 times using 10-fold cross-validation.The accuracy of all method, while they are executed on Iris Flowers dataset, is shown in Table 4.The highest accuracy is 100% obtained from all methods except ANFIS, while the lowest accuracy is 93.33% obtained from the ANFIS method.The average of accuracy calculated from all testing on Iris Flowers dataset is shown in the right column of Table 3.The best average of accuracy is 99.85% obtained from the ANFIS-PSOGA.The second best average of accuracy is 99.70% obtained from the ANFIS-GA.Thereafter, the averages of accuracy are 99.56% and 94.22%, those are achieved by ANFIS-PSO and ANFIS, respectively.
Table 4 shows the experimental results of all methods on Haberman dataset.Different from the experiment on Iris Flowers dataset which capable to achieve 100% accuracy, there is no method capable to get perfect accuracy on Haberman dataset.The averages of accuracy ranked from the best to the worst are 84.52%,84.15%, 83.78%, and 74.22%, which are obtained from ANFIS-PSOGA, ANFIS-GA, ANFIS-PSO, and ANFIS, respectively.Performances of the four methods on Iris Flower dataset are better than on Haberman dataset.The experimental results on Vertebral dataset are presented in Table 4.The highest accuracy is obtained from the ANFIS-PSOGA method with a value of 94.62%.The averages of accuracy ranked from the highest to the lowest are 91.38%,84.23%, 65.30%, and 65.23%, obtained from ANFIS-PSOGA, ANFIS, ANFIS-PSO, and ANFIS-GA, respectively.In the experiment on Vertebral dataset, PSO and GA algorithms are unable to improve the performance of ANFIS.On the other side, the modification of hybrid PSOGA is able to improve the performance of the traditional ANFIS.
As shown in Table 4, the ANFIS-PSOGA achieves the highest average of accuracy on all dataset.While working on Iris Flower dataset, which has 3 classes, 4 features, not too many amounts of data (150), and balanced amount of data in each class, ANFIS-PSOGA gives the best average of accuracy, i.e. 99.85%.ANFIS-GA and ANFIS-PSO achieve better performance compared to the traditional ANFIS.accuracy, 84.52%.This accuracy is obtained using optimal parameters, 0.5 for inertia, 0.9 for crossover ratio, and 0.3 for mutation ratio.ANFIS-GA and ANFIS-PSO achieve better accuracy than the traditional ANFIS.While working on Vertebral dataset which has 2 classes, 6 features, not too many amounts of data (310), and unbalanced amount of data in each class with 100 data in normal class and 210 data in the abnormal class, the ANFIS-PSOGA give superior accuracy compared to the other methods, i.e. 91.83%.On the other hand, not very good results are obtained by ANFIS-PSO and ANFIS-GA, with accuracy below 70%.ANFIS-PSO gets the average of accuracy 65.30% and ANFIS-GA gets 65.23%.ANFIS-PSO and ANFIS-GA are unable to find a global solution, which produces not better accuracy compared to the traditional ANFIS, which is 84.23%.The proposed ANFIS-PSOGA achieves the best average of accuracy compared to three other ANFIS-based methods, i.e. traditional ANFIS, ANFIS-GA, and ANFIS-PSO.
ISSN:1411-7010Jurnal IPTEK e-ISSN:2477-507XVol.22 No.2, Desember 2018 30 PSO method with the concepts of selection, crossover, and mutations in GA.The algorithm of hybrid PSOGA is shown in Figure2.Main steps in the algorithm of hybrid PSOGA are described as follows.1. Initialize of n population randomly.2. Initiate some variables of PSO and GA. 3. Evaluate each individual in the population using PSO operators.4. Calculate individual k 1 using GA operators.The value of k 1 is increment from 1 to n. 5. Evaluate each individual using the specified fitness function.6.If a termination criterion is met then the best solution is found, else back to step 3.

Figure 4 .
Figure 4.The features transformation from individual to premise parameter.

CONCLUSION
In this research, a modification of hybrid PSOGA has been developed to learn the premise parameters of ANFIS.The experiments use three datasets, i.e.Iris flowers, Haberman, and Vertebral.While working on Haberman dataset which has 2 classes, 3 features, not too many amounts of data (306), and unbalanced amount of data in each class with 255 data in the negative class and 81 data in the positive class, the ANFIS-PSOGA achieves the highest average of ISSN:1411-7010 Jurnal IPTEK e-ISSN:2477-507X Vol.22 No.2, Desember 2018

Table 1 .
(9) dataset characteristics.Figure 3.The algorithm of ANFIS with modification of hybrid PSOGA.Main steps of the algorithm of ANFIS with modification of hybrid PSOGA are described as follows.1.Initialize the PSO parameters (inertia) and GA parameters (crossover ratio Cr and mutation ratio Mr).Initialize the number of individuals (  ) and maximum iteration ().2.Generate value of all individuals ind_PSOGA with a random value.3.Calculate the cost function.4.Evaluate each individual ind_PSOGA with PSO operators: a. Determine the P best and G best .b. Calculate the velocity of each individual.c.Calculate the position of each individual change.5.Calculate the number of individuals to be processed with the GA operator k using Equation(9).Determine the value of individuals GA that is presented as variable ind_GA.The ind_GA is individuals obtained from the worst ind_PSO with amount of k individuals.8. Evaluate all individual GA using GA operators.The calculation result is stored in ind_GA.
X is the first parent and Y is the second parent.

Table 4 .
The accuracy of all experiment on the dataset.