An investigation of membership functions on performance of ANFIS for solving classification problems

Adaptive neuro-fuzzy inference system (ANFIS) is one of the efficient machine learning techniques, which has been successfully employed in wide variety of applications. The performance of ANFIS depends on the selection of the number and shape of membership functions as these two factors influence the most on computational complexity and accuracy of the designed ANFIS-based model. Mostly, an expert knowledge is required in this regard. However, there is an immense need of an investigative study for helping researchers make better decision on the number and shape of membership functions for thier ANFIS models. Hence, this study examines the role of four popular shapes of membership functions on the performance of ANFIS while solving various classification problems. According to experiments, Gaussian membership function demonstrated higher degree of accuracy with lesser computational complexity as compared to the counterparts.


Introduction
In recent years, the combination of neural network and fuzzy logic has been widely applied as effective soft computing technique. Adaptive neuro-fuzzy inference system (ANFIS), being prominent neuro-fuzzy models, has produced robust results among other fuzzy inference systems, as it is more flexible, adaptive, and effective on highly non-linear complex problems [1]. Any application of ANFIS demands expert knowledge of fuzzy logic as ANFIS structure requires better choice of suitable shape and number of membership functions. This does not only influence the efficiency of ANFIS-based model, but also the computational cost. That said, the number of membership functions for input space partitioning for each input decides about the number of rules in ANFIS rule-base. Moreover, the number of parameters in each membership function also determines the training cost of these parameters. Although, Gaussian shape of membership function has been most commonly used in literature due its smooth representation of input space, furthermore, it uses only two parameters [2].
Researchers have a wide variety of the shapes of membership functions; such as, Gaussian, triangular, bell, to choose from for the ANFIS-based model under consideration. Moreover, these membership functions can also be modified, or even hybridized, to customize the shape to best fit the fuzziness of inputs and generate maximum accuracy. This makes the implementation of neuro-fuzzy systems more crucial as the researchers are often found wondering what type of membership function suits their problem. Therefore, different studies have been found in  [3] evaluated ten different membership functions while designing fuzzy control systems. This study found triangular membership function revealed better results comparatively. In a comparative study, [4] studied the effects of triangular, trapezoidal, and Gaussian while developing an antenna control system based on fuzzy logic. Contrarily, [5] proposed a new diamond-shaped membership function in order to reduce noise in the noisy inputs of a prediction system.
As discussed previously, literature shows that different studies have been carried to experiment effects of different membership functions while designing a fuzzy inference systems. However, to the best of authors' knowledge, with regards to ANFIS, such approach of investigation is hardly found. Therefore, this study is aimed at evaluating and analyzing the effect of different shapes of membership functions on the accuracy of ANFIS while solving classification problems. Moreover, this study also highlights how the computational complexity is effected by the mentioned factors. Rest of the paper is structure as follows. The subsequent section presents brief discussion on membership function and its popular shapes. ANFIS is explained in Section 3 followed by experimental section (Section 4) where ANFIS is employed with the different membership functions while solving classification problems. Results are presented in Section 5. Section 6 concludes this investigative study.

Membership Functions
In fuzzy set theory, a membership function defines the degree of truth (partial truth instead of TRUE or FALSE, 0 or 1), of a crisp value, in a range between 0 and 1. This helps in designing systems with uncertainty or ill-defined problems in read world. Membership function is a function which returns membership degree of how a crisp value is mapped to an input space referred to as universe of discourse. Each membership function contains a curve which represents each point in a specified input partition. Depending on the shape of the curve, each membership function is given a certain name, i.e. triangular, bell-shaped, trapezoidal, and Gaussian membership functions. The basic types of membership functions are illustrated in Fig.  1.
Trapezoidal membership function (Trapmf) has four scalar parameters for defining its curve: a, b for feet and c, d for shoulders as it has truncated triangle shape. The mathematical representation of the function is given by (1).
The generalized bell shaped membership function (Gbellmf) is a symmetrical shape similar to a bell. As expressed by (2), this function employs three parameters: a determines the width of the bell like curve, b is a positive integer, while c sets the center of the curve in universe of discourse.
bell(x, a, b, c) = 1 With a triangular curve, triangular membership function (Trimf) is the simplest shape among others. It is defined by three parameters for defining three points: a and c for feet, and b for the tip of the curve. Equation (3) expresses mathematical formula of the function. Just as bell-shaped membership function, Gaussian (Gaussmf) also has a smooth curve. However, among all three membership functions mentioned above, it utilizes only two parameters: c for locating center and σ for determining the width of the curve as expressed mathematically by (4).
3. Adaptive Neuro-Fuzzy Inference System (ANFIS) ANFIS was introduced by Jang in 1993 [6], which takes crisp inputs and fuzzifies these inputs using membership functions. Furthermore, through inference mechanism and antecedent part of IF-THEN rule, it connects with the consequent part which is simple linear polynomial equation.
Since, the focus of this study is more on the role of membership functions on ANFIS performance and complexity, therefore, this section explains ANFIS architecture and its learning mechanism briefly. A detailed introduction of the network can be found in the original literature mentioned earlier.
ANFIS network is five layer architecture, illustrated in Fig. 2, where first layer contains membership functions (µ A i , µ B i ) that determine membership degree of each input according to the shape of membership function. The second and third layer is for inference between IF (antecedent) and THEN (consequent) part of the rule-base. Second layer performs product operation on the membership degrees in order to calculate the firing strength (w i ) of each rule. Third layer normalizes N the firing strength (w i ) of each rule against all the rules. Fourth layer is a linear polynomial equationw i f i , (f i = p i x + q i y + r i ). The last layer is simply summation of the outputs of rules calculated in previous layer. ANFIS learns by updating tunable parameters which are membership function parameters (a, b, c in case of triangular membership function) and consequent parameters (p, q, r). This means the nodes of first and fourth layer are trainable, whereas the nodes of rest of the layers are fixed. ANFIS employed two-pass learning algorithm as presented in Table 1. In first pass, which is forward pass, ANFIS calculates nodes outputs until fourth layer where it uses least square methods to update consequent parameters before calculating the final output. In later stage, in the second pass, error is propagated backward until first layer where ANFIS employs gradient descent to tune membership function parameters. The appropriate choice of the number and shape of membership functions determine the cost of training and computational complexity of the network. Node Outputs Error Signals ANFIS can be constructed by partitioning of the input-output data into rule. So, this can be accomplished by using a number of methods such as grid-partitioning (genfis1), subtractive clustering (genfis2), and fuzzy c-mean clustering (genfis3) [7]. Genfis1 employs grid partition technique to generate FIS by using given training data set therefore based on the total amount and category of membership function, grid partition approach divides data space into grids. Normally, the performance depends on grid, the finer the grid is, the better the performance [8,9]. Genfis2 uses subtractive clustering technique to generate a Sugeno-type FIS structure while, Genfis3 uses fuzzy c-means (FCM) to generate an FIS by extracting a set of rules that signifies the type and mode of data [10].
Now that fundamental knowledge about ANFIS and membership functions has been established, the subsequent section explains experiments performed in this study.

Experiments
In order to investigate the effects of membership functions mentioned before, this study solved several classification problems mentioned in Table 2. The benchmark datasets for classification problems were taken from well-established machine learning repository of UCI [11]. Since, the computational cost of ANFIS becomes drastically high when the number of inputs is large; therefore, datasets with not more than ten inputs were selected in our experiments. For dividing the samples in the datasets into training and testing sets were set to 70:30 ratio, respectively. Fuzzy Logic ToolboxTM in MATLAB [10] was used in this study. ANFIS models with gridpartitioning (genfis1), subtractive clustering (genfis2), and fuzzy c-mean clustering (genfis3) were employed to analyze the performance. Most of the settings for ANFIS models were used as default as mentioned in the toolbox, however the distinguishing changes are presented in Table 3. All the models were trained by standard two-pass hybrid training algorithm mentioned in Table 1. It is noteworthy to mention that ANFIS with grid-partitioning was used to test different membership functions, whereas genfis2 and genfis3 use Gaussian types of membership function by default in MATLAB toolbox.

Results and Analysis
The performance of ANFIS with different partitioning methods (i.e., grid partitioning, subtractive clustering, and fuzzy c-mean clustering) and membership functions is evaluated using Root Mean Square Error (RMSE). where T arget i is the target class in the sampled tuple i, whereas the output generated by ANFIS against the tuple i is Output i . Furthermore, for better and comprehensive comparison, ANFIS performances with different membership functions have been ranked from smallest to largest according to the sum of training and testing RMSEs in each dataset. Meaning, for ranking the performances, the training RMSE and testing RMSE was summed up and the rank formula was applied on the sum in each case. Additionally, the average of ranks is computed to generate overall rank in the group of membership functions. The comparison of the results is presented in Table 4. The results of the experiments show that ANFIS with subtractive clustering and Gaussian membership function performed best in all classification datasets among other types of ANFIS (i.e. genfis1, genfis2, and genfis3) and membership functions. However, it is obvious from results that in the group of genfis1 which is ANFIS with grid partitioning, Gaussian membership function achieved best RMSE as compared to three other shapes i.e., trapezoidal, bell, and triangular. This is because Gaussian draws smooth curve which allows representing effectively the data points with minute differences. Other than Gaussian, triangular is also a useful membership function as it proved to be ranked third in all six cases of ANFIS models. Since, grid partitioning generates all possible rules therefore, it is supposed to produce better results. Nevertheless, ANFIS models with subtractive clustering (genfis2) and fuzzy c-mean (genfis3) are also promising approaches.

Conclusion and Future Work
This study presents analysis of experiments performed over four famous different types or shapes of membership functions i.e., trapezoidal, generalized bell-shaped, triangular, and Gaussian, with ANFIS while solving various classification problems. These problems were selected based on number of features from four to ten. ANFIS with different partitioning methods i.e., grid partitioning, subtractive clustering, and fuzzy c-mean clustering were employed using fuzzy logic toolbox available in MATLAB. According to experiments, ANFIS with subtractive clustering and Gaussian membership function is an effective approach. Other than this, Gaussian membership function the most suitable among other selected membership functions when employed in ANFIS with grid partitioning method. Moreover, triangular membership function also proved to be promising way for representing the data points in the problem in hand.
Since, the computational cost of ANFIS rises dramatically as the number of inputs and training samples increase, future research may focus on modifications in ANFIS five layered architecture. Reducing steps between input and output, and employing effective input space partitioning methods will result in less computation, as well as, better accuracy. Hence, possibility of ANFIS implementation will be extended to applications with large features and data samples.