Feature Selection Using Different Transfer Functions for Binary Bat Algorithm

The selection feature is an important and fundamental step in the preprocessing of many classification and machine learning problems. The feature selection (FS) method is used to reduce the amount of data used and to create highprobability of classification accuracy (CA) based on fewer features by deleting irrelevant data that often reason confusion for the classifiers. In this work, bat algorithm (BA), which is a new metaheuristic rule, is applied as a wrapper type of FS technique. Six different types of BA (BA-S and BA-V) are proposed, where apiece used a transfer function (TF) to map the solutions from continuous space to the discrete space. The results of the experiment show that the features that use the BA-V methods (that is, the V-shaped transfer function) have proven effective and efficient in selecting subsets of features with high classification accuracy. KeywordsFeature subset selection, Bat algorithm, Transfer function, Metaheuristic algorithms.


Introduction
Feature selection plays an important role in the classification and machine learning, as it aims to remove the irrelevant or redundant features of the data to achieve better classification performance. Several metaheuristic algorithms have been proposed in conjunction with feature selection methods to reduce data processing time and improve the classification accuracy (CA) (Qasim and Algamal, 2018;Dahiya et al., 2019). The methods of feature subset depend on the evaluation of the selected subsets through two basic approaches: filter and wrapper subset methods. The filter relies on the evaluates the subgroups based on the original dataset itself whereas the wrapper method is based on external learning algorithms based on metaheuristic algorithms to evaluate subgroups (Wang and Alexander, 2016;Teng et al., 2017;Al-Tashi et al., 2020).
Evolutionary computing (EC) is nature-inspired techniques and algorithms by mimic the behavior of insects, animals, and birds (such as bats) and depends on the population in the search for possible solutions to the problem (Valdez, 2015). EC has been used extensively in recent years to address problems of feature selection in literature (Emary et al., 2016). For example, there are feature selection approaches based on swarm algorithms (Kashef and Nezamabadi-pour, 2015) , genetic algorithm (GA) (Bhunia et al., 2017) and differential evolution (DE) (Zorarpacı and Özel, 2016). The works in this field can also be found through reference to Mafarja and Abdullah, 2013a;Pramy, 2018;Qasim and Algamal, 2018;Al-Thanoon et al., 2019).
Many metaheuristic inspired by the conduct of animals, insects, and birds in nature have been proposed as good methods in the feature selection (Yang, 2010;Mafarja et al., 2020). The bat algorithm (BA) is one of the modern metaheuristic methods inspired by the social behavior of bats. BA is described by the fact that its equations are uncomplicated, therefore can be applied in many fields such as classification, feature selection, data mining, scheduling, and others (Yang and He, 2013).
Transfer functions (TFs) are important and effective parts of a binary metaheuristics algorithm when used in the classification process and feature selection (Mirjalili and Lewis, 2013), where it very affects the process of balance (between exploration and exploitation). In this research, six transfer functions, proposed in (Mafarja and Abdullah, 2013b) , were applied to the BA.
Our paper is organized as follows. The methodology of the BA and the BBA are introduced in Section 2. In section 3, the BBA for feature selection is presented. The Experimental results are listed in Section 4. The conclusion is provided in Section 5.

Bat Algorithm (BA)
BA is one of the new metaheuristic methods proposed by Yang in 2010 which have attracted the attention of many researchers from different fields because of their remarkable capability of echolocation. It is based on the echolocation ability of microbats guiding them on their foraging attitude (Yang, 2010). There are various types of bat that differ in size and weight but they have similar behaviors of navigation and hunting. The BA was designed based on the basic advantage of the bats when searching for prey. Bats have a strong sound propagation technique are different from other birds called bio-sonar. Bats when chasing prey tend to increase the rate of sound emitted by the ultrasound and decrease the loudness. The behavior of bats has been modeled as follows (Mirjalili et al., 2014;Yu et al., 2020). BA has a set of vectors representing the location, velocity, and frequency, which can be updated with equations (1), (2), and (3): where, best G represents best solution, i F is frequency of the i-th bat which is updated through the iterations as follows : where,  represents a random number of the uniform distribution in [0, 1]. Through Equations (1) & (3) we note that, the various frequencies of bats encourage them to have a tendency to the solution. However, exploitation was carried out through the use of a random walk procedure as follows (Anter & Ali, 2020): where, indicates a random number in [-1,1], and A represents the loudness of emitted sound that bats use. The pulse emission rate and the loudness A are updated by controlling the balance between these techniques as follows: where  and  are constants; t is analogous to the cooling factor, lastly, i A is equal to zero as well as the last value of i r is (0) r . The update of the loudness and rate guarantee that artificial bats move towards the best solutions (Tharwat et al., 2017). The pseudocode of BBA is presented in Figure 1.

Start
Create an initial population of the BA { i

while (t < Max number. of iterations).
Generate new solutions by updating the following positions and velocities in addition to adjusting frequency from Equations (1), (2) and (3).
Select a random value solution (between the best solutions). Create a local solution.

end if
Generate a new solution using randomly fly.
Chose the new solution and reduce i A and increase i r .
end if locate the current best G and rank the bats.

Binary Bat Algorithm (BBA)
The BBA is conceptually similar to the main bat algorithm (BA) but differs from it in the search space. In the main BA, the search is in continuous space, while in the binary algorithm is searched in the discrete binary space (Mirjalili et al., 2014). The velocity and position in the BBA algorithm does not change because the search space is constrained by 0 and 1. The binary search can be assumed to be a hypercube, In the BBA, it is possible only to shift to the corners of this hypercube by reversing the different bit numbers. Position update in discrete spaces is done by switching the values "0" and "1" (Naik et al., 2020). The position of the agent is changed in discrete space with the probability of its velocity by using a transfer function to map velocity values to probability  (Kennedy and Eberhart, 1997;Mirjalili and Hashim, 2012). This means that the vector values of the position are changing from 0 to 1 and vice versa, through the transfer function, which gives the force to move particles in a binary space. To select the transfer function, consider the following concepts (Rashedi et al., 2010). Through previous concepts, the search is transformed from continuous space into discrete binary space by the transfer functions (Mohamad et al., 2010;Teng et al., 2017). The selected transport function is defined as follows: The probability in the position of an agent in BBA is given as follows:

while (t < Max number. of iterations).
Updating velocities and adjusting frequency.
Transfer function values are calculated by Equation (7). Updating positions from Equation (8).

Select a random value solution and create a local solution. end if
Generate a new solution using randomly fly.
Chose the new solution and reduce i A and increase i r .
end if locate the current best G and rank the bats.

Binary Bat Algorithm for Feature Selection
The feature selection depends on the representation of any solution as a binary variable of asized, where representing the total number of features in the dataset. Meta-heuristics techniques are used in feature selection issues to overcome the problem of generating all possibilities for feature combinations calculated by the equation (Mirjalili and Lewis, 2013;Nakamura et al., 2013): where, AF indicate the all possible feature in the dataset.
BA is one of the most important metaheuristics algorithms to get the best feature subset by searching for a feature space well and effectively. Each subset of the dataset is evaluated by classification accuracy (CA) obtained from the selected features. The fitness function (Fit) is used to evaluate subgroups according to the accuracy of the classification as follows: When a feature selection strategy is used in the binary bat algorithm (BBA), the working mechanism of the algorithm is transformed from continuous space into discrete space by applying transfer functions (TFs), which belong to two main categories (V-shaped and S-shaped). The possibility of updating a binary solution (from 0 to 1) and vice versa is done through TFs. In BA, solutions are updated using S-shaped functions based on the following equation (Munshi and Mohamed, 2017;Al-Thanoon et al., 2018): Also, by using the V-shaped functions, solutions are updated based on the following equation: ( ) 1 The mathematical representation of BA-S shaped families are defined as:

Experimental Results and Discussion
Three binary class datasets: antimicrobial agents (pMIC) (Xing et al., 2014), anti-hepatitis C virus activity (Kang et al., 2009a;Kang et al., 2009b;Kang et al., 2010;Khatri et al., 2015) and neuraminidase inhibitors of influenza A viruses (H1N1) (Li et al., 2016), were used for evaluating the suggested algorithm. A brief attributive of the data from the point of view of the number of samples, number of features and class are summarized in Table 1. The datasets have been invested to evaluate the efficiency of the proposed method. Every dataset was split into training dataset which consist of 70% of the total number of samples and testing datasets consisting of 30% of the samples. All algorithms (BA-V and BA-S) were carried out with the Matlab programming and processed on a Quad-Core Intel Core i5 CPU (4G RAM and 3.2GHz processor).  Tables 2 and 3 show the obtained results for BA-V1, BA-V2, BA-V3, BA-S1, BA-S2,, and BA-S3, respectively, in terms of classification accuracy (CA) for both the training and testing datasets (in Table 1) . For comparison, the last column "# features" presents the average number of selected features for each transfer function. The results in Table 2 shows that by the cooperation of the situation with feature selection, the classification accuracy (CA) for all transfer functions (TFs) was greatly improved and the feature dimensionality reduced. The results exhibit that the methods have high efficiency in classifying data. Moreover, BA-V2 and BA-S1 functions attain typical results in a reduction of the features number, which are significantly different from the others.

Conclusion
In this paper, several transfer functions (Equations 13-18) have been investigated. To demonstrate the efficiency and effectiveness of transfer functions (V-shape and S-shape), they were used in the binary bat algorithm and made three comparisons of the datasets (in Table 1) by relying on two main scenarios: the number of features specified in each algorithm and the classification accuracy (CA). The experimental results in Tables 2 and 3 have shown that the BA-V2 algorithm has consistency in the feature selection process and that it outperforms other algorithms through the accuracy of the classification results.