Multi-Swarm Bat Algorithm

In this study a new Bat Algorithm (BA) based on multi-swarm technique called the Multi-Swarm Bat Algorithm (MSBA) is proposed to address the problem of premature convergence phenomenon. The problem happens when search process converges to non-optimal solution due to the loss of diversity during the evolution process. MSBA was designed with improved ability in exploring new solutions, which was essential in reducing premature convergence. The exploration ability was improved by having a number of sub-swarms watching over the best local optima. In MSBA, when the quality of best local optima does not improve after a pre-defined number of iterations, the population is split equally into several smaller sub-swarms, with one of them remains close to the current best local optima for further exploitation while the other sub-swarms continue to explore for new local optima. The proposed algorithm has been applied in feature selection problem and the results were compared against eight algorithms, which are Ant Colony Optimization (ACO), Genetic Algorithm (GA), Tabu Search (TS), Scatter Search (SS), Great Deluge Algorithm (GDA) and stander BA. The results showed that the MSBA is much more effective that it is able to find new best solutions at times when the rest of other algorithms are not able to.


INTRODUCTION
In bio-inspired algorithms, there must exists a balance between exploitation (intensification) and exploration (diversification); two of which very important components that affect the overall efficiency and performance of the particular learning algorithm.In intensification the promising regions are explored more thoroughly in the hope to find better solutions.Diversification non-explored regions must be visited to be sure that all regions of the search space are evenly explored and that the search is not confined to only a reduced number of regions.Too much intensification and too little diversification could cause the system to be trapped in local optima, which could lead to loss of the global optimum.Only a good combination of these two major components will usually ensure that the global optimality is reachable.This is essentially the problem of premature convergence suffered by many bio-inspired algorithms (Dorigo and Blum, 2005;Mallipeddi et al., 2011;Choubey and Kharat, 2013), whereby learning stops during convergence of nonoptimal solutions due to the loss of diversity in the process of evolution.
Literature has shown that BA, as a bio-inspired algorithm, also it may suffers from the premature convergence phenomenon (Lin et al., 2012;Xie et al., 2013;Wang et al., 2013a, b).In order to avoid this, Wang et al. (2013b) enhanced the bats' flight skills as well as flight modes by dynamically regulating the flight speed and direction while assuming the position of the prey continuously change with time.Meanwhile, Lin et al. (2012) proposed a synergistic approach based on chaotic sequence and chaotic Levy flight in the meta-heuristic search in efficiently generating new solutions.The study suggested chaotic sequence is able to generate several neighborhoods of suboptimal solutions and maintain the variability in the solutions.This, in turn, could prevent premature convergence.

LITERATURE REVIEW
Premature convergence has also been treated using Multi-Swarm Approach (MSA) (Brits et al., 2007;Zhao et al., 2008).Multi-swarm approach is based on the use of multiple sub-swarms instead of one standard swarm.The general approach in multi-swarm optimization is that each sub-swarm focuses on a specific region while a specific diversification method decides where and when to start the sub-swarms.The idea of MSA start with the implementation of parallel Genetic Algorithm called island model, whereby each population is located into a processor (island) with their own independent evolution process.In order to promote cooperation between islands, a new operator, called migration is created.According to some predefined strategy, individuals migrate from one island to another.The island model has been applied in various applications such as multiprocessor scheduling problem (Corcoran and Wainwright, 1994), surveillance tests policy optimization (Pereira and Lapa, 2003), airline crew scheduling (Levine, 1996), multi-track music segmentation (Rafael et al., 2013) and many others.Overall, implementation of the island model in previous studies have reported better search performance as compared to single population model.
Like-wise, multi-swarm approaches have shown similar outstanding results as in Muhlenbein (1991), Niu et al. (2005), Brits et al. (2007) and Marinakis and Marinaki (2010).The research began with application in PSO called the multi-swarm, which was originally presented by Parsopoulos and Vrahatis (2002) and later developed by Blackwell and Branke (2004).The main idea is to extend the single population in PSO into charged particle swarm optimization methods by constructing interacting multi-swarms.Niu et al. (2005) introduced a master-slave model as inspired by the phenomenon of symbiosis in natural ecosystem.In a master-slave model, the population consists of one master swarm and several slave swarms.The slave swarms execute PSO or its variants independently to maintain the diversity of particles, while the master swarm enhances its particles based on its own knowledge and also the knowledge of the particles in the slave swarms.
Several variations of multi-swarm PSO have been employed across different domains such as dynamic optimization (Blackwell and Branke, 2004;Liang and Suganthan, 2006;Liu et al., 2012) and benchmarked function (Niu et al., 2005;Liang and Suganthan, 2006).The reported results for these studies demonstrate superior performance of multi-swarm PSO as compared to the single-swarm PSO.Nonetheless, in the literature, multi-swarm algorithm has only been applied to the problem of feature selection by Liu et al. (2011), which is the Multi-Swarm PSO.To date, multi-Swarm strategy has not been considered for Bat Algorithm yet, hence motivating this research to develop a multiswarm BA that is capable avoiding premature convergence phenomenon in addition to improving exploration and exploitation ability of BA.

METHODOLOGY
Proposed multi-swarm bat algorithm: Swarm algorithms are founded based on the theory that similar animals aggregate together, generally cruising in the same direction.For instance, termites swarm to build colonies, birds swarm to migrate and bees swarm to collect food.Multi-swarm algorithms are based on the use of multiple sub-swarms instead of one swarm.In general, each sub-swarm will focus on specific region, therefore each swarm will learn from its experience in searching for the global optima.Swarm algorithms are suitable for problems where multiple local optima exist.In the BA, the searching space for promising areas is identified by the loudness and the pulse rate in bats.So in some cases the bats may mislead, because they start to slow down the speed assuming they are reaching global optimal solution area while in reality they are not.Therefore the bats may get stuck in local optima, this behavior of bats will result early convergence to non-optimal solution.
For applying single swarm or stander BA in feature selection problem it can be referred to Taha and Tang (2013) and Taha et al. (2013).In this study, implementation of the MSBA starts with single swarm that consists of a group of M bats looking for global optima.When the best solution cannot be improved after certain number of iterations, all the bats will be reinitialized randomly except the one with the best solution which will remain for further exploitation.The bats will then be grouped into S new and independent sub-swarms.
All swarms are autonomous as they have their own independent evolution process, such as the global best and the average loudness.The new swarms will discover different region of the search space looking for better solutions.If any of the sub-swarm finds a new solution better than the previous global best solutions from other swarms, the best swarm will call bats from other swarms.In this case all bats will migrate to the swarm with global best solution, each with their own loudness, pulse rates and velocity.Traveling from the old positions to the new potential positions facilitate exploration of new areas in the search space because the bats communicate with the new global best.Furthermore, crossing in the features between the new global best and the old global best provide unique solutions that single population cannot provide.If any bat from the immigrants' bats finds a new solution better than the previous potential swarm solution, the new solution can be considered as the new potential swarm and the process continues as illustrated in Fig. 1 with the union and successive movements.
After the new global optima have been recognized, all bats will facilitate more exploitation in the proposed new swarm.In the case where even the multi-swarm search fails to improve the global best solution after T iterations, then all the bats will be re-initialized except the one with global best solution.The search will then continue in form of single-swarm.The switching between single-swarm and multi-swarm search will continue until the stopping criteria are met.
The destroying and reinitialize sub-swarms serves to enhance the diversification or exploration in the search space and avoid premature convergence, while calls sub-swarms to grouping in single-swarm search

EXPERIMENTAL RESULTS
In order to demonstrate the effectiveness algorithm, a series of experiments were carried out on twelve  Reduction (GD-RSAR) (Abdullah and Jaddi, 2010) • Bat Algorithm for Attribute Reduction (BAAR) (Taha and Tang, 2013) In this study MSBA is based on rough set theory similar to the above mentioned algorithms, the experimental setups were also the same as provided in the literature.For each dataset, MSBA was run 20 times with different initial solutions and terminated after 250 iterations.All algorithms have the same number of runs for each dataset, except the results of SimRSAR using Heart, Vote and Drem2 datasets, for which the number of runs are 30, 30 and 10, respectively.Comparison on the number of features is summarized in Table 2 and 3, note that between the brackets is indicates the total number of runs that this cardinality was achieved.The number of features without brackets denotes that the method could obtain this number of features for all runs.

DISCUSSION
The experimental results showed that MSBA has a superior performance in eleven out of twelve datasets.This implies that the proposed algorithm is able to find the new best known solutions that the other algorithms fail to search.In the Exactly2 dataset, MSBA found new three best known solutions with slightly higher classification accuracy in two classifiers, which are JRip and J48.MSBA is also significantly better than the single swarm BA as well as other algorithms as shown in Table 4 and 5.In the Mushroom dataset, even though the classification accuracy achieved by MSBA is the same with other algorithms, but MSBA has the advantage of new found solutions and it is statistically better than GenRSAR and GD-RSAR.
In the Led dataset, MSBA achieved 100% classification accuracy aside from obtaining a new best known solution.In the Credit dataset, it can be noted that MSBA obtained the new best known solution with seven features in eight runs out of twenty runs.It can also be noted that MSBA is significantly better than all other algorithms.For the Credit dataset, all algorithms provided almost the same classification accuracy in all three classifiers.In the Vote dataset, MSBA found three new best known solutions with the same classification accuracy as other classifiers.In particular, the results for MSBA are significantly better than BAAR and other traditional methods.
The Derm2 and WQ dataset are considered as the most challenging datasets as no algorithm is able to obtain best solution for all runs as can be seen in Table 2 and 3. Nonetheless, MSBA still performed significantly better than all others algorithm with the same accuracy in the WQ dataset.Meanwhile, in the Exactly and Derm datasets, although the classification accuracy achieved by MSBA decreased moderately as compared to other method, but MSBA was still able to find new best solutions.For the Heart dataset, the results provided by MSBA with regards to the classification accuracy were the same with those delivered by BAAR but lower than other methods.Finally in the Lung dataset, MSBA could not find any new best known and the classification accuracy solution was also lower than average of the other algorithm results.Nevertheless, the proposed MSBA is still significantly better than BAAR and the other four others method (GenRSAR, AntRSAR, TSAR and GD-RSAR).
As the conclusion, the proposed MSBA is significantly better than other single-swarm algorithms in eight out of twelve datasets.It also outperformed other algorithms in most of the datasets.MSBA is able obtain the lowest number of features across the most data sets and the results also showed that MSBA is able to find new best solutions while maintaining a competitive classification accuracy.Nevertheless, in few cases, the accuracy results obtained by MSBA are less satisfactory than those excellent ones reported by rest three methods which are BAAR, SimRSAR and GenRSAR.This is attributed to the use of rough set theory, whereby MSBA has less sensitivity to the changes in the prediction accuracy.The superior performance of MSBA and the ability to find new solution is influenced by the multi-swarm movement in discovering new areas within the search space, beyond the reach of other classical algorithms.

CONCLUSION
MSBA is proposed to address the problem of premature convergence phenomenon that usually occurs in BA.The problem happens when search process converges to non-optimal solution due to the loss of diversity during the evolution process.MSBA was designed with improved ability in exploring new solutions, which was essential in reducing premature convergence.In MSBA, when the quality of best local optima does not improve after a pre-defined number of iterations, the population is split equally into several smaller sub-swarms, with one of them remains close to the current best local optima for further exploitation while the other sub-swarms continue to explore for new local optima.A series of experiments were carried out and the results of MSBA were compared against the eight conventional methods.MSBA achieved very good results, whereby it produced new best solutions in 11 out of 12 cases.Furthermore, the statistical analysis showed that the proposed method in most data sets performed significantly better than the other methods in terms of selecting smaller subset of features with providing competitive prediction accuracy.For future work is highly recommended to extensive study of the applications of proposed Multi-Swarm BA model in other complex practical optimization problems is necessary, to fully investigate the properties, evaluate and optimize MSBA performance.

Fig. 1 :
Fig. 1: Multi-swarm movement aims to enhance the local search or exploitation.The stopping criteria that have been set in this study are as follows; whether the algorithm reached the maximum number of iteration or single feature with maximum number of fitness function, the pseudo code is provided in below.
. The parameters is set as follow S equal to 4, M equal to 50, N equal to 30, T equal to 60.The experimental results were compared against seven well-known optimization algorithms as listed below: • Genetic algorithm for Rough Set Attribute Reduction (GenRSAR) (Jensen and Shen, 2003) • Ant colony optimization for Rough Set Attribute Reduction (AntRSAR) (Jensen and Shen, 2003) • Simulated annealing for Rough Set Attribute Reduction (SimRSAR) (Jensen and Shen, 2004) • Tabu Search for Attribute Reduction (TSAR) (Hedar et al., 2008) • Scatter Search for Attribute Reduction (SSAR) (Jue et al., 2009) • Scatter Search for Attribute Reduction (SSAR) (Jue et al., 2009) • Great Deluge algorithm for Rough Set Attribute

Table 1 :
Characteristics of datasets Each dataset has different number of features and samples as shown in Table1

Table 2 :
Number of features comparison (a)

Table 6 :
Average classification accuracy using JRip classifier