Comparative performance of an elitist teaching-learning-based optimization algorithm for solving unconstrained optimization problems

Article history: Received 11 July 2012 Received in revised format 14 August 2012 Accepted August 31 2012 Available online 1 September 2012 Teaching-Learning-based optimization (TLBO) is a recently proposed population based algorithm, which simulates the teaching-learning process of the class room. This algorithm requires only the common control parameters and does not require any algorithm-specific control parameters. In this paper, the effect of elitism on the performance of the TLBO algorithm is investigated while solving unconstrained benchmark problems. The effects of common control parameters such as the population size and the number of generations on the performance of the algorithm are also investigated. The proposed algorithm is tested on 76 unconstrained benchmark functions with different characteristics and the performance of the algorithm is compared with that of other well known optimization algorithms. A statistical test is also performed to investigate the results obtained using different algorithms. The results have proved the effectiveness of the proposed elitist TLBO algorithm. © 2012 Growing Science Ltd. All rights reserved


Introduction
Some of the recognized evolutionary algorithms are, Genetic Algorithms (GA), Differential Evolution (DE), Evolution Strategy (ES), Evolution Programming (EP), Artificial Immune Algorithm (AIA), Bacteria Foraging Optimization (BFO), etc.Among all, GA is a widely used algorithm for various applications.GA works on the principle of the Darwinian theory of the survival of the fittest and the theory of evolution of the living beings (Holland 1975).DE is similar to GA with specialized crossover and selection method (Storn & Price 1997, Price et al. 2005).ES is based on the hypothesis that during the biological evolution the laws of heredity have been developed for fastest phylogenetic adaptation (Runarsson & Yao, 2000).ES imitates, in contrast to the GA, the effects of genetic procedures on the phenotype.EP also simulates the phenomenon of natural evolution at phenotype level (Fogel et al. 1966).AIA works on the immune system of the human being (Farmer 1986).BFO is inspired by the social foraging behavior of Escherichia coli (Passino, 2002).Some of the well known swarm intelligence based algorithms are, Particle Swarm Optimization (PSO) which works on the principle of foraging behavior of the swarm of birds (Kennedy & Eberhart 1995); Ant Colony Optimization (ACO) which works on the principle of foraging behavior of the ant for the food (Dorigo et al. 1991); Shuffled Frog Leaping (SFL) algorithm which works on the principle of communication among the frogs (Eusuff & Lansey, 2003); Artificial Bee Colony (ABC) algorithm which works on the principle of foraging behavior of a honey bee (Karaboga, 2005;Basturk & Karaboga 2006;Karboga & Basturk, 2007-2008;Karaboga & Akay 2009).
There are some other algorithms which work on the principles of different natural phenomena.Some of them are: Harmony Search (HS) algorithm which works on the principle of music improvisation in a music player (Geem et al. 2001); Gravitational Search Algorithm (GSA) which works on the principle of gravitational force acting between the bodies (Rashedi et al. 2009); Biogeography-Based Optimization (BBO) which works on the principle of immigration and emigration of the species from one place to the other (Simon, 2008); Grenade Explosion Method (GEM) which works on the principle of explosion of grenade (Ahrari & Atai, 2010); and League championship algorithm which mimics the sporting competition in a sport league (Kashan, 2011).
All the evolutionary and swarm intelligence based algorithms are probabilistic algorithms and required common controlling parameters like population size and number of generations.Beside the common control parameters, different algorithm requires its own algorithm specific control parameters.For example GA uses mutation rate and crossover rate.Similarly PSO uses inertia weight, social and cognitive parameters.The proper tuning of the algorithm specific parameters is very crucial factor which affect the performance of the above mentioned algorithms.The improper tuning of algorithmspecific parameters either increases the computational effort or yields the local optimal solution.Considering this fact, recently Rao et al. (2011Rao et al. ( , 2012aRao et al. ( , 2012b) ) and Rao and Patel (2012a) introduced the Teaching-Learning-Based Optimization (TLBO) algorithm which does not require any algorithmspecific parameters.TLBO requires only common controlling parameters like population size and number of generations for its working.Thus, TLBO can be said as an algorithm-specific parameter-less algorithm.
The concept of elitism is utilized in most of the evolutionary and swarm intelligence algorithms where during every generation the worst solutions are replaced by the elite solutions.The number of worst solutions replaced by the elite solutions is depended on the size of elite.Rao and Patel (2012a) described the elitism concept while solving the constrained benchmark problems.The same methodology is extended in the present work and the performance of TLBO algorithm is investigated considering a number of unconstrained benchmark problems.The details of TLBO algorithm along with its computer program are available in Rao and Patel (2012a) and hence those details are not repeated in this paper.

Elitist TLBO algorithm
In the TLBO algorithm, after replacing worst solutions with elite solutions at the end of learner phase, if the duplicate solutions exist then it is necessary to modify the duplicate solutions in order to avoid trapping in the local optima.In the present work duplicate solutions are modified by mutation on randomly selected dimensions of the duplicate solutions before executing the next generation as was done in Rao and Patel (2012a).In the TLBO algorithm, the solution is updated in the teacher phase as well as in the learner phase.Also, in the duplicate elimination step, if duplicate solutions are present then they are randomly modified.So the total number of function evaluations in the TLBO algorithm is = {(2 × population size × number of generations) + (function evaluations required for duplicate elimination)}.In the entire experimental work of this paper, the above formula is used to count the number of function evaluations while conducting experiments with TLBO algorithm.Since the function evaluations required for duplication removal are not clearly known, experiments are conducted with different population sizes and based on these experiments it is reasonably concluded that the function evaluations required for the duplication removal are 7500, 15000, 22500 and 30000 for population sizes of 25, 50, 75 and 100 respectively when the maximum function evaluations of the algorithm is 500000.The next section deals with the experimentation of the elitist TLBO algorithm on various unconstrained benchmark functions.

Experiments on unconstrained benchmark functions
The considered unconstrained benchmark functions have different characteristics like unimodality/multimodality, separability/non-separability and regularity/non-regularity. In this section three different experiments are conducted to identify the performance of TLBO and compare the performance of TLBO algorithm with other evolutionary and swarm intelligence based algorithms.A common platform is provided by maintaining the identical function evolution for different algorithms considered for the comparison.Thus, the consistency in the comparison is maintained while comparing the performance of TLBO with other optimization algorithms.However, in general, the algorithm which requires less number of function evaluations to get the same best solution can be considered as better as compared to the other algorithms.If an algorithm gives global optimum solution within certain number of function evaluations, then consideration of more number of function evaluations will go on giving the same best result.Rao et al. (2011Rao et al. ( , 2012a) ) showed that TLBO requires less number of function evaluations as compared to the other optimization algorithms.However, in this paper, to maintain the consistency in comparison, the number of function evaluations of 500000, 100000 and 5000 is maintained for experiments 1, 2 and 3 respectively for all optimization algorithms including TLBO algorithm.

Experiment 1
In the first experiment, the TLBO algorithm is implemented on 50 unconstrained benchmark functions taken from the previous work of Karaboga and Akay (2009).The details of the benchmark functions considered in this experiment are shown in Table 1.
In each run the maximum function evaluations is set as 500000 for all the functions for fair comparison purpose and the results obtained using the TLBO algorithm are compared with the results given by other well known optimization algorithms for the same number of function evaluations.Moreover, in order to identify the effect of population size on the performance of the algorithm, the algorithm is experimented with different population sizes viz.25, 50, 75 and 100 with number of generations of 9850, 4850, 3183 and 2350 respectively so that the function evaluations in each strategy is 500000.Similarly, to identify the effect of elite size on the performance of the algorithm, the algorithm is experimented with different elite sizes, viz.0, 4, and 8.Here elite size 0 indicates no elitism consideration.The results of each benchmark function are presented in Table 2 in the form of best solution, worst solution, average solution and standard deviation obtained in 30 independent runs on each benchmark function along with the corresponding strategy (i.e.population size and elite size).Karaboga and Akay (2009) where the authors had experimented benchmark functions each with 500000 function evaluations with best setting of algorithm specific parameters.Table 3 shows the comparative results of the considered algorithm in the form of mean solution (M), standard deviation (SD) and standard error of mean (SEM).In order to maintain the consistency in the comparison the values bellow 10 -12 are assumed to be 0 as considered in the previous work of Karaboga and Akay (2009).It is observed from        It is observed from Table 4 that for 28 functions TLBO is better than GA and on two functions GA is better than TLBO while for remaining 20 functions both the algorithms showed the equal performance.
From Table 5, on 29 functions there is no significance difference between PSO and TLBO but on 20 functions TLBO is better than PSO while on one function PSO is better than TLBO.From Table 6, on 7 functions TLBO performed better than DE while on 4 functions DE is better than TLBO.On remaining 39 functions there is no significance difference between DE and TLBO.From Table 7, on 34 functions TLBO and ABC showed equal performance.On 11 functions, TLBO performed better than ABC while ABC performs better than TLBO on 5 functions.

Experiment 2
In this section, the performance of TLBO is compared with the different evolutionary algorithms like Canonical evolution strategies (CES), Fast evolution strategies (FES), Covariance matrix adaptation evolution strategies (CMA-ES) and Evolution strategies learned with automatic termination (ESLAT) along with the swarm intelligence based algorithm ABC.In this experiment the TLBO algorithm is implemented on 23 unconstrained benchmark functions taken from the previous work of Karaboga and Akay (2009).The details of the benchmark functions considered in this experiment are shown in Table 8.For the considered test problems, the TLBO algorithm is run for 50 times for each benchmark function.To maintain the consistency in the comparison between TLBO and other algorithms, in each run the algorithm is terminated when it has completed 100000 function evaluations or when it reached the global minima within the gap of 10 -3 .The results obtained using the TLBO algorithm are compared with the results obtained by other well known optimization algorithms for the same termination criteria.
Here also the TLBO algorithm is implemented with different combinations of population size, number of generation and elite size.After conducting experiments with different population sizes the function evaluations required for duplication removal considered are 2500, 5000, 7500 and 10000 for population sizes of 25, 50, 75 and 100 respectively when the maximum function evaluations of the algorithm is 100000.Table 9 shows the best results obtained using the TLBO algorithm along with its corresponding strategy.Comparative results of all the considered algorithms in the form of mean solution and standard deviation are shown in Table 10.Except TLBO, results of other algorithms are taken from the previous work of Karaboga and Akay (2009), Yao and Liu (1997) and Hedar and Fukushima (2006).The computational effort of all the considered algorithms in the form of mean number of function evaluations is shown in Table 11.Here the mean number of function evaluation indicates the function evaluations required to obtain global best solution within the gap of 10 -3 averaged over 30 independent runs.If for any function, the global best solution is not obtained in this precision then solution obtained in the last cycle is recorded.It is observed from the results that on 14 functions the computational effort of TLBO is less than the rest of the considered algorithms i.e. the convergence of TLBO is faster than rest of the algorithms.On 3 functions, ABC required minimum computational effort than the other algorithms.On 5 functions, CMA-ES and on 1 function ESLAT required less number of function evaluations to achieve the global best solution than rest of the considered algorithms.The success rate of all the algorithms for the considered benchmark functions are shown in Table 12.
It is observed from the results that ESLAT and CMA-ES achieved the best success rate on 9 functions while ABC and TLBO algorithms achieved the best success rate on 19 functions.On Schwefel and Hartman 6 functions ABC achieved higher success rate than TLBO while on Schwefel 2.21, Griewank, Shekel 5 and Shekel 10 functions success rate of TLBO is better than ABC.On Rosenbrock function success rate of CMA-ES is better than rest of the considered algorithms.

Experiment 3
In this section, the computational effort and consistency of the TLBO algorithm is compared with Selforganizing maps evolution strategy (SOM-ES), Neural gas networks evolution strategy (NG-ES), CMA-ES and ABC algorithms.In this experiment the TLBO algorithm is implemented on 3 unconstrained benchmark functions taken from the previous work of Karaboga and Akay (2009).The details of the benchmark functions considered in this experiment are shown in Table 13.

Table 13
Benchmark functions considered in experiment 3 No. Function Formulation .12, 5.12] MS For the considered test problems, the TLBO algorithm is run for 10000 times for each benchmark function.In each run the maximum function evaluation is set as 5000 per test function.To maintain the consistency in the comparison, the limiting value of the satisfactory convergence is set as 40, 0.001 and 0.001 for functions 1, 2 and 3 respectively (Karaboga andAkay 2009, Milano et al. 2004).Here also the TLBO algorithm is implemented with different combinations of population size, number of generation and elite size and the strategy which produced the best results is considered for the comparison.
Comparative results of all the considered algorithms in the form of mean and standard deviation and success rate are shown in Table 14.Except TLBO, results of other algorithms are taken from the previous work of Karaboga and Akay (2009).It is observed from the results that SOM-ES produced better convergence rate and success rate on modified Rosenbrock function than the other algorithms.On Griewank function, the TLBO produced better success rate though the convergence of the TLBO is slower than the other considered algorithm.On Rastrigin function, the success rate of ABC and TLBO are equally good but the convergence of both the algorithms is slower than the other algorithms.The TLBO algorithm has already been successfully applied by various researchers for solving complex benchmark functions and difficult engineering problems (Azizipanah-Abarghooee et al. 2012, Hosseinpour et al. 2011, Krishnanand et al. 2011, Nayak et al. 2011, Niknam et al. 2012a, 2012b, 2012c, Rao and Kalyankar 2012a, 2012b, 2012c, Rao and Savsani 2012, Rao and Patel 2012a;2012b;2012c;2012d, Satapathy and Naik 2011, Satapathy et al. 2012, Toğan 2012).Contrary to the opinion expressed by Črepinšek et al. (2012) that TLBO is not a parameter-less algorithm, this paper has clearly explained that TLBO is an algorithm-specific parameter-less algorithm and this was already stated by Rao and Patel (2012a).Common control parameters are common to run any of the optimization algorithms and algorithm-specific parameters are specific to the algorithm and different algorithms have different specific parameters to control.The TLBO algorithm does not have any algorithm-specific parameters to control and it requires only the control of the common control parameters like population size, number of generations and elite sizes.In fact, many of the comments made by Črepinšek et al. (2012) about the TLBO algorithm were already addressed by Rao and Patel (2012a).

Conclusion
The tuning of the common controlling parameters such as population size and number of generations is one of the important factors in any probabilistic algorithm.In addition to this, evolutionary and swarm intelligence based algorithms require proper tuning of algorithm-specific parameters.A change in the tuning of the algorithm-specific parameters influences the effectiveness of the algorithm.The recently proposed TLBO algorithm does not require any algorithm-specific parameters.It only requires the tuning of the common controlling parameters of the algorithm for its working.
In the present work, the concept of elitism is introduced in the TLBO algorithm and its effect on the performance of the algorithm for the unconstrained optimization problems is investigated.Furthermore, the effect of common controlling parameters on the performance of TLBO algorithm is also investigated by considering different combinations of common controlling parameters.The proposed algorithm is implemented on 76 unconstrained optimization problems having different characteristics to identify the effect of elitism and common controlling parameters.The results have shown that for some functions the strategy with elitism consideration produced better results than that without elitism consideration.The results obtained by using TLBO algorithm are compared with the other optimization algorithms available in the literature for the considered benchmark problems.Results have shown the satisfactory performance of TLBO algorithm for the unconstrained optimization problems.

Table 2
Results Obtained by the TLBO algorithm for 50 bench mark functions over 30 independent runs with function evaluations , strategy with population size of 50 and number of generations of 4850 gave the best results.For functions 22, 37 and 50, strategy with population size of 75 and number of generations of 3183 and for functions 25, 26, 46 and 47 strategy with population size of 100 and number of generations of 2350 produced the best results.For function 12, strategy with population size 25, 50 and 75 while for function 9 and 14, strategy with population size 25 and 50 produced the identical results.For rest of the functions all the strategies produced the same results and hence there is no effect of population size on these functions to achieve their respective global optimum values with same number of function evaluations.Similarly, it is observed from Table2 that for functions 2-4, 12-16, 37, 38, 40, 41, 48 and 49, strategy with elite size 0, i.e. no elitism produced best results than the other strategies having different elite sizes.For functions22, 26, 42, 46, 47 and 50, strategy with elite size of 4 produced the best results.For functions 5, 23, and 25, strategy with elite size of 8 produced the best results.For function 44, strategy with elite size 4 and 8 produced the same results.For rest of the functions all the strategies (i.e.strategy without elitism consideration as well as strategies with different elite sizes consideration) produced the same results.The performance of TLBO algorithm is compared with the other well known optimization algorithms such as GA, PSO, DE and ABC.The results of GA, PSO, DE and ABC are taken from the previous work of PS = Population size, ES = Elite size, SD = Standard deviation It is observed fromTable 2 that for functions 5, 13, 15, and 38, strategy with population size of 25 and number of generations of 9850 produced the best results than the other strategies.For functions 16, 23 and 49 Table3that TLBO algorithm outperforms the GA, PSO, DE and ABC algorithms for Powell, Rosenbrock, Kowalik, Perm, and Power sum functions in every aspect of comparison criteria.For Rastrigin, Hartman 6, and Griewank functions, performance of the TLBO and ABC algorithms are alike and outperforms the GA, PSO and DE algorithms.For Shekel 5, Shekel 7, Shekel 10, Hartman 3, and Ackley functions, performance of the TLBO, DE and ABC algorithms are alike and outperforms the GA and PSO algorithms.For Colville function, performance of PSO and TLBO while for Zakharov function, performance of TLBO, DE and PSO are same and produce better results.For Stepint, Step, Sphere, Sum squares, Schwefel 2.22, Schwefel 1.2, Schaffer, Bohachevsky 2 and GoldStein-Price functions, performance of TLBO, ABC, DE and PSO are identical and produced better results than GA.For Michalewicz 2 and Langerman 2 functions, performances of TLBO, ABC, DE and GA are same and better results are produced than PSO algorithm.For Dixon-Price, Schwefel, Michalewicz 5, Michalewicz 10, FletcherPowell 5, FletcherPowell 10, Penalized and Penalized 2 functions, the results obtained using ABC algorithm are better than the rest of the considered algorithms.For Langerman 5 and Langerman 10 functions, the results obtained using DE are better than other algorithms though the results of TLBO are better than GA, PSO and ABC.Similarly for Quartic function, the PSO algorithm produced better results than rest of the algorithms though the results of TLBO are better than GA, DE and ABC.To investigate the results obtained using different algorithms more deeply, a statistical test is performed in the present work.t-test is performed on the pair of the algorithms to identify the significance difference between the results of the algorithms.In the present work Modified Bonferroni Correction is adopted while performing the t-test.For t-test, first the p-value is calculated for each function and then the p-values are ranked in ascending order.The inverse rank is obtained and then the significance level (α) is found out by dividing 0.05 level by inverse rank.For any function if obtained p value is less than the significance level then there is a significance difference between pair of the algorithms on that function.Tables4-7show the results of the statistical test.

Table 3
Comparative results of TLBO with other evolutionary algorithms over 30 independent runs

Table 3
Comparative results of TLBO with other evolutionary algorithms over 30 independent runs (Cont.)

Table 3
Comparative results of TLBO with other evolutionary algorithms over 30 independent runs (Cont.)

Table 4
Significance test for GA and TLBO

Table 5
Significance test for PSO and TLBO t: t-value of student t-test, SED: standard error of difference, p: p-value calculated for t-value, R: rank of p-value, IR: Inverse rank of p-value, Sign: Significance

Table 6
Significance test for DE and TLBO

Table 7
Significance test for ABC and TLBO t: t-value of student t-test, SED: standard error of difference, p: p-value calculated for t-value, R: rank of p-value, IR: Inverse rank of p-value, Sign: Significance

Table 9
Results Obtained by the TLBO algorithm for 23 bench mark functions over 50 independent runs

Table 11
Mean number of function evaluation (Mean FE) required by ESLAT, CMA-ES, ABC and TLBO algorithms for the benchmark functions considered in experiment 2

Table 14
Comparative results of different algorithms for the benchmark functions considered in experiment 3