Improved teaching learning based optimization for global function optimization

Teaching–Learning-Based Optimization (TLBO) is recently being used as a new, reliable, accurate and robust optimization technique scheme for global optimization over continuous spaces. This paper presents an improved variant of TLBO algorithm, called Improved Teaching–Learning-Based Optimization (ITLBO). A performance comparison of the proposed method is provided against the original TLBO and some other algorithms. The improved TLBO algorithm shows a marked improvement in performance over the traditional TLBO on several benchmark optimization problems.


Introduction
Constrained and unconstrained optimization problems are generally associated with many difficulties such as multi-modality, dimensionality and differentiability.Traditional optimization techniques generally fail to solve such problems, especially with nonlinear objective functions.To overcome these difficulties, there is a need to develop more powerful optimization techniques.Rao et al. (2011Rao et al. ( , 2012) ) proposed a teaching-learning based optimization (TLBO) algorithm based on the natural phenomenon of teaching and learning.The implementation of TLBO does not require the determination of any algorithm specific controlling parameters, which makes the algorithm robust and powerful.TLBO requires only common controlling parameters like population size and number of generations for its working.In this way, TLBO can be said as an algorithm specific parameter-less algorithm.Rao and Patel (2012) investigated the performance of TLBO algorithm for different elite sizes, population sizes and number of generations considering various constrained bench mark problems available in the literature to identify their effect on the exploration and exploitation capacity of the algorithm.Satapathy and Naik (2011) tried to propose a new approach to use TLBO to cluster data.They have proved that the TLBO algorithm can be used to cluster arbitrary data.Again they have shown another good characteristic of TLBO algorithm, that the cost of computations are less in handling high dimensional problems in compare to other algorithms (Chandra Satapathy et al., 2012).TLBO also has been applied to the area of computer network by optimizing the multicast tree (Naik et al., 2012).It is capable of handling power system problem easily where optimal solution of the unit maintenance scheduling problem in which the cost reduction is as important as reliability (Satapathy et al., 2012).TLBO algorithm has been adapted to solve multi objective problems of an economic load dispatch problem with incommensurable objectives (Krishnanand et al., 2011).From literature, we can see while clustering the data using fuzzy c-means (FCM) and hard c-means (HCM), the sensitivity to tune the initial clusters centers have captured the attention of the clustering communities for quite a long time.This problem has been addressed by TLBO in (Naik et al., 2012).In the area of feature selection, also TLBO is showing its performance in connection with Rough set theory.Empirical results reveal that the Rough TLBO approach could perform better in terms of finding optimal features and doing so in quick time in comparison with GA, PSO and DE (Naik et al., 2012).Again there is some modification of TLBO have been done to improve its performance in optimization for global function optimization (Chandra Satapathy et al., 2012).There are good numbers of applications of TLBO from different papers (Rao & Patel, 2012;Rao & Savsani, 2012;Vedat Toğan, 2012;Rao & Kalyankar, 2012).Looking into the diverse applications of TLBO and effectiveness of TLBO, in this paper, we present yet another modification on TLBO to improve its performance in terms of convergence speed while solving unconstrained optimization problem.Our proposed improved TLBO (ITLBO) is compared with various other algorithms like PSO, DE, TLBO, OEA, HPSO-TVAC, CLPSO and APSO on several benchmark functions are the performance characteristics are provided to show that ITLBO performs better than all other compared techniques.
The remainder of this paper is organized as follows.The conventional TLBO is explained in detail in Sections 2, ITLBO in section 3. Numerical Experiment and results demonstrating the performance of ITLBO in comparison with other optimization algorithm are presented in Section 4. Section 5 concludes this paper.

Teaching-learning-based optimization
This optimization method is based on the effect of the influence of a teacher on the output of learners in a class.It is a population based method and like other population based methods, it uses a population of solutions to proceed to the global solution.A group of learners constitutes the population in TLBO.In any optimization algorithms, there are numbers of different design variables.Different design variables in TLBO are analogous to various subjects offered to learners and the learners' result is analogous to the 'fitness', as in other population-based optimization techniques.As the teacher is considered the most learned person in the society, the best solution so far is analogous to Teacher in TLBO.The process of TLBO is divided into two parts.The first part consists of the 'Teacher Phase' and the second part consists of the 'Learner Phase'.The 'Teacher Phase' means learning from the teacher and the 'Learner Phase' means learning through the interaction between learners.In the sub-sections below, we briefly discuss the implementation of TLBO.

A Initialization
Following are the notations used for describing the TLBO: : number of learners in a class i. e. "class size" : number of courses offered to the learners : maximum number of allowable iterations The population is randomly initialized by a search space bounded by matrix of rows and columns.The ℎ parameter of the ℎ learner is assigned values randomly using the equation where rand represents a uniformly distributed random variable within the range (0, 1), and represent the minimum and maximum value for ℎ parameter.The parameters of ℎ learner for the generation g are given by ( ) = [ ( , ) , ( , ) , ( , ) , … … , ( , ) , … … , ( , ) ] (2)

B Teacher Phase
The mean parameter of each subject of the learners in the class at generation is given as The learner with the minimum objective function value is considered as the teacher for respective iteration.The Teacher phase makes the algorithm proceed by shifting the mean of the learners towards its teacher.To obtain a new set of improved learners a random weighted differential vector is formed from the current mean and the desired mean parameters and added to the existing population of learners.
is the teaching factor, which decides the value of mean to be changed.Value of can be either 1 or 2. The value of is decided randomly with equal probability as, where is not a parameter of the TLBO algorithm.The value of is not given as an input to the algorithm and its value is randomly decided by the algorithm using Eq. ( 5).After conducting a number of experiments on many benchmark functions it is concluded that the algorithm performs better if the value of is between 1 and 2. However, the algorithm is found to perform much better if the value of is either 1 or 2 and hence to simplify the algorithm, the teaching factor is suggested to take either 1 or 2 depending on the rounding up criteria given by Eq.(5).If ( ) is found to be a superior learner than ( ) in generation , than it replaces inferior learner ( ) in the matrix.

C Learner Phase
In this phase, the interaction of learners with one another takes place.The process of mutual interaction tends to increase the knowledge of the learner.The random interaction among learners improves his or her knowledge.For a given learner ( ) , another learner ( ) is randomly selected( ≠ ).The ℎ parameter of the matrix in the learner phase is given as

D Algorithm Termination
The algorithm is terminated after iterations are completed.Details of TLBO can be refereed in (Rao et al., 2011).

Improved Teaching-Learning-Based Optimizer (ITLBO)
In the traditional TLBO (Rao et al., 2011), the teacher phase makes the algorithm proceed by shifting the mean of the learners towards its teacher.
To obtain a new set of improved learners a random weighted differential vector is formed from the current mean and the desired mean parameters and added to the existing population of learners.Similarly, in the learner phase the algorithm proceeds by random interaction among learners improve his or her knowledge.To obtain a new set of improved learners a random weighted differential vector is formed from a given learner ( ) , another learner ( ) is randomly selected( ≠ ) and added to the existing learner.In our proposed algorithm we propose to vary this random weighed differential vector in a random manner in the range (0.5, 1) by using the relation 0.5*(1+rand (0, 1)), (7) where rand (0, 1) is a uniformly distributed random number within the range [0, 1].Therefore, the mean value of this weighted differential scale factor is 0.75.This allows for stochastic variations in the amplification of the difference vector and thus helps retain population diversity as the search progresses.Even when the tips of most of the population vectors point to locations clustered near a local optimum due to the randomly scaled difference vector, a new trial vector has fair chances of pointing at an even better location on the multimodal functional surface.Therefore, the fitness of the best vector in a population is much less likely to get stagnant until a truly global optimum is reached.So the new set of improved learners can be made by using equation in the teacher phase So the new set of improved learners can be made by using equation in the learner phase

Settings
For all experiments performed in this section, the values of the common parameters used in each algorithm such as population size and total evaluation number are chosen to be the same.Population size is 20 and the maximum number fitness function evaluation is 40,000 for all functions.The other specific parameters of algorithms are given below: PSO Settings: Cognitive and social components , are constants that can be used to change the weighting between personal and population experience, respectively.In our experiments cognitive and social components were both set to 2. Inertia weight, which determines how the previous velocity of the particle influences the velocity in the next iteration, was 0.5.

DE Settings:
In DE, F is a real constant which affects the differential variation between two Solutions and set to F = 0.5*(1+ rand (0, 1)) where rand (0, 1) is a uniformly distributed random number within the range [0, 1] in our experiments.Value of crossover rate, which controls the change of the diversity of the population, was chosen to be R = ( -) * ( -) / where = 1 and =0.5 are the maximum and minimum values of scale factor R, is the current iteration number and is the maximum number of allowable iterations as recommended in (Swagatam Das et al., 2008).
TLBO Settings: For TLBO there is no such constant to set.

ITLBO Settings:
For ITLBO there is no such constant to set.

Benchmark functions
We used 20 benchmark problems in order to test the performance of the PSO, DE, TLBO and the ITLBO algorithms.This set is large enough to include many different kinds of problems such as unimodal, multimodal, regular, irregular, separable, non-separable and multidimensional.Initial range, formulation, characteristics and the dimensions of these problems are listed in Table 1.

Results
In Experiments 1, we compared the PSO, DE, TLBO and ITLBO algorithms on a large set of functions described in the previous section and are listed in Tables 1.Each of the experiments in this section was repeated 30 times with different random seeds and they were terminated when they reached the maximum number of function evaluations or when they reached the global minimum value.Mean and standard deviation of fitness values on 30 different run produced by the algorithms have been recorded in the table 2 and at the same time mean value and standard deviation of no of fitness evaluation produced by the algorithms have been recorded in the Table 3.
In order to analyze the results whether there is significance between the results of each algorithm, we performed t-test on pairs of algorithms which is quite popular among researchers in evolutionary computing (Swagatam Das et al., 2009).In the table 4 we report the statistical significance level of difference of the means of PSO and ITLBO algorithm, DE and ITLBO algorithm, TLBO and ITLBO algorithm.Note that here '+' indicates the t value is significant at a 0.05 level of significance by twotailed test, '•' means the difference of means is not statistically significant and 'NA' stands for Not Applicable, covering cases for which the two algorithms achieve the same accuracy results.From the Table 4, we understand that in 15 cases ITLBO performs better than PSO, where as in 14 cases ITLBO performs better than DE and in 8 cases ITLBO performs better than TLBO.To show convergence nature of each algorithm corresponding to each function we have made graph for each function.We have drawn two different graphs for each function.In the first graph we have shown convergence nature of all four algorithms where as in second graph we have shown the convergence nature of TLBO and ITLBO, as ITLBO is modification of TLBO to improve convergence characteristic.From Figs 1-20, it is clear that in almost all cases ITLBO have faster convergence than others.

Experiments 2: ITLBO vs. OEA, HPSO-TVAC, CLPSO and APSO
The experiments in this section constitute the comparison of the ITLBO algorithm versus OEA, HPSO-TVAC, CLPSO and APSO on 8 benchmarks function described in Table 1.The experimental result for OEA, HPSO-TVAC, CLPSO and APSO are gained from (Zhan et al., 2009;Ratnaweera et al., 2004) directly , where OEA uses the number of 3.0 × 10 FEs and HPSO-TVAC, CLPSO and APSO use the number of 2.0 × 10 FEs, where as ITLBO runs for 4.0 × 10 FEs.In the last column of table 5 shows the significance level between best and second best algorithm using t test at a 0.05 level of significance by two tailed test.Note that here '+' indicates the t value is significant ,'.' means the difference of means is not statistical significant and 'NA' stands for Not applicable, covering cases for which the two algorithms achieve the same accuracy results.As can be seen from table 5, ITLBO greatly outperforms OEA, HPSO-TVAC, CLPSO and APSO with better mean and standard deviation.The experiments in this section constitute the comparison of the ITLBO algorithm versus CABC ( Alatas et al., 2010), GABC (Zhu et al., 2010), RABC (Kang et al., 2011) and IABC (Weifeng Gao et al., 2011) on 8 benchmark functions which are described in table 1 .The parameters of the algorithms are identical to (Kang et al., 2011).In the last column of table 7 shows the significance level between best and second best algorithm using t test at a 0.05 level of significance by two tailed test.Note that here '+' indicates the t value is significant ,'.' means the difference of means is not statistical significant and 'NA' stands for Not applicable, covering cases for which the two algorithms achieve the same accuracy results The results, which have been summarized in Table 7, show that ITLBO performs much better in most cases than these ABC variants.The t-test results also clearly indicate that the difference between the ITLBO algorithm and the other algorithms is statistically significant in most cases.Therefore, it is evident that the ITLBO algorithm has a good performance.

Conclusion
In this paper, we have developed a novel algorithm, ITLBO, to solve global numerical optimization problems by introducing a new search mechanism.We testify the performance of the proposed approach on a numbers of benchmark functions and provide comparisons with some other algorithms.
The results show that the ITLBO algorithm possesses superior performance in accuracy, convergence, speed, stability and robustness, as compared to the other algorithms.Hence, the ITLBO algorithm may be a good alternative to deal with complex numerical optimization problems.Practical applications of the proposed approach in areas of clustering, data mining, design and optimization of communication networks, would also be worth studying.

Table 3
No. of fitness evaluation comparisons of PSO, DE, TLBO, ITLBO (values in brackets in the table are the absolute values of FEs)

Table 4 t
value , significant at a 0.05 level of significance by two tailed test using table 2.

Table 5
Zhan et al., 2009 )son for ITLBO,OEA,HPSO-TVAC,CLPSO and APSO The experiments in this section constitute the comparison of the ITLBO algorithm versus SaDE, jDE and JADE on 8 benchmark functions which are describe in Table1.The results of JADE, jDE and SaDE are gained from (Z.H.Zhan et al., 2009 )directly.In the last column of Table6shows the significance level between best and second best algorithm using t test at a 0.05 level of significance by two tailed test.Note that here '+' indicates the t value is significant,'.' means the difference of means is not statistical significant and 'NA' stands for Not applicable, covering cases for which the two algorithms achieve the same accuracy results.It can be seen from Table6that ITLBO performs much better than these DE variants on almost all the functions .