A novel hybrid backtracking search optimization algorithm for continuous function optimization

. 2018 by the authors; licensee Growing Science, Canada ©


Introduction
Stochastic Optimization algorithms are effective and powerful tool for solving nonlinear complex optimization problem.Many nature based stochastic algorithms have been introduced and studied by many authors (e.g.Yang & Press, 2010).The success of an optimization algorithm depends on its significant development of exploration and exploitation abilities.The first attempt to start these types of studies is genetic algorithm (GA) (Holland, 1992) which actually employs the natural process of genetic evolution.After that various nature-inspired meta-heuristic approaches have been proposed such as differential evolution (DE) (Storn & Price, 1997), evolutionary strategy (ES) (Back, 1996;Beyer, 2001), particle swarm optimization (PSO) (Kennedy & Eberhart, 1995), ant colony optimization (ACO) (Dorigo, 2004), cuckoo search (CS) (Gandomi et al., 2013), firefly algorithm (FA) (Gandomi, 2011), biogeography-based optimization (BBO) (Simon, 2008) big bang-big crunch algorithm (Erol & Eksin, 2006), charged system search (CSS) (Kaveh & Talatahari, 2010) animal migration optimization (AMO) (Li et al., 2013), water cycle algorithm (WCA) (Eskandar et al., 2012), mine blast Algorithm (MBA) (Sadollaha et al., 2013), harmony search algorithm (Mahdavi et al., 2007), improvements of Symbiosis Organisms Search Algorithm (Nama et al. 2016b(Nama et al. , 2016b;;Nama & Saha, 2018).Recently Civicioglu (2013) proposed a novel algorithm called backtracking search algorithm (BSA) which is based on the return of a social group at random intervals to hunting areas that were previously found fruitful for obtaining nourishment (Civicioglu, 2013;Nama et al., 2016d) BSA's strategy for generating a trial population includes two new crossover and mutation operators which are different from other evolutionary algorithm like DE and GA.BSA uses random mutation strategy with one direction individual for each target individual and a non-uniform crossover strategy.BSA's strategies is very powerful to control the magnitude of the search-direction matrix.
In recent years, many authors have examined thet the combination of two algorithms can give better results compared with a single optimization algorithm.The combination of one meta-heuristic algorithm with another meta-heuristics algorithm is called a hybrid metaheuristic algorithm, growing interest in the field of hybrid meta-heuristics algorithm can be seen in the literature and some of its latest applications to a wide range of problems can be seen in the references (Kundra & Sood, 2010;Yildiz, 2013;Zhang et al., 2009;Nemati et al., 2009;Kao & Zahara, 2008;Shahla et al., 2011;Xiaoxia & Lixin, 2009;Nama et al., 2016aNama et al., , 2016c;;Nama & Saha, 2018a,b) for the last decade.
Since the success of an optimization algorithm depends on its significant development of exploration and exploitation abilities (Wolpert & Macready, 1997), in the proposed HBSA BSA is used to enhance the algorithm's exploitation ability and QA is used for exploration ability of the algorithm.Here, the simplified quadratic approximation (QA) with the three best points in the current population is used to reduce the computational burden and to improve the local search ability as well as the solution accuracy of the algorithm.These can maintain the algorithm's exploitation and exploration ability and at the same time can expedite its convergence.The remaining part is arranged in the following way: In Section 2 basic concept of the BSA and QA is described, in Section 3 the new method HBSA is presented.Section 4 empirically demonstrates the efficiency and accuracy of the hybrid approach in solving unconstrained optimization problems.For the validity of the proposed method, the obtained results are compared with five state-of-the-arts particle swarm optimization (PSO) variant approaches in terms of the numerical result of the solutions.However, utilizing experiences may make BSA converge slowly and prejudice exploitation on later iteration stage.The mutation operation in BSA introduces occasional changes of a random individual position with a specified mutation probability.However, the significance of amplitude control factor F in controlling BSA performance has not been acknowledged in BSA research.So the results are investigated for different value of the BSA control parameter (F).Finally, Section 5 summarizes the contribution of this paper along with some future research directions.

Overview of BSA and QA
In this section, the discussion of basic BSA and QA has been presented.

Basics of BSA
The stochastic evolutionary algorithm BSA is a population-based iterative evolutionary algorithm.BSA executes the search space into five major components: initialization, selection-I, mutation, crossover and selection-II.

Initialization:
At the initial stage, the initial population generates randomly within the uniform search space.The initial population is calculated according to the Eq. ( 1) for i=1:PS for j=1:D Popi,j = Popmin + rand (0, 1)*(Popmax -Popmin); (1) end end Here, PS is the population size, D is the dimension of the optimization problem, Pop is the initial population, Popmax and Popmin is the lower and upper bound of the population.

Selection-l:
BSA determines the historical population OldPop for calculating the search direction.The initial historical population is determined within the search boundary according to the following: for i=1:PS for j=1:D OldPopi,j = Popmin + rand (0, 1)×(Popmax -Popmin); (2) end end BSA has the option of redefining OldPop at the beginning of each iteration through the Eq. ( 3):

OldPop = Pop;
(3) end After OldPop is determined, Eq. ( 4) is used to randomly change the order of the individuals in OldPop:

Mutation:
During each generation BSA's mutation process generates the initial form of the trial population (Mutant) using Eq. ( 5) Here CF controls the amplitude of the search-direction matrix (OldPop -Pop).As the historical population is employed within the calculation of the search-direction matrix, BSA generates a trial population, taking partial advantage of its experiences from previous generations.The control parameter F=3×rndn, where rndn ϵ N (0, 1).

Crossover:
After the new mutation operation is finished, the crossover process generates the final form of the trial population T. The initial value of the trial population is Mutant, which has been set in the mutation process.Individuals with better fitness values for the optimization problem are used to evolve the target population.The first step of the crossover process calculates a binary integer-valued matrix (H) of size N×D that indicates the individuals of T to be manipulated using the relevant individuals of P.Then, the trial population T is updated as given by Algorithm 1.
Algorithm 1.The Crossover Strategy Algorithm.
If a < b; (a, b) ϵ U (0, 1) for i from 1 to HP U= permuting (D); Map (i, mixrate×rand×D) = 0 else for i from 1 to HP Map (i, randi (D)) = 0 end for i from 1 to HP for j from 1 to D If map (i, j) = 1; T (i, j) = Pop (i, j); end end end end In Algorithm 1 two predefined strategies are randomly used in defining the integer-valued matrix, which is more complex than the processes used in DE.The first strategy uses mix rate M, and the other allows only one randomly chosen individual to mutate in each trial.

Selection-ll:
In Selection-II stage, BSA updates the population (Pop) by comparing the trial population (T) with the corresponding population (Pop) based on a greedy selection.If the best individual of Pop (Pbest) has a better fitness value than the global minimum value obtained so far by BSA, the global minimizer is updated to be Pbest and the global minimum value is updated to be the fitness value of Pbest.The implementation step of the basic BSA algorithm is as follows: Step 1. Generalizes initial population, algorithm parameter; Step.2 Evaluate the fitness of each individual in the population; Step 3. Generate mutant population by Eq. ( 5) Step 4. By using Crossover Strategy given in Algorithm 1, generates the trial population.
Step 5. Evaluate fitness vector of the trial population (offspring) Step 6. Select individual between target population and trial population (offspring) and update individual.
Step 7. If the stopping criterion is not satisfied go to Step 3, else return the individual with the best fitness as the solution.

Quadratic Approximation (QA)
In this section, we discuss about three points Quadratic Approximations (Deep & Das, 2008) is calculated according to following Eq.( 6) This has been successfully used in RST of Mohan and Shanker (1994).Here, the three-point quadratic approximation in the current population is used to reduce the computational burden and to improve the local search ability as well as the accuracy of the minimum function value of the algorithm.

Proposed Hybrid Backtracking Search Optimization Algorithm (HBSA)
In this section, a new hybrid method is introduced in detail.It has been already mention that for success of an optimization algorithm,, the significant development of the intensification i.e. exploitation and diversification i.e., exploration abilities of the algorithm is the most common factor (Wolpert & Macready, 1997).Diversification means the diverse solutions is generated within the search boundary so as to explore the search space of optimization problem on the global scale, while intensification indicates that the solution search in a local area by exploiting the information of current best solution in the region.The BSA mutation operator produce a final form of the trial population through their exploration ability.After producing the new population by BSA, QA is used to each population for reproducing the population.This can balance the exploration of the proposed algorithm.If one population violates the boundary condition the population is reflected back from the violated boundary using the following rule (Gong et al., 2010): Here, i = 1,2,3, … , PS; Popmin represents the lower limit and Popmax represents the the upper limit in the search space of the ith population.So, combining the exploration of QA with the exploitation of BSA effectively, a hybrid BSA approach, called HBSA has been proposed..In the proposed approach, the algorithm initialized with a population of random solutions and searches for optima by traveling into the search space.During this travel an evolution of this solution is performed by integrating BSA and QA.The description diagram of the proposed algorithm is shown in Fig. 1 and it is described as follows: Step 1. Initialization: Initialize algorithm parameter, population set and OldPop with random positions on D-dimensions in the problem space using Eqn.(1).Evaluate the fitness value for each individual of the population set.
Step 2. Setting OldPop: Set the final form of OldPop for each target individual using Eq. ( 3) and Eq (4).
Step 4. Crossover: By using Crossover strategy given in Algorithm 1, generate the final form of the trial population.Repair each infeasible the trial population to be feasible using Eqn.( 8) and evaluate the fitness value for each trial population.
Step 5. Selection: Select individual between trial population and target population.The fitness value of each trial population is utilized for selecting target population.In the selection, if the trial population has less or equal objective function value (in a minimization problem) than the corresponding target population, the trial population will replace the target population and enter the population of the next generation.Otherwise, the target population will remain in the population for the next generation.
Step 6. Update each individual by QA: Update each individual by QA using eqn.( 6) and repair the infeasible individuals of the population to be feasible using eqn.(8).
Step 7. Stopping Criteria: This process is repeated from Step 3 until some specific stopping criterion is met.The pseudo code of the proposed Algorithm is shown in Fig. 2.

Initialization:
Initialize population size, stopping criterion (here in this paper function evaluations), mix rate parameter (mixrate), Control parameter (F) and range of design variables.

Initial population:
Generate the initial population and evaluate the fitness for each individual.

Main loop:
While stopping criteria is not meet Evaluation Set OldPop using Eq. ( 3) and Eq. ( 4).
Update each individual of the population by Mutation operator using Eq. ( 5).By using Crossover strategy given in Algorithm 1, generate the final form of each individual of the trial population Repair the infeasible individuals of the population to be feasible using Eq. ( 8) Updated individual by QA using Eq. ( 6) Repair the infeasible individuals of the population to be feasible using Eq. ( 8) End While Fig. 2. The pseudo code of the proposed HBSA

Benchmark functions used
In order to compare the performance of the proposed HBSA, a set of 20 benchmark scalable test problems is selected from the literature.The problems include Unimodal and multimodal functions which are scalable and given in Table 1.The HBSA is implemented in MATLAB R2010a and the experiments are carried out on an Acer, 2.00 GHz machine with 500 MB RAM under Windows 7 platform.A total of 30 runs is conducted for the proposed HBSA, a different seed for the generation of random numbers to start a run.Since the significance of amplitude control factor F in controlling BSA performance has not been acknowledged in BSA research, the proposed HBSA experiment of various possible value of control parameter (F) of HBSA with keeping the population size fixed at 50 and the stopping criterion is a maximum of 150000 function evaluations.

Algorithms compared
In order to evaluate the merits of the proposed new mutation strategy, HBSA has been compared with five state-of-the-art PSO variants listed below: • Unified particle swarm optimization scheme (UPSO) (Parsopoulos, 2004); • Fully informed particle swarm (FIPS) (Mendes et al., 2004);

Results on numerical benchmarks
Table 2 shows the experimental result of the mean and the standard deviation of the best-of-run errors for 30 independent runs of each of the five algorithms on twenty numerical benchmark problems for dimension D = 50.Note that the experimental results have been performed and best-of-the-run error corresponds to the absolute difference between the best-of-the-run value ) ( best X f and the actual optimum ) (  X f of a particular objective function i.e. )) The best results are marked in bold for all problems.Table 2 indicates that out of 20 in 13 cases HBSA could beat all other competitor algorithms.It is seen that for function F8 HBSA superior than FDR-PSO, FIPS, CPSO-H, for F9 HBSA superior than FDR-PSO, FIPS, UPSO, for F10 HBSA superior than FDR-PSO, FIPS, CPSO-H, for F11 HBSA superior than FDR-PSO, FIPS, UPSO, CLPSO and for function F17 HBSA superior than FIPS, UPSO, CLPSO and CPSO-H respectively.As the comparison result given in Table 2 may be concluded that the proposed method HBSA perform better than other algorithms.

A parametric study on HBSA
Table 3 shows the mean and the standard deviation of the error function value of different problem for 30 independent runs and for D = 50.In Table 3 the results have been executed with different values of control parameter F which are taken as 0.5, 0.6, 0.7, 0.8 and 0.9, respectively.From this table it is seen that, for F2, F7, F13, and F17 HBSA give the best result for control parameter F=0.8, respectively.The best result is obtained for F8 with the value of control parameter F = 0.5 and for others function the value of control parameter F = 0.9.So, it is clear that the performance of the algorithm gradually becomes better with increase of F up to 0.9.From the parametric study it is clear that the algorithm performs best when F = 0.9 for all the numerical benchmark functions.Table 4 shows the comparative study of HBSA with the basic BSA for the value of control parameter F = 3*rndn and 0.9, from this table it is seen that, except for function F6, F11 and F12 HBSA perform better than the BSA. ) it is clear that the converges speed vary with the variation of F. So, F is an important parameter of the HBSA and F = 0.9 is the better choice in this case.We do not claim that these parameter settings are the best for any problem in general, but these values are recommended since they are found to be repeat giving good results for most of the problems and hence they are an appropriate setting to choose if we talk about the overall performance of the algorithm.

Statistical results of tests
For statistical analysis, a pairwise comparison of a problem-based or multi-problem-based statistical comparison method (Derrac et al., 2011) has been applied to justify the performance of the proposed method.In multi-problem-based pairwise comparison, the mean of the optimum results calculated in different independent runs are used to obtained the which method are more appropriate than the other compared algorithm.In this study, the average of global minimum values obtained as the result of 30 runs for its Wilcoxon signed rank test multi problem based comparison of the algorithms.Table 5 presents the multi-problem-based pairwise statistical comparison results using the averages of the global minimum values obtained through 30 runs of HBSA and the comparison algorithms to solve the benchmark problems.These results show that HBSA was statistically more successful than all of the comparison algorithms, with a statistical significance value α = 0.05.An examination of the results obtained from the tests which is given in Table 5 reveals that the success of HBSA in solving the unconstrained optimization problems is better than the other comparison algorithms.

Conclusions
A new hybrid global search algorithm has been proposed by combining BSA with QA.For the balance of the exploration and the exploitation of BSA, in this paper, we have proposed a hybrid BSA approach, called HBSA, which combines the exploration of QA with the exploitation of BSA.The simplified quadratic approximation (QA) can improve the accuracy of the solution and also enhances the local search ability of the algorithm.Since the hybrid global search algorithms have a good trade-off between the exploration and the exploitation, it makes our proposed HBSA approach very effective and efficient.To verify the performance of HBSA, 20 benchmark functions have been chosen from literature.Experimental results were compared with five PSO variant's given in Table 2.For different values of control parameter, the result was also executed which were given in Table 3.For the multiproblem-based pairwise statistical comparison of HBSA and the other algorithms, the average values of the solutions obtained in Table 2 were used.Table 5 shows the p value and R+ and R-values obtained in this comparison.Analysis of these values when α = 0.05 shows that HBSA was statistically more successful than all of the comparison algorithms.Finally, the overall experimental results demonstrate the good performance of HBSA.For the feature work, this method can also apply to solve the constrained optimization problem, multiobjective optimization problem and to solve the various complex problem in the field of engineering.

Fig. 1 .
Fig.1.Diagram model of population's update Fig. (a) -(f), shows the convergence graph for different value of F with the proposed algorithm.From the figures (Fig. (a) -(f)

Table 2
Comparison of statistical result of HBSA of error function value with different state of the art PSO variants

Table 3
Comparison of statistical result of HBSA of error function value for different value of control parameter(F)

Table 4
Comparison of statistical result of error function value with BSA (For different value of control parameter)

Table 5
Multi-problem based statistical pairwise comparison of comparison algorithms and HBSA (α = 0.05)