Rao algorithms: Three metaphor-less simple algorithms for solving optimization problems

.


Introduction
In recent years the field of population based meta-heuristic algorithms is flooded with a number of 'new' algorithms based on metaphor of some natural phenomena or behavior of animals, fishes, insects, societies, cultures, planets, musical instruments, etc.Many new optimization algorithms are coming up every month and the authors claim that the proposed algorithms are 'better' than the other algorithms.Some of these newly proposed algorithms are dying naturally as there are no takers and some have received success to some extent.However, this type of research may be considered as a threat and may not contribute to advance the field of optimization (Sorensen, 2015).It would be better if the researchers focus on developing simple optimization techniques that can provide effective solutions to the complex problems instead of looking for developing metaphor based algorithms.Keeping this point in view, three simple metaphor-less and algorithm-specific parameter-less optimization algorithms are developed in this paper.The next section describes the proposed algorithms.

Proposed algorithms
Let f(x) is the objective function to be minimized (or maximized).At any iteration i, assume that there are 'm' number of design variables, 'n' number of candidate solutions (i.e.population size, k=1,2,…,n).Let the best candidate best obtains the best value of f(x) (i.e.f(x)best) in the entire candidate solutions and the worst candidate worst obtains the worst value of f(x) (i.e.f(x)worst) in the entire candidate solutions.If Xj,k,i is the value of the j th variable for the k th candidate during the i th iteration, then this value is modified as per the following equations.
X 'j,k,i = Xj,k,i + r1,j,i (Xj,best,worst,i), (1) X'j,k,i = Xj,k,i + r1,j,i (Xj,best,i -Xj,worst,i) + r2,j,i (│Xj,k,i or Xj,l,i│-│Xj,l,i or Xj,k,i│), (2) X'j,k,i = Xj,k,i + r1,j,i (Xj,best,i -│Xj,worst,i│) + r2,j,i (│Xj,k,i or Xj,l,i│-(Xj,l,i or Xj,k,i)), where, Xj,best,i is the value of the variable j for the best candidate and Xj,worst,i is the value of the variable j for the worst candidate during the i th iteration.X'j,k,i is the updated value of Xj,k,i and r1,j,i and r2,j,i are the two random numbers for the j th variable during the i th iteration in the range [0, 1].
In Eqs.( 2) and ( 3), the term Xj,k,i or Xj,l,i indicates that the candidate solution k is compared with any randomly picked candidate solution l and the information is exchanged based on their fitness values.If the fitness value of kth solution is better than the fitness value of lth solution then the term "Xj,k,i or Xj,l,i" becomes Xj,k,i.On the other hand, if the fitness value of lth solution is better than the fitness value of kth solution then the term "Xj,k,i or Xj,l,i" becomes Xj,l,i.Similarly, if the fitness value of kth solution is better than the fitness value of lth solution then the term "Xj,l,i or Xj,k,i" becomes Xj,l,i.If the fitness value of lth solution is better than the fitness value of kth solution then the term "Xj,l,i or Xj,k,i" becomes Xj,k,i.These three algorithms are based on the best and worst solutions in the population and the random interactions between the candidate solutions.Just like TLBO algorithm (Rao, 2015) and Jaya algorithm (Rao, 2016;Rao, 2019), these algorithms do not require any algorithm-specific parameters and thus the designer's burden to tune the algorithm-specific parameters to get the best results is eliminated.These algorithms are named as Rao-1, Rao-2 and Rao-3 respectively.Fig. 1 shows the flowchart of Rao-1 algorithm.The flowchart will be same for Rao-2 and Rao-3 algorithms except that the Eq. ( 1) shown in the flowchart will be replaced by Eq. ( 2) and Eq. ( 3) respectively.The proposed algorithms are illustrated by means of an unconstrained benchmark function known as Sphere function.

Demonstration of the working of proposed Rao-1 algorithm
To demonstrate the working of proposed algorithms, an unconstrained benchmark function of Sphere is considered.The objective function is to find out the values of xi that minimize the value of the Sphere function.
Benchmark function: Sphere 2 1 min ( ) The known solution to this benchmark function is 0 for all xi values of 0. Now to demonstrate the proposed algorithms, let us assume a population size of 5 (i.e.candidate solutions), two design variables x1 and x2 and two iterations as the termination criterion.The initial population is randomly generated within the ranges of the variables and the corresponding values of the objective function are shown in Table 1.As it is a minimization function, the lowest value of f(x) is considered as the best solution and the highest value of f(x) is considered as the worst solution.From Table 1 it can be seen that the best solution is corresponding the 4 th candidate and the worst solution is corresponding to the 2 nd candidate.Using the initial solutions of Table 1 and assuming random number r1 = 0.10 for x1 and r2 = 0.50 for x2, the new values of the variables for x1 and x2 are calculated using Eq.( 1) and placed in Table 2.For example, for the 1 st candidate, the new values of x1 and x2 during the first iteration are calculated as shown below.
Similarly, the new values of x1 and x2 for the other candidates are calculated.Now, the values of f(x) of Table 1 and Table 2 are compared and the best values of f(x) are considered and placed in Table 3.This completes the first iteration of the Rao-1 algorithm.From Table 3 it can be seen that the best solution is corresponding the 1 st candidate and the worst solution is corresponding to the 3 rd candidate.In the first iteration, the value of the objective function is improved from 113 to 76.84 and the worst value of the objective function is improved from 1285 to 936.Now, assuming random number r1 = 0.80 for x1 and r2 = 0.1 for x2, the new values of the variables for x1 and x2 are calculated using Eq.( 1) and are placed in Table 4. Table 4 shows the corresponding values of the objective function also.Now, the values of f(x) of Tables 3 and 4 are compared and the best values of f(x) are considered and placed in Table 5.This completes the second iteration of the Rao-1 algorithm.It can be observed that at the end of second iteration, the value of the objective function is improved from 113 to 24.0676 and the worst value of the objective function is improved from 1285 to 539.24.If we increase the number of iterations then the known value of the objective function (i.e.0) can be obtained within next few iterations.Also, it is to be noted that in the case of maximization function problems, the best value means the maximum value of the objective function and the calculations are to be proceeded accordingly.Thus, the proposed method can deal with both minimization and maximization problems.This demonstration is for an unconstrained optimization problem.However, the similar steps can be followed in the case of constrained optimization problem.The main difference is that a penalty function is used for violation of each constraint and the penalty value is operated upon the objective function.

Demonstration of the working of proposed Rao-2 algorithm
Using the initial solutions of Table 1, and assuming random numbers r1 = 0.10 and r2 = 0.50 for x1 and r1 = 0.60 and r2 = 0.20 for x2, the new values of the variables for x1 and x2 are calculated using Eq.( 2) and placed in Table 6.For example, for the 1 st candidate, the new values of x1 and x2 during the first iteration are calculated as shown below.Here the 1 st candidate has interacted with the 2 nd candidate.The fitness value of the 1 st candidate is better than the fitness value of the 2 nd candidate and hence the information exchange is from 1 st candidate to 2 nd candidate.
X'1,1,1 = X1,1,1 + r1,1,1 (X1,4,1 -X1,2,1) + r2,1,1 (│X1,1,1│ -│X1,2,1│) = -5 + 0.10 (-8-14) + 0.50 (5-14) = -11.7 X'2,1,1 = X2,1,1 + r1,2,1 (X2,4,1 -X2,2,1) + r2,2,1 (│X2,1,1│ -│X2,2,1│) = 18 + 0.60 (7-33) + 0.20 (18-33) = -0.6 Similarly, the new values of x1 and x2 for the other candidates are calculated.Here the random interactions are taken as 2 vs. 5, 3 vs. 1, 4 vs. 2 and 5 vs. 4. Table 6 shows the new values of x1 and x2 and the corresponding values of the objective function.Now, the values of f(x) of Table 1 and Table 6 are compared and the best values of f(x) are considered and placed in Table 7.This completes the first iteration of the Rao-2 algorithm.From Table 7 it can be seen that the best solution is corresponding the 4 th candidate and the worst solution is corresponding to the 3 rd candidate.Now, during the second iteration, assuming random numbers r1 = 0.01 and r2 = 0.10 for x1 and r1 = 0.10 and r2 = 0.50 for x2, the new values of the variables for x1 and x2 are calculated using Eq.( 2).Here the random interactions are taken as 1 vs. 4, 2 vs. 3, 3 vs. 5, 4 vs. 2 and 5 vs. 1.Now, the values of f(x) of Tables 7 and 8 are compared and the best values of f(x) are considered and placed in Table 9.This completes the second iteration of the Rao-2 algorithm.From Table 9 it can be seen that the best solution is corresponding the 2 nd candidate and the worst solution is corresponding to the 5 nd candidate.It can be observed that the value of the objective function is improved from 113 to 107.517 in two iterations.Similarly, the worst value of the objective function is improved from 1285 to 512.331 in just two iterations.If we increase the number of iterations then the known value of the objective function (i.e.0) can be obtained within next few iterations.Also, just like Rao-1, the proposed Rao-2 can deal with both unconstrained and constrained minimization as well as maximization problems.

Demonstration of the working of proposed Rao-3 algorithm
Now assuming random numbers r1 = 0.10 and r2 = 0.50 for x1 and r1 = 0.60 and r2 = 0.20 for x2, the new values of the variables for x1 and x2 are calculated using Eq.( 3) and placed in Table 10.For example, for the 1 st candidate, the new values of x1 and x2 during the first iteration are calculated as shown below.
Here the 1 st candidate has interacted with the 2 nd candidate.The fitness value of the 1 st candidate is better than the fitness value of the 2 nd candidate and hence the information exchange is from 1 st candidate to 2 nd candidate.
= 18 + 0.60 (7-33) + 0.20 (18-33) = -0.6 Similarly, the new values of x1 and x2 for the other candidates are calculated.Here the random interactions are taken as 2 vs. 5, 3 vs. 1, 4 vs. 2 and 5 vs. 4. Table 10 shows the new values of x1 and x2 and the corresponding values of the objective function.Now, the values of f(x) of Tables 1 and 10 are compared and the best values of f(x) are considered and placed in Table 11.This completes the first iteration of the Rao-3 algorithm.From Table 11 it can be seen that the best solution is corresponding the 4 th candidate and the worst solution is corresponding to the 3 rd candidate.Now, during the second iteration, assuming random numbers r1 = 0.01 and r2 = 0.10 for x1 and r1 = 0.10 and r2 = 0.50 for x2, the new values of the variables for x1 and x2 are calculated using Eq.( 3).Here the random interactions are taken as 1 vs. 4, 2 vs. 3, 3 vs. 5, 4 vs. 2 and 5 vs. 1.Table 12 shows the new values of x1 and x2 and the corresponding values of the objective function during the second iteration.Now, the values of f(x) of Tables 11 and 12 are compared and the best values of f(x) are considered and placed in Table 13.This completes the second iteration of the Rao-3 algorithm.From Table 13 it can be seen that the best solution is corresponding the 2 nd candidate and the worst solution is corresponding to the 5 nd candidate.It can be observed that the value of the objective function is improved from 113 to 77.853 in just two iterations.Similarly, the worst value of the objective function is improved from 1285 to 324 in just two iterations.If we increase the number of iterations then the known value of the objective function (i.e.0) can be obtained within next few iterations.Also, just like Rao-1 and Rao-2, the proposed Rao-3 can also deal with both unconstrained and constrained minimization as well as maximization problems.It may be noted that the above three demonstrations with random numbers are just to make the readers familiar with the working of the proposed algorithms.While executing the algorithms different random numbers will be generated during different iterations and the computations will be done accordingly.The next section deals with the experimentation of the proposed algorithms on the benchmark optimization problems.

Computational experiments on unimodal, multi-modal and fixed-dimension multimodal optimization problems
The computational experiments are first conducted on 23 benchmark functions including 7 unimodal, 6 multimodal and 10 fixed-dimension multimodal functions.Table 14 shows these benchmark functions.
No The benchmark functions 1-7 are the unimodal functions (for checking the exploitation capability of the algorithms), 8-13 are the multimodal functions that have many local optima which increase with the increase in the number of dimensions (for checking the exploration capability of the algorithms) and 14-23 are the fixed-dimension multimodal benchmark functions (for checking the exploration capability of the algorithms in the case of fixed dimension optimization problems).The global optimum values of the benchmark functions are also given in Table 15 to give an idea to the readers about the performances of the proposed algorithms.
The performance of the proposed algorithms is tested on the 23 benchmark functions listed in Table 14.
To evaluate the performance of the proposed algorithms, a common experimental platform is provided by setting the maximum number of function evaluations as 30000 for each benchmark function with 30 runs for each benchmark function.The results of each benchmark function are presented in Table 15 in the form of best solution, worst solution, mean solution, standard deviation obtained in 30 independent runs, mean function evaluations, and the population size used for each benchmark function.The results of the proposed algorithms are compared with the already established Grey Wolf Optimization (GWO) algorithm (Mirjalili, 2014) and Ant Lion Optimization (ALO) algorithm (Mirjalili, 2015).
It may be mentioned here that the GWO algorithm was already shown competitive to the other advanced optimization algorithms like particle swarm optimization (PSO), gravitational search algorithm (GSA), differential evolution (DE) and fast evolutionary programming (FEP) (Mirjalili, 2014).The ALO algorithm was also shown competitive to PSO, states of matter search (SMS), bat algorithm (BA), flower pollination algorithm (FPA), cuckoo search (CS) and firefly algorithm (FA) (Mirjalili, 2015).Hence in this paper the results of other advanced optimization algorithms are not shown.The GWO algorithm was used for solving 23 benchmark functions (Mirjalili, 2014) and ALO was used for solving 13 benchmark functions (Mirjalili, 2014).The results of application of the proposed algorithms are shown in Table 15.Mirjalili (2014Mirjalili ( , 2015) ) had shown the results of only mean solutions and standard deviations.However, the results of the proposed algorithms are presented in      It may be observed from Table 15 that the proposed algorithms are not origin-biased as it can be seen that these algorithms have obtained the global optimum solutions in the case of benchmark functions 8 and 14-23 whose optima are not at origin.The performance of the proposed algorithms is appreciable on the benchmark functions considered.It may also be observed that the standard deviation results of GWO for objective functions 8,16,19-23 (Mirjalili, 2014) are incorrect as the standard deviation value can not be negative.Furthermore, it seems that the values given by GWO as mean solutions for benchmark functions 21-23 may not be corresponding to the mean solutions and these may be corresponding to the best solutions of GWO.That is why, even though the "mean solutions" of GWO are shown in bold for the functions 21-23, the mean solutions of functions 21 and 22 given by Rao-2 algorithm, and the mean solution of function 23 by given by Rao-3 algorithm are also shown in bold.
In terms of the mean solutions, GWO algorithm has performed better (compared to ALO, Rao-1, Rao-2 and Rao-3 algorithms) on functions 7,11,15, 16 (and 21-23?).The results corresponding to functions 21-23 may be corresponding to the "best (B)" solutions of GWO algorithm.The mean results of ALO algorithm are comparatively better for functions 4,5,9,10 (and 12 and 13?).The mean results of Rao-1 algorithm are better for functions 6,14,17,18 and 19.The mean results of Rao-2 algorithm are better for functions 14,17,18,19,20 (and 21 and 22?).The mean results of Rao-3 algorithm are better for functions 1-3, 8,17,19,(and 23?).Thus, the proposed three algorithms can be said competitive to the existing advanced optimization algorithms in terms of better results for solving the unimodal, multimodal and fixed-dimension multimodal optimization problems with better exploitation and exploration potential.
If an intra-comparison is made among the proposed three algorithms in terms of the "best (B)" solutions obtained, Rao-3 algorithm has obtained the best solutions in 17 functions; Rao-2 has obtained the best solutions in 9 functions and Rao-1 in 9 functions.In terms of the 'worst (W)" solutions obtained, Rao-3 performs better in 14 functions, Rao-2 in 8 functions and Rao-1 in 7 functions.
The MATLAB codes of Rao-1, Rao-2 and Rao-3 algorithms are given in Appendix-1, Appendix-2 and Appendix-3 respectively.The code is developed for the objection function "Sphere function".The user may copy and paste this code in a MATLAB file and run the program.The user may replace the portion of the code corresponding to the Sphere function with the objective function of the optimization problem considered by him/her to get the results.

Additional experiments on unconstrained optimization problems
The performance of the proposed three algorithms is tested further on 25 unconstrained benchmark functions well documented in the optimization literature.These unconstrained functions have different characteristics like unimodality, multimodality, separability, non-separability, regularity, non-regularity, etc.The number of design variables and their ranges are different for each problem.Table 16 shows the details of 25 unconstrained benchmark functions.30 [-32, 32] MN  -65.536, 65.536] MS 24 Hartman 3 ( ,5,100, 4), ( , , , ) To evaluate the performance of the proposed algorithms, a common experimental platform is provided by setting the maximum number of function evaluations as 500000 for each benchmark function with 30 runs for each benchmark function.The results of each benchmark function are presented in Table 17 in the form of best solution, worst solution, mean solution, standard deviation obtained in 30 independent runs and the mean function evaluations on each benchmark function.The global optimum values of the benchmark functions are also given in Table 17 to give an idea to the readers about the performances of the proposed algorithms.Table 18 shows the number of instances the results of each algorithm are either better or equal to the performance other algorithms in terms of best solution (B), worst solution (W), mean solution (M), standard deviation (SD) and mean function evaluations (MFE).It can be observed from Tables 17 and 18 that the algorithms are not origin-biased as it can be seen that these algorithms have obtained the global optimum solutions in the case of benchmark functions 4,7,8,13,18,19,20,23 and 24 whose optima are not at origin.The performance of the proposed algorithms is appreciable on 25 unconstrained benchmark functions considered.Out of the 25 unconstrained benchmark functions, the proposed algorithms have obtained the same results in 14 functions (i.e., in terms of best solution, worst solution, mean solution, standard deviation and mean function evaluations).Even though Rao-2 has obtained the best solution in the case of function nos.8 and 20 but the worst solutions obtained are not good and hence the mean solution values are increased.In the case of function no. 12, Rao-1 and Rao-3 have not obtained the best solution but the best solution obtained by Rao-2 is comparatively better.

Experiments on constrained optimization problems
The performance of the proposed three algorithms is tested further on 2 constrained benchmark functions as part of the investigations.The details of the functions are given below.The results of application of the proposed algorithms on the above two benchmark functions are given in Table 19.The number of runs is 30 and the maximum function evaluations are 500000.In the case of constrained benchmark functions, it can be observed from Table 19 that Rao-2 and Rao-3 have obtained comparatively better results than Rao-1.It may be noted that Rao-1 algorithm, given by Eq. ( 1), is a very simple algorithm and is based only on the difference between the best and worst solutions.Even then, it can be observed that its performance is appreciable in quite a good number of unconstrained and constrained functions.

Conclusions
It is proved in this paper that it is possible to develop potential optimization algorithms without the need of using metaphors related to the behavior of animals, birds, insects, societies, cultures, planets, musical instruments, chemical reactions, physical reactions, etc.The proposed three optimization algorithms are not based on any metaphor or algorithm-specific parameters.These require only the tuning of the common controlling parameters of the algorithm for working (e.g., population size and the number of iterations).The proposed algorithms are implemented first on 23 unconstrained optimization problems including 6 unimodal, 7 multimodal and 10 fixed-dimension multimodal problems.Additional computational experiments are carried out on 25 well defined unconstrained optimization problems having different characteristics and 2 standard constrained optimization problems.The proposed three simple algorithms have given satisfactory performance and are believed to have potential to solve the complex optimization problems as well.
The results of the proposed algorithms presented in this paper are based on the preliminary investigations.Detailed investigations are planned to be carried out in the coming days.These investigations will include testing the performance of the proposed algorithms on various complex and computationally expensive benchmark functions involving a large number of dimensions.The results of detailed experimentation will be compared with the results of other existing well established optimization algorithms and the statistical tests will also be conducted.The researchers working in the field of optimization are requested to make improvements to these three algorithms so that these algorithms will become much more powerful.If these algorithms are found having certain limitations then the researchers may suggest the ways to overcome the limitations, instead of making destructive criticism, to further strengthen the algorithms.

Table 1
Initial population Table 2 shows the new values of x1 and x2 and the corresponding values of the objective function.

Table 2
New values of the variables and the objective function during first iteration

Table 3
Updated values of the variables and the objective function based on fitness comparison at the end of first iteration

Table 4
New values of the variables and the objective function during second iteration

Table 5
Updated values of the variables and the objective function based on fitness comparison at the end of second iteration

Table 6
New values of the variables and the objective function during first iteration

Table 7
Updated values of the variables and the objective function based on fitness comparison at the end of first iteration Table 8 shows the new values of x1 and x2 and the corresponding values of the objective function during the second iteration.

Table 8
New values of the variables and the objective function during second iteration

Table 10
New values of the variables and the objective function during first iteration

Table 11
Updated values of the variables and the objective function based on fitness comparison at the end of first iteration

Table 13 Updated
values of the variables and the objective function based on fitness comparison at the end of second iteration

Table 15
Results of the proposed algorithms for 23 benchmark functions considered (30000 function evaluations)

Table 15
Results of the proposed algorithms for 23 benchmark functions considered (30000 function evaluations)

Table 15
Results of the proposed algorithms for 23 benchmark functions considered (30000 function evaluations)

Table 15
Results of the proposed algorithms for 23 benchmark functions considered (30000 function evaluations) Global optimum value; *: This may be the W value of GWO (as the standard deviation can not be negative);; **:This may be the B value of GWO; ***:This may be the B value of ALO; The results of ALO are available only for 1-13 benchmark functions.

Table 16
Unconstrained benchmark functions considered(Continued)

Table 17
Results of the proposed algorithms for the unconstrained benchmark functions

Table 17
Results of the proposed algorithms for the unconstrained benchmark functions(Continued)

Table 17
Results of the proposed algorithms for the unconstrained benchmark functions(Continued) B: Best Solution; W: Worst Solution; M: Mean Solution; SD: Standard Deviation; MFE: Mean Function Evaluations.

Table 19
Results of constrained benchmark functions