Modiﬁed Evolutionary Algorithm and Chaotic Search for Bilevel Programming Problems

: Bi-level programming problem (BLPP) is an optimization problem consists of two interconnected hierarchical optimization problems. Solving BLPP is one of the hardest tasks facing the optimization community. This paper proposes a modiﬁed genetic algorithm and a chaotic search to solve BLPP. Firstly, the proposed algorithm solves the upper-level problem using a modiﬁed genetic algorithm. The genetic algorithm has modiﬁed with a new selection technique. The new selection technique helps the upper-level decision-maker to take an appropriate decision in anticipation of a lower level’s reaction. It distinguishes the proposed algorithm with a very small number of solving the lower-level problem, enhances the algorithm performance and fasts convergence to the solution. Secondly, a local search based on chaos theory has applied around the modiﬁed genetic algorithm solution. Chaotic local search enables the algorithm to escape from local solutions and increase convergence to the global solution. The proposed algorithm has evaluated on forty di ﬀ erent test problems to show the proposed algorithm e ﬀ ectiveness. The results have analyzed to illustrate the new selection technique e ﬀ ect and the chaotic search e ﬀ ect on the algorithm performance. A comparison between the proposed algorithm results and other state-of-the-art algorithms results has introduced to show the proposed algorithm superiority


Introduction
Optimization problems have classified into single-level optimization problems and multi-level optimization problems. In single-level optimization problems, there is just one decision-maker tries to find an optimal solution to an optimization problem [1,2]. In multi-level optimization problems, there are hierarchically nested optimization problems. Every optimization problem of these hierarchical nested systems has its decision-maker that tries to find his/her optimal solution [3,4]. Many applications appeared in the real-world in engineering design, traffic problems, problems, economic policy and so on are multi-level optimization problems [5][6][7][8][9]. Multi-level optimization problems consist of "N" optimization problems. The first problem is the upper-level problem. The leader is the upper-level decision-maker. The next problems are lower-level problems. The followers are the lower levels decision-makers. A leader knows the followers' objective function and constraints while the followers may know or not leader's objective functions and constraints. Leaders and followers are independent and often have conflicting objectives although their decisions affect each other. The hierarchical decision process starts when the leader takes his/her decisions. Then, the first follower takes his/her own decisions upon the leader's decisions and so on for the remainder followers; every follower determines his/her own decisions upon his/her upper's level decisions. The followers' decisions affect Symmetry 2020, 12, 767 3 of 29 theory with a genetic algorithm helps to solve BLPP. In [38] Zhongping et al. hybridize particle swarm optimization with chaos searching technique for solving nonlinear bilevel programming problems. Zhao et al. establishes a chaotic search method to solve the bilevel programming problems directly in [39]. They stated that the ergodicity and the regularity of chaotic behavior lead the search to be asymptotical convergence search.
This paper proposes a new approach based on a genetic algorithm and a chaos theory to solve BLPP. The cause of choosing the genetic algorithm is that BLPP solution requirements produce difficulties in solving such as disconnectedness and non-convexity even to the simple problems. The genetic algorithm is presented as an efficient global method for optimization problems, especially for difficult optimization problems. In addition, we choose to use one of the recent theories, chaos theory, that has been applied for solving optimization problems. Using these two optimization techniques offers the advantages of both techniques, the genetic algorithm as a global search technique suitable for hard optimization problems and chaos searching techniques as an effective local search technique that can easily jump of locally optimal solutions. The difficulty in solving BLPP is the need to solve the lower level problem to know the feasible space of the upper-level problem. To reduce the number of solving a lower problem, a genetic algorithm has modified with a new selection technique named bi-level selection technique (BST). BST helps the upper-level decision-maker to take an appropriate decision in anticipation of the lower level's decision. The proposed algorithm operates in two phases. In the first phase, the upper-level problem has solved using a modified genetic algorithm. In the second phase, a local search based on chaos theory is applied around the upper-level solution. BST doesn't replace solving the lower-level problem to solve the BLPP. It only helps the algorithm to predict the lower-level decision on the upper-level decision. Therefore, the lower level problem is then solved using a genetic algorithm for two phase solutions. The two phase's solutions are compared and the best solution for the bilevel problem is the algorithm solution. The proposed algorithm has been evaluated on three sets of test problems; a TP set [40], and a SMD set [27,41] and some chosen test problems from the literature. TP set is a relatively smaller number of variables constrained problems and SMD set are high-dimensional unconstrained problems. SMD problems are BLPP that contain problems with controllable complexities. The third set is nonlinear standard test problems collected from the literature [42]. The results of the proposed algorithm have been discussed and compared with other state-of-the-art algorithms results to show the proposed algorithm effectiveness and efficiency to solve BLLP.
The rest of the paper is as follows. Bilevel problem formulation is proposed in Section 2. Section 3 introduces the proposed algorithm to solve BLPP. In Section 4, numerical experiments are discussed. Finally, Section 5 concludes the paper and proposes some notes for future work.

Bi-Level Programming Problems
BLPP composed of two interacted optimization problems with two decision-makers; leader and follower. The leader knows the followers' objective function and constrains while the follower may know or not the leader's objective function and constraints. The leader needs to make a decision that leads the follower to take his/her decision that benefits the leader's problem. Contrarily, the follower doesn't care about the leader's decision and takes his/her optimal decision upon the leader's decision [5,43]. In BLPP, the lower-level problem appeared as a constraint to upper-level problem BLPP are formulated as follows: Upper level BLPP: Min where F(x, y), f (x, y) : R n1 × R n2 → R are upper and lower level object functions. G(x, y) : R n1 × R n2 → R p , g(x, y) : R n1 × R n2 → R q are the constraint functions. x ∈ R n1 , y ∈ R n2 are upper and lower level decision variables. n1 is the dimensional decision vectors for the upper problem and n2 is the dimensional decision vectors for the lower problem. If vector v • = (x • , y • ) satisfies both levels constrains and y • is an optimal point of the lower level problem solved for the upper level solution x • then v • = (x • , y • ) is a feasible solution for the BLPP [44].

Proposed Algorithm (CGA-BS)
Our proposed algorithm for BLPP is based on genetic algorithm and chaos theory. In this section, we introduce basic concepts of genetic algorithm and chaos theory then we introduce our proposed algorithm CGA-BS in detail.

Basic Concepts of Genetic Algorithm
Genetic Algorithm (GA) is one of the powerful evolutionary algorithms proposed in the early 1970s [45]. GA mimics the theory of survival of the fittest. It is inspired by chromosomes and genes in nature. GA has been widely applied for solving different types of the optimization problem. It has an advantage over classical techniques that it can treat with a non-differentiable and non-convex optimization problem. GA algorithm represents optimization problems as a set of variables. Each solution for an optimization problem corresponds to a chromosome and each gene corresponds to a variable of the problem. The chromosomes improved by applying a genetic operator's selection, crossover, and mutation. The selection operator works on determining better chromosomes to be the parents. There are several selection techniques as roulette wheel selection, rank selection, steady-state selection, stochastic universal sampling [46,47]. Parents then crossover to produce new chromosomes (children chromosomes). Children's chromosomes are expected to be better than parents' chromosomes. There are several crossover techniques as one-point crossover, two-point crossover, cut and splice, uniform crossover and half uniform crossover. Figure 1 illustrates the one-point crossover. As shown in the figure, two chromosomes are split and then combined one part of one with the other pair to produce new chromosomes.

Proposed Algorithm (CGA-BS)
Our proposed algorithm for BLPP is based on genetic algorithm and chaos theory. In this section, we introduce basic concepts of genetic algorithm and chaos theory then we introduce our proposed algorithm CGA-BS in detail.

Basic Concepts of Genetic Algorithm
Genetic Algorithm (GA) is one of the powerful evolutionary algorithms proposed in the early 1970s [45]. GA mimics the theory of survival of the fittest. It is inspired by chromosomes and genes in nature. GA has been widely applied for solving different types of the optimization problem. It has an advantage over classical techniques that it can treat with a non-differentiable and non-convex optimization problem. GA algorithm represents optimization problems as a set of variables. Each solution for an optimization problem corresponds to a chromosome and each gene corresponds to a variable of the problem. The chromosomes improved by applying a genetic operator's selection, crossover, and mutation. The selection operator works on determining better chromosomes to be the parents. There are several selection techniques as roulette wheel selection, rank selection, steady-state selection, stochastic universal sampling [46,47]. Parents then crossover to produce new chromosomes (children chromosomes). Children's chromosomes are expected to be better than parents' chromosomes. There are several crossover techniques as one-point crossover, two-point crossover, cut and splice, uniform crossover and half uniform crossover. Figure 1 illustrates the one-point crossover. As shown in the figure, two chromosomes are split and then combined one part of one with the other pair to produce new chromosomes.
Sometimes, the genetic algorithm produces very similar chromosomes. This problem is considered the symmetric problem in the genetic algorithm. Symmetric chromosomes weak the genetic algorithm and worse its performance. The symmetric problem occurs when highly fit parent chromosomes in the population breed in early evolution time. To overcome the symmetric problem, the mutation operator applied on a small part of parent chromosomes. Figure 2 illustrates the real value interchanging mutation. As shown in Figure 2, some of the genes in the chromosomes are randomly replaced. The new generation then is evaluated. New generations are still produced until the termination criterion is met, and the best-obtained chromosome is the solution [46,47]. Figure 3 proposes GA cycle to obtain the optimal solution.  Sometimes, the genetic algorithm produces very similar chromosomes. This problem is considered the symmetric problem in the genetic algorithm. Symmetric chromosomes weak the genetic algorithm and worse its performance. The symmetric problem occurs when highly fit parent chromosomes in the population breed in early evolution time. To overcome the symmetric problem, the mutation operator applied on a small part of parent chromosomes. Figure 2 illustrates the real value interchanging mutation. As shown in Figure 2, some of the genes in the chromosomes are randomly replaced. The new generation then is evaluated. New generations are still produced until the termination criterion is met, and the best-obtained chromosome is the solution [46,47]. Figure 3 proposes GA cycle to obtain the optimal solution.

Chaos Theory for Optimization Problems
Chaos in common usage means "A condition or place of great disorder or confusion". Chaos theory studies the behavior of systems that follow deterministic laws but appear random and unpredictable or we can say a dynamical system that has a sensitive dependence on its initial conditions; small changes in those conditions can lead to quite different outcomes [10]. This dependency of a dynamical system on its initial condition is popularly referred to as the butterfly effect. Henon initially described chaos theory in 1976 and Lorenz summarized it in 1993 [48,49]. Chaos theory could be introduced into the optimization strategy to accelerate the optimum seeking operation and find the global optimal solution. Chaos theory has an advantage more than other optimization techniques that it can efficiently search the space and escape from local optimal solutions. Chaotic maps are used to build up chaotic sequences. Different chaotic maps are proposed to describe distinct chaotic behaviors. Chaotic maps are divided to discrete-time parameterized and continuous-time parameterized. In this section, we offer some well-known chaotic maps found in the literature. Figure 4 proposes chaotic behavior produced using some chaotic maps.
The Chebyshev map is defined as follows: The circle map is represented as follows: where 0.5 a = and 0.2 b = . The iterative chaotic map with infinite collapses is defined as follows:

Chaos Theory for Optimization Problems
Chaos in common usage means "A condition or place of great disorder or confusion". Chaos theory studies the behavior of systems that follow deterministic laws but appear random and unpredictable or we can say a dynamical system that has a sensitive dependence on its initial conditions; small changes in those conditions can lead to quite different outcomes [10]. This dependency of a dynamical system on its initial condition is popularly referred to as the butterfly effect. Henon initially described chaos theory in 1976 and Lorenz summarized it in 1993 [48,49]. Chaos theory could be introduced into the optimization strategy to accelerate the optimum seeking operation and find the global optimal solution. Chaos theory has an advantage more than other optimization techniques that it can efficiently search the space and escape from local optimal solutions. Chaotic maps are used to build up chaotic sequences. Different chaotic maps are proposed to describe distinct chaotic behaviors. Chaotic maps are divided to discrete-time parameterized and continuous-time parameterized. In this section, we offer some well-known chaotic maps found in the literature. Figure 4 proposes chaotic behavior produced using some chaotic maps.
The Chebyshev map is defined as follows: The circle map is represented as follows: where a = 0.5 and b = 0.2. The iterative chaotic map with infinite collapses is defined as follows: where a ∈ (0, 1).
where a 2.3 = . The basic idea of applying chaos search for optimization problems is transforming problems variable to chaos space using one of the chaotic maps [47]. Followingly, chaotic variables are evaluated by the fitness function. The chaotic variables that achieve the best fitness function are the solution. Chaotic search is continued for a specified number of iterations or until obtaining a satisfied solution. Some researches compare different chaotic maps for optimization problems.

Modified Genetic Algorithm and Chaotic Search for BLPP (CGA-BS)
The proposed algorithm to solve the BLPP is a modified genetic algorithm by a new selection technique (BST) and chaos searching technique. The role of BST is that the selection of individuals for the next generation is according to individuals' fitness to both upper and lower objective functions. BST distinguishes our proposed algorithm with a very small number of solving the lower level problem. We have solved lower level problem two times only and dispensing solving the lower level problem at every generation during solving the upper-level problem. This is not meaning that using BST replaces solving the lower level problem to have a solution of the BLPP. We solve lower level problem two times; one for the upper-level genetic algorithm based on BST solution and other for upper level chaotic local search solution. The steps of the proposed algorithm are listed as follows: 3.3.1. Genetic Algorithm Based on BST The logistic map demonstrates how complex behavior arises from a simple deterministic system without the need of any random sequence. It is based on a simple polynomial equation which describes the dynamics of biological population.  The Singer map is formulated as: where µ ∈ (0.9, 1.08). The Sinusoidal map is generated by the following equation: where a = 2.3. The basic idea of applying chaos search for optimization problems is transforming problems variable to chaos space using one of the chaotic maps [47]. Followingly, chaotic variables are evaluated by the fitness function. The chaotic variables that achieve the best fitness function are the solution. Chaotic search is continued for a specified number of iterations or until obtaining a satisfied solution. Some researches compare different chaotic maps for optimization problems.

Modified Genetic Algorithm and Chaotic Search for BLPP (CGA-BS)
The proposed algorithm to solve the BLPP is a modified genetic algorithm by a new selection technique (BST) and chaos searching technique. The role of BST is that the selection of individuals for the next generation is according to individuals' fitness to both upper and lower objective functions. BST distinguishes our proposed algorithm with a very small number of solving the lower level problem. We have solved lower level problem two times only and dispensing solving the lower level problem at every generation during solving the upper-level problem. This is not meaning that using BST replaces solving the lower level problem to have a solution of the BLPP. We solve lower level problem two times; one for the upper-level genetic algorithm based on BST solution and other for upper level chaotic local search solution. The steps of the proposed algorithm are listed as follows:

Genetic Algorithm Based on BST
In this Step, the upper-level problem is solved using a modified genetic algorithm. The modified genetic algorithm steps are:

1.
Constrains handling: before starting the algorithm, constraints of the lower level problem is added to the constraints of the upper-level problem. The feasible solutions for the upper-level problem must be feasible and optimal solutions for the lower level problem. Therefore, constrains of both levels are considered as upper level constrains to guarantee the feasibility of solutions. 2.
Initial population: variables are randomly initialized within the search space bounds of upper-level bounds and lower-level bounds [50].

3.
Reference point: the repairing process needs one feasible point considered as a reference point to be entered and the algorithm procedure is completed.

4.
Repairing process: for constrained problems, some generated solutions don't satisfy constraints and need to be repaired. The repairing process transforms the infeasible solutions to be feasible solutions [51].

5.
Evaluation: both upper-level objective function and lower-level objective function are used for evaluation of individuals and for determining the fitness of each solution to both upper-level problem and lower-level problem. 6.
Producing a new population: new selection technique, crossover operator and mutation operator have applied to produce the new population.
Bilevel selection technique: The main role of BST is that the selected individuals for the next generation are more appropriate for both levels. BST starts by evaluating individuals using upper-level objective function and then are selected according to their fitness to upper-level objective function using a suitable selection technique. After that, these selected individuals are evaluated using lower-level problem objective function and are selected according to their fitness to lower-level objective function using a suitable selection technique. Bilevel selection technique can also select individuals according to upper-level objective function only or to lower-level objective function only choosing objective that increases the solution quality.
Crossover: crossover operation recombines pairs of individuals and returns the new individuals after mating two individuals. In the proposed algorithm, a one-point crossover as shown in Figure 1 which has applied [52].
Mutation: In the proposed algorithm, the real valued mutation has applied. Real valued mutation produces randomly created values and then these random values are added to the variables [53,54].

7.
Migration: the parent population and child population are migrated to produce the new generation and the best individual is chosen as the solution [53][54][55]. 8.
GA termination: GA is terminated at the maximum number of generations or when population convergence occurs. If the termination condition doesn't satisfy, return to 6.

Chaotic Local Search
In this step, a chaotic local search is proceeds around upper-level variables obtained in Section 3.3.1. The following is a description of a chaotic local search in details:

1.
Define a range of the upper-level variables for chaotic local search.

2.
Produce chaotic numbers using the logistic map: El-Shorbagy et al. in [46] presented a comparison between different chaotic maps for general nonlinear optimization problems and result that logistic map gives better performance than other chaotic maps and increase the solution quality rather than other chaotic maps. Therefore, in the proposed algorithm, we choose to apply the logistic map 3.
Generate the chaotic variables into the determined range.

4.
Find the best value: The produced chaotic variables are evaluated according to upper-level objective-function. The chaotic variables obtained best objective-function values are the best value.

5.
Update the local search boundary value for the best-obtained value. 6.
Stopping chaos search: chaotic local search is stopped at the specified iterations and put out the best solution as a local search solution.

Algorithm Termination
To terminate the algorithm and obtain the algorithm result, the lower level problem is solved for upper-level decision variables obtained in Section 3.3.1 and Section Section 3.3.2. The lower-level problem is solved using a genetic algorithm with the same steps as Section 3.3.2 except that selection is according to the lower level objective function only. The two phases' results are compared, and the better result is the algorithm solution. Algorithm 1 illustrates pseudo code of the proposed algorithm. The flow chart of the proposed algorithm is shown in Figure 5. Evaluate p k according to upper level objective function; Select p ksu from p k using selection operator; Evaluate p ksu according to lower level objective function; Select p ksl from p ksu using selection operator; Apply crossover operator with CP and mutation operators with MP on p ksl and produce children population p k ; Evaluate the fitness of the children p k ; Select p ksl from p ksu using selection operator; Apply crossover operator with CP and mutation operators with MP on p ksl and produce children population p k ; Evaluate the fitness of the children p k ; Keep the best from p k ∪ p k to form p k+1 Archive p Max_Generation_U. = p k+1 ; k = k + 1 6: End while 7: Return to the individual (x * , y * ) from p Max_Generation_U. with best fitness values and archive x * 1 = x * 8: Chaotic local search around modified genetic algorithm solution (x * , y * ) 9: Begin i ← 1 10:

End if If termination criteria satisfied, Break End if
i ← i + 1 11: End while 12: Solve lower level problem using genetic algorithm for x = x * 1 13: Begin j ← 1 14: Generate an initial population p j with size N l ; 15: Check feasibility and repair out unfeasible individuals of the population p j ; 16: While j < Max_gen l Evaluate p j according to lower level objective function; Select p js from p j using selection operator; Apply crossover operator with CP and mutation operators with MP on p js and produce children population Evaluate the fitness of the children p j ; Keep the best from p j ∪ p j to form p j+1 ; Archive p Max_Generation_l. = p j+1 ; j = j + 1 17: End while 18: Return to the individual (x * 1 , y * ) from p Max_Generation_l. with best fitness values; 19: Evaluate the fitness of (x * 1 , y * ) 20: Repeat from step 13 to step 18 for x = x *

Numerical Experiments
The proposed algorithm has been evaluated on three sets of well-known test problems including linear and nonlinear problems, constrained and unconstrained problems, low-dimensional and highdimensional problems. The first set of problems is 10 constrained problems with a relatively smaller number of variables named TP problems [27]. TP problems have been extensively used test-suite for comparing different algorithms that have been solved BLPP. The second set is the SMD test set constructed in [41]. The used set of SMD problems are un-constrained high-dimensional problems that are recently proposed, which contains problems with controllable complexities. SMD test problems cover a wide range of difficulties associated with BLPP as non-linearity, multi-modality, and conflict in upper and lower level objectives. The third set is 24 nonlinear standard test problems collected from the literature [42].
For our study, the algorithm has been run 30 times for each test function. The results are median results from the runs. We use Matlab 7.8 to code the proposed algorithm CGA-BS. The simulations have been operated on an Intel Core (TM) i7-4510u CPU @2.00GHZ 2.60 GHz processor. The performance of our proposed algorithm has several parameters that affect it. The GA parameters applied on both upper and lower level problem are

Numerical Experiments
The proposed algorithm has been evaluated on three sets of well-known test problems including linear and nonlinear problems, constrained and unconstrained problems, low-dimensional and high-dimensional problems. The first set of problems is 10 constrained problems with a relatively smaller number of variables named TP problems [27]. TP problems have been extensively used test-suite for comparing different algorithms that have been solved BLPP. The second set is the SMD test set constructed in [41]. The used set of SMD problems are un-constrained high-dimensional problems that are recently proposed, which contains problems with controllable complexities. SMD test problems cover a wide range of difficulties associated with BLPP as non-linearity, multi-modality, and conflict in upper and lower level objectives. The third set is 24 nonlinear standard test problems collected from the literature [42].
For our study, the algorithm has been run 30 times for each test function. The results are median results from the runs. We use Matlab 7.8 to code the proposed algorithm CGA-BS. The simulations have been operated on an Intel Core (TM) i7-4510u CPU @2.00GHZ 2.60 GHz processor. The performance of our proposed algorithm has several parameters that affect it. The GA parameters applied on both upper and lower level problem are GA generation: 200-1000 Generation gap: 0.9 Selection operator: Stochastic universal sampling Crossover operator: Single point Mutation operator: Real-value Crossover rate: 0.9 Mutation rate: 0.07 Number of Chaotic iterations: 10,000 Chaotic rang: 10 −3

Test Problems
For our study, the algorithm has been run 30 times for each test function. The results are median results CGA-BS is tested by 40 test problems. In addition, 10 constrained test problems (TP1-TP10), 6 unconstrained high dimension test problems (SMD1-SMD6) and 24 nonlinear standard test problems collected from the literature (P1-P24). The dimension of TP problems and 24 nonlinear standard test problems is relatively small. The formulation for both TP problems and SMD problems can found at [27,41]. The formulation of standard test problems from P1 to P24 have introduced in Table A1 in Appendix A.

Results Analyses
In this section, we first compare CGA-BS results and other algorithms in the literature which solved the same chosen test set to show the effectiveness and efficiency of our algorithm. Following this, results for the TP test set have been compared with applying BST and without applying it to show how BST increases the algorithm's performance quality. The results for the SMD test set before chaos search and after chaos search have been compared to show how chaos search increases the algorithm's performance quality and accelerate reaching the optimal solution. In addition, proposed algorithm results and other compared algorithms results for the SMD test set have compared in terms of computational complexity to show the proposed algorithm computational time in reaching a solution.

TP Test Set Results Analyses
To demonstrate the effectiveness of CGA-BS, we compare CGA-BS results with results in references [27,36,42,56]. In [27] Sinha et al. presented an improved evolutionary algorithm based on quadratic approximations of lower level named it as BLEAQ. In [42], Wang et al. presented a new evolutionary algorithm based on a new constraint-handling scheme named it NEA. Wang et al. presented in [56] an evolutionary algorithm named it as NEA1. Their algorithm uses an effective crossover operator and they also propose a new mutation operator. They executed NEA1 for 50 independent runs on each of the test problems and recorded the best-obtained solutions and the worst obtained solution. To demonstrate the effectiveness of our proposed algorithm, we compare our results with the best-obtained solutions in [56]. In [36], Zhongpin et al. presented a combination between particle swarm optimization and chaos search technique PSO-CST. [36] named one of the compared results as the corresponding reference. Tables 1 and 2 propose the values of the leader's and the follower's objective functions for CGA-BS and other algorithms. CGA-BS results are introduced in the second column. Third, fourth, fifth, sixth and seventh columns are the results of BLEAQ, NEA, NEA1, PSO-CST, and corresponding reference.  Solving BLPP is executed by solving the lower level problem as a constraint to the upper-level problem. The main aim is to solve the upper-level problem. Therefore, in a comparison between CGA-BS and other algorithms, we are concerned about upper-level results. As Table 1 indicates, our proposed algorithm has the best results for the upper-level problem than other algorithms for six problems TP1, TP2, TP3, TP4, TP6, and TP9. The remainder of problems have also a small difference with the best solutions of compared algorithms. Figure 6 shows the difference between the upper-level objective-function result and the best upper-level objective-function result of the compared algorithms results for TP test problems. This difference is denoted by upper-level accuracy. From Figure 6, the proposed algorithm has the smallest upper-level accuracy which means that the proposed algorithm more converges to the optimal solution. We rank the different algorithms according to the best upper-level solution for all algorithms in Table A2 (see Appendix B). Figure 7 illustrates the ranks obtained with CGA-BS and other algorithms. From the tables and figures, the CGA-BS algorithm obtained the first rank for six problems and reach to the second rank for four problems. From results, we can consider CGA-BS performs well and finds good solutions to TP test problems. Symmetry 2020, 12, x FOR PEER REVIEW 14 of 31

SMD Test Set Results Analyses
The results of SMD test problems are presented in Table 3. The second column presents the optimal solution for the upper level and lower-level problems. CGA-BS results for upper level and lower level problems are in the third columns. As indicated in Table 3, our proposed algorithm reaches optimal solutions for two problems SMD1, SMD3 and reaches near to optimal solutions for the remaining problems. To evaluate the CGA-BS algorithm for SMD problems, we compare CGA-BS results with the other four algorithms listed in [57] and three algorithms listed in [58]. Table 4 illustrates a comparison between CGA-BS and other algorithms. We have been compared algorithms with CGA-BS according to upper-level accuracy. Accuracy is the difference between the algorithm result and an optimal solution. Figure 8 illustrates the upper-level accuracy for compared results. From Figure 8, that proposed algorithm more converges to the optimal solution than other compared algorithms for SMD test problems. In addition, we rank the different algorithms and CGA-ES algorithm in Table A3 (see Appendix B). Figure 9 illustrates the ranks obtained with CGA-BS and other algorithms. From the tables and figures, the CGA-BS algorithm obtained the first rank for four problems from six problems. CGA-BS performs well for SMD test and more convergence to the optimal solution concerning to compared algorithms. It can be concluded that CGA-BS can manage

SMD Test Set Results Analyses
The results of SMD test problems are presented in Table 3. The second column presents the optimal solution for the upper level and lower-level problems. CGA-BS results for upper level and lower level problems are in the third columns. As indicated in Table 3, our proposed algorithm reaches optimal solutions for two problems SMD1, SMD3 and reaches near to optimal solutions for the remaining problems. To evaluate the CGA-BS algorithm for SMD problems, we compare CGA-BS results with the other four algorithms listed in [57] and three algorithms listed in [58]. Table 4 illustrates a comparison between CGA-BS and other algorithms. We have been compared algorithms with CGA-BS according to upper-level accuracy. Accuracy is the difference between the algorithm result and an optimal solution. Figure 8 illustrates the upper-level accuracy for compared results. From Figure 8, that proposed algorithm more converges to the optimal solution than other compared algorithms for SMD test problems. In addition, we rank the different algorithms and CGA-ES algorithm in Table A3 (see Appendix B). Figure 9 illustrates the ranks obtained with CGA-BS and other algorithms. From the tables and figures, the CGA-BS algorithm obtained the first rank for four problems from six problems. CGA-BS performs well for SMD test and more convergence to the optimal solution concerning to compared algorithms. It can be concluded that CGA-BS can manage

SMD Test Set Results Analyses
The results of SMD test problems are presented in Table 3. The second column presents the optimal solution for the upper level and lower-level problems. CGA-BS results for upper level and lower level problems are in the third columns. As indicated in Table 3, our proposed algorithm reaches optimal solutions for two problems SMD1, SMD3 and reaches near to optimal solutions for the remaining problems. To evaluate the CGA-BS algorithm for SMD problems, we compare CGA-BS results with the other four algorithms listed in [57] and three algorithms listed in [58]. Table 4 illustrates a comparison between CGA-BS and other algorithms. We have been compared algorithms with CGA-BS according to upper-level accuracy. Accuracy is the difference between the algorithm result and an optimal solution. Figure 8 illustrates the upper-level accuracy for compared results. From Figure 8, that proposed algorithm more converges to the optimal solution than other compared algorithms for SMD test problems. In addition, we rank the different algorithms and CGA-ES algorithm in Table A3 (see Appendix B). Figure 9 illustrates the ranks obtained with CGA-BS and other algorithms. From the tables and figures, the CGA-BS algorithm obtained the first rank for four problems from six problems. CGA-BS performs well for SMD test and more convergence to the optimal solution concerning to compared algorithms. It can be concluded that CGA-BS can manage the difficulties resulting from the increasing problem dimension. In addition, results show that algorithm FOA also performs well and more convergence to the optimal solution. FOA treat BLPP by combining two improved fruit fly optimization algorithms. Fruit fly algorithm is one of the powerful evolutionary algorithms. It has a huge advantage in that it has a very small control parameter. The results illustrated that CGA-BS and FOA algorithms is an effective technique to solve a large dimension BLPP. Table 3. CGA-BS results and optimal solutions for SMD1-SMD6.

Problem
Optimal Solutions CGA-BS Results    3.47 × 10 −6 ---   Table 5 presents CGA-BS results for P1-P24 problems and the best-obtained solution of the upper level and lower level objective functions in references [36,56]. In Table 6, CGA-BS find solutions better than best-obtained solutions in [36] for all problems solved in it except P17. It can be considered that CGA-BS has superiority over the algorithm PSO-CST algorithm. Figure 10 illustrates the upperlevel accuracy for CGA-BS, NEA1 and PSO-CST algorithms. It can be observed from Figure 10 that CGA-BS more converges to the optimal solution. Ranks CGA-ES, NEA1 algorithm in Table A4 (see Appendix B). Figure 11 illustrates the ranks obtained with CGA-BS and other algorithms. CGA-BS find solutions better than a best-obtained solution in [56] for eleven problems and reach to same upper-level solutions in [56] three problems from twenty-one problems has solved in [56]. CGA-ES obtained the first rank for fourteen problems from the twenty-one problem. It can be considered that CGA-BS performs well than the algorithm in [36,56] for P1-P24.   Table 5 presents CGA-BS results for P1-P24 problems and the best-obtained solution of the upper level and lower level objective functions in references [36,56]. In Table 6, CGA-BS find solutions better than best-obtained solutions in [36] for all problems solved in it except P17. It can be considered that CGA-BS has superiority over the algorithm PSO-CST algorithm. Figure 10 illustrates the upper-level accuracy for CGA-BS, NEA1 and PSO-CST algorithms. It can be observed from Figure 10 that CGA-BS more converges to the optimal solution. Ranks CGA-ES, NEA1 algorithm in Table A4 (see Appendix B). Figure 11 illustrates the ranks obtained with CGA-BS and other algorithms. CGA-BS find solutions better than a best-obtained solution in [56] for eleven problems and reach to same upper-level solutions in [56] three problems from twenty-one problems has solved in [56]. CGA-ES obtained the first rank for fourteen problems from the twenty-one problem. It can be considered that CGA-BS performs well than the algorithm in [36,56] for P1-P24. Table 5. CGA-BS results and best obtained solutions in [35] for P1-P24.   The proposed algorithm has evaluated 40 test problems. A total of 37 test problems weere solved with other algorithms. Our proposed algorithm obtained the best solutions compared to the mentioned references for nineteen problems and reach to same best solutions for five problems. The difference between our results and the best solutions for remaining problems ranges from 1E-1 to 1E-12 except for P4 and P8. Figure 12 indicates the difference between CGA-BS and best-compared solutions in the mentioned references for the upper-level results. The negative difference indicates to CGA-BS has better solutions than best-compared solutions. Figure 13 presents the statistics of the ranks of CGA-BS. Statistics of ranking in Table A5 (see Appendix B). Figure 14 presents the rank's percentage for the proposed algorithm. As indicated in Figure 14, our proposed algorithm obtained first rank for about 65% of solved problems and obtained second rank for about 32.5% of solved problems.

CGA-BS NEA1 [56] PSO-CST [36]
The simulations results of various numerical studies have been demonstrated that CGA-BS effectively solves BLPP and it can manage the difficulties resulting from increasing dimension of the problem. In addition, results have been demonstrated the superiority of CGA-BS on compared algorithms solved the same test problems. The proposed algorithm has evaluated 40 test problems. A total of 37 test problems weere solved with other algorithms. Our proposed algorithm obtained the best solutions compared to the mentioned references for nineteen problems and reach to same best solutions for five problems. The difference between our results and the best solutions for remaining problems ranges from 1E-1 to 1E-12 except for P4 and P8. Figure 12 indicates the difference between CGA-BS and best-compared solutions in the mentioned references for the upper-level results. The negative difference indicates to CGA-BS has better solutions than best-compared solutions. Figure 13 presents the statistics of the ranks of CGA-BS. Statistics of ranking in Table A5 (see Appendix B). Figure 14 presents the rank's percentage for the proposed algorithm. As indicated in Figure 14, our proposed algorithm obtained first rank for about 65% of solved problems and obtained second rank for about 32.5% of solved problems.

Algorithm Performance Analyses with Bilevel Selection Technique
The basic idea of the bi-level selection technique is selecting more appropriate solutions that can increase the solution quality. The individuals are selected for producing the new generation during solving upper-level problem according to both the upper and lower level. Table 6 illustrates the algorithm results for TP test problems with applying the bilevel selection technique (CGA-BS) and without applying it (CGA). Figure 15 illustrates the upper-level accuracy for TP set with and without the bilevel selection technique. The results illustrate that using an effective selection technique improve the solution quality for TP1, TP5, and TP6. We note from results BST does not improve results for all problems. That is due to the conflict between upper and lower level objectives. As a result, applying an effective selection technique enables the algorithm to increase its performance quality and accelerate reaching the optimal solution especially for not conflicted upper and lower objective functions.

Algorithm Performance Analyses with Bilevel Selection Technique
The basic idea of the bi-level selection technique is selecting more appropriate solutions that can increase the solution quality. The individuals are selected for producing the new generation during solving upper-level problem according to both the upper and lower level. Table 6 illustrates the algorithm results for TP test problems with applying the bilevel selection technique (CGA-BS) and without applying it (CGA). Figure 15 illustrates the upper-level accuracy for TP set with and without the bilevel selection technique. The results illustrate that using an effective selection technique improve the solution quality for TP1, TP5, and TP6. We note from results BST does not improve results for all problems. That is due to the conflict between upper and lower level objectives. As a result, applying an effective selection technique enables the algorithm to increase its performance quality and accelerate reaching the optimal solution especially for not conflicted upper and lower objective functions.

Algorithm Performance Analyses with Chaos Search
To analyze our proposed algorithm performance, a comparison between algorithm results before chaos search (GA-BS) and after applying (CGA-BS). Table 7 illustrates algorithm results for SMD test set before applying chaos search and after applying it for both upper level and lower level problem. As illustrated in table 7, the results after applying chaos search more converges to optimal

Algorithm Performance Analyses with Chaos Search
To analyze our proposed algorithm performance, a comparison between algorithm results before chaos search (GA-BS) and after applying (CGA-BS). Table 7 illustrates algorithm results for SMD test set before applying chaos search and after applying it for both upper level and lower level problem. As illustrated in Table 7, the results after applying chaos search more converges to optimal solutions. Figure 16 illustrate the upper level accuracy for SMD set before applying chaos search and after applying it. The results reveal that applying chaos search helps to more converges to optimal solutions and the integrating between genetic algorithm and a chaos search accelerates the optimum seeking operation and helps to more converges to optimal solutions. Table 7. GA-BS and CGA-BS results for SMD1-SMD6.

Problem
Upper

Computational Expense
In BLPP, the feasible space is very large. BLPP consists of two nested optimization problem. The upper-level problem feasible space is the optimal solution of lower-level problem. With another meaning, finding one feasible solution of the upper-level problem requires solving lower-level problem. This huge feasible space makes the computational complexity for BLPP is NP. To evaluate the computational complexity of BLPP, number of function evaluations is used as a measure of the computational time. CGA-BS functions evaluations and other algorithms function evaluations for SMD test set are compared to show proposed algorithm speed convergence to reach to the optimal solution. The upper-level function evaluations of proposed algorithm and BLMA, NBLE, BLEAQ, BIDE algorithms in [57] set are presented in Table 8. The lower-level function evaluations of proposed algorithm and BLMA, NBLE, BLEAQ, BIDE algorithms in [57] are presented in Table 9. The total function evaluations for both upper-level and lower-level of proposed algorithm and other compared algorithms are presented in Table 10. Figure 17 proposed the total function evaluations of proposed algorithm and compared algorithm. As illustrated, proposed algorithm needs a smaller number of total function evaluation than compared algorithms for three problems. In addition, the compared algorithms and CGA-ES algorithm are ranked according to total function evaluations in Table A6 (see Appendix B). Figure 18 illustrates the ranks percentage obtained with CGA-BS. CGA-BS reaches to first rank for fifty percent of SMD test problems with compared algorithms. From results, the

Computational Expense
In BLPP, the feasible space is very large. BLPP consists of two nested optimization problem. The upper-level problem feasible space is the optimal solution of lower-level problem. With another meaning, finding one feasible solution of the upper-level problem requires solving lower-level problem. This huge feasible space makes the computational complexity for BLPP is NP. To evaluate the computational complexity of BLPP, number of function evaluations is used as a measure of the computational time. CGA-BS functions evaluations and other algorithms function evaluations for SMD test set are compared to show proposed algorithm speed convergence to reach to the optimal solution. The upper-level function evaluations of proposed algorithm and BLMA, NBLE, BLEAQ, BIDE algorithms in [57] set are presented in Table 8. The lower-level function evaluations of proposed algorithm and BLMA, NBLE, BLEAQ, BIDE algorithms in [57] are presented in Table 9. The total function evaluations for both upper-level and lower-level of proposed algorithm and other compared algorithms are presented in Table 10. Figure 17 proposed the total function evaluations of proposed algorithm and compared algorithm. As illustrated, proposed algorithm needs a smaller number of total function evaluation than compared algorithms for three problems. In addition, the compared algorithms and CGA-ES algorithm are ranked according to total function evaluations in Table A6 (see  Appendix B). Figure 18 illustrates the ranks percentage obtained with CGA-BS. CGA-BS reaches to first rank for fifty percent of SMD test problems with compared algorithms. From results, the proposed algorithm reaches to the best results in a much shorter time. It can be considered as an effective and comparable algorithm for BLPP.

Conclusions
This paper proposed a genetic algorithm supported by new selection technique and chaos theory to solve bilevel programming problems. The combination between genetic algorithm and chaos search offer the advantages of both genetic algorithm as a powerful global searching technique and chaos search as an efficient and fast searching technique. The new effective selection technique increases algorithm performance quality and accelerate reaching to the optimal solution. Extensive numerical experiments for multimodal test functions including ten constrained problems with relatively smaller number of variables, six unconstrained high-dimensional problems and twentyfour nonlinear standard test problems are performed to verify the performance of CGA-BS. The CGA-BS results have analyzed and compared with other state-of-the-art algorithms to show demonstration and efficiency of our proposed algorithm. A careful observation of proposed algorithm results and other state-of-the-art results will reveal to the following benefits: CGA-BS can be considered as a global powerful efficient technique to solve BLPP and can treat with the difficulties associated with BLPP as non-linearity, multi-modality and confliction between the upper and lower level objectives.
CGA-BS deals with several sets of problems including linear and nonlinear problems, constrained and unconstrained problems, low-dimensional and high-dimensional problems CGA-BS has better ability and precision than other proposed methods in literature and was demonstrated to be extremely efficient at locating optimal solutions.
The new selection technique helps the upper level decision maker to choose a solution appropriated to lower level problem and more convergence to the optimal solution by dispensing solving lower level problem for every generation during solving upper level problem The new effective selection technique enables the algorithm to increase its performance quality

Conclusions
This paper proposed a genetic algorithm supported by new selection technique and chaos theory to solve bilevel programming problems. The combination between genetic algorithm and chaos search offer the advantages of both genetic algorithm as a powerful global searching technique and chaos search as an efficient and fast searching technique. The new effective selection technique increases algorithm performance quality and accelerate reaching to the optimal solution. Extensive numerical experiments for multimodal test functions including ten constrained problems with relatively smaller number of variables, six unconstrained high-dimensional problems and twenty-four nonlinear standard test problems are performed to verify the performance of CGA-BS. The CGA-BS results have analyzed and compared with other state-of-the-art algorithms to show demonstration and efficiency of our proposed algorithm. A careful observation of proposed algorithm results and other state-of-the-art results will reveal to the following benefits: CGA-BS can be considered as a global powerful efficient technique to solve BLPP and can treat with the difficulties associated with BLPP as non-linearity, multi-modality and confliction between the upper and lower level objectives.
CGA-BS deals with several sets of problems including linear and nonlinear problems, constrained and unconstrained problems, low-dimensional and high-dimensional problems.
CGA-BS has better ability and precision than other proposed methods in literature and was demonstrated to be extremely efficient at locating optimal solutions.
The new selection technique helps the upper level decision maker to choose a solution appropriated to lower level problem and more convergence to the optimal solution by dispensing solving lower level problem for every generation during solving upper level problem.
The new effective selection technique enables the algorithm to increase its performance quality and accelerate reaching to the optimal solution.
The search using chaos theory enhances the search characteristics, helps in faster convergence of the algorithm.
CGA-BS not only reaches to good solution for BLPP. It can reach to it with acceptable running time.
In our future works, the following will be researched:  Table A1 proposes a formulation of standard test problem from P1 to P24. Problem dimensions is denoted by (n, m). Dimensions (n) denotes to the upper level dimensions. Dimensions (m) denotes to the lower level dimensions. Table A1. BLPP formulation for problems P1-P24.