Abstract
With the rise of big data in cloud computing, many optimization problems have gradually developed into high-dimensional large-scale optimization problems. In order to address the problem of dimensionality in optimization for genetic algorithms, an adaptive dimensionality reduction genetic optimization algorithm (ADRGA) is proposed. An adaptive vector angle factor is introduced in the algorithm. When the angle of an individual’s adjacent dimension is less than the angle factor, the value of the smaller dimension is marked as 0. Then, the angle between each individual dimension is calculated separately, and the number of zeros in the population is updated. When the number of zeros of all individuals in a population exceeds a given constant in a certain dimension, the dimension is considered to have no more information and deleted. Eight high-dimensional test functions are used to verify the proposed adaptive dimensionality reduction genetic optimization algorithm. The experimental results show that the convergence, accuracy, and speed of the proposed algorithm are better than those of the standard genetic algorithm (GA), the hybrid genetic and simulated annealing algorithm (HGSA), and the adaptive genetic algorithm (AGA).
1. Introduction
High-dimensional optimization problems have become increasingly prevalent in many fields, for example, radar waveform optimization [1, 2] and water quality monitoring [3]. Recently, major breakthroughs have been made with respect to solving such problems. Tuo et al. [4] proposed a global optimization algorithm that used the membrane calculation principle to solve high-dimensional functions in 2011. Their algorithm, by implementing high-dimensional segmentation, achieved the segmentation of a high-dimensional space into a low-dimensional one, thereby improving the performance. Also addressing the problem of high-dimensional space processing, Chen et al. [5] described an approach that established a sparse regression model by utilizing the geometric structural features of the approximate solution set, mapping the high-dimensional target space into a low-dimensional one. In 2015, Chen et al. [6] designed a congestion control strategy based on the concept of open angles and compared it against the indication-based evolutionary algorithm (IBEA) [7], NSGA III (nondominated sorting genetic algorithm III) [8], and the grid-based evolutionary algorithm (GrEA) [9]. The results indicated a significant improvement by the proposed algorithm. In the same year, Zheng et al. [10] proposed a high-dimensional multiobjective evolutionary algorithm based on information separation. Although this algorithm decomposed the high-dimensional space into a low-dimensional one, it did not remove the excess dimensions. In order to maintain a balance between the convergence and distributed features of the objective evolutionary algorithms, Bi and Wang [11] proposed a high-dimensional objective evolutionary algorithm based on an angle penalty. In 2018, He et al. [12] combined a dimensionality reduction and differential evolution algorithm to solve the knapsack problem. The experimental results showed that this algorithm achieved good accuracy and stability and is suited for solving large-scale problems. Liang et al. [13] first conducted an analysis of the features of large-scale high-dimensional problems and developed a coevolutionary dynamic particle swarm optimization algorithm, dividing the whole swarm into different groups. Unlike the improvement targeting the number of dimensions, Xia et al. [14] proposed a fitness-based multiplayer swarm optimization algorithm (FMPSO), with the addition of a new component into the particle velocity update rules, called the “subspace learning” component. During the evolutionary process, two adjustment operators were introduced to adjust the roles and number of objective dimensions of the particles. Xu et al. [15] proposed a multidimensional learning strategy based on the experience of the best individuals, which was used to discover and integrate valuable information from the best swarm solution. In their experimentation, 16 classical benchmark functions, 30 CEC 2014 testing functions, and one actual optimization problem were used. This algorithm demonstrated higher convergence, accuracy, and speed. In 2015, Wu et al. [16] described a nondominated sorting genetic algorithm utilizing fractal dimensions and an elitist strategy for feature selection. This algorithm could successfully reduce the number of dimensions for the objective problems. This algorithm, however, only improved the classification accuracy as compared with the standard genetic algorithms to a limited extent. He and Yen [17] used the minimum included angle between two vectors to look for similar individuals, among which the individuals with poor convergence were deleted. However, it was still difficult to set the difference threshold. Xiang et al. [18] implemented priority selection based on the maximum included angle between two vectors, thereby ensuring the diversity of the swarm. However, a good strategy for determining the angle threshold was still not provided. Furthermore, the U-model methodology, which is a generic systematic approach to convert a nonlinear polynomial model into a controller output u(t − 1) -based time-varying polynomial model, has been studied for facilitating a nonlinear control system design over the last decade [19–22]. Given the aforementioned, we define the concept of an adaptive vector angle factor, which changes with the swarm size and number of dimensions. In this paper, the adaptive vector angle factor is used for realizing dimensionality reduction with a GA. At the same time, it also provides a reference for U-model approaches in algorithm implementation.
2. The Proposed ADRGA Algorithm
2.1. Vector Angle Factor
Dimensionality reduction is a good approach for solving large-scale problems. However, most dimensionality reduction algorithms are random processes; those dimensions carrying important information or those crucial for objective problem solving may be deleted. Consider a 2D vector coordinate (a, b) in Figure 1. When the included angle θ between this vector and the x-axis (y-axis) is infinitely small, the vector coordinate is approximated to (a, 0) or (0, b). In this way, the 2D vector is approximately mapped into a one-dimensional one, as illustrated in Figure 2. Similarly, a 3D vector can be approximately mapped into a 2D or one-dimensional one, depending on the set threshold for the included angle between the two vectors. Apparently, setting an appropriate threshold for the included angle between the two vectors is very important.
The dimension describes the precision of the required solution target characteristics. For example, when describing a pen in one dimension, only its length or width can be described, and so it is impossible to perform dimensionality reduction through mapping. In this case, the threshold for the included angle θ between the two vectors must be close to 0. When a pen has two dimensions, both its length and width can be described. At this point, if the included angle between the two vectors is smaller than a certain threshold, the dimension where the smaller values are located can be neglected. In this case, the threshold for the included angle θ between the two vectors may be close to a value above 0. The threshold range can be set according to an individual problem. When three dimensions are used for the description, not only the pen’s length and width but also its cross-sectional radius can be described. Therefore, the threshold for the included angle θ between the two vectors is larger than that for the 2D situation. As the number of dimensions increases, the threshold range of the included angle θ between the two vectors increases as well. Experiments have shown that the threshold range cannot be infinitely large. Whatever the number of dimensions, the threshold must be maintained within the interval . In this paper, the vector included angle factor is represented by the function in equation (1), which is a monotonically increasing function and also a bounded function:where .
The vector included angle factor proposed in this study must be within the interval . An adaptive vector angle factor can be obtained by transformation of the function, as shown in equation (2), where is the control parameter, , is the adjustment parameter, , and is the number of dimensions. When the tangent of the included angle between the two vectors in adjacent dimensions , the dimension where the smaller value is located is denoted as 0 (, is the -th dimension of the -th individual; is the -th dimension of the -th individual).
2.2. ADRGA Workflow
ADRGA targets high-dimensional swarms. First, the tangent value of the included angle between the two vectors in adjacent dimensions is calculated for each individual and is compared against the adaptive vector angle factor . If it is smaller than the latter, the dimension where the smaller value of the individual is located is denoted as 0. Then, using the same method, the tangent values of the included angle between the two vectors in adjacent dimensions for all individuals are compared against . Finally, the number of 0 elements in each dimension in the updated swarm is determined; if it is above the critical value , then this dimension is deleted.
Adaptive dimensionality reduction is feasible following the principle mentioned above. The pseudocode for ADRGA is shown in Algorithm 1 ( is the probability of crossover, is the probability of mutation, is the swarm size, is the number of generations upon termination of evolution, is the number of tests, is the critical value).
|
3. Experimental Results
Simulation was carried out using MATLAB 2014b under Windows 7 (also applicable to a higher version). A comparison was made between ADRGA, the standard GA [1], the hybrid genetic and simulated annealing algorithm (HGSA) [23], and the adaptive genetic algorithm (AGA) [24].
In this study, 8 standard composite testing functions were used, namely, ∼, as shown in Table 1. ∼ are high-dimensional unimodal functions, which only have one global best solution; ∼ are high-dimensional multimodal functions, which have several local best solutions, but only one global best solution. The latter are generally hard to optimize. The global best solutions of all testing functions were 0. Since some testing functions have several optimal solutions, the algorithm is very likely to be trapped in a local optimum. This aspect may be challenging for the proposed ADRGA algorithm.
In the experimentation, the value of the fitness function was the function value at the current position. The parameters of the GA were configured as follows: crossover probability = 0.70, mutation probability = 0.05, number of iterations = 1000, swarm size = 50, number of dimensions = 1000, and the critical value = 1.
Each of the four algorithms was run 20 times on the 8 testing functions, and the means and standard deviations are shown in Table 2. Table 3 presents the improvement percentages of the optimal solutions using ADRGA as compared with the other three algorithms, calculated as follows: (mean of other algorithm−mean of ADRGA)(mean of other algorithm). When optimizing these 8 classical testing functions, ADRGA could obtain a solution closer to the global best solution compared to the other algorithms, as shown in Table 2. This was especially true for the testing functions , , and . But on the other testing functions, although a better solution was obtained using ADRGA, it failed to find the global optimum and was still far from achieving this goal. As shown in Table 3, the higher the improvement percentage, the greater the amplitude of performance improvement of ADRGA would be, and vice versa. Although a solution closer to the global optimum was obtained for in Table 2, its improvement percentage was smaller than for the other 7 functions, indicating that was less influenced by the number of dimensions.
The number of dimensions after dimensionality reduction and the average number of dimensions after running each algorithm 20 times are shown in Table 4. The number of residual dimensions varied after each run, indicating that the dimensionality reduction with the proposed algorithm changed with the initialized swarm in each run. Therefore, valuable information could be preserved when reducing the number of dimensions. Under the similar minimum included angle between the two vectors, the larger the included angle for the newly generated swarm, the smaller the number of dimensions deleted would be, and vice versa. As shown in Table 3, the maximum number of dimensions deleted was 40%, and the minimum was 34%. The maximum number of dimensions deleted was not above 50%, but it was ensured that the dimensions were not deleted randomly. Table 4 provides a better proof of the adaptive features of the proposed algorithm.
Figure 3 shows the comparison of convergence for the eight testing functions ∼. The x-axis is the number of fitness calculations, and the y-axis is the mean fitness value. As shown in Figure 3, ADRGA had a faster convergence speed and accuracy on the functions , , , , and , though it was more likely to be trapped in a local optimum. For the testing function , the convergence curves of ADRGA and GA nearly coincided. This was because the value range of the independent variable in function was [−1, 1]. Therefore, when the numbers of dimensions were 600 and 1000, the difference was negligibly small. This also demonstrated that the impact on was small if the number of dimensions did not change significantly. For the multimodal function , the convergence curve of ADRGA not only had a faster convergence speed but also showed a more apparent decreasing trend as compared with the other six testing functions (except for ). This suggested that ADRGA was not trapped in the local optimum on . For the testing function , ADRGA outperformed all the other algorithms. There was a dramatic improvement in the convergence accuracy and speed with ADRGA, and the convergence curve displayed a more apparent decreasing trend. For the five high-dimensional multimodal functions, ADRGA achieved better results on two testing functions, indicating that this algorithm was more appropriate for solving high-dimensional multimodal targets.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
4. Conclusion
This paper aims to find the optimal solutions for high-dimensional testing functions based on GAs. Similarity, the minimum included angle between the two vectors is utilized, and a GA approach for adaptive dimensionality reduction is proposed. The specific steps for dimensionality reduction are provided. The critical value Q is defined to ensure the diversity of the swarm information. It was then demonstrated via eight testing functions that ADRGA displays faster convergence speed and higher accuracy than other algorithms. However, a major problem with ADRGA is that the critical value Q is prespecified instead of changing with the swarm size and number of dimensions in real-time. In the future, the critical value Q will be the focus in order to achieve better control of the number of dimensions in the swarm and more accurate dimensionality reduction.
Data Availability
The data used to support the study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
This work was supported in part by the National Natural Science Foundation of China under Grant U1809209 and the Zhejiang Provincial Natural Science Foundation (LZ15F030002) and the Zhejiang Provincial Natural Science Foundation under Grant LY16F020022.