A Novel Prediction Strategy Based on Change Degree of Decision Variables for Dynamic Multi-Objective Optimization

Effectively balancing the convergence and diversity in dynamic environments is a challenging task. In order to handle the issue, this paper proposes a novel prediction strategy based on change degree of decision variables for dynamic multi-objective optimization (CDDV), which has the ability to detect the change degree in the decision space and design the different prediction strategy to make the population adapt to the new environment. The proposed method can adaptively increase population diversity according to the analysis of change degree, when an environmental change is detected. In order to study the efficacy and usefulness of the novel change degree on evolutionary algorithms, a range of dynamic multi-objective benchmark problems are selected to evaluate the performance of the proposed algorithm. The results demonstrate the effectiveness of proposed algorithm in compared with four other state-of-the-art methods.


I. INTRODUCTION
Dynamic multi-objective optimization problems (DMOPs) [1]- [4] not only have multiple conflicting objectives, but also need to deal with the tradeoffs and the time-dependent objective functions, constraints and/or parameters.Although static multi-objective optimization evolutionary algorithms (MOEAs) [5], [6] have the superior capability on addressing the static multi-objective optimization problems, it still is a challenging task as static MOEAs lack the capability of dynamic optimization [7]- [9] on dealing with DMOPs.Nevertheless, there has been an increasing amount of research interest in the field of dynamic multi-objective optimization (DMO) for better solving DMOPs since many real-world applications, like dynamic scheduling [10]- [13], path planning [14]- [16], resource allocation [17], [18], and machine learning [19].Moreover, the effective EAs must be created to attain the goals [20] of convergence and diversity with high The associate editor coordinating the review of this manuscript and approving it for publication was Xin Luo .efficacy over time in dealing with DMOPs, this kind of EAs is called DMOEAs to distinguish MOEAs.
DMOEAs are different from the MOEAs [21], DMOEAs pose a higher requirement for efficiency due to the changing environment (i.e., MOEAs once converged, cannot adapt quickly to environmental changes [22]).Particularly, DMOEAs is not to find a set of trade-off solutions, but to find different set of trade-off solutions in different time instants during the same optimization in dealing with DMOPs.Because the training period at each time instants is usually short (which is set as 10/20/30 generation evolution in most of literatures [23]- [25]).Furthermore, DMOEAs are required to have the ability of guiding the new population toward Pateto-optimal solutions (POS) after a environment change.The new population is likely to replace the best population at previous time instant.For attaining this goal above, the existed static algorithms by combining with change response [26], [27] to create efficient DMOEAs seemed to have become increasingly popular.Thereinto, the design of the change response mainly consider the two key problems [28] as follows: 1) tracking ability of the population in dynamic optimization and 2) addressing diversity loss ability whenever there is an environmental change.
In most of DMO literature, the main change response can be classified into the following four categories: memory approaches [29], prediction approaches [8], [30], [31], diversity approaches [32], [33], multipopulation approaches [20], [28] and hybrid strategy [34].These approaches can handle the existing DMOPs better than static MOEAs and for the future development of the DMO have made tremendous research contributions and reference value.However, they hardly refer to the influence generated by the environmental change in the decision space and/or the objective space.To the best of the authors knowledge, this idea was proposed to solve the single-objective optimization problems by Woldesenbet and Yen [35], which relocates individuals of the population based on their change in function value due to the change in the environment and the average sensitivities of their decision variables.In the DMO field, Liu et al. [36] proposed an adaptive diversity introduction method, which defined the extent of environmental change as the degree of deviation between the old POF and the new POF.
Although the approaches aforementioned have been recognized by the DMO field, the DMO field still have the prodigious room of improvement for realizing the DMOEAs goal of possessing fast convergence capabilities to track the varying optimum solutions effectively either through inherent design or by incorporating additional dynamic handling techniques [22].Therefore, we propose dynamic multi-objective optimization algorithm based on change degree1 of decision variable for dynamic multi-objective optimization, which adaptive increases the diversity of population for handling DMOPs.The proposed algorithm is different from the existed prediction approaches, it can be adjusted adaptively according to the change degree of the environmental change in the decision space.The main contributions of this study are summarized as follows: 1) How to detect the effect of the environmental change on the each dimension of the decision space?This implementation is called 'the change degree of the decision space'.Here, we calculate the change degree of the decision space according to the previous two consecutive non-dominated solutions (that are gained by the proposed algorithm), which aims to discover the affection of the environmental change to each dimension during the decision space by numerical manner.2) How to make the decision variables divide into two categories according to the change degree?what are its advantages?For the first problem, we fist introduce the concept of the middle change degree (that is introduced by Eq 7).Then if the change degree of one dimension is less or equality than the middle change degree, this dimension is regard as having the smaller change degree; else this dimension is regard as having the bigger change degree.This two categories point out that the affection of the current environmental change is bigger or smaller for each dimension of the decision space, respectively.3) How to structure an efficient change response according to the two categories aforementioned?The dynamic environment may have different affection on each dimension of the decision space.Therefore, the biggish change degree of the decision variables is assumed that they hardly are optimized to the best status only by the evolutionary algorithms.Thus, for improving the performance of algorithms, the optimization intensity should be increased in dynamic optimization for these decision variables with having the biggish change degree.On the contrary, the optimization intensity is relatively weak for these decision variables with having the small change degree.Therefore, the proposed algorithm owns the bigger competitiveness than the other algorithms on the speed of the convergence and the maintained diversity.The paper is structured as follows.Section II gives the background of this work, including some definitions of dynamic multi-objective optimization and a brief review of related work.Section III introduces the proposed algorithm.Section IV provides test instances, performance metrics, compared algorithms, the algorithm setting and the experimental results.Section V presents some discussion, including the influence of change degree and an analysis of computational complexity.Finally, Section VI concludes this study and introduces some potential future directions.

II. BACKGROUND A. DYNAMIC MULTI-OBJECTIVE OPTIMIZATION
In this paper, we consider the minimization DMOPs [8], which can be presented as follows: where t represents the time variable, and T are the lower and upper bounds, respectively.
The definition of DMOPs is a standard formula proposed by Farina et al. [3] and the formula is used in most literature [1].The time instance, t, is a discrete time defined [9] as where τ is the generation number, n t is the number of distinct steps in t (i.e., the severity of change) and τ t is the number of generations (i.e., the frequency of change) for which t remains the same.Definition 1 (Pareto Dominance [5]): Assume that p and q are any two individuals in the population; p is said to Algorithm 1 The Overall Framework of CDDV Input: N (the size of the population), t (time step) Output: P t (a series of approximated POSs and POFs) 1: Initialize a population P t , set time instance t = 0; 2: while the stopping criterion is not met do smc, bmc 1: Calculate the change degree of the decision space cs using Eq.6; 2: Calculate the middle change degree of the decision space mcs using 7; 3: for cs j ∈ cs do dominate q, written as Definition 2 (Pareto-Optimal Set (POS) [3], [25]): x is the decision vector; is the decision space; F is the objective function.A solution is said to be nondominated if it is not dominated by any other solutions in .Thus, the POS is the set of all nondominated solutions and can be defined mathematically as follows: Definition 3 [Pareto Optimal-Front (POF)]: x is the decision vector; is the decision space; F is the objective func-Algorithm 3 ChangeResponse Input: P t (population at t-th time step) Output: P t (the reinitialized population) 1: (smc, bmc) ←− ChangeSeverity(); 2: for x i t ∈ P t do 3: if cs j ∈ smc then 5: if rand[0, 1] ≤ 0.5 then 6: Reinitialize x i,j t using Eq.8; 7: end if tion.Thus, the POF is the set of all nondominated solutions with respect to the objective space and can be defined mathematically as follows: B. RELATED WORKS Many DMOEAs have been proposed in recent years for dealing with DMOPs well, and most existing algorithms can be classified into the different categories (i.e., memory approaches [37], [38], prediction approaches [7], [9], [22], [31], diversity approaches [32], [33], multipopulation approaches [20], [28] and hybrid strategy [34]), not restricted to evolutionary DMO.This section mainly focuses on the prediction approaches.Meanwhile, we also display some computations [35], which are used in detecting the affection of the environmental changes on the population.Why design the prediction approaches on dealing with DMOPs?For the first problem, changes in dynamic environments may exhibit some predictable patterns [7].Consequently, most of literatures have been reported according to this idea to exploit the predictability of dynamic environments.For example, Zhang et al. put forward novel prediction strategies [9]  prediction method [8] by detecting the type of the POS change; Cao et al. put forward a novel prediction model combined with a multi-objective evolutionary algorithm based on decomposition [39]; Sun et al. enhanced NSGA-II with evolving directions prediction [40]; and so on.These kind of algorithms have solved the DMOPs to some extend, but their effectiveness is still questionable if the change in the environment is, as DMOPs with different the challenge characteristic lead to mostly happens or hardly predictable.
To the best of the authors' knowledge, there are relatively few studies in the literature focusing on the influence of the environmental changes.This idea is proposed [35] to solve the single-objective optimization problems by Woldesenbet et al, which relocates individuals of the population based on their change in function value due to the change in the environment and the average sensitivities of their decision variables.In the DMO field, there is little literature on the influence of the environmental changes, and this work has been done primarily with respect to the objective space.For example, Liu et al. [36] proposed an adaptive diversity introduction method, which defined the extent of environmental change as the degree of deviation between the old POF and the new POF.Rong et al. [7] put forward multidirectional prediction approach, which the number of clusters is adapted according to the intensity of the environmental change; And Azzouz et al. [29] proposed a dynamic multi-objective evolutionary algorithm using a change severity-based adaptive population management strategy, which adjusts the number of memory and random solutions to be used according to the change severity.Consequently, more research needs to be done about the influence of the environmental changes on the decision variables of the decision space.And how to use this kind of technique to help the optimization for dealing with DMOPs should be considered.

III. PROPOSED ALGORITHM
The dynamic optimization merely happens after detecting an environmental change [9] in the whole optimization process.Therefore, how to successfully promote the competition and cooperation between the forces of multi-objective optimization and dynamic optimization is a key design point for EAs with change response.When the balance between multi-objective optimization and dynamic optimization is not appropriate, the hybrid EAs is not to achieve the goal of convergence and diversity with high efficacy during the short time.To address the above problems, this paper mainly consider the following several important issues of dynamic multi-objective optimization (DMO): 1) How to calculate the change degree of the solutions in two consecutive POS in the decision space?2) How to reflect the change degree of the current population by the change degree of solutions in the decision space?3) Why is the concept of change degree put forward in the decision space, what are its advantages?4) What are the innovations about the prediction strategy based on change degree of the decision variable compared to the other strategies?The overall framework of the CDDV is presented in Algorithm 1.In the following subsection, some key constituent of the proposed algorithm is described in detail.

A. THE CALCULATION OF CHANGE DEGREE
Dynamic multi-objective optimization may be affected significantly, due to the change of dynamic environments it leads to the optimization algorithm that cannot readily find the search directions of the POS and/or POF.In short, the environmental change will cause different degree of influence to each dimension of the decision space.Therefore, how to reflect change degree of the current population caused by the dynamic environments under each dimension of decision space is an important assignment for enhancing the performance of an algorithm.This section aims to discover the effect of the environmental change to each dimension during the decision space by numerical manner.This success laid a foundation for the key design point in the change response of Section III-B.
We assume that POS t and POS t−1 are the nondominated solutions at time t and t −1, respectively.Meanwhile, suppose that x i t = (x i,1 t , x i,2 t , . . ., x i,n t ) is the i-th nondominated solution in POS t , where K (i.e., i ∈ (1, 2, • • • , K )) is the number of the solutions in POS t .For each solution x i t (x i t ∈ POS t ), its nearest solution y near (y near ∈ POS t−1 ) is defined as follow: where y − x i t 2 represents the Euclidean distances between y and x i t ; y is a solution of POS t−1 ; i is gained by a set (1, 2, • • • , K ), which is expressed as i ∈ (1, 2, • • • , K ).Then, the change degree of the decision space is exhibited by the cs = (cs 1 , cs 2 , . . ., cs n ) T , which can be calculated by the following formulate: where j is gained by a set (1, 2, is the j-th dimension value of the solution x i t in the decision space; cs j represents the change degree of the j-th dimension in the decision space.
According to the above calculation about the change degree of the decision space, we can make the decision variables divide as the different types.This is an advantage for us to analyze the influence degree of each dimension in the decision space due to the environmental change.In DMOPs, if a decision variable has bigger change, which indicates that it cannot gain the best results only by EAs during the short time.Hence, those kind of the decision variables need be improved by the change response, which distinctly differs from most of the existed other strategies of the change response when the environment changes.In Section III-B, we will introduce its advantage in detail.
At first, we make rank form low to high the change degree of the decision variables.And the smallest and the biggest change degree are saved scs and bcs, respectively.The middle change degree mcs is calculated as follows: Afterwards, if cs j < mcs, we make cs j save to the set smc, implying that the j-th dimension of the decision space has the smaller change degree; else cs j is saved to the set bmc, signifying that the j-th dimension of the decision space has the larger change degree.To illustrate the process of classification of the decision space, an example is given in Fig. 1.The straight line represents cs after the rank form low to high the member cs j .The mcs make all cs j divided into two parts.cs j is placed in the set smc when cs j between the scs and mcs.Else it is put in the set bmc. Algorithm 2 provides the implementation about the change degree of the decision space.

B. CHANGE RESPONSE
When a change is detected, the diversity of population is loss which may make population trap in local optima and/or not find the optimization search region, which may lead to the current population to fail to find the POF and/or POS.For the above issue, the most of methods were proposed to address it.However, they often ignore the influence of dynamic environments on the each dimensions in the decision space.Therefore, to achieve the optimization goal of tracking the moving POF and/or POS and obtain a sequence of approximations with high efficacy during the short time, we propose a method based on the change degree to improve the tracking speed.
Firstly, for the j-th dimension of the decision space having the smaller change degree, it may be re-initialized by the preset calculation model at a certain probability.The preset calculation model is presented as follows: where ε 1 ∼ N (0, a 1 ) is a Gaussian noise.a 1 is defined as follows: Secondly, if a decision variable cs j has bigger change degree, it not only increase diversity of population.Therefore, it can be reinitialized by following formula: where ε 2 ∼ N (0, a 2 ) is a Gaussian noise.a 2 is defined as follows: It is worth noting that the Gaussian noise from Eq. 8 and Eq. 10 can increase the probability of the reinitialized population to cover the POS in the new environment.This benefits have been testified by Jiang and Yang [21] and Zhang et al. [9].This change response carried out different tactics for the different dimensions of the same solution according to the two kind of decisions with different change degree.Its advantage is to gain a population with keeping a good balance between the convergence and the diversity in the whole optimization.The variables with the bigger change degree of the decision space should be emphasized.This is because those variables hardly be optimized to the best condition only using NSGA-II [5].And if those variables of the decision space are regarded as the same as the variables with the smaller change degree, the variables with the smaller change degree may be excessively optimized so that the current population still not is improved in dynamic optimization.Therefore, the different degree optimization for the variables with the bigger and smaller change degree of the decision space is very important and necessary.

IV. EXPERIMENTAL DESIGN A. TEST INSTANCES AND PERFORMANCE METRICS
Similar to the other DMOEAs [8], [9], the proposed algorithm is tested on the JY test problem [1] with different dynamic characteristics and difficulties.In the JY test suit, mixed POFs in terms of convexity and concavity and complicated diversity-resistant schemes are considered.Furthermore, nonmonotonic and time-varying relationship between variables is introduced for facilitating theoretical analysis.Afterward, we adopt a number of the performance metrics, including the mean inverted generational distance (MIGD) [9], [31], hypervolume difference (HVD) [21], mean Schott's spacing metric (MSP) [9], [20] and mean generational distance (MGD) [20].As they can help deeply investigate the competition of the proposed algorithms performance in the comparison with state-of-the-art algorithms [21], [31], [32] regarding convergence and distribution for dynamic multi-objective optimization.

B. COMPARED ALGORITHMS AND PARAMETER SETTINGS
In the paper, SGEA [21], PPS [31], DNSGA-II-A and DNSGA-II-B [32] are served as benchmark algorithms in the comparative studies.Several algorithms are selected because of the good performance in DMO.In order to ensure the fair comparative studies, the parameter settings of all algorithms are same, and all algorithms have the same number of fitness evaluations for each experimental setting.Furthermore, some key parameters in these algorithms are set in Table 1.The Wilcoxon ranksum test [41] is conducted to test the significant performance difference between the CDDV and other algorithms at the 0.05 significance level.‡ and † indicate CDDV performs significantly better than and equivalently to the corresponding algorithm, respectively.

C. EXPERIMENTAL RESULTS AND ANALYSIS
In order to compare the effect of change frequency on the compared algorithms in dynamic environments, the severity of change (n t ) was fixed to 10, and the frequency of change (τ t ) was set to 10, 20 and 30, respectively.Meanwhile, the obtained average MIGD, HVD, MGD and MSP results over a series of time steps and their standard deviation values are presented in Tables 2, 3, 4 and 5, respectively, where the best metric results obtained by one of five algorithms are highlighted in the bold face.In the following paragraphs, we will explain these results in detail by the overall metrics MIGD and HVD together with GD and MS to deeply and extensively reveal the algorithms' performance on each test instances.
Table 2 displays all MIGD values obtained by five algorithms, which clearly show that CDDV has the best results on most of test instants.For JY4, the performance of CDDV has some fluctuation when τ t = 30.The reason is that the solutions obtained by CDDV do not uniformly distribute to the whole POF (i.e., the values of the SP metric obtained by CDDV are slightly worse than SGEA in Table 5) under a low frequency of change.It is worth noting that, for JY8, the performance of CDDV is worse than SGEA (i.e., the MIGD values obtained by CDDV are bigger SGEA).Clearly, the uncompetitive distribution (i.e., slightly large SP metric in Table 5) and poor convergence (i.e., relatively large GD metric in Table 4) of obtained approximations are the main reasons for the low performance of CDDV on JY8, when τ t changes from high to low, respectively.It is very interesting that, when τ t = 10 and τ t = 20 for JY6, SGEA realizes the better distribution (i.e., the small SP metric in Table 5) and the good convergence (i.e., the small GD metric in Table 4), but it does not have satisfying IGD metric.This may be because the extent of the best solutions covered POF is not enough.For JY4, JY7 and JY9, although CDDV dose not have the best SP metric, its overall performance is the best than the other algorithms.This may be because other algorithms have poor convergence; the difference between CDDV and the best methods is very small.Even if there exist different levels of change severity (i.e., τ t from 10 to 30), CDDV can still achieve better performance than other algorithms in most of test instances.This demonstrates that the proposed prediction technique has the excellent tracking ability.
Table 3 presents the HVD metric obtained by five algorithms on the JY problems.That has shown that the obtained values of the HVD metric by CDDV are basically consistent with the MIGD values in Table 2.It is obvious that CDDV is more competitive than the other compared algorithms on most of test instances.However, it is worth noting that the results of the MIGD and HVD metrics are conflicting in Tables 2 and 3 on JY6 and JY8.Particularly, for JY6, the inconsistency between these two measures suggests that, CDDV may find dominance resistant solutions (DRS) that affect the calculation of HVD metric; for JY8, the IGD metric obtained by CDDV is inferior to PPS, but the HVD metric is the smallest.This is likely due to the fact that HVD is sensitive to knee points in the population, suggesting CDDV seeds effective for identifying knee points.
It can also be observed from the results of the four selected metrics that, the frequency of change has a significant effect on algorithms' performance.The metrics' values gained by the different algorithms are better and better with the change of τ t from 10 to 30.This is because the frequency of change is smaller and smaller as it increased from 10 to 30.During the whole optimization process, the proposed algorithm is better on the majority of the test instances than the other start-of-the-art algorithms compared from the four metrics used.
To give readers a more intuitive understanding of the performance of the algorithm, we provide evolution curves of the average IGD values on the test instances in Fig. 2. It can be clearly seen that, the curve obtained by CDDV usually is under that the other curves gained by the compared algorithms.This implies that it has effectively response ability when the environmental change is detected, such as tracking the time-dependent POF and/or POS; and providing diversified solutions that enable a timely adaptation to the changing environments.For a graphical view of algorithms' tracking ability, we select JY1 and JY6 to plot the obtained POFs over six time step as shown in Fig. 3 and 4. Fig. 3 indicates that CDDV has better convergence distribution in the early stages, implying that it is very capable of tracking environmental changes.For multimodal problem JY6, CDDV performs better than other algorithms, which CDDV is suitable to handle the multimodal problem.

V. MORE DISCUSSIONS A. INFLUENCE OF CHANGE DEGREE
To examine the influence of the different severity of change level on algorithms' performance, JY1-JY9 test problems are handled as the different algorithms with τ t fixed to 10 and n t set to 5, 10, 20 for severe, moderate, and slight environmental changes, respectively.This relevant experimental results are presented in the Table 6.
It can be observed from the Table 6 that all the algorithms are very sensitive to the severity of change.In most case, CDDV significantly was superior to the compared algorithms on the majority of the instances.For JY6, as the n t decreases, the CDDV has a slightly worse performance than SGEA.The reason is that CDDV needs be improved to handle the complex test problem.In general, the CDDV has more robust performance compared with other algorithms in different changing level.

B. COMPUTATIONAL COMPLEXITY OF CDDV
The overall framework of CDDV is based on NSGA-II [5], which is a representative of algorithm.In the algorithm, NSGA-II consumes the most computational resources according the original version.The overall computational complexity is O(MN 2 ), M is the number of objectives and N is the population size.Here, we mainly analyze the computational complexity of the CDDV strategy.In the algorithm 2, calculating the change degree of the decision space takes O(N 2 ) and dividing the decision variables spends O(n); so the total computational complexity of algorithm 2 is O(N 2 + n).The algorithm 3 spends O(Nn), where n is the number of decision variables.In total, the whole computational complexity of CDDV is O(N 2 + n).

VI. CONCLUSION AND FUTURE WORK
In this paper, a novel prediction strategy based on change degree of decision variables, called CDDV, is proposed for handling dynamic multi-objective optimization.When an environmental change is detected, the change degree of each dimension of the decision variables is calculated by a calculation model.That aims to detect that the environmental change will cause different degree of influence to each dimension of the decision space.Afterward, we propose a novel change response based on the change degree to improve the tracking speed and obtain a sequence of approximations with high efficacy during the short time.
Nine test problems with different characteristic are selected to test the overall performance of the proposed algorithm.The experimental results clearly demonstrate the effectiveness of the proposed algorithm in compared with four other state-of-the-art methods on most of the test instances.This implies that the dynamic handling technique has a bigger ability of tracking the moving POF and/or POS efficiently.It also has fast convergence speed and maintains the good diversity.However, the disadvantage of the proposed algorithm cannot be overlooked.For example, the JY6 and JY8 hardly are addressed well by the the proposed method (although it gained the best IGD metric on JY6 and the best HVD metric on JY8.).
In the future work, we aim to discover the novel change responses to handle dynamic optimization, new self-adaption operators to replace the change response, new environment detection to detect the dynamic environments.

4 : 10 :FIGURE 1 .
FIGURE 1.An example of the classification of the decision space according to cs.
, which combines a new prediction-based reaction mechanism and a popular regularity model-based multi-objective estimation of distribution algorithm; Rong et al. proposed a multi-model

FIGURE 2 .
FIGURE 2. Evolution curves of average IGD values for problems with n t = 10 and τ t = 20.

FIGURE 3 .
FIGURE 3. Solution sets obtained by five algorithms at six different time steps on JY1.

FIGURE 4 .
FIGURE 4. Solution sets obtained by five algorithms at six different time steps on JY6.

TABLE 2 .
Mean and standard deviation values of MIGD obtained by five algorithms.

TABLE 3 .
Mean and standard deviation values of MHVD obtained by five algorithms.

TABLE 4 .
Mean and standard deviation values of MGD obtained by five algorithms.

TABLE 5 .
Mean and standard deviation values of MSP obtained by five algorithms.

TABLE 6 .
Mean and standard deviation values of MHVD obtained by five algorithms under different n t .