Robust Design Optimization and Improvement by Metamodel

The robust design optimization (RDO) is an effective method to improve product performance with uncertainty factors. The robust optimal solution should be not only satisfied the probabilistic constraints but also less sensitive to the variation of design variables. There are some important issues in RDO, such as how to judge robustness, deal with multi-objective problem and black-box situation. In this paper, two criteria are proposed to judge the deterministic optimal solution whether satisfies robustness requirment. The robustness measure based on maximum entropy is proposed. Weighted sum method is improved to deal with the objective function, and the basic framework of metamodel assisted robust optimization is also provided for improving the efficiency. Finally, several engineering examples are used to verify the advantages.


Introduction
The purpose of engineering optimization is to make the cost of structure as low as possible or make some properties to achieve optimal state under constraints. Traditional optimization problems are based on deterministic parameters and model, which are also solved by classical deterministic optimization methods. However, the deterministic optimal solution may violate the imposed constraints due to the existence of variations on design variables or cause the system performance (named as the objective function) to be varied drastically [1]. The uncertainty-based design optimization can overcome the shortcoming of deterministic optimization which neglects the parameter uncertainty. The uncertaintybased design optimization mainly contains reliability-based design optimization (RBDO) and robust design optimization (RDO). The purpose of RBDO is to obtain the optimal solution satisfying the probabilistic constraints. The robust optimal solution should be not only satisfied the probabilistic constraints but also less sensitive to variations and tolerances of design variables. The intuitive comparison chart among three design optimization methods are shown in Fig. 1.
sensitive to the variability of design variables than RDO. Therefore, RDO is widely used in the design optimization field.
Traditional robust design is an efficient tool based on experimental design to improve the product quality, which was initially proposed by Taguchi [2]. The three stages of robust design include system design, parameter design and tolerance design. The core of robust design is to improve robustness by adjusting design parameters reasonably. Vining et al. [3] utilized the dual response approach into Taguchi design, in order to achieve the design goal with fewer tests and cost during the process of designing. However, in abovementioned methodologies, the variations of design variables were simply viewed as changes in an interval, which ignored the probabilistic distributions of variables. Because of the variables defined in discrete space, the traditional robust design is also difficult to deal with the constraints.
With the development of computer engineering techniques, probability statistics and optimization algorithms are introduced into robust design, and RDO based on mathematical model has been gradually formulated [4,5]. Huang et al. [6] summarized several main issues of RDO problems, containing robustness assessment, objective function processing, mathematical model and solution strategy.
Several robustness assessments have been proposed, such as moment assessment, quantile, information entropy [1,[7][8][9][10][11]. The most widely applied assessment is the moment assessment, e.g., a combination of mean and variance. Song [11] used the first four order moments to measure the robustness of objective function, since only mean and variance cannot accurately describe the statistical characteristics of response. Quantile is another robustness measure [7]. Quantile contains more information than variance, and it is only applicable for unimodal probability distribution. In fuzzy RDO problems, Beer et al. [8] utilized the entropy ratio of input and output to measure robustness.
RDO is not only to make the design objective as best as possible but also to make the optimal solution insensitive to the disturbance of design parameters. Therefore, RDO is a typical multi-objective and multiconstraint problem considering the trade-off between objective performance and robustness as well as subject to probabilistic constraints meanwhile.
For the multi-objective optimization problem [12], weighted sum method can transform multi-objective optimization into single-objective optimization, which is commonly used in engineering [13]. However, due to the introduction of decision-maker's preference, it inevitably brings some errors [14]. Genetic algorithm (GA) is a representative global optimization method and can directly deal with multi-objective problems to  [15], which can also effectively deal with highly nonlinear, non-convex, discontinuous, non-derivative and other complex situations despite its computational burden.
As we all know, the dimension of uncertain parameters is very high in practical engineering. If all the uncertain parameters are taken into consideration, the computational burden and data storage space will be too large to accept. Hence, in order to reduce the variables' dimension, sensitivity analysis (also called importance analysis) should be employed to distinguish important and unimportant uncertain variables [16,17]. And then these unimportant uncertainties can be screened out according to the ranking of sensitivity indices, thus providing useful guidance in respect to RDO problems.
In RDO of complex engineering system, it is difficult to derive the explicit expressions of objective function and constraints accurately. In order to avoid expensive and time-consuming experiment or simulation, RDO is supposed to be assisted by metamodel, which can approximate the input-output relationship. Various techniques have been studied, such as response surface method [18], Kriging [19], support vector machines [20], artificial neural network [21], etc. A brief introduction of abovementioned and other metamodels is listed in [22]. Among neural network algorithms, group method of data handling combined neural network (GMDH-NN) is self-organized network and the polynomial expression of model output can be obtained. Song et al. [23] modified GMDH-NN method further and utilized it to estimate variance-based sensitivity indices efficiently.
This paper focuses on some main issues of RDO. The remaining of the paper is organized as follows: Section 2 reviews the fundamental mathematical models of deterministic design, RBDO and RDO. Two new criteria are defined to judge the optimal solution whether satisfies robustness in Section 3. A novel robustness assessment based on maximum entropy is proposed in Section 4. Section 5 improves adaptive weighted sum method, a kind of multi-objective optimization method, by hyper-plane method. The basic framework of metamodel assisted robust optimization is provided in Section 6 and several engineering examples are used to illustrate the results of the proposed RDO in Section 7. Finally, conclusions of this paper are drawn in Section 8.

Review of Design Optimization Problems
A general single-objective deterministic design problem can be given by the following conventional optimization model, where d represents the vector of design variables, and d L and d U denote the corresponding lower and upper bounds of design variables. f d ð Þ is the objective function to be minimized, such as the structural weight and the initial cost. g i d ð Þ is the ith inequality constraint function, e.g., displacement constraints, geometric constraints and so on.
Ignoring the uncertainty of input parameters, the deterministic optimal solution may fall into unfeasible region due to small perturbations of parameters. So in RBDO, there are two types of variables are considered, e.g., design variables d and random parameters x. The mathematical model of RBDO is where l d denotes the mean vector of design variable vector d, g i (d, x) is the ith constraint function, P ð Þ is the probability operator and P Ã fi is the superior limit of failure probability. Admittedly, the RBDO optimum can ensure that structure satisfies the reliability requirements when considering the uncertainty of constraints, but the sensitivity of RBDO optimum to parameters' perturbations may be high as well. The robust optimal solution should be not only satisfied the probabilistic constraints but also less sensitive to variation of design variables, and RDO is also viewed as reliability-based robust design optimization (RBRDO) [24]. The mathematical optimization model of RDO is Þ means the robustness measures of objective function f d ð Þ. For example, robustness has been widely measured by variance or standard deviation, then the corresponding objective function of RDO can be written as which is the combination of first two order statistical moment of original objective f d ð Þ.
RBDO as well as RDO are both double loop optimization, including inner reliability assessment loop and outer design optimization loop. Various approaches have been developed with the aim of mitigating the unbearable computational cost from double-loop process [25,26], especially from the reliability assessment of constraints, which are not comprehensively investigated here. Since statistical moments can also be used to estimate the reliability index, the constraint is approximately written as where β i is admissible reliability index, which is normally chosen as 3 or bigger value, and a kg i k ¼ 1; 2; 3; 4 ð Þ is the kth central moment [11]. Obviously, using first four order moments to approximate reliability is more precise than only using mean and variance. Besides, moments can be efficiently estimated by sparse grids method, which contributes to alleviating computational burden [27].
function which is second-order derivative at optimum point, and Criterion 2 is to deal with the objective function which is second-order non-derivative at optimum point.
Before using these criteria to judge the robustness, it is necessary to standardize all random variables. For example, general uniform distribution need to be transformed into the interval of [0,1], normal distribution should be transformed into standard normal distribution, other distribution variables should be transformed into equivalent normal distribution by using Rackwitz-Fiessler method.

Criterion 1 (Second-Order Derivative at the Optimum Point)
Curvature can reflect the bending degree of objective function, which can be also regarded as the fluctuation or variation of response. So when the curvature can be calculated at optimum point of objective function, its absolute value can be utilized to represent its robustness. For one-dimensional function f(x), the robustness criterion based on curvature can be written as For multi-dimensional function f(x), the second-order Taylor expansion in standardized normal space at optimum point x* can be expressed as [27] f where b jj (j = 1, 2, ···, n) are the diagonal elements of B and n is the dimension of variables. So the robustness criterion for multi-dimensional function is

Criterion 2 (Second-Order Non-Derivative at the Optimum Point)
Since Criterion 1 is not available when objective function does not have the second-order derivative at optimum point, Criterion 2 is proposed to deal with second-order non-derivative case.
Firstly, consider the variable x k k ¼ 1; 2; Á Á Á n ð Þseparately, and set an interval x k L ; x k R ½ which contains the optimum point x Ã k . Then divide the interval equally into N parts and calculate the partial derivative @f x ð Þ @x k x k ¼x k i ði ¼ 1; 2; …; N Þ on the midpoint of each small interval. Thereby, the robustness criterion of kth variable is noted as where the bounds of each interval are set as where Dx k is a preset parameter. For example, Dx k can be equal to 0.1 for standard uniform distribution and equal to 3 for standard normal distribution (from 3 Sigma criterion). Finally, the biggest one Ro k is chosen as the final value of Criterion 2.
If the criterion is smaller than given threshold value, the deterministic optimal solution can be considered as a robust solution. Next, there are two examples to illustrate above criteria.

Illustrative Examples
Example 3.1: Consider a nonlinear function: It is concluded that point A is more robust than point B. When the threshold is set as 100, point A is robust and point B is not robust. Consider a piecewise function: where x is uniformly distributed at interval [0, 1] and the threshold is set as 100. Because the onlyminimum point A (0.5, 5) is not derivative, as shown in Fig. 3, the Criterion 2 is used to judge its robustness and the value is Ro A = 122.3460. Thus, point A is not a robust solution.

Robustness Assessment Based on Maximum Entropy
As mentioned before, one of the main issue about RDO is its robustness assessment. Robustness assessments, including moment assessment, quantile assessment, information entropy assessment, have been studied till now. Moment assessment is most widely used, but only mean and variance cannot accurately describe the statistical characteristics of response and their weight coefficients are determined by human. Quantile contains more information than variance, but it is only applicable for unimodal probability distribution. Entropy assessment will be mainly discussed in this part, whose drawback is huge calculation cost. Later, a new robust assessment based on maximum entropy will be introduced in detail.
Information entropy can be used to measure the degree of uncertainty, which was firstly proposed by Shannon [28]. The smaller the degree of uncertainty, the smaller the entropy is. The entropy of a continuous distribution is defined by where q x ð Þ represents probability density function (PDF). Obviously, information entropy can serve as robustness assessment. However, for a group of random samples, the probability density function cannot be known in advance, so it needs to be inferred according to statistical characteristics. The approximate calculation formula of entropy can be obtained According to the large number theorem, it needs a lot of random samples to get accurate entropy, which is not conducive to application.
Jaynes [29,30] introduced the principal of maximum entropy into statistical decision theory in 1957, i.e., there exists a distribution represents the best that can be done with the given information. Since the minimization of entropy is accordance with the minimization of maximum entropy, robustness assessment based on maximum entropy can be proposed in RDO.
The typical solution strategy of maximum entropy is to use Lagrange multipliers method and variational approach under the classical moment constraints.
where l k is k-order moment and K is the biggest order. In general cases, first four order moments, i.e., K = 4, are used as constraints to meet accuracy requirement.
Introduce Lagrange multipliers k (k = 0,1, ···,K), and then Lagrange function L of Eq. (15) can be obtained as After obtaining the optimal solution of Lagrange function by unconstrained optimization methods, such as Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm, the PDF conforming to maximum entropy is Finally, the maximum entropy is expressed as [31] max If the magnitude of samples is too large, the integral of the exponential function in Eq. (18) may be out of memory by MATLAB, resulting in the solution impossible. Therefore, the expression of PDF can be converted into Eq. (20) to accelerate the convergence.
The proposed robustness assessment based on maximum entropy can not only remain the feature of using entropy to reflect the uncertainty but also greatly shorten the calculation time compared with entropy assessment. Meanwhile, it is obvious that Lagrange multipliers in the objective function are not fixed, which automatically achieves the best with the change of design variables in the process of robust optimization.

Adaptive Weighted Sum Method for RDO
RDO is not only to make the design objective as best as possible, but also to make the optimal solution insensitive to the disturbance of design parameters. Therefore, RDO is a typical multi-objective optimization problem. For example, when moment assessment is chose as the robust assessment, mean value and standard deviation of design objective need to be minimized at the same time. Weighted sum method is widely used to deal with such problems, but its coefficients are usually determined by designers' preference. In this section, an adaptive weighted sum method is proposed.
Weighted sum method is a classical method for multi-objective optimization, which seeks optimal solution by transforming multiple objectives into an aggregated single objective function. Although there are some drawbacks, the weighted sum method is still widely used because it is simple to understand and easy to implement [32].
Choosing mean and variance as the robustness measure and using weighted sum method to deal with the objective function, the mathematical model of RDO can be written as where w 1 and w 2 represent for weight factors and w 1 þ w 2 ¼ 1. l Ã f and r Ã f are the mean and standard deviation of objective function at the deterministic optimum considering uncertainty, respectively. Generally, weight factors are determined by the designers' preference (e.g., w 1 ¼ w 2 ¼ 0:5), which might not be the optimal values. In order to overcome the limitation, a new adaptive weighted sum method is proposed as follows.
Assume there are m objective functions h 1 x ð Þ; h 2 x ð Þ; Á Á Á ; h m x ð Þ of a multi-objective optimization problem, if only one objective function h i x ð Þ i ¼ 1; 2; Á Á Á ; m ð Þis considered sequentially, the optimal solution of each single objective is separately noted as Þ , which can serve as normalization factors in weighted sum method. Then an optimization model for solving optimal weights is formulated as follows: where The optimization model of Eq. (22) can be solved to obtain the weight coefficients, and then the final form of optimization objective of adaptive weighted sum method is The principle of adaptive weighted sum method is to find an ideal hyper-plane, where each objective function can be minimized. However, in many practical engineering, we can only find the hyper-plane with the minimum deviation from ideal one.

Robust Design Optimization by Metamodel
Although RDO can be finished by abovementioned procedures, it is hard to conduct RDO in complex engineering system. Because it is difficult to derive the explicit expressions of objective function and constraints accurately in such cases. In order to avoid expensive and time-consuming experiment or simulation, RDO is supposed to be assisted by metamodel, which can approximate the input-output relationship. In the iterative process of RDO, the Most Probable Point of Inverse Reliability (MPPIR) [7] is introduced to determine whether update the metamodel.
Sensitivity analysis can be used to rank variables, simplify models, establish priorities for research and so on. It is widely carried out to distinguish important and unimportant uncertain variables. And then these unimportant uncertainties can be screened out according to the ranking of sensitivity indices, thus providing useful guidance in respect to RDO problems.
The flowchart of metamodel assisted RDO is shown in Fig. 4 and main procedures are summarized as follows.
Step 1. Carry out sensitivity analysis. Sensitivity analysis distinguishes important and unimportant variables, and then set unimportant variables as constant.
Step 2. Separately build metamodels of objective and constraints.
Step 3. Re-sample and select points in design region to update metamodels.
Step 4. Iterative optimization process of RDO to search the new optimum point.
Step 5. Find the inverse MMP according to target reliability and optimum point. If the inverse MMP does not satisfy certain constraint, we need repeat Steps 3-5 to build updated metamodels. If all constraints are satisfied, the whole optimization process will be finished.

Examples
Example 7.1: Cantilever beam A rectangle cross-section cantilever beam is considered, which is shown in Fig. 5. The beam with the length L is loaded at the tip by vertical load P. The elastic modulus of beam is noted as E. S represents the random yield strength, and w represents admissible reflection. They are all normally distributed variables and distribution parameters are listed in Tab. 1. The width a and thickness b of the cross-section are random design variables with the standard deviation r ¼ 0:1; 0:1 ½ T . The optimization objective is to minimize the cross-section area of the beam. The main failure modes of cantilever beam includes strength failure 392 CMES, 2020, vol.125, no.1 Figure 5: The schematic diagram of cantilever beam  Figure 4: The flowchart of metamodel assisted RDO and stiffness failure, which also serve as constraints, and the reliability index of each reliability constraints is required to satisfy b ! 3.
According to mechanical knowledge, the expressions of reliability constraints corresponding to strength failure and stiffness failure are Besides, the size of cantilever beam should satisfy design requirements, which are When the randomness of variables is not considered, the deterministic optimization model is Then we conduct deterministic optimization, reliability-based design optimization and robust design optimization with different methods for cantilever beam. The results are shown in Tab   An electric tower structure can be simplified as 25-truss structure. All the 25 trusses are divided into six categories, which are represented by six different colors in Fig. 7. Among them, the modulus of elasticity E 6 obeys normal distribution with l E 6 ¼ 10 7 Pa; r E 6 ¼ 1:5 Â 10 6 Pa, and other modulus of elasticity obeys normal distribution with The coordinates of all nodes 1-10 are listed in Tab. 4. Nodes 7-10 at the bottom are fixed, and nodes 1 and 2 are subject to static load, P 1y = P 2y = P 1z = P 2z = 10 7 N, and nodes 3 and 6 are subject to random load P 3x = P 6x with l P ¼ 500N;r P ¼ 50N.
In order to ensure the strength of structure, the tensile and compressive stress of all trusses σ i (i = 1,2, …,25) shall not exceed 15000 Pa. The design variables are the cross sectional area A 1 ∼A 6 , and they are independent normal variables with coefficient of variation is 0.05. Taking the minimum quality of truss structure as the objective function, the deterministic optimization model is constructed as follows.     There is no doubt that the calculation burden is huge if the design optimizations only depends on calling finite element models. In order to mitigate this problem, design optimizations are assisted by metamodel as mentioned in Section 6. In this example, adaptive Kriging metamodel is used to replace the performance function of constraints. What's more, all constraints g i d ð Þ ¼ r i À 15000 0ði ¼ 1; 2; …; 25Þ can be transform into one constraint, i.e., g d ð Þ ¼ maxðr i Þ À 15000 0, which can be surrogated by Kriging model. For comparison, optimization results of example 7.3 by using finite element model and metamodel are separately listed in Tabs. 5 and 6.
From the optimization results of three engineering examples, it can be found that the deterministic optimum is at the boundary of constraint condition, and it may fall into the unfeasible region if there is a little disturbance. That is to say, the structure is on the edge of failure. Considering the volatility of design and random variables, the structure is very vulnerable to damage. And the RBDO optimum is at the boundary of probabilistic constraints. In contrast, the robust optimal solution is much more conservative. It has incomparable significance for the normal performance function and the guarantee of the core efficiency. Comparing moment assessment with maximum entropy assessment, the RDO optimal solutions are basically consistent. Also, through Tabs. 5 and 6, it is obviously that RDO assisted by metamodel saves much calculation time, compared with calling finite element model directly.

Conclusions
The main issues of RDO are studied, containing robustness assessment, objective function handling, mathematical models and solution strategies. Firstly, there are two criteria to judge the deterministic solution whether satisfies robustness. If the criterion values are small, the deterministic optimal solution can be considered as a robust solution. Secondly, robustness assessment based maximum entropy is introduced, it can not only remain the feature of using entropy to reflect the uncertainty but also greatly shorten the calculation time compared with entropy assessment. Thirdly, the RDO multi-objective optimization is transformed into the single-objective optimization by adaptive weighted sum method, whose weights are decided by hyper-plane method. Finally, RDO is coupled with sensitivity analysis and metamodel, which can reduce dimension, reduce the computational load and realize single loop iterative optimization process. Several engineering examples are used to verify the advantages of improved RDO.

Conflicts of Interest:
The authors declare that they have no conflicts of interest to report regarding the present study.