Abstract
With rapid development of advanced manufacturing technologies and high demands for innovative lightweight constructions to mitigate the environmental and economic impacts, design optimization has attracted increasing attention in many engineering subjects, such as civil, structural, aerospace, automotive, and energy engineering. For nonconvex nonlinear constrained optimization problems with continuous variables, evaluations of the fitness and constraint functions by means of finite element simulations can be extremely expensive. To address this problem by algorithms with sufficient accuracy as well as less computational cost, an extended multipoint approximation method (EMAM) and an adaptive weighting-coefficient strategy are proposed to efficiently seek the optimum by the integration of metamodels with sequential quadratic programming (SQP). The developed EMAM stems from the principle of the polynomial approximation and assimilates the advantages of Taylor’s expansion for improving the suboptimal continuous solution. Results demonstrate the superiority of the proposed EMAM over other evolutionary algorithms (e.g., particle swarm optimization technique, firefly algorithm, genetic algorithm, metaheuristic methods, and other metamodeling techniques) in terms of the computational efficiency and accuracy by four well-established engineering problems. The developed EMAM reduces the number of simulations during the design phase and provides wealth of information for designers to effectively tailor the parameters for optimal solutions with computational efficiency in the simulation-based engineering optimization problems.
1. Introduction
Solving nonlinear optimization problems is a hot issue in design optimization of practical engineering systems. In this class of optimization problems, both the objective function and the constraints are nonlinear and extremely expensive when solved using numerical methods, for example, finite element methods. In order to obtain solutions with high computational accuracy in reasonable time, the hybrid optimization method has become increasingly popular for solving nonlinear optimization problems because it can reduce the computational burden during the analysis by replacing the complex physical systems with the mathematical models and improve the accuracy of the optimal solution with the use of the combined heuristic methods and mathematical programming techniques.
The multipoint approximation method (MAM) [1, 2] is one of the best-known metamodel-based optimization methods with the integration of sequential quadratic programming (SQP) technique, and it replaces the original optimization problem with a sequence of mathematical approximations that use much simpler objective and constraint functions. MAM stemmed from previous work [3, 4] and was further generalized to multipoint approximations [2, 5]. Recently, Liu and Toropov [6] have implemented the discrete capability into the MAM to solve mixed continuous-discrete optimization problems. In MAM, the process of constructing metamodels can be described as an assembly of multiple metamodels into a single metamodel using linear regression. The coefficients of the model assembly are not weights of the individual models but tuning parameters determined by the least squares method.
In inexpensive engineering design problems, such as a cantilever beam design [7], the hypersonic wing [8], and the wind farm layout design [9], evolutionary algorithms can be a good choice to find globally optimal solutions. Genetic algorithm (GA) [10, 11] is inspired by natural evolution in biology, and the population of candidate solutions experiences a process similar to natural selection and genetic variation. GA has been well-recognized as an optimization method handling nonsmooth and nonlinear problems, where traditional methods generally fail.
Similarly, intrigued by the group foraging such as fish schooling and bird flocking, particle swarm optimization (PSO) technique developed by Kennedy [12] has become one of the dominant optimization algorithms in many fields. The advantages of using various variants of this technique have been validated in the civil engineering applications in terms of convergence rate and success rate. Chen et al. [13] proposed an improved particle swarm optimization- (IPSO-) based form-finding method for suspension bridge design and construction with the test on the design analysis on Yingwuzhou Yangtze River Bridge. Ghamisi and Benediktsson [14] applied integration PSO on feature selection and demonstrated the usefulness on road detection. Meanwhile, PSO has also been widely applied for solving structural mechanics problems [15]. Firefly algorithm (FA), inspired by social behavior of fireflies which is related to the rate and rhythmic of flash [16], is another very promising method. This novel technique has played an important role in the research of truss structures [17] and composite reinforced bridges [18]. Furthermore, the integration of numerical algorithms with neural networks to solve complex problems has been recently investigated by Moghadas et al. [19], Cao et al. [20], and Li et al. [21].
Besides the aforementioned metaheuristic algorithms, metamodel-based algorithms have become increasingly popular in recent years. Widely used metamodels include polynomial regression (PR) [22], radial basis function (RBF) [23], Kriging [24], multivariate adaptive regression splines (MARS) [25], artificial neural networks (ANNs) [26], and support vector regression (SVR) [27]. Currently, there are lots of novel techniques and approaches in the area of metamodel-based optimization. Jones et al. [28] proposed efficient global optimization (EGO), which employs the Kriging metamodel for solving black-box problems. The optimization progress is guided by both the prediction and error estimations. Regis [29] developed COBRA, an efficient solver which makes use of RBF interpolation to approximate objective and constraint functions. A new iterate in COBRA is selected according to the violation of constraints within some small margins. An application of multifidelity metamodel can be found in [30–32], where genetic algorithms are responsible for exploring the global design space.
As stated in Haftka et al. [33], there is still much room for the development of efficient and accurate algorithms to tackle high-fidelity design optimization. To address complex nonlinear optimization problems involving multiscale/multilevel/multidisciplinary analysis within reasonable time, much attention has been paid to the research in relevant fields. Taking into account the above situations, MAM has been gradually developed and become one of the algorithms demonstrating good performance on efficiently solving mid-range constrained engineering optimization problems with the use of the combined heuristic methods and SQP technique. Based on the authors’ previous work [2, 6], an extended MAM (EMAM) framework is proposed in this paper to further improve the computational efficiency during the entire simulation process. First, a novel metamodel model inspired by Taylor’s expansion technique is developed to effectively construct the approximations. To implement Taylor’s expansion metamodel into the framework of MAM, the function of Euclidean distance for the determination of weighting coefficients during the process of approximations is replaced by a proposed strategy for adaptive selection of weighting coefficients. Then, the SQP technique is applied on the approximations to obtain the optimal solutions. The correctness of this enhanced EMAM is validated by comparing with the results from several nonconvex benchmark problems [34, 35], which were successfully solved by researchers in use of the state-of-the-art algorithms, such as genetic algorithms (GAs) [36–38], evolution strategies (ESs) [39], particle swarm optimization (PSO) [40], charged system search (CSS) [41], colliding bodies optimization (CBO) [42], and firefly algorithm (FA) [43]. Although there were some primary tests and rudimentary findings in previous work [44], robust numerical results are found in this paper to extensively demonstrate the advantages and superiority of the developed hybrid algorithm over evolutionary algorithms and MAM in terms of the computational efficiency and accuracy during the optimization process. With the implementation of the effective Taylor’s expansion in the current MAM optimization framework, the developed EMAM does not deteriorate the ability to solve the mid-range optimization problems, which is the distinctive feature of MAM optimization framework.
2. Multipoint Approximation Method (MAM)
Based on response surface methodology [22], the multipoint approximation method (MAM) aims at constructing mid-range approximations [4, 5] and is suitable to solve large-scale optimization problems by producing better-quality approximations that are sufficiently accurate in a current trust region and inexpensive in terms of computational costs required for their building. These approximation functions have a relatively small number (N + 1, where N is the number of design variables) of regression coefficients to be determined and the corresponding least squares problem can be solved easily.
In general, an optimization problem can be formulated aswhere refers to the vector of design variables; and are the given lower and upper bounds on the design variable ; is the total number of the design variables; is an objective function; is the constraint function; and is the total number of the constraint functions.
In order to present the detailed physical model using the response functions and reduce the number of calls for the response function evaluations, the MAM replaces the optimization problem by a sequence of approximate optimization problems:where and are the functions which approximate the functions and defined in equation (1), and are the side constraints of a trust subregion, and k is the iteration number.
The selection strategy of the approximate response functions outlines that their evaluations are inexpensive as compared to the evaluations of the actual response functions and are intended to be adequate in a current trust region. This is achieved by appropriate planning of numerical experiments and use of the trust region defined by the side constraints and .
In the present work, constructing the metamodels for the objective and constraint functions includes two stages. In the first stage, the parameters involved in building a single metamodel are formulated as follows:where is the function to be approximated; P means the total number of sampling points; the coefficient denotes the weight of each point ; in other words, it represents the inequality of each sample point in the sample space [45]; and indicates the tuning parameter associated with the specific metamodel in equation (4), and it is determined by the weighted least squares method.
In the second stage, different approximate models are assembled into one metamodel described by equations (5) and (6). Also, equation (5) is built in the same manner as equation (3). It should be noted here that the design of experiments is fixed when a different approximate model is constructed.where the assembly metamodel is expressed aswhere is the number of regressors in the model bank and the coefficients are regression coefficients that should not be considered as weight factors, e.g., could be positive or negative.
Finally, the above two-step metamodel building strategy leads to solving the linear system of equations with unknowns .
3. Extended MAM and Adaptive Selection of Weighting Coefficients
Moving least-squares method (MLSM) is a metamodel building technique that has been suggested for use in the meshless form of the finite element method [46] and then advocated to build the highly dependent metamodels around the specific point in the local space for design optimization [47–49]. Intrigued by MLSM, an extended MAM (EMAM) is proposed in this paper to explore the full potential of the polynomial regression-based metamodels through the entire optimization process. Since MLSM can more accurately predict the response function around the point at which the approximation is made, EMAM has the ability to capture values of the response function around the point with a high level of accuracy.
In current research, a novel metamodel called Taylor’s expansion metamodel is developed to construct the linearly combined metamodel and it is given as follows:where is the initial function value at the starting point , which is the suboptimal point obtained in the previous iteration during the optimization loop, and N is the number of design variables. It is noted that the quality of the above metamodel highly depends on the suboptimal point because the approximation around is constructed with high levels of accuracy and efficiency by linear expansion. This enables EMAM to outperform other metamodel methods in local search for the optimal solution and also improve the quality of the optimal solution.
As described in Section 2, the explicit metamodel will be determined on the basis of implicit response evaluations by the weighted least-squares fitting. Apparently, values of the weighting coefficient in equations (3) and (5) directly control the quality of the approximation. Generally, the optimal solution in an optimization problem lies on the boundary of the feasible region. In other words, there is at least one constraint to be activated when the optimum is found in the constrained optimization problems. Therefore, it is necessary to propose a strategy for adaptive selection of weighting coefficients in the current optimization framework so that the approximation function could improve its accuracy near the promising region. As a result, the optimal solution will be more likely to locate in the vicinity of a boundary.
To implement Taylor’s expansion metamodel into this approximation-based framework, the weighting coefficient is defined as follows:where and are user-defined positive constants and and are determined by the author’s experience from a lot of tests performed. It is noted that the bigger weightings should be adaptively assigned to the points which are located more closely to the boundaries between the feasible and infeasible regions. As can be seen from equation (8), the maximum constraint weighting factor is assigned when the constraint evaluation equals 1. With , the quality of the metamodel to approximate response functions in the feasible region is much ‘better’ than the one in the infeasible region.
As compared to the formulations of the weighting coefficient in [45], which is defined in the following equation: and .
The objective weighting factor has not been used in the proposed strategy. There are two reasons: (1) The weighting factor will sometimes be allocated a wrong weight value for an infeasible point. Considering an infeasible solution with an extremely low objective value , this weight would approach infinity. As a result, the quality of the metamodel is severely damaged. (2) Even if the objective weighting factor is well defined, the influence of on the quality of metamodels is much less than that of the constraint weighting factor .
In equation (9), the constraint weighting coefficient only considers the contribution from each constraint during the process of the constraint metamodel building. Therefore, is only affected by a single constraint for a given design point. In the proposed strategy for adaptive selection of weighting coefficients, all information of different constraints is considered by the product of a bunch of weighting factor . Obviously, one design point and its corresponding constraints are not isolated one from another. The optimal behavior of a design point should be judged by the information of the whole set of constraints, rather than the information from a single constraint. By combining the constraints with multiplication shown in equation (8), the more the constraints are active, the larger the weighting of a point. On the contrary, the less weighting value is given when the point is far away from the feasible region. It is noted that the proposed strategy for adaptive selection of weighting coefficients leads to the approximation function with a high level of accuracy in the feasible region, which results in a high probability of identifying feasible solutions during the optimization process.
Based on these facts, the flowchart of the EMAM as an enhanced optimization framework is shown in Figure 1. At the beginning, an initial feasible design is given to trigger the entire optimization process and the corresponding trust region is defined. Then, a number of sampling points (N + 5, where N means the number of design variables) are uniformly distributed over the trust region. The objective and constraint values at these points are obtained by evaluating the response functions. In this paper, we assume the response functions are computationally expensive in simulation-based optimization and any design point will never cause a crash during the simulations. Based on the obtained data about design variables and responses, Taylor’s expansion regressor defined by equation (7) and five other forms of regressors represented by equation (4) are built in sequence. Following that, these six regressors are assembled into one metamodel for the evaluations of all response functions of the interests. Thus, numerical simulations are performed on the metamodels inside the trust region and the optimal solution of the subproblem is found by SQP (sequential quadratic programming) solver. To update the trust region in next iteration, it will be resized and moved based on several indicators [2] and then, the next iteration starts. When the size of the trust region is small enough, the entire optimization process will terminate and the final solution will be achieved.
4. Examples
4.1. Design of a Tension/Compression Spring
This problem first described by Belegundu [34] and Arora [50] has arisen from the wide applications of vibration-resistant structures in civil engineering. The design objective is to minimize the weight of a tension/compression spring subject to constraints on shear stress, surge frequency, and minimum deflections as shown in Figure 2. The design variables include the wire diameter , the mean coil diameter , and the number of active coils . Detailed information on constraint functions , , , and can be found in reference [50].
In Table 1, the results obtained by extended MAM are compared to those by other methods, such as mathematical programming methods [34, 50], genetic algorithms (GAs) [36–38], evolution strategies (ESs) [39], and charged system search (CSS) [41]. As is shown in Table 1, the optimal design (0.0126653) found by extended MAM has a good agreement with the one by MAM and it also represents the lightest weight design among all the feasible solutions indicated in Table 2. Actually, Kaveh and Talatahari [41] obtained a slightly better design (0.0126384) using CSS. However, this optimal design can be noted that at least 0.11% design constraint violation () was clearly observed in Table 2.
By choosing different starting points that are randomly generated for each example in this section, both extended MAM and MAM have the ability to obtain the lightest design (0.0126653) shown in Table 3, when eight sampling points are selected to build the metamodels in each iteration of the optimization process. Taking into account the randomness in the developed algorithm, the mean value and standard deviation (SD) of the results have also been provided in Table 3 to reveal the method’s robustness. To compare methods using a probabilistic metric, more details can be found in [51]. In general, the number of evaluations called by the extended MAM is less than the number of analyses by MAM and the former can obtain the more robust optimum as well. In conclusion, the extended MAM effectively enhances search performance with the higher robustness and accuracy of the optimal solution than metaheuristic algorithms.
4.2. A Reactor Pressure Vessel Example
The second case study focuses on the design optimization of a cylindrical pressure vessel capped at both ends by hemispherical heads (Figure 3). The main purpose of this research is to minimize the total manufacturing cost of the vessel including the combination of welding, material, and forming costs. The design variables consist of the shell thickness , the spherical head thickness , the radius of cylindrical shell , and the shell length . The detailed problem formulation is given in [35].
The comparison of results obtained by the extended MAM and other metamodel-based methods (SCGOSR [52], eDIRECT-C [53], ConstrLMSRBF [53], CORBA [53], and CiMPS [53]) has been presented in Table 4. The cost computed using the extended MAM or MAM has been further reduced to 5885.268 by 0.0009% from 5885.33, which was the best design referred in [52]. To build the metamodels at each iteration of the optimization process, nine sampling points are used in this example. It is noted that the optimized solutions by MAM and extended MAM are the best feasible designs since no violated constraints are observed in Table 5. SCGOSR [52] could find a near-optimal design with the cost of 5885.3653, which is slightly heavier than the result by extended MAM, but the first and second constraints are violated. In average, the extended MAM outperforms the MAM to seek the optimum in terms of the number of iterations used in the case studies with different starting points shown in Table 6. Taking into account the above advantages of extended MAM for seeking optimal solutions, the superiority of the proposed method over other metamodel-based techniques has been demonstrated in terms of the accuracy and efficiency. It is concluded that hybrid algorithms, such as the extended MAM and MAM, are quite robust algorithms to consistently achieve higher accuracy of the solution than other metamodel-based algorithms used for solving problems with multiple local optima, and the extended MAM has a slightly faster rate of convergence than MAM.
4.3. Welded Beam Design
Design optimization of a welded beam shown in Figure 4 is a complex and challenging problem in nature with many variables and constraints. Usually, conventional optimization methods fail to find global optimal solution. Hence, the welded beam design problem is often used to evaluate the performance of different optimization methods. To determine the best set of design variables for minimizing the total fabrication cost of the structure, the minimum cost optimization is performed considering shear stress (), bending stress (), buckling load (), and end deflection () constraints. The constants in this study are chosen as follows:
Taking into account design variables , , , and , the mathematical optimization of the problem can be formulated as follows.
Objective: minimize the cost
The bounds on the design variables are
Subject towhere
In this example, the best combination of design variables and the lowest cost by hybrid algorithms (MAM and extended MAM) are compared with those obtained using GA [36–38], PSO [40, 54], FA [43], colliding bodies optimization (CBO) [42], CMA-ES [55], and differential evolution [56] in Table 7. Nine sampling points are applied to construct the metamodels in each iteration of the optimization process. Although Kaveh and Mahdavi [42] claimed that the minimum cost design was 1.724663 indicated in Table 7, the corresponding fabrication cost of the structure was actually 1.724983, which can be easily evaluated by substituting the values of design variables for the optimal design into the objective function . The cost of this design is higher than the result (1.724852) by the extended MAM, which is one of the best feasible designs shown in Table 7. CMA-ES, IAPSO, and iDEaSm could also find the best design; however, the required number of function evaluations is 4658, 12500, and 4425, respectively. MAM and EMAM only need about 120 function evaluations (about 13 iterations) as shown in Table 8, where the statistical results of four randomly tests are given to demonstrate the robustness of the solution. It is concluded that both extended MAM and MAM demonstrate the superiority over the other methods to solve the complex optimization problem with respect to the efficiency and accuracy of the solution.
4.4. A Ten-Bar Truss Structure
To further demonstrate the computational efficiency of the extended MAM, the well-known ten-bar truss benchmark problem [6] shown in Figure 5 is used to explore the potential. The optimization formulation of this problem is defined to minimize the weight of the structure by varying the cross-sectional areas (from 0.1 to 12.7 ) of the truss members subject to stress constraints. The allowable stress in each truss member is the same in tension and compression and is set to 25 for all members except member 9 for which it is 75 . The density of the truss material is 0.1 , the member size L = 360 , the loads P1 = P2 = 100 , and P3 = 0.
In order to demonstrate the superiority of the extended MAM over other optimization methods such as PSO [57], FA [58], and SQP in HyperStudy [59], a comparison of optimal designs of ten-bar truss structure has been given in Table 9. It should be noted that for PSO, FA, MAM, and the extended MAM, the objective function value, the number of iterations, and the number of response analyses are actually the average results over 5 independent runs. The best design (1497.0) was achieved by Haftka [60]; however, some constraints indicated in Table 10 had been violated. The same conclusion can be drawn for the optimal design (1497.6) by SQP in HyperStudy. The results by PSO (1519.2) and FA (1558.1) are feasible solutions; however, they are not the best design. For the best feasible design (1497.6) by extended MAM and MAM, the higher efficiency and accuracy of these two algorithms have been demonstrated, for example, the average number of iterations used by the extended MAM has been reduced by an order of magnitude from 520 (PSO) or 400 (FA) to 28. It is also noted that the average number of response analyses (420) called by extended MAM is 24% less than the one (555) by MAM. In summary, the extended MAM outperforms the other methods in seeking the optimal solution of the complex engineering design problems in terms of the efficiency and accuracy.
5. Conclusions
The paper focuses on obtaining the high efficiency and accuracy solution of complex simulation-based optimization problems by developing an extended multipoint approximation method. A novel metamodel inspired by Taylor’s expansion technique is proposed, as well as a strategy for adaptive selection of weighting coefficients, so that the approximation of responses of the interests in the computationally expensive design problems can be performed more efficiently. The superiority of the extended MAM over SQP, metaheuristic algorithms, metamodel-based algorithms, and MAM has been demonstrated by four nonconvex benchmark examples in terms of the computational efficiency and accuracy. In the current implementation, there are some limitations of EMAM. First, the optimization performance needs improvement to solve mixed-variable optimization problems. Second, the moving trust region strategy has certain drawbacks of balancing exploration and exploitation. Finally, the metamodel has difficulty in modelling highly multimodal and high-dimensional responses. However, possessing the potential of remarkably reducing the computational effort in the simulation-based optimization, the extended MAM can pose great influence on solving highly nonlinear engineering problems and provide valuable insights into the development of effective algorithms applied during the simulation-driven design process.
Data Availability
The underlying data used in this paper can be made available from the corresponding author upon request.
Conflicts of Interest
The authors declare that they have no conflicts of interest.