Abstract
Many optimization methods for simulation-based design rely on the sequential use of metamodels to reduce the associated computational burden. In particular, kriging models are frequently used in variable fidelity optimization. Nevertheless, such methods may become computationally inefficient when solving problems with large numbers of design variables and/or sampled data points due to the expensive process of optimizing the kriging model parameters in each iteration. One solution to this problem would be to replace the kriging models with traditional Taylor series response surface models. Kriging models, however, were shown to provide good approximations of computer simulations that incorporate larger amounts of data, resulting in better global accuracy. In this paper, a metamodel update management scheme (MUMS) is proposed to reduce the cost of using kriging models sequentially by updating the kriging model parameters only when they produce a poor approximation. The scheme uses the trust region ratio (TR-MUMS), which is a ratio that compares the approximation to the true model. Two demonstration problems are used to evaluate the proposed method: an internal combustion engine sizing problem and a control-augmented structural design problem. The results indicate that the TR-MUMS approach is very effective; on the demonstration problems, it reduced the number of likelihood evaluations by three orders of magnitude compared to using a global optimizer to find the kriging parameters in every iteration. It was also found that in trust region-based method, the kriging model parameters need not be updated using a global optimizer—local methods perform just as well in terms of providing a good approximation without affecting the overall convergence rate, which, in turn, results in a faster execution time.
Similar content being viewed by others
References
Alexandrov NM, Lewis RM (2003) Large-scale PDE-constrained optimization. In: Biegler LT, Ghattas O, Heinkenschloss M, Bloemen Waanders Bv (eds) Lecture notes in computational science and engineering, vol. 30. Springer, Berlin Heidelberg New York
Alexandrov NM, Lewis RM, Gumbert CR, Green LL, Newman PA (2000a) Optimization with variable-fidelity models applied to wing design. In: Proceedings of the 38th AIAA Aerospace Sciences Meeting and Exhibit, No. AIAA 2000-0841, Reno, Nevada, 2000
Alexandrov NM, Nielsen EJ, Lewis RM, Anderson WK (2000b) First-order model management with variable-fidelity physics applied to multi-element airfoil optimization. In: 8th AIAA/NASA/USAF/ISSMO Symposium on multidisciplinary analysis and optimization, No. AIAA paper 2000-4886, Long Beach, CA, 2000
Alexandrov NM, Dennis JE, Lewis RM, Torczon V (2001) A trust region framework for managing the use of approximation models in optimization. ICASE Report 97-50, NASA/CR-201745, October 2001
Booker AJ (1998) Design and analysis of computer experiments. In: Proceedings of the 7th AIAA/USAF/NASA ISSMO Symposium on multidisciplinary analysis and optimization, vol 1. St. Louis, MO, 1998, pp 118–128
Booker AJ, Dennis JE, Frank PD, Serafini DB, Torczon V, Trosset MW (1999) A rigorous framework for optimization of expensive function by surrogates. Struct Optim 17(1):1–13
Chang KJ, Haftka RT, Giles GL, Kao P-J (1993) Sensitivity-based scaling for approximating structural response. J Aircr 30(2):283–288
Chung H-S, Alonso JJ (2002a) Using gradients to construct cokriging approximation models for high-dimensional design optimization problems. In: 40th AIAA aerospace sciences meeting and exhibit, No. AIAA 2002-0317, Reno, NV, 2002
Chung H-S, Alonso JJ (2002b) Design of a low-boom supersonic business jet using cokriging approximation models. In: 9th AIAA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, No. AIAA 2002-5598, Atlanta, GA, 2002
Conn AR, Gould NIM, Toint PL (1988) Global convergence of a class of trust region algorithms for optimization with simple bounds. SIAM J Numer Anal 25(2):433–464
Currin C, Mitchell T, Morris M, Ylvisaker D (1988) A Bayesian approach to the design and analysis of computer experiments. Technical Report ORNL-6498, Oak Ridge National Laboratory, Oak Ridge, TN
Denison DGT, Holmes CC, Mallick BK, Smith AFM (2002) Bayesian methods for nonlinear classification and regression. Wiley, New York
Gano SE, Renaud JE, Martin JD, Simpson TW (2005a) Update strategies for kriging models for use in variable fidelity optimization. In: Proceedings of the 1st multidisciplinary design optimization specialist conference, No. AIAA 2005-2057, Austin, TX, 2005
Gano SE, Renuad JE, Sanders B (2005b) Hybrid variable fidelity optimization using a kriging-based scaling function. AIAA J 43(11):2422–2430
Giunta AA, Watson LT (1998) A comparison of approximation modeling techniques: polynomial versus interpolating models. In: Proceedings of the 7th AIAA/USAF/NASA/ISSMO Symposium on multidisciplinary analysis and optimization, No. AIAA-98-4758, St. Louis, MO, 1998, pp. 392–404
Giunta AA, Eldred MS (2000) Implementation of a trust region model management strategy in the DAKOTA optimization toolkit. In: Proceeedings of the 8th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, No. AIAA-2000-4935, Long Beach, CA
Haftka RT (1991) Combining global and local approximations. AIAA J 29(9):1523–1525
Iman RL, Helton JC, Cambpell JE (1981a) An approach to sensitivity analysis of computer models, Part I: introduction, input variable selection and preliminary variable assessment. J Qual Technol 13:174–183
Iman RL, Helton JC, Cambpell JE (1981b) An approach to sensitivity analysis of computer models, Part II: ranking of input variables, response surface validation, distribution effect and technique synopsis. J Qual Technol 13:232–240
Ingber L (1989) Very fast simulated re-annealing. J Math Comput Model 12:967–973
Ingber L (1992) Genetic algorithms and very fast simulated reannealing: a comparison. J Math Comput Model 16(11):87–100
Ingber L (1993) Simulated annealing: practice versus theory. J Math Comput Model 18(11):29–57
Ingber L (1996) Adaptive simulated annealing (ASA): lessons learned. J Control Cybern 25(1):33–54
Jones DR, Schonlau M, Welch WJ (1998) Efficient global optimization of expensive black-box functions. J Glob Optim 13(4):455–492
Kowalik J, Osborne MR (1968) Methods for unconstrained optimization problems. American Elsevier, New York
Lewis RM, Nash SG (2000) A multigrid approach to the optimization of systems governed by differential equations In: 8th AIAA/NASA/USAF/ISSMO Symposium on multidisciplinary analysis and optimization, No. AIAA2000-4890, Long Beach, CA, 6–8 September 2000
Lim YB, Sacks JJSW, Welch WJ (1997) Design and analysis of computer experiments when the output is highly correlated over the input space. Technical report 62, National Institute of Statistical Sciences, NC, USA
Lophaven SN, Nielsen HB, Søndergaard J (2002a) DACE a Matlab kriging toolbox. Technical Report IMM-REP-2002-12, Technical University of Denmark, version 2.0
Lophaven SN, Nielsen HB, Søndergaard J (2002b) Aspects of the Matlab toolbox DACE. Technical report IMM-REP-2002-13, Technical University of Denmark
Martin JD, Simpson TW (2003) A study on the use of kriging models to approximate deterministic computer models. In: Proceedings of ASME 2003 design engineering technical conferences and computers and information in engineering conference, No. DET2003/DAC-48762, Chicago, IL, 2003
Martin JD, Simpson TW (2004) A Monte Carlo simulation of the kriging model. In: Proceedings of the 11th AIAA/ISSMO multidisciplinary analysis & optimization conference, No. AIAA 2004-4483, Albany, NY, 30 September 2004
Martin JD, Simpson TW (2005) On the use of kriging models to approximate deterministic computer models. AIAA J 43(4):853–863
Matheron G (1971) The theory of regionalized variables and its applications. Ecole des Mines, Fontainebleau
McAllister CD, Simpson TW (2003) Multidisciplinary robust design optimization of an internal combustion engine. ASME J Mech Des 125:124–130
Myers RH, Montgomery DC (1995) Response surface methodology: process and product optimization using designed experiments. Wiley, New York
Ripley BD (1988) Statistical inference for spatial processes. Cambridge University Press, Cambridge, UK
Rodríguez JF, Renaud JE (1998) Convergence of trust region augmented Lagrangian methods using variable fidelity approximation data. Struct Optim 15:141–156
Rodriguez JF, Pérez VM, Padmanabhan D, Renaud JE (2001) Sequential approximate optimization using variable fidelity response surface approximations. Struct Multidisc Optim 22(1):24–34
Sacks J, Welch WJ, Mitchell TJ, Wynn HP (1989) Design and analysis of computer experiments. Stat Sci 4(4):409–423
Sasena MJ, Papalmbros P, Goovaerts P (2002) Exploration of metamodeling sampling criteria for constrained global optimization. Eng Optim 34(3):263–278
Simpson TW, Maurey TM, Korte JJ, Mistree F (2001a) Kriging metamodels for global approximation in simulation-based design. Eng Comput 17(2):129–150
Simpson TW, Mauery TM, Korte JJ, Mistree F (2001b) Kriging metamodels for global approximation in simulation-based multidisciplinary design optimization. AIAA J 39(12):2233–2241
Simpson TW, Peplinski JD, Koch PN, Allen JK (2001c) Metamodels for computer-based engineering design: survey and recommendations. Eng Comput 17:129–150
Simpson TW, Booker AJ, Gosh D, Gunita AA (2004) Approximation methods in multidisciplinary analysis and optimization: a panel discussion. Struct Multidisc Optim 27(5):302–313
Sobieszczanski-Sobieski J, Bloebaum CL, Hajela P (1991) Sensitivity of control-augmented structure obtained by a system decomposition method. AIAA J 29(2):264–270
Stein ML (1999) Interpolation of spatial data: some theory for kriging. Springer, Berlin Heidelberg New York
Wagner TC, Papalambros PY (1991) Optimal engine design using nonlinear programming and the engine system assessment model. Technical report, Ford Motor Company Scientific Research Laboratories, Dearborn, MI and Department of Mechanical Engineering at the University of Michigan, Ann Arbor, MI
Warnes JJ, Ripley BD (1987) Problems with likelihood estimation of covariance function of spatial Gaussian processes. Biometrika 74(3):640–642
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Gano, S.E., Renaud, J.E., Martin, J.D. et al. Update strategies for kriging models used in variable fidelity optimization. Struct Multidisc Optim 32, 287–298 (2006). https://doi.org/10.1007/s00158-006-0025-y
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00158-006-0025-y