Abstract
Global optimization of expensive functions has important applications in physical and computer experiments. It is a challenging problem to develop efficient optimization scheme, because each function evaluation can be costly and the derivative information of the function is often not available. We propose a novel global optimization framework using adaptive radial basis functions (RBF) based surrogate model via uncertainty quantification. The framework consists of two iteration steps. It first employs an RBF-based Bayesian surrogate model to approximate the true function, where the parameters of the RBFs can be adaptively estimated and updated each time a new point is explored. Then it utilizes a model-guided selection criterion to identify a new point from a candidate set for function evaluation. The selection criterion adopted here is a sample version of the expected improvement criterion. We conduct simulation studies with standard test functions, which show that the proposed method has some advantages, especially when the true function has many local optima. In addition, we also propose modified approaches to improve the search performance for identifying optimal points.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Andrieu, C., De Freitas, N., Doucet, A.: Robust full Bayesian learning for radial basis networks. Neural Comput. 13(10), 2359–2407 (2001)
Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, New York (2006)
Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press, New York (2004)
Buhmann, M.D.: Radial Basis Functions: Theory and Implementations, vol. 12. Cambridge University Press, New York (2003)
Chen, R.B., Wang, W., Wu, C.F.J.: Building surrogates with overcomplete bases in computer experiments with applications to bistable laser diodes. IIE Trans. 43(1), 39–53 (2011)
Chen, R.-B., Wang, W., Wu, C.F.J.: Sequential designs based on bayesian uncertainty quantification in sparse representation surrogate modeling. Technometrics 59(2), 139–152 (2017)
Chipman, H., Hamada, M., Wu, C.F.J.: A Bayesian variable-selection approach for analyzing designed experiments with complex aliasing. Technometrics 39(4), 372–381 (1997)
Chipman, H., Ranjan, P., Wang, W.: Sequential design for computer experiments with a flexible Bayesian additive model. Can. J. Stat. 40, 663–678 (2012)
Fasshauer, G.E., Zhang, J.G.: On choosing “optimal” shape parameters for RBF approximation. Numer. Algorithms 45(1–4), 345–368 (2007)
George, E.I., McCulloch, R.E.: Variable selection via Gibbs sampling. J. Am. Stat. Assoc. 88(423), 881–889 (1993)
Gutmann, H.M.: A radial basis function method for global optimization. J. Glob. Opt. 19(3), 201–227 (2001)
Jones, D.R.: A taxonomy of global optimization methods based on response surfaces. J. Glob. Opt. 21(4), 345–383 (2001)
Jones, D.R., Schonlau, M., Welch, W.J.: Efficient global optimization of expensive Black-Box functions. J. Glob. Opt. 13(4), 455–492 (1998)
Koutsourelakis, P.S.: Accurate uncertainty quantification using inaccurate computational models. SIAM J. Sci. Comput. 31(5), 3274–3300 (2009)
Liang, J., Qu, B., Suganthan, P.: roblem definitions and evaluation criteria for the CEC 2014 special session and competition on single objective real-parameter numerical optimization. Technical report 201311 (2013)
Mockus, J., Tiesis, V., Zilinskas, A.: The application of Bayesian methods for seeking the extremum. Towards Glob. Optim. 2, 117–129 (1978)
Morris, M.D., Mitchell, T.J.: Exploratory designs for computer experiment. J. Stat. Plan. Inference 43, 381–402 (1995)
Oefelein, J.C., Yang, V.: Modeling high-pressure mixing and combustion processes in liquid rocket engines. J. Propul. Power 14(5), 843–857 (1998)
Picheny, V., Wagner, T., Ginsbourger, D.: A benchmark of kriging-based infill criteria for noisy optimization. Struct. Multidiscip. Optim. 48(3), 607–626 (2013)
Regis, R.G., Shoemaker, C.A.: A stochastic radial basis function method for the global optimization of expensive functions. INFORMS J. Comput. 19(4), 497–509 (2007)
Rönkkönen, J., Li, X., Kyrki, V.: A framework for generating tunable test function for multimodal optimization. Soft. Comput. 15, 1689–1706 (2011)
Santner, T.J., Williams, B.J., Notz, W.I.: The Design and Analysis of Computer Experiments. Springer, New York (2013)
Torn, A., Žilinskas, A.: Global Optimization. Springer, Berlin (1989)
Žilinskas, A.: On similarities between two models of global optimization: statistical models and radial basis functions. J. Glob. Opt. 48, 173–182 (2010)
Žilinskas, A., Zhigljavsky, A.: Stochastic global optimization: a review on the occasion of 25 years of informatica. Informatica 27, 229–256 (2016)
Zellner, A.: On assessing prior distributions and Bayesian regression analysis with g-prior distributions. Bayesian Inference Decis. Tech. Essays Honor Bruno De Finetti 6, 233–243 (1986)
Acknowledgements
Chen’s research is supported by Ministry of Science and Technology (MOST) of Taiwan 104-2918-I-006-005 and the Mathematics Division of the National Center for Theoretical Sciences in Taiwan. Wu’s research is supported by ARO W911NF-17-1-0007 and NSF DMS-1564438.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Chen, RB., Wang, Y. & Wu, C.F.J. Finding optimal points for expensive functions using adaptive RBF-based surrogate model via uncertainty quantification. J Glob Optim 77, 919–948 (2020). https://doi.org/10.1007/s10898-020-00916-w
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10898-020-00916-w