Skip to main content

Advertisement

SpringerLink
Log in

GOSAC: global optimization with surrogate approximation of constraints

  • Published:
Journal of Global Optimization Aims and scope Submit manuscript

Abstract

We introduce GOSAC, a global optimization algorithm for problems with computationally expensive black-box constraints and computationally cheap objective functions. The variables may be continuous, integer, or mixed-integer. GOSAC uses a two-phase optimization approach. The first phase aims at finding a feasible point by solving a multi-objective optimization problem in which the constraints are minimized simultaneously. The second phase aims at improving the feasible solution. In both phases, we use cubic radial basis function surrogate models to approximate the computationally expensive constraints. We iteratively select sample points by minimizing the computationally cheap objective function subject to the constraint function approximations. We assess GOSAC’s efficiency on computationally cheap test problems with integer, mixed-integer, and continuous variables and two environmental applications. We compare GOSAC to NOMAD and a genetic algorithm (GA). The results of the numerical experiments show that for a given budget of allowed expensive constraint evaluations, GOSAC finds better feasible solutions more efficiently than NOMAD and GA for most benchmark problems and both applications. GOSAC finds feasible solutions with a higher probability than NOMAD and GOSAC.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Abbreviations

GOSAC:

Global optimization with surrogate approximation of constraints

NOMAD:

Nonlinear optimization by mesh adaptive direct search, version 3.6.2

RBF:

Radial basis function

\({\mathbf {x}}\) :

Decision variable vector (may be continuous, integer, binary, mixed-integer)

\({\mathbf {x}}^T\) :

Transpose of \({\mathbf {x}}\)

d :

Problem dimension

\(x_i^l\), \(x_i^u\), \(i = 1\ldots d\) :

Variables’ lower and upper bounds

\(\varOmega \) :

Feasible domain defined by variables’ lower and upper bounds

\(\varOmega _c\) :

Feasible domain defined by all constraints

\({{\mathbb {I}}}\) :

Index set of integer variables

\(f(\cdot )\) :

Objective function, computationally cheap

\(c_j(\cdot ), j = 1\ldots m\) :

Computationally expensive black-box constraint functions

\(s_j(\cdot )\), \(j = 1\ldots m\) :

Response surface for the jth constraint function

\({\mathcal {S}}\) :

Set of already evaluated points, \({\mathcal {S}}=\{{\mathbf {x}}_1,\ldots ,{\mathbf {x}}_n\}\)

\(n_0\) :

Number of points in initial experimental design

References

  1. Arnold, J., Srinivasan, R., Muttiah, R., Williams, J.: Large area hydrologic modeling and assessment part 1: model development. J. Am. Water Resour. Assoc. 34, 73–89 (1998)

    Article  Google Scholar 

  2. Audet, C., Dennis Jr., J.: Mesh adaptive direct search algorithms for constrained optimization. SIAM J. Optim. 17(1), 188–217 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  3. Audet, C., Dennis Jr., J.: A progressive barrier approach for derivative-free nonlinear programming. SIAM J. Optim. 20(1), 445–472 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  4. Basudhar, A., Dribusch, C., Lacaze, S.S.M.: Constrained efficient global optimization with support vector machines. Struct. Multidiscip. Optim. 46(2), 201–221 (2012)

    Article  MATH  Google Scholar 

  5. Bertsekas, D.: Constrained Optimization and Lagrange Multiplier Methods. Academic Press, New York (1982)

    MATH  Google Scholar 

  6. Booker, A., Dennis Jr., J., Frank, P., Serafini, D., Torczon, V., Trosset, M.: A rigorous framework for optimization of expensive functions by surrogates. Struct. Multidisc. Optim. 17, 1–13 (1999)

    Article  Google Scholar 

  7. Conn, A., Scheinberg, K., Vicente, L.: Introduction to Derivative-Free Optimization. SIAM. 1, 1–12 (2009). doi:10.1137/1.9780898718768.ch1

    Google Scholar 

  8. Currie, J., Wilson, D.: Foundations of Computer-Aided Process Operations, Chap. OPTI: Lowering the Barrier Between Open Source Optimizers and the Industrial MATLAB User. Savannah, Georgia, USA (2012)

  9. Fletcher, R., Leyffer, S.: Nonlinear programming without a penalty function. Math. Progr. Ser. A 91(2), 239–269 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  10. Forrester, A., Sóbester, A., Keane, A.: Engineering Design via Surrogate Modelling: A Practical Guide. Wiley, London (2008)

    Book  Google Scholar 

  11. Friedman, J.: Multivariate adaptive regression splines. Ann. Stat. 19, 1–141 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  12. Goel, T., Haftka, R.T., Shyy, W., Queipo, N.V.: Ensemble of surrogates. Struct. Multidiscip. Optim. 33, 199–216 (2007)

    Article  Google Scholar 

  13. Gramacy, R., Gray, G., Le Digabel, S., Lee, H., Ranjan, P., Wells, G., Wild, S.: Modeling an augmented Lagrangian for blackbox constrained optimization. Technometrics 58(1), 1–11 (2016)

    Article  MathSciNet  Google Scholar 

  14. Gutmann, H.: A radial basis function method for global optimization. J. Global Optim. 19, 201–227 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  15. Harbaugh, A., McDonald, M.: User’s documentation for MODFLOW-96: an update to the U.S. Geological Survey modular finite-difference ground-water flow model. U.S. Geological Survey Open-File Report 96-485, p. 56 (1996)

  16. Jones, D., Schonlau, M., Welch, W.: Efficient global optimization of expensive black-box functions. J. Global Optim. 13, 455–492 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  17. Le Digabel, S.: Algorithm 909: NOMAD—nonlinear optimization with the MADS algorithm. ACM Trans. Math. Softw. 37, 44 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  18. Le Digabel, S., Wild, S.: A Taxonomy of Constraints in Simulation-Based Optimization (2015). ArXiv:1505.07881

  19. Liuzzi, G., Lucidi, S., Rinaldi, F.: Derivative-free methods for mixed-integer constrained optimization problems. J. Optim. Theory Appl. 164, 933–965 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  20. Martin, J.D., Simpson, T.W.: Use of kriging models to approximate deterministic computer models. AIAA J. 43, 853–863 (2005)

    Article  Google Scholar 

  21. Matheron, G.: Principles of geostatistics. Econ. Geol. 58, 1246–1266 (1963)

    Article  Google Scholar 

  22. Moré, J., Wild, S.: Benchmarking derivative-free optimization algorithms. SIAM J. Optim. 20(1), 172–191 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  23. Müller, J.: MISO: mixed-integer surrogate optimization framework. Optim. Eng. 17(1), 177–203 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  24. Müller, J., Krityakierne, T., Shoemaker, C.: SO-MODS: Optimization for high dimensional computationally expensive multi-modal functions with surrogate search. In: 2014 IEEE Congress on Evolutionary Computation (CEC), Beijing, China (2014)

  25. Müller, J., Piché, R.: Mixture surrogate models based on Dempster–Shafer theory for global optimization problems. J. Global Optim. 51, 79–104 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  26. Müller, J., Shoemaker, C.: Influence of ensemble surrogate models and sampling strategy on the solution quality of algorithms for computationally expensive black-box global optimization problems. J. Global Optim. 60(2), 123–144 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  27. Müller, J., Shoemaker, C., Piché, R.: SO-I: a surrogate model algorithm for expensive nonlinear integer programming problems including global optimization applications. J. Global Optim. 59(4), 865–889 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  28. Müller, J., Shoemaker, C., Piché, R.: SO-MI: a surrogate model algorithm for computationally expensive nonlinear mixed-integer black-box global optimization problems. Comput. Oper. Res. 40, 1383–1400 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  29. Myers, R., Montgomery, D.: Response Surface Methodology, Process and Product Optimization Using Designed Experiments. Wiley, Hoboken (1995)

    MATH  Google Scholar 

  30. Nocedal, J., Wright, S.: Numerical Optimization. Springer, Berlin (2006)

    MATH  Google Scholar 

  31. Parr, J.: Improvement criteria for constraint handling and multiobjective optimization. Ph.D. thesis, University of Southampton, University Road, Southampton SO17 1BJ (2013)

  32. Powell, M.: The theory of radial basis function approximation in 1990. Advances in Numerical Analysis, vol. 2: Wavelets, Subdivision Algorithms and Radial Basis Functions. Oxford University Press, Oxford, pp. 105-210 (1992)

  33. Regis, R.: Stochastic radial basis function algorithms for large-scale optimization involving expensive black-box objective and constraint functions. Comput. Oper. Res. 38, 837–853 (2011)

    Article  MathSciNet  Google Scholar 

  34. Regis, R., Shoemaker, C.: A stochastic radial basis function method for the global optimization of expensive functions. INFORMS J. Comput. 19, 497–509 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  35. Regis, R., Shoemaker, C.: Combining radial basis function surrogates and dynamic coordinate search in high-dimensional expensive black-box optimization. Eng. Optim. 45, 529–555 (2013)

    Article  MathSciNet  Google Scholar 

  36. Regis, R., Shoemaker, C.: A quasi-multistart framework for global optimization of expensive functions using response surface models. J. Global Optim. 56(4), 1719–1753 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  37. Talgorn, B., Le Digabel, S., Kokkolaras, M.: Statistical surrogate formulations for simulation-based design optimization. J. Mech. Des. 137(2), MD-14-1128 (2015)

  38. Tolson, B., Asadzadeh, M., Maier, H., Zecchin, A.: Hybrid discrete dynamically dimensioned search (HD-DDS) algorithm for water distribution system design optimization. Water Resour. Res. 45(12), W12416 (2009)

    Article  Google Scholar 

  39. Tolson, B., Shoemaker, C.: Cannonsville reservoir watershed SWAT2000 model development, calibration and validation. J. Hydrol. 337, 68–86 (2007)

    Article  Google Scholar 

  40. Tolson, B., Shoemaker, C.: Dynamically dimensioned search algorithm for computationally efficient watershed model calibration. Water Resour. Res. W01413(413), 16 (2007)

    Google Scholar 

  41. Viana, F.A.: Multiple Surrogates for Prediction and Optimization. Ph.D. thesis, University of Florida (2011)

  42. Wild, S., Regis, R., Shoemaker, C.: ORBIT: optimization by radial basis function interpolation in trust-regions. SIAM J. Sci. Comput. 30, 3197–3219 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  43. Wild, S., Shoemaker, C.: Global convergence of radial basis function trust-region algorithms for derivative-free optimization. SIAM Rev. 55, 349–371 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  44. Zheng, C., Wang, P.: MT3DMS: A Modular Three-Dimensional Multispecies Transport Model. US Army Corps of Engineers (1999)

Download references

Author information

Authors and Affiliations

  1. Lawrence Berkeley National Laboratory, Computational Research Division, Center for Computational Sciences and Engineering, Berkeley, CA, USA

    Juliane Müller

  2. Swiss Re, Armonk, NY, USA

    Joshua D. Woodbury

Authors
  1. Juliane Müller
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Joshua D. Woodbury
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Juliane Müller.

Additional information

This material is based upon work supported by the U.S. Department of Energy, Office of Science, Office of Advanced Scientific Computing Research, Applied Mathematics program under Contract Number DE-AC02005CH11231.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 226 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Müller, J., Woodbury, J.D. GOSAC: global optimization with surrogate approximation of constraints. J Glob Optim 69, 117–136 (2017). https://doi.org/10.1007/s10898-017-0496-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10898-017-0496-y

Keywords

Mathematics Subject Classification

Search

Navigation