Skip to main content
SpringerLink
Log in

A Versatile Surrogate-Assisted Memetic Algorithm for Optimization of Computationally Expensive Functions and its Engineering Applications

  • Chapter
Book cover Success in Evolutionary Computation

Part of the book series: Studies in Computational Intelligence ((SCI,volume 92))

A core element of modern engineering is optimization with computationally expensive ‘black-box’ functions. We identify three open critical issues in optimization of such functions: (a) how to generate accurate surrogate-models (b) how to handle points where the simulation fails to converge and (c) how to balance exploration vs. exploitation.

In this work we propose a novel surrogate-assisted memetic algorithm which addresses these issues and analyze its performance. The proposed algorithm contains four novelties: (a) it autonomously generates an accurate surrogate-models using multiple cross-validation tests (b) it uses interpolation to generate a global penalty function which biases the global search away from points where the simulation did not converge (c) it uses a method based on the Metropolis criterion from statistical physics to manage exploration vs. exploitation and (d) it combines an EA with the trust-region derivative-free algorithm. The proposed algorithm generates global surrogate-models of the expensive objective function and searches for an optimum using an EA; it then uses a trust-region local-search which combines local quadratic surrogate-models and sequential quadratic programming with the trust-region framework, and converges to an accurate optimum of the true objective function. While the global-local search approach has been studied previously, the improved performance in our algorithm is due to the four novelties just mentioned.

We present a detailed performance analysis, in which the proposed algorithm significantly outperformed four variants of a real-coded EA on three difficult optimization problems which include mathematical test functions with a complicated landscape, discontinuities and a tight limit of only 100 function evaluations. The proposed algorithm was also benchmarked against a recent memetic algorithm and a recent EA and significantly outperformed both, showing it is competitive with some of the best available algorithms.

Lastly, the proposed algorithm was also applied to a difficult real-world engineering problem of airfoil shape optimization; also in this experiment it consistently performed well, and in spite of a many points where the simulation did not converge, and the tight limit on function evaluations it obtained an airfoil which satisfies the requirements very well.

Overall, the proposed algorithm offers a solution to important open issues in optimization of computationally-expensive black-box functions: generating accurate surrogate-models, handling points where the simulation does not converge and managing exploration-exploitation to improve the optimization search when only a small number of function evaluations are allowed, and it efficiently obtains a good approximation to an optimum of such functions in difficult real-world optimization problems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. N. Alexandrov. Robustness properties of a trust region framework for managing approximations in engineering optimization. In Proceedings of the Sixth AIAA/NASA/ISSMO Symposium on Multidisciplinary Analysis and Design, number AIAA-96-4102-CP, pages 1056–1059. AIAA, 1996.

    Google Scholar 

  2. J. F. M. Barthelemy and R. T. Haftka. Approximation concepts for optimum structural design — a review. Structural optimization, 5:129–144, 1993.

    Article  Google Scholar 

  3. A. J. Booker. Case studies in design and analysis of computer experiments. In Proceedings of the Section on Physical and Engineering Sciences, pages 244–248. American Statistical Association, 1996.

    Google Scholar 

  4. V. Černy. Thermodynamical approach to the traveling salesman problem: An efficient simulation algorithm. Journal of Optimization Theory and Applications, 45(1):41–51, 1985.

    Article  MATH  MathSciNet  Google Scholar 

  5. K. Chiba, S. Jeong, O. Shigeru, and H. Morino. Data mining for multidisciplinary design space of regional-jet wing. In Proceedings the 2005 IEEE Congress on Evolutionary Computation–CEC 2005, pages 2333–2340. IEEE, 2005.

    Google Scholar 

  6. A. Chipperfield, P. Fleming, H. Pohlheim, and C. Fonseca. Genetic Algorithm TOOLBOX for use with MATLAB, Version 1.2. Department of Automatic Control and Systems Engineering, University of Sheffield.

    Google Scholar 

  7. A. R. Conn, K. Scheinberg, and P. L. Toint. Recent progress in unconstrained nonlinear optimization without derivatives. Mathematical Programming, 79:397–414, 1997.

    MathSciNet  Google Scholar 

  8. A. R. Conn, K. Scheinberg, and P. L. Toint. A derivative free optimization algorithm in practice. In Proceedings of the Seventh AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization. AIAA, 1998.

    Google Scholar 

  9. J. E. Dennis and L. N. Vicente. Trust-region interior-point algorithms for minimization methods with simple bounds. In H. Fischer, B. Riedmuller, and S. Schaffler, editors, Applied Mathematics and Parallel Computing, Festschrift for Klaus Ritter, pages 97–107. Physica, Heidelberg, 1996.

    Google Scholar 

  10. M. Derla and H. Youngren. XFOIL 6.9 User Primer. Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, Cambridge, MA, 2001.

    Google Scholar 

  11. J. G. Digalakis and K. G. Margaritis. An experimental study of benchmarking functions for genetic algorithms. International Journal of Computer Mathematics, 79(4):403–416, 2002.

    Article  MATH  MathSciNet  Google Scholar 

  12. L. C. W. Dixon. The choice of step length, a crucial factor in the performance of variable metric algorithms. In F. A. Lootsma, editor, Numerical Methods for Non-linear Optimization, chapter 10, pages 149–170. Academic, London New York, 1972.

    Google Scholar 

  13. J. W. Eaton. Octave: An Interactive Language for Numerical Computations, University of Wisconsin–Madison, Madison, 1997.

    Google Scholar 

  14. D. Eby, R. C. Averill, W. F. I. Punch, and E. D. Goodman. Evaluation of injection island GA performance on flywheel design optimization. In Proceedings of the Third Conference on Adaptive Computing in Design and Manufacturing, Plymouth, England, 1998.

    Google Scholar 

  15. P. D. Frank. Global modeling for optimization. SIAM SIAG/OPT Views-and-News, 7:9–12, 1995.

    Google Scholar 

  16. R. Franke. Scattered data interpolation: Tests of some method. Mathematics of Computation, 38(157):181–200, 1982.

    Article  MATH  MathSciNet  Google Scholar 

  17. K. C. Giannakogolu. Design of optimal aerodynamic shapes using stochastic optimization methods and computational intelligence. International Review Journal Progress in Aerospace Sciences, 38(1):43–76, 2002.

    Article  Google Scholar 

  18. R. L. Hardy. Theory and applications of the multiquadric-biharmonic method. Computers and Mathematics with Applications, 19(8/9):163–208, 1990.

    Article  MATH  MathSciNet  Google Scholar 

  19. T. L. Holst and T. H. Pulliam. Aerodynamic shape optimization using a real-number-encoded genetic algorithm. Technical Report 2001-2473, AIAA, 2001.

    Google Scholar 

  20. R. Jin, W. Chen, and T. W. Simpson. Comparative studies of metamodeling techniques under multiple modeling criteria. Journal of Structural Optimization, 23(1):1–13, 2001.

    Article  Google Scholar 

  21. Y. Jin. A comprehensive survey of fitness approximation in evolutionary computation. Soft Computing, 9(1):3–12, 2005.

    Article  Google Scholar 

  22. Y. Jin, M. Olhofer, and B. Sendhoff. A framework for evolutionary optimization with approximate fitness functions. IEEE Transactions on evolutionary computation, 6(5):481–494, 2002.

    Article  Google Scholar 

  23. H.-S. Kim and S.-B. Cho. An efficient genetic algorithm with less fitness evaluation by clustering. In Proceedings of 2001 IEEE Conference on Evolutionary Computation, pages 887–894. IEEE, 2001.

    Google Scholar 

  24. S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi. Optimization by simulated annealing. Science, 220(4598):471–480, 1983.

    Article  MathSciNet  Google Scholar 

  25. K.-H. Liang, X. Yao, and C. Newton. Evolutionary search of approximated N-dimensional landscapes. International Journal of Knowledge-Based Intelligent Engineering Systems, 4(3):172–183, 2000.

    Google Scholar 

  26. M. D. McKay, R. J. Beckman, and W. J. Conover. A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics, 21(2):239–245, 1979.

    Article  MATH  MathSciNet  Google Scholar 

  27. N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, and T. Edward. Equation of state calculations by fast computing machines. Journal of Chemical Physics, 21(6):1087–1092, 1953.

    Article  Google Scholar 

  28. T. Mitchell and M. Morris. Bayesian design and analysis of computer experiments: Two examples. Statistica Sinica, 2:359–379, 1992.

    MATH  Google Scholar 

  29. H. Mühlenbein and D. Schlierkamp-Voosen. Predictive models for the breeder genetic algorithm I: Continuous parameter optimization. Evolutionary Computations, 1(1):25–49, 1993.

    Article  Google Scholar 

  30. S. Obayashi. Airfoil shape optimization for evolutionary computation. In Genetic Algorithms for Optimization in Aeronautics and Turbomachinery, VKI Lecture Series 2000-07. Rhode Saint Genese, Belgium and Von Karman Institute for Fluid Dynamics, 2000.

    Google Scholar 

  31. Y.-S. Ong, P. B. Nair, and A. J. Keane. Evolutionary optimization of computationally expensive problems via surrogate modeling. American Institute of Aeronautics and Astronautics Journal, 41(4):687–696, 2003.

    Google Scholar 

  32. Y.-S. Ong, P. B. Nair, and K. Y. Lum. Max-min surrogate-assisted evolutionary algorithm for robust aerodynamic design. IEEE Transactions on Evolutionary Computation, 10(4):392–404, 2006.

    Article  Google Scholar 

  33. Y.-S. Ong, Z. Zong, and D. Lim. Curse and blessing of uncertainty in evolutionary algorithm using approximation. In Proceedings of the IEEE Congress on Evolutionary Computation–WCCI 2006, pages 2928–2935. IEEE, 2006.

    Google Scholar 

  34. A. Oyama, S. Obayashi, and T. Nakamura. Real-coded adaptive range genetic algorithm applied to transonic wing optimization. In M. Schoenauer, K. Deb, R. Günter, X. Yao, E. Lutton, J. J. M. Guervós, and H.-P. Schwefel, editors, The 6th Parallel Problem Solving from Nature International Conference–PPSN VI, volume 1917 of Lecture Notes in Computer Science, pages 712–721. Springer, Berlin Heidelberg New York, 2000.

    Chapter  Google Scholar 

  35. M. J. D. Powell. Direct search algorithms for optimization calculations. Acta Numerica, 7:287–336, 1998.

    Article  MathSciNet  Google Scholar 

  36. M. J. D. Powell. UOBYQA: Unconstrained optimization by quadratic approximation. Mathematical Programming, Series B, 92:555–582, 2002.

    Article  MATH  MathSciNet  Google Scholar 

  37. K. Rasheed, H. Hirsh, and A. Gelsey. Genetic algorithm for continuous design space search. Artificial Intelligence in Engineering, 11(3):295–305, 1996.

    Article  Google Scholar 

  38. A. Ratle. Accelerating the convergence of evolutionary algorithms by fitness landscape approximations. In A. E. Eiben, B. T., M. Schoenauer, and H. P. Schwefel, editors, Proceedings of the 5th International Conference on Parallel Problem Solving from Nature–PPSN V, volume 1498 of Lecture Notes in Computer Science, pages 87–96. Springer, Berlin Heidelberg New York, 1998.

    Chapter  Google Scholar 

  39. J.-M. Renderes and S. P. Flasse. Hybrid methods using genetic algorithms for global optimization. IEEE Transcation on Systems, Man and Cybernetics–Part B, 26(2):243–258, 1996.

    Article  Google Scholar 

  40. H. H. Rosenbrock. An automated method for finding the greatest of least value of a function. The Computer Journal, 3:175–184, 1960.

    Article  MathSciNet  Google Scholar 

  41. J. Sacks, W. J. Welch, T. J. Mitchell, and H. P. Wynn. Design and analysis of computer experiments. Statistical Science, 4(4):409–435, 1989.

    Article  MATH  MathSciNet  Google Scholar 

  42. T. Sauer and Y. Xu. On multivariate Lagrange interpolation. Mathematics of Computation, 64(211):1147–1170, 1995.

    Article  MATH  MathSciNet  Google Scholar 

  43. H.-P. Schewefel. Numerical Optimization of Computer Models, volume 26 of Interdisciplinary Systems Research. Wiley, Chichester, 1981.

    Google Scholar 

  44. M. Sefrioui and J. Périaux. A hierarchical genetic algorithm using multiple models for optimization. In M. Schoenauer, K. Deb, G. Rudolph, X. Yao, E. Lutton, J. J. M. Guervós, and H.-P. Schwefel, editors, Proceedings of the 6th International Conference on Parallel Problem Solving from Nature–PPSN VI, volume 1917 of Lecture Notes in Computer Science, pages 879–888. Springer, Berlin Heidelberg New York, 2000.

    Chapter  Google Scholar 

  45. T. W. Simpson, D. K. J. Lin, and W. Chen. Sampling strategies for computer experiments. International Journal of Reliability and Applications, 2(3):209–240, 2001.

    Google Scholar 

  46. R. E. Smith, B. Dike, and S. Stegmann. Fitness inheritance in genetic algorithms. In K. M. George, editor, Proceedings of the 1995 ACM Symposium on Applied Computing–SAC 95, pages 345–350. ACM, 1995.

    Google Scholar 

  47. H. Sobieckzy. Parametric airfoils and wings. In K. Fujii and G. S. Dulikravich, editors, Recent Development of Aerodynamic Design Methodologies-Inverse Design and Optimization, volume 68 of Notes on Numerical Fluid Mechanics, pages 71–88. Vieweg, Germany, 1999.

    Google Scholar 

  48. J. Sobieszczansk-Sobieski and R. Haftka. Multidisciplinary aerospace design optimization: Survey of recent developments. Structural Optimization, 14(1): 1–23, 1997.

    Article  Google Scholar 

  49. Y. Tenne and S. W. Armfield. A memetic algorithm using a trust-region derivative-free optimization with quadratic modelling for optimization of expensive and noisy black-box functions. In S. Yang, Y.-S. Ong, and Y. Jin, editors, Evolutionary Computation in Dynamic and Uncertain Environments, volume 51 of Studies in Computational Intelligence, pages 389–415. Springer, Berlin Heidelberg New York, 2007.

    Chapter  Google Scholar 

  50. Y. Tenne, S. Obayashi, and S. Armfield. Airfoil shape optimization using an algorithm for minimization of expensive and discontinuous black-box functions. In Proceedings of AIAA InfoTec 2007, number AIAA-2007-2874. AIAA, 2007.

    Google Scholar 

  51. Y. Tenne, S. Obayashi, S. Armfield, Y.-S. Ong, and R. Tapabrata. A surrogate-assisted memetic algorithm for handling computationally expensive functions with ill-defined constraints. In preparation.

    Google Scholar 

  52. A. Törn. Cluster analysis using seed points and density-determined hyperspheres as an aid to global optimization. IEEE Transcation on Systems, Man and Cybernetics, 7(8):610–616, 1977.

    Article  MATH  Google Scholar 

  53. A. A. Törn and A. Žilinskas. Global Optimization, volume 350 of Lecture Notes In Computer Science. Springer, Berlin Heidelberg New York, 1989.

    MATH  Google Scholar 

  54. X. Yao, G. Lin, and Y. Liu. An analysis of evolutionary algorithms based on neighbourhood and step sizes. In P. Angeline, R. Reynolds, J. McDonnel, and R. Eberhart, editors, Evolutionary Programming VI:Proceedings of the Sixth Annual Conference on Evolutionary Programming, volume 1213 of Lecture Notes on Computer Science, pages 297–307. Springer, Berlin Heidelberg New York, 1997.

    Google Scholar 

  55. Z. Z. Zhou, Y.-S. Ong, P. B. Nair, A. J. Keane, and K. Y. Lum. Combining global and local surrogate models to accelerate evolutionary optimization. IEEE Transactions On Systems, Man and Cybernetics-Part C, 37(1):66–76, 2007.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. The School of Aerospace, Mechanical and Mechatronic Engineering, The University of Sydney, Australia

    Yoel Tenne & S. W. Armfield

Authors
  1. Yoel Tenne
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. S. W. Armfield
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. CSIRO Land and Water, Clunies Ross Street, ACT 2901, Canberra, Australia

    Ang Yang

  2. Medicare Australia, ACT 2901, Canberra, Australia

    Yin Shan

  3. University of New South Wales, Northcott Drive, ACT 2600, Canberra, Australia

    Lam Thu Bui

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Tenne, Y., Armfield, S.W. (2008). A Versatile Surrogate-Assisted Memetic Algorithm for Optimization of Computationally Expensive Functions and its Engineering Applications. In: Yang, A., Shan, Y., Bui, L.T. (eds) Success in Evolutionary Computation. Studies in Computational Intelligence, vol 92. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-76286-7_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-76286-7_3

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-76285-0

  • Online ISBN: 978-3-540-76286-7

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation