Abstract
The development and identification of effective optimization algorithms for non-convex real-world problems is a challenge in global optimization. Because theoretical performance analysis is difficult, and problems based on models of real-world systems are often computationally expensive, several artificial performance test problems and test function generators have been proposed for empirical comparative assessment and analysis of metaheuristic optimization algorithms. These test problems however often lack the complex function structures and forthcoming difficulties that can appear in real-world problems. This communication presents a method to systematically build test problems with various types and degrees of difficulty. By weighted composition of parameterized random fields, challenging test functions with tunable function features such as, variance contribution distribution, interaction order, and nonlinearity can be constructed. The method is described, and its applicability to optimization performance analysis is described by means of a few basic examples. The method aims to set a step forward in the systematic generation of global optimization test problems, which could lead to a better understanding of the performance of optimization algorithms on problem types with particular characteristics. On request an introductive MATLAB implementation of a test function generator based on the presented method is available.
Similar content being viewed by others
Notes
When “difficulty” or “hardness” is averaged over all possible search or optimization algorithms no problems are intrinsically harder than others [25]. However, for a particular optimization algorithm, some problem classes can be more difficult than others.
Such a passive map would be very memory intensive since the required memory scales with the number of elements \(m=\mathop \prod \nolimits _{d=1}^n r_d\) or \(m=r^n\) for a uniform resolution r and field dimension n. which already becomes problematic at modest resolutions and problem dimensions. A discrete field array of resolution \(r=10\) and dimension \(n=12\) would already require 8 TB (terabyte) of memory when each element takes 8 bit of storage.
Although the above statement is true for all \(p > 0\) the author recommends to use as a rule of thumb \(p \ge 1/d\), since for very small values of p the smoothness vanishes.
Explanation to the multi-index notation: The expression \(\mathop \sum \nolimits _{1\le i<j\le n} f_{i,j} ( {x_i,x_j })\) indicates a sum over all function decomposition terms with two variables for which \(1\le i<j\le n\). This applies similarly to all pairs of higher order interactions \(f_{i,j,\ldots ,n} \).
The variance for the terms expression 6, w.r.t. the corresponding sub domain in the unit hypercube can be expressed as: \(\text{ Var }( {f_{i,j,\ldots ,n} ({x_1,x_2,\ldots ,x_n })}) = \int f^2_{i,j,\ldots ,n} ( {x_1,x_2,\ldots ,x_n })\text{ d }x_i \ldots \text{ d }x_n \).
References
Bonnans, J.F., Gilbert, J.C., Lemaréchal, C., Sagastizábal, C.A.: Numerical Optimization: Theoretical and Practical Aspects. Springer Science & Business Media, Berlin (2006)
Floudas, C.A., Gounaris, C.E.: A review of recent advances in global optimization. J. Glob. Optim. 45, 3–38 (2009)
Floudas, C.A., Pardalos, P.M., Adjiman, C.S., Esposito, W.R., Gümü, Z.H., Harding, S.T., Klepeis, J.L., Meyer, C.A., Schweiger, C.A.: Handbook of Test Problems in Local and Global Optimization. Springer, Boston (1999)
van Leeuwen, J.: Handbook of Theoretical Computer Science. Elsevier, MIT Press, Amsterdam; Cambridge, Mass (1990)
Ahmadi, A.A., Olshevsky, A., Parrilo, P.A., Tsitsiklis, J.N.: NP-hardness of deciding convexity of quartic polynomials and related problems. Math. Program. 137, 453–476 (2013)
Jones, D.R., Schonlau, M., Welch, W.J.: Efficient global optimization of expensive black-box functions. J. Glob. Optim. 13, 455–492 (1998)
Osman, I.H., Laporte, G.: Metaheuristics: A bibliography. Ann. Oper. Res. 63, 511–623 (1996)
Yang, X.-S.: Review of meta-heuristics and generalised evolutionary walk algorithm. Int. J. Bio-Inspired Comput. 3, 77–84 (2011)
Borenstein, Y., Poli, R.: Fitness distributions and GA hardness. In: Parallel Problem Solving from Nature-PPSN VIII, pp. 11–20. Springer, Berlin (2004)
De Jong, K.A.: Analysis of the behavior of a class of genetic adaptive systems (1975)
Hock, W., Schittkowski, K.: Test examples for nonlinear programming codes. J. Optim. Theory Appl. 30(1), 127–129 (1980)
Schittkowski, K.: More Test Examples for Nonlinear Programming Codes, vol. 282. Springer Science & Business Media, Berlin (2012)
Andrei, N.: An unconstrained optimization test functions collection. Adv. Model. Optim. 10, 147–161 (2008)
Gould, N.I., Orban, D., Toint, P.L.: CUTEr and SifDec: a constrained and unconstrained testing environment, revisited. ACM Trans. Math. Softw. (TOMS) 29(4), 373–394 (2003)
Shcherbina, O., Neumaier, A., Sam-Haroud, D., Vu, X.H., Nguyen, T.V.: Benchmarking Global Optimization and Constraint Satisfaction Codes. Springer, Berlin (2003)
Gould, N.I., Orban, D., Toint, P.L.: CUTEst: a constrained and unconstrained testing environment with safe threads. Cahier du GERAD G 2013(27), 30 (2013)
Domes, F., Fuchs, M., Schichl, H., Neumaier, A.: The optimization test environment. Optim. Eng. 15(2), 443–468 (2014)
Barrera, J., Coello, C.A.C.: Test function generators for assessing the performance of PSO algorithms in multimodal optimization. In: Handbook of Swarm Intelligence, pp. 89–117. Springer, Berlin (2011)
Dieterich, J.M., Hartke, B.: Empirical review of standard benchmark functions using evolutionary global optimization. Appl. Math. 03, 1552–1564 (2012)
Liang, J.J., Suganthan, P.N., Deb, K.: Novel composition test functions for numerical global optimization. In: Swarm Intelligence Symposium, 2005. SIS 2005. Proceedings 2005 IEEE, pp. 68–75. IEEE, New York (2005)
Balasundaram, B., Butenko, S.: Constructing test functions for global optimization using continuous formulations of graph problems. Optim. Methods Softw. 20, 439–452 (2005)
Addis, B., Locatelli, M.: A new class of test functions for global optimization. J. Glob. Optim. 38, 479–501 (2007)
Gallagher, M.: Bo Yuan: a general-purpose tunable landscape generator. IEEE Trans. Evol. Comput. 10, 590–603 (2006)
Ahrari, A., Ahrari, R.: On the utility of randomly generated functions for performance evaluation of evolutionary algorithms. Optim. Lett. 4, 531–541 (2010)
Macready, W.G., Wolpert, D.H.: What makes an optimization problem hard? Complexity 1, 40–46 (1996)
Sala, R., Baldanzini, N., Pierini, M.: Representative surrogate problems as test functions for expensive simulators in multidisciplinary design optimization of vehicle structures. Under Rev. Struct. Multidiscip (2016). doi:10.1007/s00158-016-1410-9
Mahdavi, S., Shiri, M.E., Rahnamayan, S.: Metaheuristics in large-scale global continues optimization: a survey. Inf. Sci. 295, 407–428 (2015)
Potter, M.A., De Jong, K.A.: A cooperative coevolutionary approach to function optimization. In: Parallel Problem Solving from Nature—PPSN III, pp. 249–257. Springer, Berlin (1994)
Adler, R.J., Taylor, J.E.: Random Fields and Geometry. Springer Science & Business Media, Berlin (2009)
Hoeffding, W.: A class of statistics with asymptotically normal distribution. Ann. Math. Stat., 19(3), 293–325 (1948)
Sobol’, I.M.: On sensitivity estimation for nonlinear mathematical models. Mat. Model. 2, 112–118 (1990)
Multi-index notation. Encyclopedia of Mathematics. http://www.encyclopediaofmath.org/index.php?title=Multi-index_notation&oldid=30979
Chipperfield, A.J., Fleming, P.J.: The MATLAB genetic algorithm toolbox. In: IEE Colloquium on Applied Control Techniques Using MATLAB, pp. 10/1–10/4. IET, New York (1995)
Kolmogorov, A.N.: Three approaches to the quantitative definition of information. Probl. Inf. Transm. 1, 1–7 (1965)
Acknowledgments
The presented method is a generalized derivative of strategies developed for efficient optimization on structural simulations in vehicle design in the scope of the GRESIMO Project Funded by the European Commission under the 7th framework program under Grant Agreement 290050.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Sala, R., Baldanzini, N. & Pierini, M. Global optimization test problems based on random field composition. Optim Lett 11, 699–713 (2017). https://doi.org/10.1007/s11590-016-1037-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11590-016-1037-1