Skip to main content
SpringerLink
Log in

Fourier Analysis Meets Runtime Analysis: Precise Runtimes on Plateaus

  • Published:
Algorithmica Aims and scope Submit manuscript

Abstract

We propose a new method based on discrete Fourier analysis to analyze the time evolutionary algorithms spend on plateaus. This immediately gives a concise proof of the classic estimate of the expected runtime of the \((1+1)\) evolutionary algorithm on the Needle problem due to Garnier et al. (Evol Comput 7:173–203, 1999). We also use this method to analyze the runtime of the \((1+1)\) evolutionary algorithm on a benchmark consisting of \(n/\ell \) plateaus of effective size \(2^\ell -1\) which have to be optimized sequentially in a LeadingOnes fashion. Using our new method, we determine the precise expected runtime both for static and fitness-dependent mutation rates. We also determine the asymptotically optimal static and fitness-dependent mutation rates. For \(\ell = o(n)\), the optimal static mutation rate is approximately 1.59/n. The optimal fitness dependent mutation rate, when the first k fitness-relevant bits have been found, is asymptotically \(1/(k+1)\). These results, so far only proven for the single-instance problem LeadingOnes, thus hold for a much broader class of problems. We expect similar extensions to be true for other important results on LeadingOnes. We are also optimistic that the Fourier analysis approach can be applied to other plateau problems as well.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. We thank the anonymous reviewer who suggested that this hypothesis might be needed. It indeed is needed. In his paper [91], Zhang had forgotten this hypothesis in his statement of Theorem 3.1 but used it in his Lemma 3.3. The authors of the present paper had only considered a probability distribution whose support generates G.

  2. https://mathoverflow.net/questions/17202/sum-of-the-first-k-binomial-coefficients-for-fixed-n

References

  1. Afshani, P., Agrawal, M., Doerr, B., Doerr, C., Larsen, K.G.: The query complexity of a permutation-based variant of Mastermind. Discrete Appl. Math. 260, 28–50 (2019)

    Article  MathSciNet  Google Scholar 

  2. Auger, A., Doerr, B. (eds.): Theory of Randomized Search Heuristics. World Scientific Publishing, Singapore (2011)

    Google Scholar 

  3. Antipov, D., Doerr, B.: Precise runtime analysis for plateau functions. ACM Trans. Evol. Learn. Optim. 1, 13:1-13:28 (2021)

    Article  Google Scholar 

  4. Antipov, D., Doerr, B., Karavaev, V.: A tight runtime analysis for the \({(1 + (\lambda ,\lambda ))}\) GA on LeadingOnes. Foundations of Genetic Algorithms, FOGA 2019, pp. 169–182. ACM (2019)

  5. Bambury, H., Bultel, A., Doerr, B.: Generalized jump functions. In: Genetic and Evolutionary Computation Conference, GECCO 2021, pp. 1124–1132. ACM (2021)

  6. Buzdalov, M., Doerr, B., Doerr, C., Vinokurov, D.: Fixed-target runtime analysis. Algorithmica 84, 1762–1793 (2022)

    Article  MathSciNet  Google Scholar 

  7. Böttcher, S., Doerr, B., Neumann, F.: Optimal fixed and adaptive mutation rates for the LeadingOnes problem. In: Parallel Problem Solving from Nature, PPSN 2010, pp. 1–10. Springer (2010)

  8. Brockhoff, D., Friedrich, T., Hebbinghaus, N., Klein, C., Neumann, F., Zitzler, E.: Do additional objectives make a problem harder? In: Genetic and Evolutionary Computation Conference, GECCO 2007, pp. 765–772. ACM (2007)

  9. Badkobeh, G., Lehre, P.K., Sudholt, D.: Unbiased black-box complexity of parallel search. In: Parallel Problem Solving from Nature, PPSN 2014, pp. 892–901. Springer (2014)

  10. Bian, C., Qian, C., Jiang, W., Tang, K.: Towards a running time analysis of the (1+1)-EA for OneMax and LeadingOnes under general bit-wise noise. In: Parallel Problem Solving from Nature, PPSN 2018, Part II, pp. 165–177. Springer (2018)

  11. Corus, D., Dang, D.C., Eremeev, A.V., Lehre, P.K.: Level-based analysis of genetic algorithms and other search processes. IEEE Trans. Evol. Comput. 22, 707–719 (2018)

    Article  Google Scholar 

  12. Chicano, F., Sutton, A.M., Whitley, L.D., Alba, E.: Fitness probability distribution of bit-flip mutation. Evol. Comput. 23, 217–248 (2015)

    Article  Google Scholar 

  13. Doerr, B., Doerr, C., Lengler, J.: Self-adjusting mutation rates with provably optimal success rules. Algorithmica 83, 3108–3147 (2021)

    Article  MathSciNet  Google Scholar 

  14. Doerr, B., Doerr, C., Neumann., F.: Fast re-optimization via structural diversity. In: Genetic and Evolutionary Computation Conference, GECCO 2019, pp. 233–241. ACM (2019)

  15. Doerr, B., Goldberg, L.A.: Adaptive drift analysis. Algorithmica 65, 224–250 (2013)

    Article  MathSciNet  Google Scholar 

  16. Doerr, B., Happ, E., Klein, C.: Crossover can provably be useful in evolutionary computation. Theoret. Comput. Sci. 425, 17–33 (2012)

    Article  MathSciNet  Google Scholar 

  17. Doerr, B., Hebbinghaus, N., Neumann, F.: Speeding up evolutionary algorithms through asymmetric mutation operators. Evol. Comput. 15, 401–410 (2007)

    Article  Google Scholar 

  18. Droste, S., Jansen, T., Wegener, I.: On the analysis of the (1+1) evolutionary algorithm. Theoret. Comput. Sci. 276, 51–81 (2002)

    Article  MathSciNet  Google Scholar 

  19. Droste, S., Jansen, T., Wegener, I.: Upper and lower bounds for randomized search heuristics in black-box optimization. Theory Comput. Syst. 39, 525–544 (2006)

    Article  MathSciNet  Google Scholar 

  20. Doerr, B., Johannsen, D., Winzen, C.: Multiplicative drift analysis. Algorithmica 64, 673–697 (2012)

    Article  MathSciNet  Google Scholar 

  21. Doerr, B., Jansen, T., Witt, C., Zarges, C.: A method to derive fixed budget results from expected optimisation times. In: Genetic and Evolutionary Computation Conference, GECCO 2013, pp. 1581–1588. ACM (2013)

  22. Doerr, B., Künnemann, M.: Royal road functions and the (1 + \(\lambda \)) evolutionary algorithm: almost no speed-up from larger offspring populations. In: Congress on Evolutionary Computation, CEC 2013, pp. 424–431. IEEE (2013)

  23. Doerr, B., Krejca, M.S.: Significance-based estimation-of-distribution algorithms. IEEE Trans. Evol. Comput. 24, 1025–1034 (2020)

    Article  Google Scholar 

  24. Doerr, B., Kötzing, T.: Lower bounds from fitness levels made easy. In: Genetic and Evolutionary Computation Conference, GECCO 2021, pp. 1142–1150. ACM (2021)

  25. Doerr, B., Kötzing, T.: Multiplicative up-drift. Algorithmica 83, 3017–3058 (2021)

    Article  MathSciNet  Google Scholar 

  26. Doerr, B., Kelley, A.J.: Fourier analysis meets runtime analysis: precise runtimes on plateaus. In: Genetic and Evolutionary Computation Conference, GECCO 2023. ACM (2023). To appear

  27. Doerr, B., Le, H.P., Makhmara, R., Nguyen, T.D.: Fast genetic algorithms. In: Genetic and Evolutionary Computation Conference, GECCO 2017, pp. 777–784. ACM (2017)

  28. Dang, D.C., Lehre, P.K., Nguyen, P.T.: Level-based analysis of the univariate marginal distribution algorithm. Algorithmica 81, 668–702 (2019)

    Article  MathSciNet  Google Scholar 

  29. Doerr, B., Lissovoi, A., Oliveto, P.S., Warwicker, J.A.: On the runtime analysis of selection hyper-heuristics with adaptive learning periods. In: Genetic and Evolutionary Computation Conference, GECCO 2018, pp. 1015–1022. ACM (2018)

  30. Doerr, B., Neumann, F., (eds).: Theory of Evolutionary Computation—Recent Developments in Discrete Optimization. Springer (2020). http://www.lix.polytechnique.fr/Labo/Benjamin.Doerr/doerr_neumann_book.html

  31. Dang-Nhu, R., Dardinier, T., Doerr, B., Izacard, G., Nogneng, D.: A new analysis method for evolutionary optimization of dynamic and noisy objective functions. In: Genetic and Evolutionary Computation Conference, GECCO 2018, pp. 1467–1474. ACM (2018)

  32. Doerr, B., Neumann, F., Sudholt, D., Witt, C.: Runtime analysis of the 1-ANT ant colony optimizer. Theoret. Comput. Sci. 412, 1629–1644 (2011)

    Article  MathSciNet  Google Scholar 

  33. Doerr, B.: Analyzing randomized search heuristics via stochastic domination. Theoret. Comput. Sci. 773, 115–137 (2019)

    Article  MathSciNet  Google Scholar 

  34. Doerr, B.: Probabilistic tools for the analysis of randomized optimization heuristics. In: Doerr, B., Neumann, F. (eds), Theory of Evolutionary Computation: Recent Developments in Discrete Optimization, pp. 1–87. Springer (2020). arXiv:1801.06733

  35. Doerr, B.: Exponential upper bounds for the runtime of randomized search heuristics. Theoret. Comput. Sci. 851, 24–38 (2021)

    Article  MathSciNet  Google Scholar 

  36. Doerr, B.: Lower bounds for non-elitist evolutionary algorithms via negative multiplicative drift. Evol. Comput. 29, 305–329 (2021)

    Article  Google Scholar 

  37. Doerr, B.: The runtime of the compact genetic algorithm on Jump functions. Algorithmica 83, 3059–3107 (2021)

    Article  MathSciNet  Google Scholar 

  38. Droste, S.: Analysis of the (1+1) EA for a dynamically changing OneMax-variant. In: Congress on Evolutionary Computation, CEC 2002, pp. 55–60. IEEE (2002)

  39. Doerr, B., Sudholt, D., Witt, C.: When do evolutionary algorithms optimize separable functions in parallel? In: Foundations of Genetic Algorithms, FOGA 2013, pp. 48–59. ACM (2013)

  40. Doerr, B., Winzen, C.: Black-box complexity: breaking the O(n log n) barrier of LeadingOnes. In: International Conference on Artificial Evolution, EA 2011, pp. 205–216. Springer (2012)

  41. Doerr, C., Wagner, M.: Simple on-the-fly parameter selection mechanisms for two classical discrete black-box optimization benchmark problems. In: Genetic and Evolutionary Computation Conference, GECCO 2018, pp. 943–950. ACM (2018)

  42. Doerr, C., Ye, F., Horesh, N., Wang, H., Shir, O.M., Bäck, T.: Benchmarking discrete optimization heuristics with iohprofiler. Appl. Soft Comput. 88, 106027 (2020)

    Article  Google Scholar 

  43. Doerr, B., Zheng, W.: Theoretical analyses of multi-objective evolutionary algorithms on multi-modal objectives. In: Conference on Artificial Intelligence, AAAI 2021, pp. 12293–12301. AAAI Press (2021)

  44. Eremeev, A.V.: On non-elitist evolutionary algorithms optimizing fitness functions with a plateau. In: Mathematical Optimization Theory and Operations Research, MOTOR 2020, pp. 329–342. Springer (2020)

  45. Eremeev, A.V., Spirov, A.V.: Modeling SELEX for regulatory regions using Royal Road and Royal Staircase fitness functions. Biosystems 200, 104312 (2021)

    Article  Google Scholar 

  46. Friedrich, T., Hebbinghaus, N., Neumann, F.: Comparison of simple diversity mechanisms on plateau functions. Theoret. Comput. Sci. 410, 2455–2462 (2009)

    Article  MathSciNet  Google Scholar 

  47. Friedrich, T., Hebbinghaus, N., Neumann, F.: Plateaus can be harder in multi-objective optimization. Theoret. Comput. Sci. 411, 854–864 (2010)

    Article  MathSciNet  Google Scholar 

  48. Friedrich, t., Kötzing, t., Krejca, M.S.: EDAs cannot be balanced and stable. In: Genetic and Evolutionary Computation Conference, GECCO 2016, pp. 1139–1146. ACM (2016)

  49. Garrett, P.: Fourier analysis on finite abelian groups. Preprint. Available at https://www-users.cse.umn.edu/~garrett/m/mfms/notes_c/fin_ab_fourier.pdf (2012)

  50. Gießen, C., Kötzing, T.: Robustness of populations in stochastic environments. Algorithmica 75, 462–489 (2016)

    Article  MathSciNet  Google Scholar 

  51. Garnier, J., Kallel, L., Schoenauer, M.: Rigorous hitting times for binary mutations. Evol. Comput. 7, 173–203 (1999)

    Article  Google Scholar 

  52. Giel, O., Wegener, I.: Evolutionary algorithms and the maximum matching problem. In: Symposium on Theoretical Aspects of Computer Science, STACS 2003, pp. 415–426. Springer (2003)

  53. He, J., Yao, X.: Drift analysis and average time complexity of evolutionary algorithms. Artif. Intell. 127, 51–81 (2001)

    Article  MathSciNet  Google Scholar 

  54. Jansen, T.: Analyzing Evolutionary Algorithms—The Computer Science Perspective. Springer (2013)

  55. Jansen, T.: On the black-box complexity of example functions: the real jump function. In: Foundations of Genetic Algorithms, FOGA 2015, pp. 16–24. ACM (2015)

  56. Jansen, T., De Jong, K.A., Wegener, I.: On the choice of the offspring population size in evolutionary algorithms. Evol. Comput. 13, 413–440 (2005)

    Article  Google Scholar 

  57. Jansen, T., Wegener, I.: Evolutionary algorithms—how to cope with plateaus of constant fitness and when to reject strings of the same fitness. IEEE Trans. Evol. Comput. 5, 589–599 (2001)

    Article  Google Scholar 

  58. Jansen, T., Zarges, C.: Performance analysis of randomised search heuristics operating with a fixed budget. Theoret. Comput. Sci. 545, 39–58 (2014)

    Article  MathSciNet  Google Scholar 

  59. Kötzing, T., Witt, C.: Improved fixed-budget results via drift analysis. In: Parallel Problem Solving from Nature, PPSN 2020, Part II, pp. 648–660. Springer (2020)

  60. Lehre, P.K.: Fitness-levels for non-elitist populations. In: Genetic and Evolutionary Computation Conference, GECCO 2011, pp. 2075–2082. ACM (2011)

  61. Lehre, P.K., Nguyen, P.T.H.: On the limitations of the univariate marginal distribution algorithm to deception and where bivariate EDAs might help. In: Foundations of Genetic Algorithms, FOGA 2019, pp. 154–168. ACM (2019)

  62. Lehre, P.K., Nguyen, P.T.H.: Runtime analyses of the population-based univariate estimation of distribution algorithms on LeadingOnes. Algorithmica 83, 3238–3280 (2021)

  63. Lissovoi, A., Oliveto, P.S., Warwicker, J.A.: Simple hyper-heuristics control the neighbourhood size of randomised local search optimally for LeadingOnes. Evol. Comput. 28, 437–461 (2020)

    Article  Google Scholar 

  64. Lehre, P.K., Qin, X.: More precise runtime analyses of non-elitist EAs in uncertain environments. In: Genetic and Evolutionary Computation Conference, 2021, pp. 1160–1168. ACM (2021)

  65. Lässig, J., Sudholt, D.: General upper bounds on the runtime of parallel evolutionary algorithms. Evol. Comput. 22, 405–437 (2014)

    Article  Google Scholar 

  66. Lehre, P.K., Witt, C.: Tail bounds on hitting times of randomized search heuristics using variable drift analysis. Comb. Probab. Comput. 30, 550–569 (2021)

    Article  MathSciNet  Google Scholar 

  67. Mitchell, M., Forrest, S., Holland, J.H.: The royal road for genetic algorithms: fitness landscapes and GA performance. In: European Conference on Artificial Life (ECAL 1991), pp. 245–254. MIT Press (1992)

  68. Metropolis, N., Rosenbluth, A.W., Rosenbluth, M.N., Teller, A.H., Teller, E.: Equation of state calculations by fast computing machines. J. Chem. Phys. 21, 1087–1092 (1953)

    Article  Google Scholar 

  69. Moraglio, A., Sudholt, D.: Principled design and runtime analysis of abstract convex evolutionary search. Evol. Comput. 25, 205–236 (2017)

    Article  Google Scholar 

  70. Mühlenbein, H.: How genetic algorithms really work: mutation and hillclimbing. In: Parallel Problem Solving from Nature, PPSN 1992, pp. 15–26. Elsevier (1992)

  71. Neumann, F.: Expected runtimes of evolutionary algorithms for the Eulerian cycle problem. Comput. OR 35, 2750–2759 (2008)

    Article  MathSciNet  Google Scholar 

  72. Neumann, F., Sudholt, D., Witt, C.: Analysis of different MMAS ACO algorithms on unimodal functions and plateaus. Swarm Intell. 3, 35–68 (2009)

    Article  Google Scholar 

  73. Neumann, F., Wegener, I.: Randomized local search, evolutionary algorithms, and the minimum spanning tree problem. Theoret. Comput. Sci. 378, 32–40 (2007)

    Article  MathSciNet  Google Scholar 

  74. Neumann, F., Witt, C.: Bioinspired Computation in Combinatorial Optimization—Algorithms and Their Computational Complexity. Springer (2010)

  75. Oliveto, P.S., Witt, C.: Simplified drift analysis for proving lower bounds in evolutionary computation. Algorithmica 59, 369–386 (2011)

    Article  MathSciNet  Google Scholar 

  76. Qian, C., Bian, C., Jiang, W., Tang, K.: Running time analysis of the \({(1+1)}\)-EA for OneMax and LeadingOnes under bit-wise noise. Algorithmica 81, 749–795 (2019)

    Article  MathSciNet  Google Scholar 

  77. Rudolph, G.: Convergence Properties of Evolutionary Algorithms. Verlag Dr, Kovǎc (1997)

    Google Scholar 

  78. Rowe, J.E., Vose, M.D., Wright, A.H.: Structural search spaces and genetic operators. Evol. Comput. 12, 461–493 (2004)

    Article  Google Scholar 

  79. Sutton, A.W., Chicano, F., Whitley, L.D.: Fitness function distributions over generalized search neighborhoods in the q-ary hypercube. Evol. Comput. 21, 561–590 (2013)

    Article  Google Scholar 

  80. Scharnow, J., Tinnefeld, K., Wegener, I.: The analysis of evolutionary algorithms on sorting and shortest paths problems. J. Math. Model. Algorithms 3, 349–366 (2004)

    Article  MathSciNet  Google Scholar 

  81. Sudholt, D.: A new method for lower bounds on the running time of evolutionary algorithms. IEEE Trans. Evol. Comput. 17, 418–435 (2013)

    Article  Google Scholar 

  82. Sudholt, D.: Analysing the robustness of evolutionary algorithms to noise: refined runtime bounds and an example where noise is beneficial. Algorithmica 83, 976–1011 (2021)

    Article  MathSciNet  Google Scholar 

  83. van Nimwegen, E., Crutchfield, J.P.: Optimizing epochal evolutionary search: population-size dependent theory. Mach. Learn. 45, 77–114 (2001)

    Article  Google Scholar 

  84. Vose, M.D., Wright, A.H.: The simple genetic algorithm and the walsh transform: part I, theory. Evol. Comput. 6, 253–273 (1998)

    Article  Google Scholar 

  85. Wegener, I.: Theoretical aspects of evolutionary algorithms. In: Automata, Languages and Programming, ICALP 2001, pp. 64–78. Springer (2001)

  86. Witt, C.: Runtime analysis of the (\(\mu \) + 1) EA on simple pseudo-Boolean functions. Evol. Comput. 14, 65–86 (2006)

    Google Scholar 

  87. Witt, C.: Fitness levels with tail bounds for the analysis of randomized search heuristics. Inf. Process. Lett. 114, 38–41 (2014)

    Article  MathSciNet  Google Scholar 

  88. Witt, C.: How majority-vote crossover and estimation-of-distribution algorithms cope with fitness valleys. Theoret. Comput. Sci. 940, 18–42 (2023)

    Article  MathSciNet  Google Scholar 

  89. Wegener, I., Witt, C.: On the optimization of monotone polynomials by simple randomized search heuristics. Combin. Probab. Comput. 14, 225–247 (2005)

    Article  MathSciNet  Google Scholar 

  90. Wang, S., Zheng, W., Doerr, B.: Choosing the right algorithm with hints from complexity theory. In: International Joint Conference on Artificial Intelligence, IJCAI 2021, pp. 1697–1703. ijcai.org (2021)

  91. Zhang, C.: Formulas for hitting times and cover times for random walks on groups. arXiv:2302.01963 (2023)

  92. Zhou, Z.-H., Yang, Y., Qian, C.: Advances in Theories and Algorithms. Springer, Evolutionary Learning (2019)

Download references

Acknowledgements

We would like to thank Marcin Mazur for help with the statement and proof of Lemma 5.10. We also would like to thank the reviewers for their helpful comments, in particular, pointing us to several previous works we were not aware of. Also, one reviewer helped us notice a missing hypothesis in Theorem 3.1. This work was supported by a public grant as part of the Investissements d’avenir project, reference ANR-11-LABX-0056-LMH, LabEx LMH.

Author information

Authors and Affiliations

  1. Laboratoire d’Informatique (LIX), École Polytechnique and CNRS, Palaiseau, France

    Benjamin Doerr

  2. Cobham Advanced Electronic Solutions, Colorado Springs, USA

    Andrew James Kelley

Authors
  1. Benjamin Doerr
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrew James Kelley
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Andrew James Kelley.

Ethics declarations

Conflict of interest

The authors have no conflict of interest to declare that are relevant to the content of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Doerr, B., Kelley, A.J. Fourier Analysis Meets Runtime Analysis: Precise Runtimes on Plateaus. Algorithmica (2024). https://doi.org/10.1007/s00453-024-01232-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00453-024-01232-5

Keywords

Search

Navigation