Abstract
This chapter explains how structural learning performed by multi-variate estimation of distribution algorithms (EDAs) while building their probabilistic models is a form of linkage learning. We then show how multi-variate EDAs linkage learning mechanisms can be misled with the help of two test problems; the concatenated parity function (CPF), and the concatenated parity/trap function (CP/TF). Although these functions are separable, with bounded complexity and uniformly scaled sub-function contributions, the hierarchical Bayesian Optimization Algorithm (hBOA) scales exponentially on both. We argue that test problems containing parity functions are hard for EDAs because there are no interactions in the contribution to fitness between any strict subset of a parity function’s bits. This means that as population sizes increase the dependency between variable values for any strict subset of a parity function’s bits decreases. Unfortunately most EDAs including hBOA search for their models by looking for dependencies between pairs of variables (at least at first). We make suggestions on how EDAs could be adjusted to handle parity problems, but also comment on the apparently inevitable computational cost.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Ackley, D.H.: A Connectionist Machine for Genetic Hillclimbing. Kluwer Academic Publishers, Boston (1987)
Baluja, S., Caruana, R.: Removing the Genetics from the Standard Genetic Algorithm. School of Computer Science, Carnegie Mellon University (1995)
Chen, Y.P., Yu, T.L., Sastry, K., Goldberg, D.E.: A survey of linkage learning techniques in genetic and evolutionary algorithms. IlliGAL Tech. Rep. 2007014 (2007)
Chickering, D., Geiger, D., Heckerman, D.: Learning Bayesian networks is NP-hard. Microsoft Research, 94–17 (1994)
Coffin, D., Smith, R.: The limitations of distribution sampling for linkage learning. Evolutionary Computation. In: CEC 2007. IEEE Congress on 2007, pp. 364–369 (2007)
Deb, K., Goldberg, D.E.: Analyzing deception in trap functions. In: Foundations of Genetic Algorithms - 2, pp. 93–108. Morgan Kaufmann, San Francisco (1992)
Etxeberria, R., Larrañaga, P.: Global optimization using Bayesian networks. In: Ochoa, A., Soto, M.R., Santana, R. (eds.) Proceedings of the Second Symposium on Artificial Intelligence (CIMAF 1999), Havana, Cuba, pp. 151–173 (1999)
Harik, G.: Linkage Learning via Probabilistic Modeling in the ECGA. Tech. Rep. 99010, IlliGAL (1999)
Harik, G., Lobo, F., Goldberg, D.: The compact genetic algorithm. Evolutionary Computation. IEEE Transactions 3(4), 287–297 (1999)
Harik, G.R., Goldberg, D.E.: Learning linkage. In: Belew, R.K., Vose, M.D. (eds.) Foundations of Genetic Algorithms, vol. 4, pp. 247–262. Morgan Kaufmann, San Francisco (1997)
Heckendorn, R.B., Wright, A.H.: Efficient linkage discovery by limited probing. Evol. Comput. 12(4), 517–545 (2004)
Larrañaga, P., Lozano, J.A. (eds.): Estimation of Distribution Algorithms. A New Tool for Evolutionary Computation. Kluwer Academic Publishers, Boston (2002)
Mühlenbein, H., Mahnig, T., Ochoa, A.: Schemata, distributions and graphical models in evolutionary optimization. Journal of Heuristics 5(2), 213–247 (1999)
Mühlenbein, H., Paas, G.: From recombination of genes to the estimation of distributions I. Binary parameters. In: Ebeling, W., Rechenberg, I., Voigt, H.M., Schwefel, H.P. (eds.) PPSN 1996. LNCS, vol. 1141, pp. 178–187. Springer, Heidelberg (1996)
Mühlenbein, H., Paas, G.: From Recombination of Genes to the Estimation of Distributions I. Binary Parameters. In: Proceedings of the 4th International Conference on Parallel Problem Solving from Nature pp. 178–187 (1996)
Munetomo, M., Goldberg, D.: Identifying linkage by nonlinearity check (1998), citeseer.ist.psu.edu/munetomo98identifying.html
Pearl, J.: Causality: Models, Reasoning, and Inference. Cambridge University Press, Cambridge (2000)
Pelikan, M.: Hierarchical Bayesian Optimization Algorithm: Toward a New Generation of Evolutionary Algorithms. Springer, Heidelberg (2005)
Pelikan, M., Goldberg, D.: Hierarchical Bayesian Optimization Algorithm= Bayesian Optimization Algorithm+ Niching+ Local Structures. In: Optimization by Building and Using Probabilistic Models, pp. 217–221 (2001)
Pelikan, M., Goldberg, D., Cantu-Paz, E.: BOA: The Bayesian optimization algorithm. In: Proceedings of the Genetic and Evolutionary Computation Conference GECCO 1999, vol. 1, pp. 525–532 (1999)
Pelikan, M., Goldberg, D.E., Lobo, F.G.: A survey of optimization by building and using probabilistic models. Computational Optimization and Applications 21(1), 5–20 (2002)
Pelikan, M., Sastry, K., Butz, M.V., Goldberg, D.E.: Hierarchical boa on random decomposable problems. Tech. Rep. 2006002, IlliGAL (2006)
Santana, R., Larrañaga, P., Lozano, J.A.: Challenges and open problems in discrete edas. Tech. Rep. EHU-KZAA-IK-1/07, Department of Computer Science and Artificial Intelligence, University of the Basque Country (2007), http://www.sc.ehu.es/ccwbayes/technical.htm
Smith, R.E.: An iterative mutual information histogram technique for linkage learning in evolutionary algorithms. In: Proceedings of CEC 2005, pp. 2166–2173 (2005)
Thierens, D., Goldberg, D., Pereira, A.: Domino convergence, drift, and the temporal-salience structure of problems. In: The IEEE International Conference on Evolutionary Computation Proceedings, IEEE World Congress on Computational Intelligence, 1998, pp. 535–540 (1998)
Tsuji, M., Munetomo, M., Akama, K.: Modeling dependencies of loci with string classification according to fitness differences. In: GECCO (2), pp. 246–257 (2004)
Wolpert, D.H., MacReady, W.G.: No free lunch theorems for optimization. IEEE Transactions on Evolutionary Computation, 67–82 (1997)
Wright, A., Poli, R., Stephens, C., Langdon, W., Pulavarty, S.: An Estimation of Distribution Algorithm based on maximum entropy. In: Deb, K., et al. (eds.) GECCO 2004. LNCS, vol. 3103, pp. 343–354. Springer, Heidelberg (2004)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Coffin, D., Smith, R.E. (2008). Linkage Learning in Estimation of Distribution Algorithms. In: Chen, Yp., Lim, MH. (eds) Linkage in Evolutionary Computation. Studies in Computational Intelligence, vol 157. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-85068-7_7
Download citation
DOI: https://doi.org/10.1007/978-3-540-85068-7_7
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-85067-0
Online ISBN: 978-3-540-85068-7
eBook Packages: EngineeringEngineering (R0)