Skip to main content
SpringerLink
Log in
Book cover

Scalable Optimization via Probabilistic Modeling pp 121–160Cite as

A Survey of Probabilistic Model Building Genetic Programming

  • Chapter

Part of the book series: Studies in Computational Intelligence ((SCI,volume 33))

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. H. A. Abbass, N. X. Hoai, and R. I. McKay. AntTAG: A new method to compose computer programs using colonies of ants. In The IEEE Congress on Evolutionary Computation, pages 1654-1659, 2002.

    Google Scholar 

  2. P. J. Angeline. Genetic programming and emergent intelligence. In K. E. Kinnear, Jr., editor, Advances in Genetic Programming, chapter 4, pages 75-98. MIT, Cambridge, MA, 1994.

    Google Scholar 

  3. P. J. Angeline. Two self adaptive crossover operations for genetic programming. In Advances in Genetic Programming II, pages 89-110. MIT Press, Cambridge, MA, 1995.

    Google Scholar 

  4. P. J. Angeline and J. B. Pollack. Coevolving high-level representations. In C. G. Langton, editor, Artificial Life III, volume XVII, pages 55-71, Santa Fe, New Mexico. Addison-Wesley, Reading, MA, 1994.

    Google Scholar 

  5. D. Angluin. Negative results for equivalence queries. Mach. Learn. 5 (2):121-150, 1990.

    Google Scholar 

  6. S. Baluja. Population-based incremental learning: A method for in-tegrating genetic search based function optimization and competitive learning. Technical Report CMU-CS-94-163, Carnegie Mellon University, Pittsburgh, PA, 1994.

    Google Scholar 

  7. S. Baluja. Using a priori knowledge to create probabilistic models for optimization. Int. J. Approx. Reason., 31(3):193-220, 2002.

    Article  MATH  MathSciNet  Google Scholar 

  8. W. Banzhaf, P. Nordin, R. E. Keller, and F. D. Francone. Genetic Pro-gramming - An Introduction; On the Automatic Evolution of Computer Programs and its Applications. Morgan Kaufmann, dpunkt.verlag, 1998.

    Google Scholar 

  9. E. Bengoetxea, P. Larrañaga, I. Bloch, A. Perchant, and C. Boeres. Inexact graph matching by means of estimation of distribution algo-rithms. Pattern Recognit., 35(12):2867-2880, 2002.

    Article  MATH  Google Scholar 

  10. R. Blanco, I. Inza, and P. Larrañaga. Learning Bayesian networks in the space of structures by estimation of distribution algorithms. Int. J. Intell. Syst. 18(2):205-220, 2003.

    Article  MATH  Google Scholar 

  11. E. Bonabeau, M. Dorigo, and T. Theraulaz. From Natural to Artificial Swarm Intelligence. Oxford University Press, New York, 1999.

    MATH  Google Scholar 

  12. M. Boryczka and Z. J. Czech. Solving approximation problems by ant colony programming. In W. B. Langdon, E. Cantú-Paz, K. Mathias, R. Roy, D. Davis, R. Poli, K. Balakrishnan, V. Honavar, G. Rudolph, J. Wegener, L. Bull, M. A. Potter, A. C. Schultz, J. F. Miller, E. Burke, and N. Jonoska, editors, GECCO 2002: Proceedings of the Genetic and Evolutionary Computation Conference, page 133, New York. Morgan Kaufmann, Los Altos, CA, 2002.

    Google Scholar 

  13. P. Bosman and D. Thierens. An algorithmic framework for density estimation based evolutionary algorithms. Technical Report Technical Report UU-CS-1999-46, Utrecht University, 1999.

    Google Scholar 

  14. P. A. N. Bosman and E. D. de Jong. Grammar transformations in an eda for genetic programming. In Special session: OBUPM - Optimization by Building and Using Probabilistic Models, GECCO, Seattle, Washington, USA, 2004.

    Google Scholar 

  15. S. F. Chen. Bayesian grammar induction for language modeling. In Meeting of the Association for Computational Linguistics, pages 228-235, 1995.

    Google Scholar 

  16. S. F. Chen. Buidling Probabilistic Models for Natural Language. PhD thesis, Harvard University Press, Cambridge, MA, USA, 1996.

    Google Scholar 

  17. N. Cramer. A representation for the adaptive generation of simple sequential programs. In Proceedings of an International Conference on Genetic Algorithms and their Applications, pages 183-187, CarnegieMellon University, Pittsburgh, PA, USA, 1985.

    Google Scholar 

  18. N. L. Cramer. A representation for the adaptive generation of simple sequential programs. In J. J. Grefenstette, editor, Proceedings of an Inter-national Conference on Genetic Algorithms and the Applications, pages 183-187, Carnegie-Mellon University, Pittsburgh, PA, USA, 1985.

    Google Scholar 

  19. A. S. d’Avila Garcez, K. Broda, and D. M. Gabbay. Symbolic knowledge extraction from trained neural networks: a sound approach. Artif. Intell. 125 (1-2):155-207, 2001.

    Article  MATH  MathSciNet  Google Scholar 

  20. P. D’haeseleer. Context preserving crossover in genetic programming. In Proceedings of the 1994 IEEE World Congress on Computational Intelligence, volume 1, pages 256-261, Orlando, Florida, USA. IEEE, New York, 1994.

    Chapter  Google Scholar 

  21. D. E. Goldberg. Genetic algorithms in search, optimization, and machine learning. Addison-Wesley, Reading, MA, 1989.

    MATH  Google Scholar 

  22. J. Green, J. L. Whalley, and C. G. Johnson. Automatic programming with ant colony optimization. In M. Withall and C. Hinde, editors, Proceedings of the 2004 UK Workshop on Computational Intelligence, pages 70-77. Loughborough University, 2004.

    Google Scholar 

  23. F. Gruau. On using syntactic constraints with genetic programming. In P. J. Angeline and K. E. Kinnear, Jr., editors, Advances in Genetic Programming 2, chapter 19, pages 377-394. MIT, Cambridge, MA, USA, 1996.

    Google Scholar 

  24. P. Grunwald. A minimum description length approach to grammar inference. In Connectionist, Statistical and Symbolic Approaches to Learning for Natural Language, volume 1004 of Lecture Notes in AI, pages 203-216. Springer, Berlin Heidelberg New York, 1994.

    Google Scholar 

  25. P. Haddawy. Generating Bayesian networks from probability logic knowledge bases. In Proceedings of the Tenth Conference on Uncertainty in Articial Intelligence, 1994.

    Google Scholar 

  26. G. Harik. Linkage learning via probabilistic modeling in the ECGA. Technical Report IlliGAL Report No. 99010, University of Illinois at UrbanaChampaign, 1999.

    Google Scholar 

  27. G. R. Harik, F. G. Lobo, and D. E. Goldberg. The compact genetic algorithm. IEEE Trans. Evol. Comput. 3(4):287-297, 1999.

    Article  Google Scholar 

  28. J. H. Holland. Adaptation in Natural and Artificial Systems. University of Michigan Press, 1975.

    Google Scholar 

  29. H. Iba and H. de Garis. Extending genetic programming with recombinative guidance. In Advances in Genetic Programming II, pages 69-88. MIT, Cambridge, MA, 1996.

    Google Scholar 

  30. A. K. Joshi, L. S. Levy, and M. Takahashi. Tree adjunct grammars. J. Comput. Syst. Sci. 10(1):136-163, 1975.

    Article  MATH  MathSciNet  Google Scholar 

  31. C. Keber and M. G. Schuster. Option valuation with generalized ant programming. In Proceedings of the Genetic and Evolutionary Computation Conference, pages 74-81. Morgan Kaufmann, Los Altos, CA, 2002.

    Google Scholar 

  32. B. Keller and R. Lutz. Evolving stochastic context-free grammars from examples using a minimum description length principle. In Proceedings of Workshop on Automata Induction Grammatical Inference and Language Acquisition, ICML-97, Nashville, Tennessee, 1997.

    Google Scholar 

  33. J. R. Koza. Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT, Cambridge, MA, USA, 1992.

    Google Scholar 

  34. W. B. Langdon and R. Poli. Foundations of Genetic Programming. Springer, Berlin Heidelberg New York, 2002.

    MATH  Google Scholar 

  35. K. Lari and S. J. Young. The estimation of stochastic context-free grammars using the inside-outside algorithm. Comput. Speech Lang. 4:35-56, 1990.

    Article  Google Scholar 

  36. K. Lari and S. J. Young. Applications of stochastic context-free grammars using the inside-outside algorithm. Comput. Speech Lang. 5: 237-257, 1991.

    Article  Google Scholar 

  37. P. Larrañaga and J. A. Lozano. Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation. Kluwer, Dordrecht, 2001.

    Google Scholar 

  38. P. P. Le, A. Bah, and L. H. Ungar. Using prior knowledge to improve genetic network reconstruction from microarray data. Silico Biol. 4(27), 2004.

    Google Scholar 

  39. C. Manning and H. Schütze. Foundations of Statistical Natural Language Processing. MIT, Cambridge, MA, 1999.

    MATH  Google Scholar 

  40. R. S. Michalski. Learnable evolution model: Evolutionary processes guided by machine learning. Mach. Learn. 38:9-40, 2000.

    Article  MATH  Google Scholar 

  41. D. J. Montana. Strongly typed genetic programming. BBN Technical Report #7866, Bolt Beranek and Newman, Inc., 10 Moulton Street, Cambridge, MA 02138, USA, 1993.

    Google Scholar 

  42. H. Müehlenbein and G. Paaß. From recombination of genes to the estimation of distributions i.binary parameters. In Lecture Notes in Computer Science 1411: Parallel Problem Solving from Nature, PPSN IV, pages 178-187. Springer, Berlin Heidelberg New York, 1996.

    Chapter  Google Scholar 

  43. H. Mühlenbein and T. Mahnig. The factorized distribution algorithm for additively decompressed functions. In 1999 Congress on Evolutionary Computation, pages 752-759, Piscataway, NJ. IEEE Service Center, 1999.

    Google Scholar 

  44. P. Nordin. A compiling genetic programming system that directly manipulates the machine code. In K. E. KinnearJr, ., editor, Advances in Genetic Programming, chapter 14, pages 311-331. MIT, Cambridge, MA, 1994.

    Google Scholar 

  45. M. O’Neill and C. Ryan. Grammatical evolution. IEEE Trans. Evol. Comput. 5(4):349-358, 2001.

    Article  Google Scholar 

  46. U.-M. O’Reilly and F. Oppacher. The troubling aspects of a building block hypothesis for genetic programming. In L. D. Whitley and M. D. Vose, editors, Foundations of Genetic Algorithms 3, pages 73-88, Estes Park, Colorado, USA. Morgan Kaufmann, Los Altos, CA, 1995.

    Google Scholar 

  47. M. Osborne. DCG induction using MDL and parsed corpora. In J. Cussens, editor, Proceedings of the 1st Workshop on Learning Language in Logic, pages 63-71, Bled, Slovenia, 1999.

    Google Scholar 

  48. T. K. Paul and H. Iba. Identification of informative genes for molecular classification using probabilistic model building genetic algorithm. In e. a. Kalyanmoy Deb, editor, Proceedings of Genetic and Evolutionary Computation (GECCO 2004) (Lecture Notes in Computer Science), volume 3102, pages 414-425, Seattle, USA, 2004.

    Google Scholar 

  49. M. Pelikan. Bayesian optimization algorithm: From single level to hierarchy. PhD thesis, University of Illinois at Urbana-Champaign, Urbana, IL, 2002. Also IlliGAL Report No. 2002023.

    Google Scholar 

  50. M. Pelikan, D. E. Goldberg, J. Ocenasek, and S. Trebst. Robust and scalable black-box optimization, hierarchy, and Ising spin glasses. IlliGAL Report No. 2003019, Illinois Genetic Algorithms Laboratory, University of Illinois at Urbana-Champaign, Urbana, IL, 2003.

    Google Scholar 

  51. F. Pereira and Y. Schabes. Inside-outside reestimation from partially bracketed corpora. In Proceedings of the 30th conference on Association for Computational Linguistics, pages 128-135. Association for Computational Linguistics, 1992.

    Google Scholar 

  52. P. Prusinkiewicz and A. Lindenmayer. The Algorithmic Beauty of Plants. Springer, Berlin Heidelberg New York, 1990.

    MATH  Google Scholar 

  53. A. Ratle and M. Sebag. Avoiding the bloat with probabilistic grammarguided genetic programming. In P. Collet, C. Fonlupt, J.-K. Hao, E. Lutton, and M. Schoenauer, editors, Artificial Evolution 5th International Conference, Evolution Artificielle, EA 2001, volume 2310 of LNCS, pages 255-266, Creusot, France. Springer, Berlin Heidelberg New York, 2001.

    Google Scholar 

  54. J. Rissanen. Stochastic Complexity in Statistical Inquiry. World Scientific, Singapore, 1989.

    MATH  Google Scholar 

  55. S. A. Rojas and P. J. Bentley. A grid-based ant colony system for automatic program synthesis. In M. Keijzer, editor, Late Breaking Papers at the 2004 Genetic and Evolutionary Computation Conference, Seattle, Washington, USA, 2004.

    Google Scholar 

  56. J. P. Rosca. Analysis of complexity drift in genetic programming. In J. R. Koza, K. Deb, M. Dorigo, D. B. Fogel, M. Garzon, H. Iba, and R. L. Riolo, editors, Genetic Programming 1997: Proceedings of the Second Annual Conference, pages 286-294, Stanford University, CA, USA. Morgan Kaufmann, Los Altos, CA, 1997.

    Google Scholar 

  57. J. P. Rosca and D. H. Ballard. Genetic programming with adaptive representations. Technical Report TR 489, University of Rochester, Computer Science Department, Rochester, NY, USA, 1994.

    Google Scholar 

  58. O. Roux and C. Fonlupt. Ant programming: or how to ants for automatic programming. In Proceedings of the Second International Conference on Ant Algorithms (ANTS2000), Belgium, 2000.

    Google Scholar 

  59. R. Sagarna and J. Lozano. On the performance of estimation of distribution algorithms applied to software testing. Appl. Artif. Intell. 19(5): 457-489, 2004.

    Article  Google Scholar 

  60. Y. Sakakibara. Learning contextfree grammars from structural data in polynomial time. Theor. Comput. Sci. 76:223-242, 1990.

    Article  MATH  MathSciNet  Google Scholar 

  61. Y. Sakakibara. Efficient learning of context-free grammars from positive structural examples. Inf. Comput. 97(1):23-60, 1992.

    Article  MATH  MathSciNet  Google Scholar 

  62. Y. Sakakibara. Recent advances of grammatical inference. Theor.Comput. Sci. 185(1):15-45, 1997.

    Article  MATH  MathSciNet  Google Scholar 

  63. R. P. Salustowicz and J. Schmidhuber. Probabilistic incremental program evolution. Evol. Comput. 5(2):123-141, 1997.

    Article  Google Scholar 

  64. K. Sastry and D. E. Goldberg. Probabilistic model building and competent genetic programming. In R. L. Riolo and B. Worzel, editors, Genetic Programming Theory and Practise, chapter 13, pages 205-220. Kluwer, Dordecht, 2003.

    Google Scholar 

  65. J. Schmidhuber. Evolutionary principles in self-referential learning. on learning now to learn: The meta-meta-meta…-hook. Diploma thesis, Technische Universitat Munchen, Germany, 1987.

    Google Scholar 

  66. Y. Shan. Program Distribution Estimation with Grammar Models. PhD thesis, University of New South Wales, Australia, 2005.

    Google Scholar 

  67. Y. Shan, R. McKay, R. Baxter, H. Abbass, D. Essam, and H. Nguyen. Grammar model-based program evolution. In Proceedings of The Congress on Evolutionary Computation, Portland, USA. IEEE, New York, 2004.

    Google Scholar 

  68. Y. Shan, R. I. McKay, H. A. Abbass, and D. Essam. Program evolution with explicit learning: a new framework for program automatic synthesis. In Proceedings of 2003 Congress on Evolutionary Computation, Canberra, Australia. University College, University of New South Wales, Australia, 2003.

    Google Scholar 

  69. W. M. Spears, K. A. D. Jong, T. Bäck, D. B. Fogel, and H. de Garis. An overview of evolutionary computation. In P. Brazdil, editor, Machine Learning: ECML-93, European Conference on Machine Learning, pages 442-459, Vienna, Austria. Springer, Berlin Heidelberg New York, 1993.

    Google Scholar 

  70. A. Stolcke. Bayesian Learning of Probabilistic Language Models. PhD thesis, University of California, Berkeley, CA, 1994.

    Google Scholar 

  71. I. Tanev. Implications of incorporating learning probabilistic contextsensitive grammar in genetic programming on evolvability of adaptive locomotion gaits of snakebot. In Proceedings of GECCO 2004, Seattle, Washington, USA, 2004.

    Google Scholar 

  72. A. Teller and M. Veloso. PADO: A new learning architecture for object recognition. In K. Ikeuchi and M. Veloso, editors, Symbolic Visual Learning, pages 81-116. Oxford University Press, Oxford, 1996.

    Google Scholar 

  73. G. G. Towell and J. W. Shavlik. Extracting refined rules from knowledge-based neural networks. Mach. Learn. 13(1):71-101, 1993.

    Google Scholar 

  74. C. S. Wallace and D. M. Boulton. An information measure for classification. Comput. J. 11(2):185-194, 1968.

    MATH  Google Scholar 

  75. C. S. Wallace and D. L. Dowe. Minimum message length and kolmogorov complexity. Comput. J. 42(4):270-283, 1999.

    Article  MATH  Google Scholar 

  76. C. S. Wallace and P. R. Freeman. Estimation and inference by compact coding. J. R. Statist. Soc. Ser. B (Methodological), 49(3):240-265, 1987.

    MATH  MathSciNet  Google Scholar 

  77. P. Whigham. Grammatically-based genetic programming. In J. P. Rosca, editor, Proceedings of the Workshop on Genetic Programming: From Theory to Real-World Applications, pages 33-41, Tahoe City, California, USA, 1995.

    Google Scholar 

  78. P. Whigham. Inductive bias and genetic programming. In Proceedings of First International Conference on Genetic Algorithms in Engineer-ing Systems: Innovations and Applications, pages 461-466. IEE, London, 1995.

    Chapter  Google Scholar 

  79. P. Whigham. A schema theorem for context-free grammars. In 1995 IEEE Conference on Evolutionary Computation, volume 1, pages 178-181, Perth, Australia. IEEE, New York, 1995.

    Chapter  Google Scholar 

  80. P. Whigham and F. Recknagel. Predicting chlorophyll-a in freshwater lakes by hybridising process-based models and genetic algorithms. Ecol. Modell. 146(1-3):243-252, 2001.

    Article  Google Scholar 

  81. M. L. Wong and K. S. Leung. Genetic logic programming and applications. IEEE Expert, 10(5):68-76, 1995.

    Article  Google Scholar 

  82. K. Yanai and H. Iba. Estimation of distribution programming based on Bayesian network. In Proceedings of Congress on Evolutionary Computation, pages 1618-1625, Canberra, Australia, 2003.

    Chapter  Google Scholar 

  83. M. Zlochin, M. Birattari, N. Meuleau, and M. Dorigo. Model-based search for combinatorial optimization: A critical survey. Ann. Oper. Res. 131:373-395, 2004.

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of New South Wales University College, ADFA Canberra, ACT 2600, Australia

    Yin Shan

  2. University of New South Wales University College, ADFA Canberra, ACT 2600, Australia

    Robert I. McKay

  3. University of New South Wales University College, ADFA Canberra, ACT 2600, Australia

    Daryl Essam

  4. University of New South Wales University College, ADFA Canberra, ACT 2600, Australia

    Hussein A. Abbass

Authors
  1. Yin Shan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Robert I. McKay
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daryl Essam
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hussein A. Abbass
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Math. And Computer Science, University of Missouri at St. Louis, One University Blvd., 320 CCB, MO 63121, St. Louis, USA

    Martin Pelikan

  2. Department of Industrial and Enterprise Systems Engineering, Illinois Genetic Algorithms Laboratory, 414 TB, 104 S. Mathews Ave., IL 61801, Urbana, USA

    Kumara Sastry

  3. Yahoo! Inc., 701 First Avenue, CA 94809, Sunnyvale, USA

    Erick CantúPaz

Rights and permissions

Reprints and permissions

Copyright information

© 2006 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Shan, Y., McKay, R.I., Essam, D., Abbass, H.A. (2006). A Survey of Probabilistic Model Building Genetic Programming. In: Pelikan, M., Sastry, K., CantúPaz, E. (eds) Scalable Optimization via Probabilistic Modeling. Studies in Computational Intelligence, vol 33. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-34954-9_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-34954-9_6

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-34953-2

  • Online ISBN: 978-3-540-34954-9

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation