Sara Silva
ABSTRACT
Operator equalisation was recently proposed as a new bloat control technique for genetic programming. By controlling the distribution of program lengths inside the population, it can bias the search towards smaller or larger programs. In this paper we propose a new implementation of operator equalisation and compare it to a previous version, using a hard real-world regression problem where bloat and overfitting are major issues. The results show that both implementations of operator equalisation are completely bloat-free, producing smaller individuals than standard genetic programming, without compromising the generalization ability. We also show that the new implementation of operator equalisation is more efficient and exhibits a more predictable and reliable behavior than the previous version. We advance some arguable ideas regarding the relationship between bloat and overfitting, and support them with our results.
- ]]F. Archetti, E. Messina, S. Lanzeni, and L. Vanneschi. Genetic programming for computational pharmacokinetics in drug discovery and development. Genetic Programming and Evolvable Machines, 8(4):17--26, 2007. Google Scholar
Digital Library
- ]]S. Dignum and R. Poli. Generalisation of the limiting distribution of program sizes in tree-based genetic programming and analysis of its effects on bloat. In D. Thierens, et al., editors, GECCO '07: Proceedings of the 9th annual conference on Genetic and evolutionary computation, volume 2, pages 1588--1595, London, 7-11 July 2007. ACM Press. Google ScholarDigital Library
- ]]S. Dignum and R. Poli. Crossover, sampling, bloat and the harmful effects of size limits. In M. O'Neill, et al., editors, Proceedings of the 11th European Conference on Genetic Programming, EuroGP 2008, volume 4971 of Lecture Notes in Computer Science, pages 158--169, Naples, 26-28 Mar. 2008. Springer. Google ScholarDigital Library
- ]]S. Dignum and R. Poli. Operator equalisation and bloat free GP. In M. O'Neill, et al., editors, Proceedings of the 11th European Conference on Genetic Programming, EuroGP 2008, volume 4971 of Lecture Notes in Computer Science, pages 110--121, Naples, 26--28 Mar. 2008. Springer. Google ScholarDigital Library
- ]]F. Archetti, S. Lanzeni, E. Messina and L. Vanneschi. Genetic programming for human oral bioavailability of drugs. In M. Cattolico, editor, Proceedings of the 8th annual conference on Genetic and Evolutionary Computation, pages 255--262, Seattle, Washington, USA, July 2006. Google ScholarDigital Library
- ]]F. Yoshida and J. G. Topliss. QSAR model for drug human oral bioavailability. Journal of Medicinal Chemistry, 43:2575--2585, 2000.Google ScholarCross Ref
- ]]H. Van de Waterbeemd and S. Rose. In The Practice of Medicinal Chemistry, 2nd edition. ed. Wermuth, L. G., 1367--1385,Academic Press, 2003.Google Scholar
- ]]I. Kola and J. Landis. Can the pharmaceutical industry reduce attrition rates? Nature Reviews Drug Discovery, 3:711--716, 2004.Google ScholarCross Ref
- ]]C. Igel and K. Chellapilla. Investigating the influence of depth and degree of genotypic change on fitness in genetic programming. In W. Banzhaf, et al., editors, Proceedings of the Genetic and Evolutionary Computation Conference, volume 2, pages 1061--1068, Orlando, Florida, USA, 13--17 July 1999. Morgan Kaufmann.Google Scholar
- ]]J. R. Koza. Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge, MA, USA, 1992. Google ScholarDigital Library
- ]]W. B. Langdon and R. Poli. Foundations of Genetic Programming. Springer, Berlin, Heidelberg, New York, Berlin, 2002. Google ScholarDigital Library
- ]]S. Luke. Modification point depth and genome growth in genetic programming. Evolutionary Computation, 11(1):67--106, Spring 2003. Google ScholarDigital Library
- ]]S. Luke and L. Panait. Lexicographic parsimony pressure. In W. B. Langdon, et al., editors, GECCO 2002: Proceedings of the Genetic and Evolutionary Computation Conference, pages 829--836, New York, 9-13 July 2002. Morgan Kaufmann Publishers.Google Scholar
- ]]P. Domingos. The role of Occam's razor in knowledge discovery. Data Mining and Knowledge Discovery, 3(4):409--425, 1999. Google ScholarDigital Library
- ]]R. Poli, W. B. Langdon, and S. Dignum. On the limiting distribution of program sizes in tree-based genetic programming. In M. Ebner, et al., editors, Proceedings of the 10th European Conference on Genetic Programming, volume 4445 of Lecture Notes in Computer Science, pages 193--204, Valencia, Spain, 11-13 Apr. 2007. Springer. Google ScholarDigital Library
- ]]R. Poli, N. F. McPhee, and L. Vanneschi. The impact of population size on code growth in GP: analysis and empirical validation. In M. Keijzer, et al., editors, GECCO '08: Proceedings of the 10th annual conference on Genetic and evolutionary computation, pages 1275--1282, Atlanta, GA, USA, 12-16 July 2008. ACM. Google ScholarDigital Library
- ]]R. Poli, W. B. Langdon, and N. F. McPhee. A field guide to genetic programming. Published via http://lulu.com and freely available at http://www.gp-field-guide.org.uk, 2008. (With contributions by J. R. Koza). Google ScholarDigital Library
- ]]R. Todeschini and V. Consonni. Handbook of Molecular Descriptors. Wiley-VCH, Weinheim, 2000.Google ScholarCross Ref
- ]]J. Rissanen. Modeling by shortest data description. Automatica, 14:465--471, 1978.Google ScholarDigital Library
- ]]J. Rosca. Generality versus size in genetic programming. In J. R. Koza, et al., editors, Genetic Programming 1996: Proceedings of the First Annual Conference, pages 381--387, Stanford University, CA, USA, 28-31 July 1996. MIT Press. Google ScholarDigital Library
- ]]S. David, Wishart, C. Knox, A. C. Guo, S. Shrivastava, M. Hassanali,P. Stothard, Z. Chang and J. Woolsey. DrugBank: a comprehensive resource for in silico drug discovery and exploration. Nucleic Acids Research, 34:doi:10.1093/nar/gkj067, 2006.Google Scholar
- ]]S. Silva. GPLAB -- a genetic programming toolbox for MATLAB, version 3.0, 2007. http://gplab.sourceforge.net.Google Scholar
- ]]S. Silva and J. Almeida. Dynamic maximum tree depth. In E. Cantú-Paz, et al., editors, Genetic and Evolutionary Computation -- GECCO--2003, volume 2724 of LNCS, pages 1776---1787, Chicago, 12-16 July 2003. Springer--Verlag. Google ScholarDigital Library
- ]]S. Silva and E. Costa. Dynamic limits for bloat control in genetic programming and a review of past and current bloat theories. Genetic Programming and Evolvable Machines, 10(2):141--179, 2009. Google ScholarDigital Library
- ]]S. Silva and S. Dignum. Extending operator equalisation: Fitness based self adaptive length distribution for bloat free GP. In L. Vanneschi, et al., editors, Proceedings of the 12th European Conference on Genetic Programming, EuroGP2009. Springer, 2009. To appear. Google ScholarDigital Library
- ]]Simulation Plus Inc. a company that use both statistical methods and differential equations based simulations for ADME parameter estimation., 2006. See www.simulationsplus.com.Google Scholar
- ]]T. Kennedy. Managing the drug discovery/development interface. Drug Discovery Today, 2:436--444, 1997.Google ScholarCross Ref
- ]]L. Vanneschi, M. Tomassini, P. Collard, and M. Clergue. Fitness distance correlation in structural mutation genetic programming. In C. Ryan, et al., editors, Genetic Programming, Proceedings of EuroGP'2003, volume 2610 of LNCS, pages 455--464, Essex, 14-16 Apr. 2003. Springer-Verlag. Google ScholarDigital Library
- ]]W. B. Langdon and S. J. Barrett. Genetic Programming in data mining for drug discovery. in Evolutionary computing in data mining, pages 211--235, 2004.Google Scholar
Recommendations
Operator equalisation for bloat free genetic programming and a survey of bloat control methods
Bloat can be defined as an excess of code growth without a corresponding improvement in fitness. This problem has been one of the most intensively studied subjects since the beginnings of Genetic Programming. This paper begins by briefly reviewing the ...
Reassembling operator equalisation: a secret revealed
GECCO '11: Proceedings of the 13th annual conference on Genetic and evolutionary computationThe recent Crossover Bias theory has shown that bloat in Genetic Programming can be caused by the proliferation of small unfit individuals in the population. Inspired by this theory, Operator Equalisation is the most recent and successful bloat control ...
Operator equalisation and bloat free GP
EuroGP'08: Proceedings of the 11th European conference on Genetic programmingResearch has shown that beyond a certain minimum program length the distributions of program functionality and fitness converge to a limit. Before that limit, however, there may be program-length classes with a higher or lower average fitness than that ...
Comments