Jean-Baptiste Mouret
ABSTRACT
Encouraging exploration, typically by preserving the diversity within the population, is one of the most common method to improve the behavior of evolutionary algorithms with deceptive fitness functions. Most of the published approaches to stimulate exploration rely on a distance between genotypes or phenotypes; however, such distances are difficult to compute when evolving neural networks due to (1) the algorithmic complexity of graph similarity measures, (2) the competing conventions problem and (3) the complexity of most neural-network encodings.
In this paper, we introduce and compare two conceptually simple, yet efficient methods to improve exploration and avoid premature convergence when evolving both the topology and the parameters of neural networks. The two proposed methods, respectively called behavioral novelty and behavioral diversity, are built on multiobjective evolutionary algorithms and on a user-defined distance between behaviors. They can be employed with any genotype. We benchmarked them on the evolution of a neural network to compute a Boolean function with a deceptive fitness. The results obtained with the two proposed methods are statistically similar to those of NEAT and substantially better than those of the control experiment and of a phenotype-based diversity mechanism.
- ]]H. A. Abbass and K. Deb. Searching under multi-evolutionary pressures. Proceedings of the Fourth Conference on Evolutionary Multi-Criterion Optimization, 2003. Google Scholar
Digital Library
- ]]L. Bui, H. A. Abbass, and J. Branke. Multiobjective optimization for dynamic environments. The 2005 IEEE Congress on Evolutionary Computation (CEC), 3:2349--2356 Vol. 3, 2005.Google Scholar
- ]]H. Bunke. Recent developments in graph matching. Proc. 15th International Conference on Pattern Recognition, pages 117--124, 2000.Google ScholarCross Ref
- ]]E. Burke, S. Gustafson, and G. Kendall. A survey and analysis of diversity measures in genetic programming. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO'02), pages 716--723, 2002.Google Scholar
- ]]C. A. Coello Coello and G. B. Lamont. Applications Of Multi-Objective Evolutionary Algorithms. World Scientific, 2004.Google Scholar
- ]]E. D. de Jong, R. A. Watson, and J. B. Pollack. Reducing bloat and promoting diversity using multi-objective methods. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO'01), pages 11--18, 2001.Google Scholar
- ]]K. Deb. Multi-objectives optimization using evolutionnary algorithms. Wiley, 2001. Google ScholarDigital Library
- ]]K. Deb, S. Agrawal, A. Pratab, and T. Meyarivan. A Fast Elitist Non-Dominated Sorting Genetic Algorithm for Multi-Objective Optimization: NSGA-II. In Proceedings of the Parallel Problem Solving from Nature VI Conference, pages 849--858, 2000. Google ScholarDigital Library
- ]]K. Deb, M. Mohan, and S. Mishra. Evaluating the μ-Domination Based Multi-Objective Evolutionary Algorithm for a Quick Computation of Pareto--Optimal Solutions. Evolutionary Computation, 13(4):501--525, 2005. Google ScholarDigital Library
- ]]S. Doncieux and J.-A. Meyer. Evolving PID-like neurocontrollers for non-linear control problems. International Journal of Control and Intelligent Systems (IJCIS). Special Issue on nonlinear adaptive PID control, 33(1):55--62, 2005.Google Scholar
- ]]C. M. Fonseca and P. J. Fleming. Genetic algorithms for multiobjective optimization: formulation, discussion and generalization. In Proceedings of the Fourth International Conference on Evolutionary Programming, pages 416--423, 1993. Google ScholarDigital Library
- ]]D. E. Goldberg. Simple genetic algorithms and the minimal, deceptive problem. Genetic Algorithms and Simulated Annealing, 74, 1987.Google Scholar
- ]]D. E. Goldberg, K. Deb, and J. Horn. Massive multimodality, deception, and genetic algorithms. In Proceedings of the Second Conference on Parallel Problem Solving from Nature, pages 37--48, 1992.Google Scholar
- ]]D. E. Goldberg and J. Richardson. Genetic algorithms with sharing for multimodal function optimization. In Proceedings of the Second International Conference on Genetic Algorithms, pages 148--154, 1987. Google ScholarDigital Library
- ]]F. Gruau. Automatic definition of modular neural networks. Adaptive Behaviour, 3(2):151--183, 1995. Google ScholarDigital Library
- ]]J. Handl, S. C. Lovell, and J. Knowles. Multiobjectivization by decomposition of scalar cost functions. In Proceedings of the 10th international conference on Parallel Problem Solving from Nature: PPSN X, pages 31--40. Springer, 2008.Google ScholarCross Ref
- ]]J. H. Holland. Adaptation in Natural and Artificial Systems. MI: University of Michigan Press, 1975. Google ScholarDigital Library
- ]]N. Kashtan and U. Alon. Spontaneous evolution of modularity and network motifs. Proceedings of the National Academy of Sciences, 102(39):13773--13778, 2005.Google ScholarCross Ref
- ]]J. D. Knowles, R. A. Watson, and D. W. Corne. Reducing Local Optima in Single-Objective Problems by Multi-objectivization. First International Conference on Evolutionary Multi-Criterion Optimization, 1993:268--282, 2001. Google ScholarDigital Library
- ]]J. Lehman and K. Stanley. Exploiting open-endedness to solve problems through the search for novelty. In Artificial Life XI: Proceedings of the Eleventh International Conference on the Simulation and Synthesis of Living Systems, pages 329--336, 2008.Google Scholar
- ]]D. Lopresti and G. Wilfong. A fast technique for comparing graph representations with applications to performance evaluation. International Journal on Document Analysis and Recognition, 6(4):219--229, 2003. Google ScholarDigital Library
- ]]J.-B. Mouret and S. Doncieux. Mennag: a modular, regular and hierarchical encoding for neural-networks based on attribute grammars. Evolutionary Intelligence, 1:187--207, 2008.Google ScholarCross Ref
- ]]J.-B. Mouret and S. Doncieux. Overcoming the bootstrap problem in evolutionary robotics using behavioral diversity. In IEEE Congress Evolutionary Computation (CEC)., page to appear, 2009. Google ScholarDigital Library
- ]]K. Praditwong and X. Yao. How well do multi-objective evolutionary algorithms scale to large problems. In IEEE Congress on Evolutionary Computation (CEC)., pages 3959--3966, 2007.Google ScholarCross Ref
- ]]R. C. Purshouse and P. J. Fleming. On the Evolutionary Optimization of Many Conflicting Objectives. IEEE Transactions on Evolutionary Computation, 11(6):770--784, 2007. Google ScholarDigital Library
- ]]R. Reveaux, B. Eugen, H. Locteau, S. Adam, P. Heroux, and E. Trupin. A Graph Classification Approach Using a Multi-objective Genetic Algorithm Application to Symbol Recognition. In IAPR GBR Workshop on Graph-based Representations in Pattern Recognition, volume 4538, page 361, 2007. Google ScholarDigital Library
- ]]K. O. Stanley and R. Miikkulainen. Evolving neural networks through augmenting topologies. Evolutionary Computation, 10(2)(99--127), 2002. Google ScholarDigital Library
- ]]A. Toolo and E. Benini. Genetic diversity as an objective in multi-objective evolutionary algorithms. Evolutionary Computation, 11(2):151--167, 2003. Google ScholarDigital Library
- ]]X. Yao. Evolving artificial neural networks. Proceedings of the IEEE, 87(9):1423--1447, 1999.Google ScholarCross Ref
Index Terms
- Using behavioral exploration objectives to solve deceptive problems in neuro-evolution
Recommendations
An analysis of multi-chromosome GAs in deceptive problems
GECCO '11: Proceedings of the 13th annual conference on Genetic and evolutionary computationThis paper discusses a new approach to using GAs to solve deceptive fitness landscapes by incorporating mechanisms to control the convergence direction instead of simply increasing the population diversity. In order to overcome some of the difficulties ...
Constrained differential evolution with multiobjective sorting mutation operators for constrained optimization
The proposed constrained differential evolution framework uses nondominated sorting mutation operator based on fitness and diversity information for constrained optimization. This study proposes a new constraint differential evolution framework.Parents ...
Why and how to measure exploration in behavioral space
GECCO '11: Proceedings of the 13th annual conference on Genetic and evolutionary computationExploration and exploitation are two complementary aspects of Evolutionary Algorithms. Exploration, in particular, is promoted by specific diversity keeping mechanisms generally relying on the genotype or the fitness value. Recent works suggest that, in ...
Comments