ABSTRACT
The effectiveness of evolutionary test case generation based on Genetic Algorithms (GAs) can be seriously impacted by genetic drift, a phenomenon that inhibits the ability of such algorithms to effectively diversify the search and look for alternative potential solutions. In such cases, the search becomes dominated by a small set of similar individuals that lead GAs to converge to a sub-optimal solution and to stagnate, without reaching the desired objective. This problem is particularly common for hard-to-cover program branches, associated with an extremely large solution space. In this paper, we propose an approach to solve this problem by integrating a mechanism for orthogonal exploration of the search space into standard GA. The diversity in the population is enriched by adding individuals in orthogonal directions, hence providing a more effective exploration of the solution space. To the best of our knowledge, no prior work has addressed explicitly the issue of evolution direction based diversification in the context of evolutionary testing. Results achieved on 17 Java classes indicate that the proposed enhancements make GA much more effective and efficient in automating the testing process. In particular, effectiveness (coverage) was significantly improved in 47% of the subjects and efficiency (search budget consumed) was improved in 85% of the subjects on which effectiveness remains the same.
- M. Alshraideh, L. Bottaci, and B. A. Mahafzah. Using program data-state scarcity to guide automatic test data generation. Software Quality Journal, 18(1):109–144, 2010. Google ScholarDigital Library
- A. Baresel, D. Binkley, M. Harman, and B. Korel. Evolutionary testing in the presence of loop-assigned flags: a testability transformation approach. In Proceedings of the 2004 ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA ’04, pages 108–118, Boston, USA, 2004. Google ScholarDigital Library
- R. Battiti and G. Tecchiolli. The reactive tabu search. ORSA Journal on Computing, 6(2):126–140, 1994.Google ScholarCross Ref
- H. G. Cobb and J. J. Grefenstette. Genetic algorithms for tracking changing environments. In Proceedings of the 5th International Conference on Genetic Algorithms, pages 523–530, San Francisco, USA, 1993. Google ScholarDigital Library
- J. Cohen. Statistical power analysis for the behavioral sciences. Lawrence Earlbaum Associates, 2nd edition edition, 1988.Google Scholar
- W. J. Conover. Practical Nonparametric Statistics. Wiley, 3rd edition edition, 1998.Google Scholar
- K. A. De Jong. An analysis of the behavior of a class of genetic adaptive systems. PhD thesis, University of Michigan, 1975. Google ScholarDigital Library
- R. Feldt, R. Torkar, T. Gorschek, and W. Afzal. Searching for cognitively diverse tests: Towards universal test diversity metrics. In Software Testing Verification and Validation Workshop, 2008. ICSTW’08. IEEE International Conference on, pages 178–186. IEEE, 2008. Google ScholarDigital Library
- G. Fraser and A. Arcuri. Evosuite: automatic test suite generation for object-oriented software. In Proceedings of the 19th ACM SIGSOFT Symposium and the 13th European Conference on Foundations of Software Engineering, ESEC/FSE ’11, pages 416–419, Szeged, Hungary, 2011. Google ScholarDigital Library
- G. Fraser and A. Arcuri. Whole test suite generation. IEEE Transactions on Software Engineering, 39(2), 2013. Google ScholarDigital Library
- D. Goldberg. Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley Professional, 1989. Google ScholarDigital Library
- D. E. Goldberg and J. Richardson. Genetic algorithms with sharing for multimodal function optimization. In Proceedings of the 2nd International Conference on Genetic algorithms and their application, pages 41–49, Cambridge, USA, 1987. Google ScholarDigital Library
- L. Hansheng and K. Lishan. Balance between exploration and exploitation in genetic search. Wuhan University Journal of Natural Sciences, 4(1):28–32, 1999.Google ScholarCross Ref
- M. Harman. The current state and future of search based software engineering. In 2007 Future of Software Engineering, FOSE ’07, pages 342–357, Washington, DC, USA, 2007. Google ScholarDigital Library
- J. H. Holland. Adaptation in Natural and Artificial Systems. University of Michigan Press, 1975.Google ScholarDigital Library
- S. Holm. A simple sequentially rejective Bonferroni test procedure. Scandinavian Journal on Statistics, 6:65–70, 1979.Google Scholar
- S. G. E. Hyunsook Do and G. Rothermel. Supporting controlled experimentation with testing techniques: An infrastructure and its potential impact. Empirical Software Engineering: An International Journal, 10:405–435, 2005. Google ScholarDigital Library
- Y. Jia and M. Harman. An analysis and survey of the development of mutation testing. IEEE Transactions on Software Engineering, 37(5), 2011. Google ScholarDigital Library
- A. D. Lucia, M. D. Penta, R. Oliveto, and A. Panichella. Estimating the evolution direction of populations to improve genetic algorithms. In Proceedings of the Genetic and Evolutionary Computation Conference, GECCO ’12, pages 617–624, Philadelphia, USA, 2012. Google ScholarDigital Library
- S. W. Mahfoud. Niching methods for genetic algorithms. Technical report, Illinois Genetic Algorithms Laboratory, 1995.Google Scholar
- P. McMinn. Search-based software test data generation: a survey. Software Testing, Verification and Reliability, 14:105–156, 2004. Google ScholarDigital Library
- C. Michael, G. McGraw, and M. A. Schatz. Generating software test data by evolution. IEEE Transactions on Software Engineering, 27(12), 2001. Google ScholarDigital Library
- A. Rogers and A. Prugel-Bennett. Genetic drift in genetic algorithm selection schemes. IEEE Transactions on Evolutionary Computation, 3(4):298 –303, 1999. Google ScholarDigital Library
- S. N. Sivanandam and S. N. Deepa. Introduction to Genetic Algorithms. Springer Publishing Company, Incorporated, 1st edition, 2007. Google ScholarDigital Library
- G. Strang. Introduction to Linear Algebra. Wellesley-Cambridge Press and SIAM, 4th edition, 2009.Google Scholar
- P. Tonella. Evolutionary testing of classes. In Proceedings of the 2004 ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA ’04, pages 119–128, Boston, USA, 2004. Google ScholarDigital Library
- J. Wegener, A. Baresel, and H. Sthamer. Evolutionary test environment for automatic structural testing. Information and Software Technology, 43(14):841–854, 2001.Google ScholarCross Ref
- D. Whitley. The genitor algorithm and selection pressure: why rank-based allocation of reproductive trials is best. In Proceedings of the 3rd International Conference on Genetic algorithms, pages 116–121, San Francisco, USA, 1989. Google ScholarDigital Library
Index Terms
- Orthogonal exploration of the search space in evolutionary test case generation
Recommendations
Random or Genetic Algorithm Search for Object-Oriented Test Suite Generation?
GECCO '15: Proceedings of the 2015 Annual Conference on Genetic and Evolutionary ComputationAchieving high structural coverage is an important aim in software testing. Several search-based techniques have proved successful at automatically generating tests that achieve high coverage. However, despite the well- established arguments behind ...
Intelligent exploration for genetic algorithms: using self-organizing maps in evolutionary computation
GECCO '05: Proceedings of the 7th annual conference on Genetic and evolutionary computationExploration vs. exploitation is a well known issue in Evolutionary Algorithms. Accordingly, an unbalanced search can lead to premature convergence. GASOM, a novel Genetic Algorithm, addresses this problem by intelligent exploration techniques. The ...
Parallel evolutionary test case generation for web applications
Abstract Context:Web applications follow a client–server schema, so it is more appropriate for evolutionary test case generation considering both client and server. However, test cases from the client-side are composed of event ...
Highlights- We first propose the parallel evolutionary test case generation for web applications.
Comments