skip to main content
10.1145/2483760.2483789acmconferencesArticle/Chapter ViewAbstractPublication PagesisstaConference Proceedingsconference-collections
research-article

Orthogonal exploration of the search space in evolutionary test case generation

Published:15 July 2013Publication History

ABSTRACT

The effectiveness of evolutionary test case generation based on Genetic Algorithms (GAs) can be seriously impacted by genetic drift, a phenomenon that inhibits the ability of such algorithms to effectively diversify the search and look for alternative potential solutions. In such cases, the search becomes dominated by a small set of similar individuals that lead GAs to converge to a sub-optimal solution and to stagnate, without reaching the desired objective. This problem is particularly common for hard-to-cover program branches, associated with an extremely large solution space. In this paper, we propose an approach to solve this problem by integrating a mechanism for orthogonal exploration of the search space into standard GA. The diversity in the population is enriched by adding individuals in orthogonal directions, hence providing a more effective exploration of the solution space. To the best of our knowledge, no prior work has addressed explicitly the issue of evolution direction based diversification in the context of evolutionary testing. Results achieved on 17 Java classes indicate that the proposed enhancements make GA much more effective and efficient in automating the testing process. In particular, effectiveness (coverage) was significantly improved in 47% of the subjects and efficiency (search budget consumed) was improved in 85% of the subjects on which effectiveness remains the same.

References

  1. M. Alshraideh, L. Bottaci, and B. A. Mahafzah. Using program data-state scarcity to guide automatic test data generation. Software Quality Journal, 18(1):109–144, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. A. Baresel, D. Binkley, M. Harman, and B. Korel. Evolutionary testing in the presence of loop-assigned flags: a testability transformation approach. In Proceedings of the 2004 ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA ’04, pages 108–118, Boston, USA, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. R. Battiti and G. Tecchiolli. The reactive tabu search. ORSA Journal on Computing, 6(2):126–140, 1994.Google ScholarGoogle ScholarCross RefCross Ref
  4. H. G. Cobb and J. J. Grefenstette. Genetic algorithms for tracking changing environments. In Proceedings of the 5th International Conference on Genetic Algorithms, pages 523–530, San Francisco, USA, 1993. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. J. Cohen. Statistical power analysis for the behavioral sciences. Lawrence Earlbaum Associates, 2nd edition edition, 1988.Google ScholarGoogle Scholar
  6. W. J. Conover. Practical Nonparametric Statistics. Wiley, 3rd edition edition, 1998.Google ScholarGoogle Scholar
  7. K. A. De Jong. An analysis of the behavior of a class of genetic adaptive systems. PhD thesis, University of Michigan, 1975. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. R. Feldt, R. Torkar, T. Gorschek, and W. Afzal. Searching for cognitively diverse tests: Towards universal test diversity metrics. In Software Testing Verification and Validation Workshop, 2008. ICSTW’08. IEEE International Conference on, pages 178–186. IEEE, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. G. Fraser and A. Arcuri. Evosuite: automatic test suite generation for object-oriented software. In Proceedings of the 19th ACM SIGSOFT Symposium and the 13th European Conference on Foundations of Software Engineering, ESEC/FSE ’11, pages 416–419, Szeged, Hungary, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. G. Fraser and A. Arcuri. Whole test suite generation. IEEE Transactions on Software Engineering, 39(2), 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. D. Goldberg. Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley Professional, 1989. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. D. E. Goldberg and J. Richardson. Genetic algorithms with sharing for multimodal function optimization. In Proceedings of the 2nd International Conference on Genetic algorithms and their application, pages 41–49, Cambridge, USA, 1987. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. L. Hansheng and K. Lishan. Balance between exploration and exploitation in genetic search. Wuhan University Journal of Natural Sciences, 4(1):28–32, 1999.Google ScholarGoogle ScholarCross RefCross Ref
  14. M. Harman. The current state and future of search based software engineering. In 2007 Future of Software Engineering, FOSE ’07, pages 342–357, Washington, DC, USA, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. J. H. Holland. Adaptation in Natural and Artificial Systems. University of Michigan Press, 1975.Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. S. Holm. A simple sequentially rejective Bonferroni test procedure. Scandinavian Journal on Statistics, 6:65–70, 1979.Google ScholarGoogle Scholar
  17. S. G. E. Hyunsook Do and G. Rothermel. Supporting controlled experimentation with testing techniques: An infrastructure and its potential impact. Empirical Software Engineering: An International Journal, 10:405–435, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Y. Jia and M. Harman. An analysis and survey of the development of mutation testing. IEEE Transactions on Software Engineering, 37(5), 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. A. D. Lucia, M. D. Penta, R. Oliveto, and A. Panichella. Estimating the evolution direction of populations to improve genetic algorithms. In Proceedings of the Genetic and Evolutionary Computation Conference, GECCO ’12, pages 617–624, Philadelphia, USA, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. S. W. Mahfoud. Niching methods for genetic algorithms. Technical report, Illinois Genetic Algorithms Laboratory, 1995.Google ScholarGoogle Scholar
  21. P. McMinn. Search-based software test data generation: a survey. Software Testing, Verification and Reliability, 14:105–156, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. C. Michael, G. McGraw, and M. A. Schatz. Generating software test data by evolution. IEEE Transactions on Software Engineering, 27(12), 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. A. Rogers and A. Prugel-Bennett. Genetic drift in genetic algorithm selection schemes. IEEE Transactions on Evolutionary Computation, 3(4):298 –303, 1999. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. S. N. Sivanandam and S. N. Deepa. Introduction to Genetic Algorithms. Springer Publishing Company, Incorporated, 1st edition, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. G. Strang. Introduction to Linear Algebra. Wellesley-Cambridge Press and SIAM, 4th edition, 2009.Google ScholarGoogle Scholar
  26. P. Tonella. Evolutionary testing of classes. In Proceedings of the 2004 ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA ’04, pages 119–128, Boston, USA, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. J. Wegener, A. Baresel, and H. Sthamer. Evolutionary test environment for automatic structural testing. Information and Software Technology, 43(14):841–854, 2001.Google ScholarGoogle ScholarCross RefCross Ref
  28. D. Whitley. The genitor algorithm and selection pressure: why rank-based allocation of reproductive trials is best. In Proceedings of the 3rd International Conference on Genetic algorithms, pages 116–121, San Francisco, USA, 1989. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Orthogonal exploration of the search space in evolutionary test case generation

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in
    • Published in

      cover image ACM Conferences
      ISSTA 2013: Proceedings of the 2013 International Symposium on Software Testing and Analysis
      July 2013
      381 pages
      ISBN:9781450321594
      DOI:10.1145/2483760

      Copyright © 2013 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 15 July 2013

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article

      Acceptance Rates

      Overall Acceptance Rate58of213submissions,27%

      Upcoming Conference

      ISSTA '24

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader