Reducing the cost of mutation testing: An empirical study

https://doi.org/10.1016/0164-1212(94)00098-0Get rights and content

Abstract

Of the various testing strategies, mutation testing has been empirically found to be effective in detecting faults. However, mutation often imposes unacceptable demands on computing and human resources because of the large number of mutants that need to be compiled and executed on one or more test cases. In addition, the tester needs to examine many mutants and analyze these for possible equivalence with the program under test. For these reasons, mutation is generally regarded as too expensive to use. Because one significant component of the cost of mutation is the execution of mutants against test cases, we believe that this cost can be reduced dramatically by reducing the number of mutants that need to be examined. We report results from a case study designed to investigate two alternatives for reducing the cost of mutation. The alternatives considered are randomly selected x% mutation and constrained abs/ror mutation. We provide experimental data indicating that both alternatives lead to test sets that distinguish a significant number of nonequivalent mutants and provide high all-uses coverage.

References (30)

  • A.T. Acree

    On Mutation

  • A.T. Acree et al.

    Mutation Analysis

  • H. Agrawal et al.

    Design of Mutant Operators for the C Programming Language

  • R.S. Boyer et al.

    SELECT—A Formal System for Testing and Debugging Programs by Symbolic Execution

    Sigplan Not.

    (1975)
  • T.A. Budd

    Mutation Analysis of Program Test Data

  • T.A. Budd

    Mutation analysis: Ideas, examples, problems and prospect

  • T.A. Budd et al.

    The design of a prototype mutation system for program testing

  • T.A. Budd et al.

    Theoretical and empirical studies on using program mutation to test the functional correctness of programs

  • B.J. Choi et al.

    The Mothra toolset

  • L.A. Clarke

    A System to Generate Test Data and Symbolically Execute Programs

    IEEE Trans. Software Eng.

    (1976)
  • L.A. Clarke et al.

    A Formal Evaluation of Data Flow Path Selection Criteria

    IEEE Trans. Software Eng.

    (1989)
  • R.A. DeMillo et al.

    Constraint Based Automatic Test Data Generation

    IEEE Trans. Software Eng.

    (1991)
  • R.A. DeMillo et al.

    Hints on Test Data Selection: Help for the Practicing Programmer

    IEEE Comp.

    (1978)
  • R.A. DeMillo et al.

    An extended overview of the Mothra software testing environment

  • R.A. DeMillo et al.

    The Mothra Software Testing Environment User's Manual

  • Cited by (0)

    View full text