Skip to main content
SpringerLink
Log in
Book cover

International Conference on Tests and Proofs

TAP 2008: Tests and Proofs pp 66–83Cite as

A Logic-Based Approach to Combinatorial Testing with Constraints

  • Conference paper

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 4966))

Abstract

Usage of combinatorial testing is wide spreading as an effective technique to reveal unintended feature interaction inside a given system. To this aim, test cases are constructed by combining tuples of assignments of the different input parameters, based on some effective combinatorial strategy. The most commonly used strategy is two-way (pairwise) coverage, requiring all combinations of valid assignments for all possible pairs of input parameters to be covered by at least one test case. In this paper a new heuristic strategy developed for the construction of pairwise covering test suites is presented, featuring a new approach to support expressive constraining over the input domain. Moreover, it allows the inclusion or exclusion of ad-hoc combinations of parameter bindings to let the user customize the test suite outcome. Our approach is tightly integrated with formal logic, since it uses test predicates to formalize combinatorial testing as a logic problem, and applies an external model checker tool to solve it. The proposed approach is supported by a prototype tool implementation, and early results of experimental assessment are also presented.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Atlee, J.M., Buckley, M.A.: A logic-model semantics for SCR software requirements. In: International Symposium on Software Testing and Analysis, ACM, New York (1996)

    Google Scholar 

  2. Bose, R.C., Bush, K.A.: Orthogonal arrays of strength two and three. The Annals of Mathematical Statistics 23(4), 508–524 (1952)

    Article  MathSciNet  Google Scholar 

  3. Bryce, R.C., Colbourn, C.J.: Prioritized interaction testing for pair-wise coverage with seeding and constraints. Information & Software Technology 48(10), 960–970 (2006)

    Article  Google Scholar 

  4. Bryce, R.C., Colbourn, C.J., Cohen, M.B.: A framework of greedy methods for constructing interaction test suites. In: ICSE 2005, pp. 146–155. ACM, New York (2005)

    Google Scholar 

  5. Cohen, D.M., Dalal, S.R., Fredman, M.L., Patton, G.C.: The AETG system: An approach to testing based on combinatorial design. IEEE Transactions On Software Engineering 23(7) (1997)

    Google Scholar 

  6. Cohen, M.B., Colbourn, C.J., Gibbons, P.B., Mugridge, W.B.: Constructing test suites for interaction testing. In: ICSE 2003, pp. 38–48 (2003)

    Google Scholar 

  7. Cohen, M.B., Dwyer, M.B., Shi, J.: Interaction testing of highly-configurable systems in the presence of constraints. In: ISSTA International symposium on Software testing and analysis, pp. 129–139. ACM Press, New York (2007)

    Google Scholar 

  8. Czerwonka, J.: Pairwise testing in real world. In: 24th Pacific Northwest Software Quality Conference (2006)

    Google Scholar 

  9. Dalal, S.R., Jain, A., Karunanithi, N., Leaton, J.M., Lott, C.M., Patton, G.C., Horowitz, B.M.: Model-based testing in practice. In: International Conference on Software Engineering ICSE, May 1999, pp. 285–295. Association for Computing Machinery, New York (1999)

    Google Scholar 

  10. de Moura, L., Owre, S., Rueß, H., Shankar, J.R.N., Sorea, M., Tiwari, A.: SAL 2. In: Alur, R., Peled, D.A. (eds.) CAV 2004. LNCS, vol. 3114, pp. 496–500. Springer, Heidelberg (2004)

    Google Scholar 

  11. Gargantini, A., Heitmeyer, C.: Using model checking to generate tests from requirements specifications. In: Nierstrasz, O., Lemoine, M. (eds.) ESEC 1999 and ESEC-FSE 1999. LNCS, vol. 1687, Springer, Heidelberg (1999)

    Chapter  Google Scholar 

  12. Gargantini, A., Riccobene, E.: Asm-based testing: Coverage criteria and automatic test sequence generation. JUCS 10(8) (November 2001)

    Google Scholar 

  13. Gargantini, A.: Using model checking to generate fault detecting tests. In: Gurevich, Y., Meyer, B. (eds.) TAP 2007. LNCS, vol. 4454, pp. 189–206. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  14. Gargantini, A., Riccobene, E., Rinzivillo, S.: Using spin to generate tests from ASM specifications. In: Börger, E., Gargantini, A., Riccobene, E. (eds.) ASM 2003. LNCS, vol. 2589, Springer, Heidelberg (2003)

    Chapter  Google Scholar 

  15. Grindal, M., Offutt, J., Andler, S.F.: Combination testing strategies: a survey. Softw. Test, Verif. Reliab 15(3), 167–199 (2005)

    Article  Google Scholar 

  16. Hartman, A.: Ibm intelligent test case handler: Whitch, http://www.alphaworks.ibm.com/tech/whitch

  17. Hartman, A.: Graph Theory, Combinatorics and Algorithms Interdisciplinary Applications. Chapter Software and Hardware Testing Using Combinatorial Covering Suites, pp. 237–266. Springer, Heidelberg (2005)

    Google Scholar 

  18. Hartman, A., Raskin, L.: Problems and algorithms for covering arrays. DMATH: Discrete Mathematics 284(1-3), 149–156 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  19. Hnich, B., Prestwich, S.D., Selensky, E., Smith, B.M.: Constraint models for the covering test problem. Constraints 11(2-3), 199–219 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  20. Kobayashi, N., Tsuchiya, T., Kikuno, T.: Non-specification-based approaches to logic testing for software. Journal of Information and Software Technology 44(2), 113–121 (2002)

    Article  MathSciNet  Google Scholar 

  21. Kuhn, D.R., Okum, V.: Pseudo-exhaustive testing for software. In: SEW 2006: IEEE/NASA Software Engineering Workshop, pp. 153–158. IEEE Computer Society, Los Alamitos (2006)

    Chapter  Google Scholar 

  22. Kuhn, D.R., Wallace, D.R., Gallo, A.M.: Software fault interactions and implications for software testing. IEEE Trans. Software Eng. 30(6), 418–421 (2004)

    Article  Google Scholar 

  23. Lott, C., Jain, A., Dalal, S.: Modeling requirements for combinatorial software testing. In: A-MOST 2005: Proceedings of the 1st international workshop on Advances in model-based testing, pp. 1–7. ACM Press, New York (2005)

    Chapter  Google Scholar 

  24. McDowell, A.: All-pairs testing, http://www.mcdowella.demon.co.uk/allpairs.html

  25. Nurmela, K.: Upper bounds for covering arrays by tabu. Discrete Applied Mathematics 138(1-2), 143–152 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  26. Pairwise web site, http://www.pairwise.org/

  27. Tai, K.C., Lie, Y.: A test generation strategy for pairwise testing. IEEE Trans. Softw. Eng. 28(1), 109–111 (2002)

    Article  Google Scholar 

  28. TestCover tool, http://www.testcover.com/

  29. The asmeta project, http://asmeta.sourceforge.net

  30. Jenny Combinatorial Tool, http://www.burtleburtle.net/bob/math/jenny.html

  31. Williams, A.W., Probert, R.L.: A measure for component interaction test coverage. In: AICCSA, pp. 304–312. IEEE Computer Society, Los Alamitos (2001)

    Google Scholar 

  32. Williams, A.W.: Determination of test configurations for pair-wise interaction coverage. In: Proceedings of the 13th International Conference on the Testing of Communicating Systems (TestCom 2000), August 2000, pp. 59–74 (2000)

    Google Scholar 

  33. Yilmaz, C., Cohen, M.B., Porter, A.A.: Covering arrays for efficient fault characterization in complex configuration spaces. IEEE Trans. Software Eng. 32(1), 20–34 (2006)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Catania, Italy

    Andrea Calvagna

  2. University of Bergamo, Italy

    Angelo Gargantini

Authors
  1. Andrea Calvagna
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Angelo Gargantini
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Bernhard Beckert Reiner Hähnle

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Calvagna, A., Gargantini, A. (2008). A Logic-Based Approach to Combinatorial Testing with Constraints. In: Beckert, B., Hähnle, R. (eds) Tests and Proofs. TAP 2008. Lecture Notes in Computer Science, vol 4966. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-79124-9_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-79124-9_6

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-79123-2

  • Online ISBN: 978-3-540-79124-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation