Skip to main content
SpringerLink
Log in

Model-Based Testing for Verification Back-Ends

  • Conference paper
Tests and Proofs (TAP 2013)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 7942))

Included in the following conference series:

Abstract

Many verification tools used in practice rely on sophisticated SAT and SMT solvers. These reasoning engines are assumed and expected to be correct, but, in general, too complex to be fully verified. Therefore, effective testing techniques have to be employed. In this paper, we show how to employ model-based testing (MBT) to test sequences of application programming interface (API) calls and different system configurations. We applied this approach to our SAT solver Lingeling and compared it to existing testing approaches, revealing the effectiveness of MBT for the development of reliable SAT solvers.

This work was partially funded by the Vienna Science and Technology Fund (WWTF) under grant ICT10-018 and by the Austrian Science Fund (FWF) under NFN Grant S11408-N23 (RiSE).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 54.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 72.00
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. D’Silva, V., Kroening, D., Weissenbacher, G.: A survey of automated techniques for formal software verification. TCAD 27(7), 1165–1178 (2008)

    Google Scholar 

  2. Biere, A., Heule, M., van Maaren, H., Walsh, T. (eds.): Handbook of Satisfiability. IOS Press (2009)

    Google Scholar 

  3. Prasad, M.R., Biere, A., Gupta, A.: A survey of recent advances in SAT-based formal verification. STTT 7(2), 156–173 (2005)

    Article  Google Scholar 

  4. Brummayer, R., Lonsing, F., Biere, A.: Automated Testing and Debugging of SAT and QBF Solvers. In: Strichman, O., Szeider, S. (eds.) SAT 2010. LNCS, vol. 6175, pp. 44–57. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  5. Biere, A.: Lingeling and Friends at the SAT Competition 2011. FMV Report Series Technical Report 11(1) (2011)

    Google Scholar 

  6. Davis, M., Logemann, G., Loveland, D.: A machine program for theorem-proving. Communications of the ACM 5(7), 394–397 (1962)

    Article  MathSciNet  MATH  Google Scholar 

  7. Brummayer, R., Biere, A.: Fuzzing and delta-debugging SMT solvers. In: Proc. of the Workshop on Satisfiability Modulo Theories, pp. 1–5. ACM (2009)

    Google Scholar 

  8. Brummayer, R., Järvisalo, M.: Testing and debugging techniques for answer set solver development. TPLP 10(4-6), 741–758 (2010)

    MATH  Google Scholar 

  9. Cuoq, P., Monate, B., Pacalet, A., Prevosto, V., Regehr, J., Yakobowski, B., Yang, X.: Testing static analyzers with randomly generated programs. In: Goodloe, A.E., Person, S. (eds.) NFM 2012. LNCS, vol. 7226, pp. 120–125. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  10. Creignou, N., Egly, U., Seidl, M.: A Framework for the Specification of Random SAT and QSAT Formulas. In: Brucker, A.D., Julliand, J. (eds.) TAP 2012. LNCS, vol. 7305, pp. 163–168. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  11. Nudelman, E., Leyton-Brown, K., Hoos, H.H., Devkar, A., Shoham, Y.: Understanding Random SAT: Beyond the Clauses-to-Variables Ratio. In: Wallace, M. (ed.) CP 2004. LNCS, vol. 3258, pp. 438–452. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  12. Pérez, J.A.N., Voronkov, A.: Generation of Hard Non-Clausal Random Satisfiability Problems. In: Proc. of AAAI/IAAA, AAAI, pp. 436–442. The MIT Press (2005)

    Google Scholar 

  13. Misherghi, G., Su, Z.: HDD: hierarchical Delta Debugging. In: Proc. of ICSE, pp. 142–151. ACM (2006)

    Google Scholar 

  14. Artho, C., Biere, A., Hagiya, M., Potter, R., Ramler, R., Tanabe, Y., Yamamoto, F.: Modbat: A model-based API tester for event-driven systems. In: Dependable Systems Workshop (2012)

    Google Scholar 

  15. Claessen, K., Hughes, J.: Quickcheck: a lightweight tool for random testing of haskell programs. ACM Sigplan Notices 35(9), 268–279 (2000)

    Article  Google Scholar 

  16. Utting, M., Pretschner, A., Legeard, B.: A taxonomy of model-based testing approaches. Softw. Test., Verif. Reliab. 22(5), 297–312 (2012)

    Article  Google Scholar 

  17. Dai, H., Murphy, C., Kaiser, G.E.: Confu: Configuration fuzzing testing framework for software vulnerability detection. IJSSE 1(3), 41–55 (2010)

    Google Scholar 

  18. Cichos, H., Oster, S., Lochau, M., Schürr, A.: Model-Based Coverage-Driven Test Suite Generation for Software Product Lines. In: Whittle, J., Clark, T., Kühne, T. (eds.) MODELS 2011. LNCS, vol. 6981, pp. 425–439. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  19. Barnett, M., Grieskamp, W., Nachmanson, L., Schulte, W., Tillmann, N., Veanes, M.: Towards a Tool Environment for Model-Based Testing with AsmL. In: Petrenko, A., Ulrich, A. (eds.) FATES 2003. LNCS, vol. 2931, pp. 252–266. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  20. Liu, L(L.), Meyer, B., Schoeller, B.: Using Contracts and Boolean Queries to Improve the Quality of Automatic Test Generation. In: Gurevich, Y., Meyer, B. (eds.) TAP 2007. LNCS, vol. 4454, pp. 114–130. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  21. Jalbert, N., Sen, K.: A trace simplification technique for effective debugging of concurrent programs. In: Proc. of FSE, pp. 57–66. ACM (2010)

    Google Scholar 

  22. Jia, Y.: Milu (2012), http://www0.cs.ucl.ac.uk/staff/Y.Jia/Milu/

  23. Holzleitner, J.: Using feedback to improve black box fuzz testing of SAT solvers. Master’s thesis, Johannes Kepler University Linz (2009)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Research Institute for Secure Systems (RISEC), AIST, National Institute of Advanced Industrial Science and Technology (AIST), Amagasaki, Japan

    Cyrille Artho

  2. Institute for Formal Models and Verification, Johannes Kepler University, Linz, Austria

    Armin Biere & Martina Seidl

  3. Business Informatics Group, Vienna University of Technology, Vienna, Austria

    Martina Seidl

Authors
  1. Cyrille Artho
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Armin Biere
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Martina Seidl
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Microsoft Research, 98052-5321, Redmond, WA, USA

    Margus Veanes

  2. Università di Verona, 37134, Verona, Italy

    Luca Viganò

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Artho, C., Biere, A., Seidl, M. (2013). Model-Based Testing for Verification Back-Ends. In: Veanes, M., Viganò, L. (eds) Tests and Proofs. TAP 2013. Lecture Notes in Computer Science, vol 7942. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-38916-0_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-38916-0_3

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-38915-3

  • Online ISBN: 978-3-642-38916-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation