Skip to main content
SpringerLink
Log in

SMT Sampling via Model-Guided Approximation

  • Conference paper
  • First Online:
Formal Methods (FM 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14000))

Included in the following conference series:

Abstract

We investigate the domain of satisfiable formulas in satisfiability modulo theories (SMT), in particular, automatic generation of a multitude of satisfying assignments to such formulas. Despite the long and successful history of SMT in model checking and formal verification, this aspect is relatively under-explored. Prior work exists for generating such assignments, or samples, for Boolean formulas and for quantifier-free first-order formulas involving bit-vectors, arrays, and uninterpreted functions (QF_AUFBV). We propose a new approach that is suitable for a theory T of integer arithmetic and to T with arrays and uninterpreted functions. The approach involves reducing the general sampling problem to a simpler instance of sampling from a set of independent intervals, which can be done efficiently. Such reduction is carried out by expanding a single model—a seed—using top-down propagation of constraints along the original first-order formula.

This work is supported by the Israeli Science Foundation Grant No. 243/19 and the Binational Science Foundation (NSF-BSF) Grant No. 2018675.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Available at: https://github.com/chaosite/MeGASampler.

  2. 2.

    The arity of functions is restricted to 1 for simplicity of presentation; an extension of our algorithm to functions with arbitrary arity is straightforward.

  3. 3.

    The “interesting” operation in \(T_{\text {NIA}}\) is multiplication; most benchmarks in this directory are actually in \(T_{\text {MIA}}\), which is supported by MeGASampler.

References

  1. Achlioptas, D., Hammoudeh, Z.S., Theodoropoulos, P.: Fast sampling of perfectly uniform satisfying assignments. In: Beyersdorff, O., Wintersteiger, C.M. (eds.) SAT 2018. LNCS, vol. 10929, pp. 135–147. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-94144-8_9

    Chapter  MATH  Google Scholar 

  2. Agbaria, S., Carmi, D., Cohen, O., Korchemny, D., Lifshits, M., Nadel, A.: SAT-based semiformal verification of hardware. In: Formal Methods in Computer Aided Design, pp. 25–32 (2010)

    Google Scholar 

  3. Barrett, C., et al.: CVC4. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 171–177. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-22110-1_14

    Chapter  Google Scholar 

  4. Barrett, C.W., Fontaine, P., Tinelli, C.: The satisfiability modulo theories library (SMT-LIB) (2021). https://smtlib.cs.uiowa.edu

  5. Baumert, S., Ghate, A., Kiatsupaibul, S., Shen, Y., Smith, R.L., Zabinsky, Z.B.: Discrete hit-and-run for sampling points from arbitrary distributions over subsets of integer hyperrectangles. Oper. Res. 57(3), 727–739 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  6. Borralleras, C., Brockschmidt, M., Larraz, D., Oliveras, A., Rodríguez-Carbonell, E., Rubio, A.: Proving termination through conditional termination. In: Legay, A., Margaria, T. (eds.) Tools and Algorithms for the Construction and Analysis of Systems, pp. 99–117. Springer, Berlin, Heidelberg (2017)

    Chapter  Google Scholar 

  7. Borzacchiello, L., Coppa, E., Demetrescu, C.: Fuzzing symbolic expressions. In: 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE) (2021). https://doi.org/10.1109/icse43902.2021.00071

  8. Bryant, R.E., Kroening, D., Ouaknine, J., Seshia, S.A., Strichman, O., Brady, B.: Deciding bit-vector arithmetic with abstraction. In: Grumberg, O., Huth, M. (eds.) TACAS 2007. LNCS, vol. 4424, pp. 358–372. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-71209-1_28

    Chapter  Google Scholar 

  9. Chakraborty, S., Meel, K.S., Vardi, M.Y.: A scalable and nearly uniform generator of SAT witnesses. In: Sharygina, N., Veith, H. (eds.) CAV 2013. LNCS, vol. 8044, pp. 608–623. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39799-8_40

    Chapter  Google Scholar 

  10. Choi, J., Jang, J., Han, C., Cha, S.K.: Grey-box concolic testing on binary code. In: 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE), pp. 736–747. IEEE (2019)

    Google Scholar 

  11. Cimatti, A., Griggio, A., Schaafsma, B.J., Sebastiani, R.: The MathSAT5 SMT solver. In: Piterman, N., Smolka, S.A. (eds.) Tools and Algorithms for the Construction and Analysis of Systems. Lecture Notes in Computer Science, pp. 93–107. Springer, Cham (2013)

    Chapter  MATH  Google Scholar 

  12. Clarke, E., Kroening, D., Yorav, K.: Behavioral consistency of C and Verilog programs using bounded model checking. In: Proceedings of the Design Automation Conference, pp. 368–371. IEEE (2003)

    Google Scholar 

  13. Cousot, P., Cousot, R.: Static determination of dynamic properties of programs. In: Proceedings of the Second International Symposium on Programming, pp. 106–130. Dunod, Paris, France (1976)

    Google Scholar 

  14. Dutertre, B.: Yices 2.2. In: Biere, A., Bloem, R. (eds.) CAV 2014. LNCS, vol. 8559, pp. 737–744. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-08867-9_49

    Chapter  Google Scholar 

  15. Dutra, R., Bachrach, J., Sen, K.: SMTSampler: efficient stimulus generation from complex SMT constraints. In: 2018 IEEE/ACM International Conference on Computer-Aided Design (ICCAD), pp. 1–8 (2018). https://doi.org/10.1145/3240765.3240848

  16. Dutra, R., Bachrach, J., Sen, K.: Guidedsampler: coverage-guided sampling of SMT solutions. In: 2019 Formal Methods in Computer Aided Design (FMCAD), pp. 203–211 (2019). https://doi.org/10.23919/FMCAD.2019.8894251

  17. Dutra, R., Laeufer, K., Bachrach, J., Sen, K.: Efficient sampling of SAT solutions for testing. In: Chaudron, M., Crnkovic, I., Chechik, M., Harman, M. (eds.) Proceedings of the 40th International Conference on Software Engineering, ICSE 2018, Gothenburg, Sweden, 27 May–03 June 2018, pp. 549–559. ACM (2018). https://doi.org/10.1145/3180155.3180248

  18. Ermon, S., Gomes, C.P., Sabharwal, A., Selman, B.: Embed and project: discrete sampling with universal hashing. In: NIPS, pp. 2085–2093 (2013)

    Google Scholar 

  19. Ermon, S., Gomes, C.P., Selman, B.: Uniform solution sampling using a constraint solver as an oracle. arXiv preprint arXiv:1210.4861 (2012)

  20. Giesl, J., Thiemann, R., Schneider-Kamp, P., Falke, S.: Automated termination proofs with AProVE. In: van Oostrom, V. (ed.) RTA 2004. LNCS, vol. 3091, pp. 210–220. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-25979-4_15

    Chapter  Google Scholar 

  21. Glynn, P.W., Iglehart, D.L.: Importance sampling for stochastic simulations. Manag. Sci. 35(11), 1367–1392 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  22. Gurfinkel, A., Kahsai, T., Komuravelli, A., Navas, J.A.: The seahorn verification framework. In: CAV (2015)

    Google Scholar 

  23. Hastings, W.K.: Monte carlo sampling methods using markov chains and their applications. Biometrika 57(1), 97–109 (1970)

    Article  MathSciNet  MATH  Google Scholar 

  24. Heizmann, M., Hoenicke, J., Podelski, A.: Software model checking for people who love automata. In: Sharygina, N., Veith, H. (eds.) CAV 2013. LNCS, vol. 8044, pp. 36–52. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39799-8_2

    Chapter  Google Scholar 

  25. Huang, H., Yao, P., Wu, R., Shi, Q., Zhang, C.: Pangolin: incremental hybrid fuzzing with polyhedral path abstraction. In: 2020 IEEE Symposium on Security and Privacy (SP), pp. 1613–1627. IEEE (2020)

    Google Scholar 

  26. Kitchen, N.: Markov Chain Monte Carlo Stimulus Generation for Constrained Random Simulation. Ph.D. thesis, University of California, Berkeley, USA (2010). http://www.escholarship.org/uc/item/6gp3z1t0

  27. Kitchen, N., Kuehlmann, A.: Stimulus generation for constrained random simulation. In: Gielen, G.G.E. (ed.) 2007 International Conference on Computer-Aided Design, ICCAD 2007, San Jose, CA, USA, 5–8 November 2007, pp. 258–265. IEEE Computer Society (2007). https://doi.org/10.1109/ICCAD.2007.4397275

  28. Komuravelli, A., Gurfinkel, A., Chaki, S.: SMT-based model checking for recursive programs. Formal Methods Syst. Des. 48(3), 175–205 (2016)

    Article  MATH  Google Scholar 

  29. Liu, J.S.: Metropolized independent sampling with comparisons to rejection sampling and importance sampling. Stat. Comput. 6(2), 113–119 (1996)

    Article  Google Scholar 

  30. Meel, K.S.: Sampling techniques for Boolean satisfiability. CoRR abs/1404.6682 (2014). http://arxiv.org/abs/1404.6682

  31. Meel, K.S., et al.: Constrained sampling and counting: Universal hashing meets sat solving. In: Workshops at the Thirtieth AAAI Conference on Artificial Intelligence (2016)

    Google Scholar 

  32. de Moura, L., Bjørner, N.: Z3: an efficient SMT solver. In: Ramakrishnan, C.R., Rehof, J. (eds.) TACAS 2008. LNCS, vol. 4963, pp. 337–340. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-78800-3_24

    Chapter  Google Scholar 

  33. Nadel, A.: Generating diverse solutions in SAT. In: Sakallah, K.A., Simon, L. (eds.) SAT 2011. LNCS, vol. 6695, pp. 287–301. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-21581-0_23

    Chapter  MATH  Google Scholar 

  34. Naveh, Y., et al.: Constraint-based random stimuli generation for hardware verification. AI Mag. 28(33), 13–13 (2007)

    Google Scholar 

  35. Ozols, M., Roetteler, M., Roland, J.: Quantum rejection sampling. ACM Trans. Comput. Theory 5(3), 11:1–11:33 (2013)

    Google Scholar 

  36. Peled, M., Rothenberg, B.C., Itzhaky, S.: SMT sampling via model-guided approximation. CoRR (arXiv) (2022)

    Google Scholar 

  37. van Ravenzwaaij, D., Cassey, P., Brown, S.D.: A simple introduction to Markov chain monte-Carlo sampling. Psychon. Bull. Rev. 25(1), 143–154 (2018)

    Article  Google Scholar 

  38. Shapiro, A.: Monte Carlo sampling methods, stochastic programming, vol. 10, pp. 353–425. Elsevier (2003)

    Google Scholar 

  39. Sharma, S., Gupta, R., Roy, S., Meel, K.S.: Knowledge compilation meets uniform sampling. In: LPAR, pp. 620–636 (2018)

    Google Scholar 

  40. Tokdar, S.T., Kass, R.E.: Importance sampling: a review. WIREs Comput. Stat. 2(1), 54–60 (2010)

    Article  Google Scholar 

  41. Wei, W., Erenrich, J., Selman, B.: Towards efficient sampling: exploiting random walk strategies. In: AAAI, Vol. 4, pp. 670–676 (2004)

    Google Scholar 

  42. Yao, P., Shi, Q., Huang, H., Zhang, C.: Fast bit-vector satisfiability. In: Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis, pp. 38–50 (2020)

    Google Scholar 

Download references

Acknowledgements

The authors would like to thank Profs. Orna Grumberg and Ofer Strichman for their valuable input and contributions to this work. We would additionally like to thank the anonymous reviewers for their time and effort.

Author information

Authors and Affiliations

  1. Technion—Israel Institute of Technology, Haifa, Israel

    Matan I. Peled, Bat-Chen Rothenberg & Shachar Itzhaky

Authors
  1. Matan I. Peled
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bat-Chen Rothenberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shachar Itzhaky
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bat-Chen Rothenberg .

Editor information

Editors and Affiliations

  1. University of Toronto, Toronto, ON, Canada

    Marsha Chechik

  2. RWTH Aachen University, Aachen, Germany

    Joost-Pieter Katoen

  3. University of Lübeck, Lübeck, Germany

    Martin Leucker

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Peled, M.I., Rothenberg, BC., Itzhaky, S. (2023). SMT Sampling via Model-Guided Approximation. In: Chechik, M., Katoen, JP., Leucker, M. (eds) Formal Methods. FM 2023. Lecture Notes in Computer Science, vol 14000. Springer, Cham. https://doi.org/10.1007/978-3-031-27481-7_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-27481-7_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-27480-0

  • Online ISBN: 978-3-031-27481-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation