skip to main content
10.1145/3092282.3092293acmconferencesArticle/Chapter ViewAbstractPublication PagesicseConference Proceedingsconference-collections
research-article

Optimizing parallel Korat using invalid ranges

Published:13 July 2017Publication History

ABSTRACT

Constraint-based input generation enables systematic testing for effective bug finding, but requires exploration of very large spaces of candidate inputs. This paper introduces a novel approach to optimize input generation using Korat – a solver for constraints written as imperative predicates in Java – when Korat is executed more than once for the same constraint solving problem. Our key insight is that in certain application scenarios the Korat search over the same state space and constraint is repeated across separate runs of Korat, and an earlier run can be summarized to optimize a later run. We introduce invalid ranges to represent parts of the exploration space that do not contain any valid inputs but must be explicitly explored by Korat. Our approach directly prunes these parts in a future run of Korat over the same search problem. We develop our approach for two settings: a sequential setting where the Korat search is run using one worker (i.e., processing unit), and a parallel setting where the search is distributed to several workers. In the parallel setting, we build on a previous technique for parallel Korat, namely SEQ-ON, and integrate invalid ranges with it. Experimental evaluation using 6 subjects show that our approach achieves: in the sequential setting, a speedup of up to 2.82X over sequential Korat (in comparison, SEQ-ON does not provide any speedup in the sequential setting); and in the distributed setting, using up to 32 workers, a speedup of up to 38.84X over sequential Korat (using 1 worker), and up to 3.04X over SEQ-ON in terms of total execution time across the workers.

References

  1. Thomas Ball, Daniel Hoffman, Frank Ruskey, Richard Webber, and Lee White. 2000. State generation and automated class testing. Software Testing, Verification and Reliability 10, 3 (2000), 149–170.Google ScholarGoogle ScholarCross RefCross Ref
  2. Dirk Beyer, Thomas A. Henzinger, M. Erkan Keremoglu, and Philipp Wendler. 2012. Conditional Model Checking: A Technique to Pass Information Between Verifiers. In Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering (FSE ’12). ACM, New York, NY, USA, 57:1–57:11. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Chandrasekhar Boyapati, Sarfraz Khurshid, and Darko Marinov. 2002. Korat: Automated testing based on Java predicates. In ISSTA. ACM, New York, NY, USA, 123–133. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Stefan Bucur, Vlad Ureche, Cristian Zamfir, and George Candea. 2011. Parallel Symbolic Execution for Automated Real-world Software Testing. In Proceedings of the Sixth Conference on Computer Systems (EuroSys). 183–198. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Cristian Cadar and Dawson R. Engler. 2005. Execution Generated Test Cases: How to Make Systems Code Crash Itself. In 12th International Conference on Model Checking Software (SPIN’05). Springer-Verlag, Berlin, Heidelberg, 2–23. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Edmund M. Clarke, Jr., Orna Grumberg, and Doron A. Peled. 1999. Model Checking. MIT Press, Cambridge, MA, USA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein. 2009. Introduction to Algorithms, Third Edition (3rd ed.). The MIT Press, Cambridge, MA, USA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Nima Dini. 2016. MKorat: A Novel Approach for Memoizing the Korat Search and Some Potential Applications.Google ScholarGoogle Scholar
  9. Matthew B. Dwyer, Sebastian Elbaum, Suzette Person, and Rahul Purandare. 2007. Parallel Randomized State-Space Search. In Proceedings of the 29th International Conference on Software Engineering (ICSE ’07), Vol. 37. IEEE Computer Society, Washington, DC, USA, 3–12. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Niklas Een and Niklas Sorensson. 2003. An Extensible SAT-solver. In 6th Conference on Theory and Applications of Satisfiability Testing (SAT). Santa Margherita Ligure, Italy.Google ScholarGoogle Scholar
  11. Antonio Filieri, Marcelo F. Frias, Corina S. Pă£Să£Reanu, and Willem Visser. 2015. Model Counting for Complex Data Structures. In Proceedings of the 22nd International Symposium on Model Checking Software (SPIN 2015), Vol. 9232. Springer-Verlag New York, Inc., New York, NY, USA, 222–241. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Diego Funes, Junaid Haroon Siddiqui, and Sarfraz Khurshid. 2012. Ranged Model Checking. ACM SIGSOFT Software Engineering Notes 37, 6 (2012), 1–5. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Milos Gligoric, Tihomir Gvero, Vilas Jagannath, Sarfraz Khurshid, Viktor Kuncak, and Darko Marinov. 2010. Test generation through programming in UDITA. In 32Nd ACM/IEEE International Conference on Software Engineering - Volume 1 (ICSE ’10). ACM, New York, NY, USA, 225–234. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Patrice Godefroid. 1997. Model Checking for Programming Languages using Verisoft. In Conference Record of POPL’97: The 24th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, Papers Presented at the Symposium. ACM, New York, NY, USA, 174–186. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Patrice Godefroid. 2007. Compositional Dynamic Test Generation. In Proceedings of the 34th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’07). ACM, New York, NY, USA, 47–54. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Patrice Godefroid, Nils Klarlund, and Koushik Sen. 2005. DART: directed automated random testing. In Proceedings of the 2005 ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’05). ACM, New York, NY, USA, 213–223. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. M.J. Harrold and M.L. Souffa. 1988. An incremental approach to unit testing during maintenance. In Conference on Software Maintenance.Google ScholarGoogle Scholar
  18. Gerald Holzmann. 1997. The Model Checker SPIN. IEEE Transactions on Software Engineering 23, 5 (May 1997). Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Gerard J. Holzmann, Rajeev Joshi, and Alex Groce. 2011. Swarm Verification Techniques. IEEE Trans. Softw. Eng. 37, 6 (Nov 2011), 845–857. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Daniel Jackson and Mandana Vaziri. 2000. Finding Bugs with a Constraint Solver. In International Symposium oAn Software Testing and Analysis (ISSTA). Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Sarfraz Khurshid, Corina S. Pasareanu, and Willem Visser. 2003. Generalized Symbolic Execution for Model Checking and Testing. In Proceedings of the 9th International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS’03). Springer-Verlag, Berlin, Heidelberg, 553–568. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. James C. King. 1976. Symbolic Execution and Program Testing. Commun. ACM 19, 7 (1976), 385–394. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. KoratWebPage. Korat Home Page. http://korat.sourceforge.net/index.html.Google ScholarGoogle Scholar
  24. Shuvendu K Lahiri, Robert Nieuwenhuis, and Albert Oliveras. 2006. SMT techniques for fast predicate abstraction. In International Conference on Computer Aided Verification. Springer, 424–437. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Steven Lauterburg, Ahmed Sobeih, Darko Marinov, and Mahesh Viswanathan. 2008. Incremental state-space exploration for programs with dynamically allocated data. In 30th international conference on Software engineering. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Barbara Liskov and John Guttag. 2000. Program Development in Java: Abstraction, Specification, and Object-Oriented Design. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Darko Marinov and Sarfraz Khurshid. 2001. TestEra: A Novel Framework for Automated Testing of Java Programs. In 16th IEEE International Conference on Automated Software Engineering (ASE ’01). IEEE Computer Society, Washington, DC, USA, 22–31. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Sasa Misailovic, Aleksandar Milicevic, Nemanja Petrovic, Sarfraz Khurshid, and Darko Marinov. 2007. Parallel test generation and execution with Korat. In 6th joint meeting of the European Software Engineering Conference and the ACM SIGSOFT International Symposium on Foundations of Software Engineering (ESEC/FSE). ACM, New York, NY, USA, 135–144. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Rui Qiu. 2016. Scaling and Certifying Symbolic Execution. Ph.D. Dissertation. University of Texas at Austin.Google ScholarGoogle Scholar
  30. Junaid Haroon Siddiqui and Sarfraz Khurshid. 2009. PKorat: Parallel Generation of Structurally Complex Test Inputs. In Second International Conference on Software Testing Verification and Validation, ICST. IEEE, Denver, CO, USA, 250–259. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. J. H. Siddiqui and S. Khurshid. 2010. ParSym: Parallel symbolic execution. In 2nd International Conference on Software Technology and Engineering. V1–405–V1– 409.Google ScholarGoogle Scholar
  32. Junaid Haroon Siddiqui and Sarfraz Khurshid. 2012. Scaling Symbolic Execution Using Ranged Analysis. In Proceedings of the 27th Annual ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA, part of SPLASH. ACM, New York, NY, USA, 523–536. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. O. Sokolsky and S. A. Smolka. 1994. Incremental model checking in the modal mu-calculus. In International Conference on Computer Aided Verification. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Matt Staats and Corina Pˇasˇareanu. 2010. Parallel Symbolic Execution for Structural Test Generation. In Proceedings of the Nineteenth International Symposium on Software Testing and Analysis (ISSTA ’10). ACM, New York, NY, USA, 183–194. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Ulrich Stern and David L. Dill. 1997. Parallelizing the Murphi Verifier. In Proceedings of the 9th International Conference on Computer Aided Verification (CAV ’97). Springer-Verlag, London, UK, UK, 256–278. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Engin Uzuncaova. 2008. Efficient Specification-based Testing Using Incremental Techniques. Ph.D. Dissertation. University of Texas at Austin. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Willem Visser, Jaco Geldenhuys, and Matthew B. Dwyer. 2012. Green: Reducing, Rusing and Recycling Constraints in Program Analysis. In ESEC/FSE. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Willem Visser, Klaus Havelund, Guillaume P. Brat, and Seungjoon Park. 2000. Model Checking Programs. In The Fifteenth IEEE International Conference on Automated Software Engineering, ASE,Grenoble, France (ASE ’00). IEEE Computer Society, Washington, DC, USA, 3–12. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. G. Yang, M. B. Dwyer, and G. Rothermel. 2009. Regression model checking. In ICSM. 115–124.Google ScholarGoogle Scholar

Index Terms

  1. Optimizing parallel Korat using invalid ranges

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in
    • Published in

      cover image ACM Conferences
      SPIN 2017: Proceedings of the 24th ACM SIGSOFT International SPIN Symposium on Model Checking of Software
      July 2017
      199 pages
      ISBN:9781450350778
      DOI:10.1145/3092282

      Copyright © 2017 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 13 July 2017

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article

      Upcoming Conference

      ICSE 2025

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader