Elia Geretto
ABSTRACT
Hybrid fuzzing, the combination between fuzzing and concolic execution, holds great promise in theory, but has so far failed to deliver all the expected advantages in practice due to its high overhead. The cause is the large amount of time spent in the SMT solver. As a result, hybrid fuzzers often lose out to simpler, yet faster techniques. This issue remains despite novel query pruning techniques that reduce the number and complexity of solver queries as they preclude other crucial optimizations like incremental solving.
We introduce Triereme, a method to speed up the hybrid fuzzer’s concolic engine by reducing the time spent in the SMT solver. Triereme uses a trie (or prefix tree) data structure to schedule and cache solver queries, exploiting common prefixes. This design is made possible by decoupling concolic tracing from concolic solving. As a result, Triereme manages to reconcile pruning with incremental solving, reaping their combined benefits. In our tests, Triereme speeds up concolic executions by 6.1x on average in FuzzBench [22] and improves coverage progress in 79% of the benchmarks.
- Cornelius Aschermann, Sergej Schumilo, Tim Blazytko, Robert Gawlik, and Thorsten Holz. 2019. REDQUEEN: Fuzzing with Input-to-State Correspondence. In Proceedings of the 2019 Network and Distributed System Security Symposium (NDSS), Vol. 19. 1–15.Google Scholar
Cross Ref
- Thanassis Avgerinos, Alexandre Rebert, Sang Kil Cha, and David Brumley. 2014. Enhancing Symbolic Execution with Veritesting. In Proceedings of the 36th International Conference on Software Engineering (Hyderabad, India) (ICSE 2014). Association for Computing Machinery, New York, NY, USA, 1083–1094. https://doi.org/10.1145/2568225.2568293Google ScholarDigital Library
- Jinsheng Ba, Marcel Böhme, Zahra Mirzamomen, and Abhik Roychoudhury. 2022. Stateful Greybox Fuzzing. In 31st USENIX Security Symposium (USENIX Security 22). USENIX Association, Boston, MA, 3255–3272. https://www.usenix.org/conference/usenixsecurity22/presentation/baGoogle Scholar
- Ferenc Bodon and Lajos Rónyai. 2003. Trie: an alternative data structure for data mining algorithms. Mathematical and Computer Modelling 38, 7-9 (2003), 739–751. Publisher: Elsevier.Google ScholarDigital Library
- Marcel Böhme, László Szekeres, and Jonathan Metzman. 2022. On the Reliability of Coverage-Based Fuzzer Benchmarking. In Proceedings of the 44th International Conference on Software Engineering (Pittsburgh, Pennsylvania) (ICSE ’22). Association for Computing Machinery, New York, NY, USA, 1621–1633. https://doi.org/10.1145/3510003.3510230Google ScholarDigital Library
- Luca Borzacchiello, Emilio Coppa, and Camil Demetrescu. 2021. Fuzzing Symbolic Expressions. In Proceedings of the 43rd International Conference on Software Engineering(ICSE ’21). https://doi.org/10.1109/ICSE43902.2021.00071Google ScholarDigital Library
- Cristian Cadar, Daniel Dunbar, and Dawson Engler. 2008. KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs. In Proceedings of the 8th USENIX Conference on Operating Systems Design and Implementation(OSDI’08). 209–224.Google Scholar
- Ju Chen, Wookhyun Han, Mingjun Yin, Haochen Zeng, Chengyu Song, Byoungyoung Lee, Heng Yin, and Insik Shin. 2022. { SYMSAN} : Time and Space Efficient Concolic Execution via Dynamic Data-flow Analysis. In 31st USENIX Security Symposium (USENIX Security 22). 2531–2548.Google Scholar
- Ju Chen, Jinghan Wang, Chengyu Song, and Heng Yin. 2022. JIGSAW: Efficient and Scalable Path Constraints Fuzzing. In 2022 IEEE Symposium on Security and Privacy (SP’22). 18–35.Google Scholar
- Vitaly Chipounov, Volodymyr Kuznetsov, and George Candea. 2011. S2E: A platform for in-vivo multi-path analysis of software systems. Acm Sigplan Notices 46, 3 (2011), 265–278.Google ScholarDigital Library
- Leonardo De Moura and Nikolaj Bjørner. 2008. Z3: An efficient SMT solver. In Tools and Algorithms for the Construction and Analysis of Systems: 14th International Conference, TACAS 2008, Held as Part of the Joint European Conferences on Theory and Practice of Software, ETAPS 2008, Budapest, Hungary, March 29-April 6, 2008. Proceedings 14. Springer, 337–340.Google ScholarCross Ref
- Andrea Fioraldi, Dominik Maier, Heiko Eißfeldt, and Marc Heuse. 2020. AFL++ : Combining Incremental Steps of Fuzzing Research. In 14th USENIX Workshop on Offensive Technologies (WOOT 20). USENIX Association. https://www.usenix.org/conference/woot20/presentation/fioraldiGoogle Scholar
- Andrea Fioraldi, Dominik Maier, Dongjia Zhang, and Davide Balzarotti. 2022. LibAFL: A Framework to Build Modular and Reusable Fuzzers. In Proceedings of the 29th ACM conference on Computer and communications security (CCS) (Los Angeles, U.S.A.) (CCS ’22). ACM.Google ScholarDigital Library
- Elia Geretto, Cristiano Giuffrida, Herbert Bos, and Erik Van Der Kouwe. 2022. Snappy: Efficient Fuzzing with Adaptive and Mutable Snapshots. In Proceedings of the 38th Annual Computer Security Applications Conference (Austin, TX, USA) (ACSAC ’22). Association for Computing Machinery, New York, NY, USA, 375–387. https://doi.org/10.1145/3564625.3564639Google ScholarDigital Library
- Patrice Godefroid, Nils Klarlund, and Koushik Sen. 2005. DART: Directed automated random testing. In Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation. 213–223.Google ScholarDigital Library
- Patrice Godefroid, Michael Y Levin, and David Molnar. 2012. SAGE: whitebox fuzzing for security testing. Commun. ACM 55, 3 (2012), 40–44. Publisher: ACM New York, NY, USA.Google ScholarDigital Library
- Xiangyang Jia, Carlo Ghezzi, and Shi Ying. 2015. Enhancing reuse of constraint solutions to improve symbolic execution. In Proceedings of the 2015 International Symposium on Software Testing and Analysis. 177–187.Google ScholarDigital Library
- George Klees, Andrew Ruef, Benji Cooper, Shiyi Wei, and Michael Hicks. 2018. Evaluating Fuzz Testing. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security (Toronto, Canada) (CCS ’18). Association for Computing Machinery, New York, NY, USA, 2123–2138. https://doi.org/10.1145/3243734.3243804Google ScholarDigital Library
- Daniel Liew, Cristian Cadar, Alastair F. Donaldson, and J. Ryan Stinnett. 2019. Just Fuzz It: Solving Floating-Point Constraints Using Coverage-Guided Fuzzing. In Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering(ESEC/FSE ’19). Association for Computing Machinery.Google ScholarDigital Library
- Stephan Lipp, Daniel Elsner, Thomas Hutzelmann, Sebastian Banescu, Alexander Pretschner, and Marcel Böhme. 2022. FuzzTastic: A Fine-Grained, Fuzzer-Agnostic Coverage Analyzer. In Proceedings of the ACM/IEEE 44th International Conference on Software Engineering: Companion Proceedings (Pittsburgh, Pennsylvania) (ICSE ’22). Association for Computing Machinery, New York, NY, USA, 75–79. https://doi.org/10.1145/3510454.3516847Google ScholarDigital Library
- Rupak Majumdar and Koushik Sen. 2007. Hybrid concolic testing. In 29th International Conference on Software Engineering (ICSE’07). IEEE, 416–426.Google ScholarDigital Library
- Jonathan Metzman, László Szekeres, Laurent Maurice Romain Simon, Read Trevelin Sprabery, and Abhishek Arya. 2021. FuzzBench: An Open Fuzzer Benchmarking Platform and Service. In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering(ESEC/FSE 2021). Association for Computing Machinery, New York, NY, USA, 1393–1403. https://doi.org/10.1145/3468264.3473932Google ScholarDigital Library
- Xianya Mi, Sanjay Rawat, Cristiano Giuffrida, and Herbert Bos. 2021. LeanSym: Efficient Hybrid Fuzzing Through Conservative Constraint Debloating. In 24th International Symposium on Research in Attacks, Intrusions and Defenses (San Sebastian, Spain) (RAID ’21). Association for Computing Machinery, New York, NY, USA, 62–77. https://doi.org/10.1145/3471621.3471852Google ScholarDigital Library
- Sebastian Poeplau and Aurélien Francillon. 2020. Symbolic execution with SymCC: Don’t interpret, compile!. In 29th USENIX Security Symposium (USENIX Security 20). USENIX Association, 181–198. https://www.usenix.org/conference/usenixsecurity20/presentation/poeplauGoogle Scholar
- Sebastian Poeplau and Aurélien Francillon. 2021. SymQEMU: Compilation-based symbolic execution for binaries. In Proceedings of the 2021 Network and Distributed System Security Symposium (NDSS).Google ScholarCross Ref
- Emil Rakadjiev, Taku Shimosawa, Hiroshi Mine, and Satoshi Oshima. 2015. Parallel SMT Solving and Concurrent Symbolic Execution. In 2015 IEEE Trustcom/BigDataSE/ISPA, Vol. 3. 17–26. https://doi.org/10.1109/Trustcom.2015.608Google ScholarDigital Library
- Nick Stephens, John Grosen, Christopher Salls, Andrew Dutcher, Ruoyu Wang, Jacopo Corbetta, Yan Shoshitaishvili, Christopher Kruegel, and Giovanni Vigna. 2016. Driller: Augmenting fuzzing through selective symbolic execution. In Proceedings of the 2016 Network and Distributed System Security Symposium (NDSS), Vol. 16. 1–16.Google ScholarCross Ref
- Jan Taljaard, Jaco Geldenhuys, and Willem Visser. 2020. Constraint Caching Revisited. In NASA Formal Methods, Ritchie Lee, Susmit Jha, Anastasia Mavridou, and Dimitra Giannakopoulou (Eds.). 251–266.Google Scholar
- Willem Visser, Jaco Geldenhuys, and Matthew B. Dwyer. 2012. Green: reducing, reusing and recycling constraints in program analysis. In Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering(FSE ’12). Association for Computing Machinery.Google ScholarDigital Library
- Mingyuan Wu, Ling Jiang, Jiahong Xiang, Yanwei Huang, Heming Cui, Lingming Zhang, and Yuqun Zhang. 2022. One Fuzzing Strategy to Rule Them All. In Proceedings of the 44th International Conference on Software Engineering (Pittsburgh, Pennsylvania) (ICSE ’22). Association for Computing Machinery, New York, NY, USA, 1634–1645. https://doi.org/10.1145/3510003.3510174Google ScholarDigital Library
- Guowei Yang, Corina S Păsăreanu, and Sarfraz Khurshid. 2012. Memoized symbolic execution. In Proceedings of the 2012 International Symposium on Software Testing and Analysis. 144–154.Google ScholarDigital Library
- Insu Yun, Sangho Lee, Meng Xu, Yeongjin Jang, and Taesoo Kim. 2018. QSYM : A Practical Concolic Execution Engine Tailored for Hybrid Fuzzing. In 27th USENIX Security Symposium (USENIX Security 18). USENIX Association, Baltimore, MD, 745–761. https://www.usenix.org/conference/usenixsecurity18/presentation/yunGoogle Scholar
- Michał Zalewski. 2013. American fuzzy lop.Google Scholar
- Lei Zhao, Yue Duan, Heng Yin, and Jifeng Xuan. 2019. Send hardest problems my way: Probabilistic path prioritization for hybrid fuzzing.. In NDSS.Google Scholar
Index Terms
- Triereme: Speeding up hybrid fuzzing through efficient query scheduling
Recommendations
Grammar-aware test case trimming for efficient hybrid fuzzing
AbstractIn recent years, hybrid fuzzing has garnered significant attention by combining the benefits of both fuzzing and concolic execution. However, existing hybrid fuzzing techniques face problems such as the performance overhead of concolic execution ...
LeanSym: Efficient Hybrid Fuzzing Through Conservative Constraint Debloating
RAID '21: Proceedings of the 24th International Symposium on Research in Attacks, Intrusions and DefensesTo improve code coverage and flip complex program branches, hybrid fuzzers couple fuzzing with concolic execution. Despite its benefits, this strategy inherits the inherent slowness and memory bloat of concolic execution, due to path explosion and ...
SymRustC: A Hybrid Fuzzer for Rust
ISSTA 2023: Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and AnalysisWe present SymRustC, a hybrid fuzzer for Rust. SymRustC is hybrid in the sense that it combines fuzzing and concolic execution. SymRustC leverages an existing tool called SymCC for its concolic execution capability and another existing tool ...
Comments