Abstract
Symbolic execution has been a popular method for detecting vulnerabilities of programs in recent years, yet path explosion has remained a significant challenge in its application. This paper proposes a method for improving the efficiency of symbolic execution and detecting heap overflow vulnerability in executable codes. Instead of applying symbolic execution to the whole program, our method initially determines test units of the program, which are parts of the code that might contain heap overflow vulnerability. This is performed through static analysis and based on the specification of heap overflow vulnerability. Then, it applies symbolic execution to the test units and extracts a constraint tree for each unit. Every node in this tree contains the path and vulnerability constraints on the unit input data for executing and overflowing heap buffers in that node. Solving these constraints gives us input values for the test unit that execute the desired nodes and cause heap overflow. Finally, we use curve fitting and treatment learning to approximate the relation between system and unit input data as a function. Using this function, we generate system inputs that enter the program, reach vulnerable instructions in the desired test unit, and cause heap overflow in those instructions. This method is implemented as a plugin for angr framework and evaluated using a group of benchmark programs. The experiments show its superiority over similar tools in accuracy and performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The smoothness of a function is a stronger case than the continuity of the function. A smooth function is a function having continuous derivatives up to a specific order.
References
Heap Overflow Detection Tool. https://github.com/SoftwareSecurityLab/Heap-Overflow-Detection
National Institute of Standards and Technology in Software Assurance Reference Dataset Project. https://samate.nist.gov/SRD. Accessed 4 Mar 2022
Arlinghaus, S.L., Arlinghaus, W.C., Drake, W.D., Nystuen, J.D.: Practical Handbook of Curve Fitting (1994)
Baldoni, R., Coppa, E., D’elia, D.C., Demetrescu, C., Finocchi, I.: A survey of symbolic execution techniques. ACM Comput. Surv. 51(3) (2018). https://doi.org/10.1145/3182657
Cadar, C., Dunbar, D., Engler, D.: KLEE: unassisted and automatic generation of high-coverage tests for complex systems programs. In: Proceedings of the 8th USENIX Conference on Operating Systems Design and Implementation, OSDI 2008, pp. 209–224. USENIX Association (2008). https://doi.org/10.5555/1855741.1855756
Cha, S., Hong, S., Bak, J., Kim, J., Lee, J., Oh, H.: Enhancing dynamic symbolic execution by automatically learning search heuristics. IEEE Trans. Softw. Engi., 1 (2021). https://doi.org/10.1109/TSE.2021.3101870
Cha, S., Lee, S., Oh, H.: Template-guided concolic testing via online learning, pp. 408–418. Association for Computing Machinery, New York (2018). https://doi.org/10.1145/3238147.3238227
Cha, S., Oh, H.: Concolic testing with adaptively changing search heuristics. In: Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2019, pp. 235–245. Association for Computing Machinery, New York (2019). https://doi.org/10.1145/3338906.3338964
Chen, J., Hu, W., Zhang, L., Hao, D., Khurshid, S., Zhang, L.: Learning to accelerate symbolic execution via code transformation. In: Millstein, T. (ed.) 32nd European Conference on Object-Oriented Programming (ECOOP 2018). Leibniz International Proceedings in Informatics (LIPIcs), vol. 109, pp. 6:1–6:27. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik, Dagstuhl (2018). https://doi.org/10.4230/LIPIcs.ECOOP.2018.6
Chen, T., Zhang, X.S., Guo, S.Z., Li, H.Y., Wu, Y.: State of the art: dynamic symbolic execution for automated test generation. Future Gener. Comput. Syst. 29(7), 1758–1773 (2013). https://doi.org/10.1016/j.future.2012.02.006
Davies, M., Păsăreanu, C.S., Raman, V.: Symbolic execution enhanced system testing. In: Joshi, R., Müller, P., Podelski, A. (eds.) VSTTE 2012. LNCS, vol. 7152, pp. 294–309. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-27705-4_23
Dong, S., Olivo, O., Zhang, L., Khurshid, S.: Studying the influence of standard compiler optimizations on symbolic execution. In: 2015 IEEE 26th International Symposium on Software Reliability Engineering (ISSRE), pp. 205–215 (2015). https://doi.org/10.1109/ISSRE.2015.7381814
Godefroid, P.: Compositional dynamic test generation. SIGPLAN Not. 42(1), 47–54 (2007). https://doi.org/10.1145/1190215.1190226
Godefroid, P., Luchaup, D.: Automatic partial loop summarization in dynamic test generation. In: Proceedings of the 2011 International Symposium on Software Testing and Analysis, ISSTA 2011, pp. 23–33. Association for Computing Machinery, New York (2011). https://doi.org/10.1145/2001420.2001424
Hansen, T., Schachte, P., Søndergaard, H.: State joining and splitting for the symbolic execution of binaries. In: Bensalem, S., Peled, D.A. (eds.) RV 2009. LNCS, vol. 5779, pp. 76–92. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-04694-0_6
Menzies, T., Hu, Y.: Data mining for very busy people. Computer 36(11), 22–29 (2003). https://doi.org/10.1109/MC.2003.1244531
Mouzarani, M., Sadeghiyan, B.: Towards designing an extendable vulnerability detection method for executable codes. Inf. Softw. Technol. 80, 231–244 (2016). https://doi.org/10.1016/j.infsof.2016.09.004
Ognawala, S., Ochoa, M., Pretschner, A., Limmer, T.: MACKE: compositional analysis of low-level vulnerabilities with symbolic execution. In: Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering, ASE 2016, pp. 780–785. Association for Computing Machinery, New York (2016). https://doi.org/10.1145/2970276.2970281
Păsăreanu, C.S., et al.: Combining unit-level symbolic execution and system-level concrete execution for testing Nasa software, ISSTA 2008, pp. 15–26. Association for Computing Machinery, New York (2008). https://doi.org/10.1145/1390630.1390635
Schwartz-Narbonne, D., Schäf, M., Jovanović, D., Rümmer, P., Wies, T.: Conflict-directed graph coverage. In: Havelund, K., Holzmann, G., Joshi, R. (eds.) NFM 2015. LNCS, vol. 9058, pp. 327–342. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-17524-9_23
Stephens, N., et al.: Driller: augmenting fuzzing through selective symbolic execution. In: In: NDSS (2016). https://doi.org/10.14722/ndss.2016.23368
Strang, G.: Linear Algebra and Its Applications. Thomson, Brooks/Cole, Belmont (2006). http://www.amazon.com/Linear-Algebra-Its-Applications-Edition/dp/0030105676
Wang, F., Shoshitaishvili, Y.: Angr - the next generation of binary analysis. In: 2017 IEEE Cybersecurity Development (SecDev), pp. 8–9 (2017). https://doi.org/10.1109/SecDev.2017.14
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Mouzarani, M., Kamali, A., Baradaran, S., Heidari, M. (2022). A Unit-Based Symbolic Execution Method for Detecting Heap Overflow Vulnerability in Executable Codes. In: Kovács, L., Meinke, K. (eds) Tests and Proofs. TAP 2022. Lecture Notes in Computer Science, vol 13361. Springer, Cham. https://doi.org/10.1007/978-3-031-09827-7_6
Download citation
DOI: https://doi.org/10.1007/978-3-031-09827-7_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-09826-0
Online ISBN: 978-3-031-09827-7
eBook Packages: Computer ScienceComputer Science (R0)