Abstract
Several malware analysis techniques suppose that the disassembled code of a piece of malware is available, which is however not always possible. This paper proposes a flexible and automated approach to extract malware behaviour by observing all the system function calls performed in a virtualized execution environment. Similarities and distances between malware behaviours are computed which allows to classify malware behaviours. The main features of our approach reside in coupling a sequence alignment method to compute similarities and leverage the Hellinger distance to compute associated distances. We also show how the accuracy of the classification process can be improved using a phylogenetic tree. Such a tree shows common functionalities and evolution of malware. This is relevant when dealing with obfuscated malware variants that have often similar behaviour. The phylogenetic trees were assessed using known antivirus results and only a few malware behaviours were wrongly classified.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abdi, H.: Distance. In Salkind, N.J. (eds.) Encyclopedia of Measurement and Statistics, pp. 280–284. Sage, Thousand Oaks
Bruschi, D., Martignoni, L., Monga, M.: Recognizing self-mutating malware by code normalization and control-flow graph analysis. IEEE Secur. Privac. (2007, in press)
Christodorescu, M., Jha, S., Kruegel, C.: Mining specifications of malicious behavior. In: ESEC-FSE ’07: Proceedings of the the 6th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering, pp. 5–14. ACM Press, New York (2007).
Christodorescu, M., Kinder, J., Jha, S., Katzenbeisser, S., Veith, H.: Malware normalization. Technical Report 1539, University of Wisconsin, Madison, Wisconsin, USA, November 2005
Ferrie, P.: Attacks on virtual machine emulators. In Proceedings AVAR (2006)
Filiol E. (2004). Les virus informatiques: théorie, pratique et applications. Springer, Heidelberg
Ford, R.: The future of virus detection. Information Security Technical Report, pp. 19–26. Elsevier, Amsterdam (2004)
Forrest, S., Hofmeyr, S.A., Somayaji, A., Longstaff, T.A.: A sense of self for Unix processes. In: Proceedinges of the 1996 IEEE Symposium on Research in Security and Privacy, pp. 120–128. IEEE Computer Society Press (1996)
Goldberg L.A., Goldberg P.W., Phillips C.A. and Sorkin G.B. (1998). Constructing computer virus phylogenies. J. Algorith. 26(1): 188–208
Hoskins M.E. (2006). User-mode linux. Linux J. 2006(145): 2
Julliard, A.: Wine. http://www.winehq.com
Karim Md.E., Walenstein A., Lakhotia A. and Parida L. (2005). Malware phylogeny generation using permutations of code. J. Comput. Virol. 1(1–2): 13–23
Kim, J., Warnow, T.: Tutorial on phylogenetic tree estimation (1999)
Kirda, E., Kruegel, C., Banks, G., Vigna, G., Kemmerer, R.: Behavior-based spyware detection. In: Proceedings of the 15th USENIX Security Symposium, Vancouver, BC, Canada, August 2006. Springer, Heidelberg
Kruegel, C., Kirda, E., Mutz, D., Robertson, W., Vigna, G.: Automating mimicry attacks using static binary analysis. In: SSYM’05: Proceedings of the 14th Conference on USENIX Security Symposium, Berkeley, CA, USA, p. 11. USENIX Association (2005)
Lyda R. and Hamrock J. (2007). Using entropy analysis to find encrypted and packed malware. IEEE Secur. Privac. 5(2): 40–45
Swimmer, A.M.M., Le Charlier, B.: Dynamic detection and classification of computer viruses using general behavior patterns. In: Proceedings of the 5th International Virus Bulletin Conference, pp. 75–88 (1995)
Nepenthes: http://nepenthes.mwcollect.org
Norman: http://sandbox.norman.no
Objdump: http://www.gnu.org/software/binutils
Peid: http://peid.tk/
Sung, A.H., Xu, J., Chavez, P., Mukkamala, S.: Static analyzer of vicious executables (save). In: ACSAC ’04: Proceedings of the 20th Annual Computer Security Applications Conference (ACSAC’04), Washington, DC, USA, pp. 326–334. IEEE Computer Society (2004)
Mazeroff, G., De Cerqueira, V., Gregor, J., Thomason, M.G.: Probabilistic trees and automata for application behavior modeling. In: Proceedings of the 43rd ACM Southeast Conference (2003)
Mody Tony Lee, J.J.: Behavioral classification. In: Proceedings Eicar’06, May 2006
Matthew Evan Wagner. Behavior oriented detection of malicious code at run-time. Master’s thesis, Florida Institute of Technology (2004)
Willems, Carsten Holz, Thorsten, Felix Freiling. Toward automated dynamic malware analysis using cwsandbox. Secur. Privac. Mag. 5(April), 32–39 (2007)
Wilson: Activity pattern analysis by means of sequence-alignment methods. Environ. Plann. 30, 1017–1038 (1998)
Ylonen, T.: SSH – secure login connections over the internet. In: Proceedings of the 6th Security Symposium, p. 37. USENIX Association, Berkeley (1996)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Wagener, G., State, R. & Dulaunoy, A. Malware behaviour analysis. J Comput Virol 4, 279–287 (2008). https://doi.org/10.1007/s11416-007-0074-9
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11416-007-0074-9