Smart Boosted Model for Behavior-Based Malware Analysis and Detection

Abstract

Malware analysis and detection are the most important activities to ensure system security. However, current attacks like polymorphic viruses and zero-day attacks that utilize signature-based methods complicate the detection process with accurate results. Therefore, this has—in turn—raised the need for more intelligent techniques to analyze the behavior of the malware rather than depending on the signature-based analyses. This paper proposes a machine learning-based model to analyze and detect the different types of malware. The system tries to determine the optimal feature representation and extraction and classification method that can lead to the best detection accuracy. Particularly, different machine learning algorithms were evaluated, including k-Nearest Neighbors (kNN), Multi-Layer Perceptron (MLP), Naive Bayes Classifier (NBC), Adaboost/XGBoost Decision Trees (ADT), and Support Vector Machines (SVM). The models were trained and tested using a new dataset that includes op-codes available in .asm format (generated using the IDA disassembler tool); it is a subset of data used in Kaggle for the Microsoft Classification challenges. Our empirical results revealed the superiority of the XGBoost-based model scoring an overall detection accuracy of 98.3%.