Abstract
Aiming at the difficulty of unknown Trojan detection in the APT flooding situation, an improved detecting method has been proposed. The basic idea of this method originates from advanced persistent threat (APT) attack intents: besides dealing with damaging or destroying facilities, the more essential purpose of APT attacks is to gather confidential data from target hosts by planting Trojans. Inspired by this idea and some in-depth analyses on recently happened APT attacks, five typical communication characteristics are adopted to describe application’s network behavior, with which a fine-grained classifier based on Decision Tree and Naïve Bayes is modeled. Finally, with the training of supervised machine learning approaches, the classification detection method is implemented. Compared with general methods, this method is capable of enhancing the detection and awareness capability of unknown Trojans with less resource consumption.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Zhou Yonglin, Wang Minghua. 2011 China internet network security situation [EB/OL]. [2012-05-21]. http://www.cert.org.cn/userfiles/file/201203192011annualreport(1).pdf .
Matrosov A, Rodionov E, Harley D, et al. Stuxnet under the microscope [EB/OL]. [2012-10-17]. http://www.eset.com/us/resources/white-papers/stuxnet_under_the_mciroscope.pdf .
Falliere N, Murchu O L, Chien E. W32. stuxnet dossier [EB/OL]. [2012-05-21]. http://www.cert.org.cn/userfiles/file/201203192011annualreport(1).pdf .
Bencsáth B, Pek G, Buttyan L, et al. Duqu: A Stuxnet-like malware found in the wild [EB/OL]. [2012-07-14]. http://www.crysys.hu/mfelegyhazi/publications/Bencsath2011duqu.pdf .
Binde B, McRee R, O’Connor T J. Assessing outbound traffic to uncover advanced persistent threat [EB/OL]. [2011-04-22]. http://www.symantec.com/content/en/us/enterprise/media/secruity_response/whitepapers/w32_stuxnet_dossier.pdf .
Perdisci R, Lee W, Feamster N. Behavioral clustering of http-based malware and signature generation using malicious network traces [EB/OL]. [2011-06-14]. http://static.usenix.org/event/nsdi10/tech/full_papers/perdisci.pdf .
Sun Haitao, Liu Shengli, Chen Jiayong, et al. Tunnel Trojan detection method Based on operation behavior [J]. Computer Engineering, 2011, 37(20): 123–126.
Sun Xiaoyan, Xing Yundong, Liu Shengli, et al. Generation of Trojan communication signatures based on support [J]. Journal on Communications, 2010, 31(9): 176–182.
Tang Zhangguo, Li Huanzhou, Zhong Mingquan, et al. Heuristic Trojan identification system based on network communication fingerprint [J]. Computer Engineering, 2011, 37(17): 119–122.
Bayer U, Comparetti P M, Hlauschek C, et al. Scalable, behavior-based malware clustering [EB/OL]. [2012-04-19]. http://citeserrix.ist.psu.edu/viewdoc/download?doi=10.1.1.148.7690&rep=rep1&type=pdf .
Jacob G, Hund R, Kruegel C, et al. JACKSTRAWS: picking command and control connections from bot traffic [EB/OL]. [2012-07-21]. https://www.usenix.org/legacy/event/sec11/tech/full_papers/jacob.pdf .
Yen T F, Reiter M K. Detection of Intrusions and Malware, and Vulnerability Assessment [M]. Berlin, Heidelberg: Springer-Verlag, 2008, 5137: 207–227.
Brumley D, Hartwig C, Liang Z, et al. Automatically identifying trigger-based behavior in malware [J]. Botnet Detection, 2008, 36: 65–88.
Jiang X, Wang X, Xu D. Stealthy malware detection through vmm-based out-of-the-box semantic view reconstruction [C]// Proceedings of the 14th ACM Conference on Computer and Communications Security. New York: ACM Press, 2007: 128–138.
Inoue D, Yoshioka K, Eto M, et al. Malware behavior analysis in isolated miniature network for revealing malware’s network activity [C]//Proceedings of the IEEE International Conference on Communications. New York: IEEE Press, 2008: 1715–1721.
Bramer M. Principles of Data Mining [M]. New York: Springer-Verlag, 2007.
Bishop C M. Pattern Recognition and Machine Learning [M]. New York: Springer-Verlag, 2006: 653–656.
Author information
Authors and Affiliations
Corresponding author
Additional information
Foundation item: Supported by the National Natural Science Foundation of China (61202387, 61103220), Major Projects of National Science and Technology of China(2010ZX03006-001-01), Doctoral Fund of Ministry of Education of China (2012014110002), China Postdoctoral Science Foundation (2012M510641), Hubei Province Natural Science Foundation (2011CDB456), and Wuhan Chenguang Plan Project(2012710367)
Biography: LIANG Yu, male, Ph.D. candidate, research direction: network and information system security.
Rights and permissions
About this article
Cite this article
Liang, Y., Peng, G., Zhang, H. et al. An unknown Trojan detection method based on software network behavior. Wuhan Univ. J. Nat. Sci. 18, 369–376 (2013). https://doi.org/10.1007/s11859-013-0944-6
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11859-013-0944-6