Abstract
An important technique for attack detection in complex company networks is the analysis of log data from various network components. As networks are growing, the number of produced log events increases dramatically, sometimes even to multiple billion events per day. The analysis of such big data highly relies on a full normalization of the log data in realtime. Until now, the important issue of full normalization of a large number of log events is only insufficiently handled by many software solutions and not well covered in existing research work. In this paper, we propose and evaluate multiple approaches for handling the normalization of a large number of typical logs better and more efficient. The main idea is to organize the normalization in multiple levels by using a hierarchical knowledge base (KB) of normalization rules. In the end, we achieve a performance gain of about 1000x with our presented approaches, in comparison to a naive approach typically used in existing normalization solutions. Considering this improvement, big log data can now be handled much faster and can be used to find and mitigate attacks in realtime.
Chapter PDF
References
United States Computer Emergency Readiness Team (US-CERT). USCERT Year in Review CY 2012. Tech. rep. US Department of Homeland Security (2012)
US Office of Management and Budget. Fiscal Year 2012 Report to Congress on the Implementation of The Federal Information Security Management Act of 2002 (March 2013)
Kent, K., Souppaya, M.: Guide to Computer Security Log Management. In: NIST special publication (September 2006). http://212.200.39.245:81/CrnaRupa/2009-2010/FIM/ZIS/Literatura/GuidetoComputerSecurityLogManagementSP800-92.pdf
Gerhards, R.: The Syslog Protocol. RFC 5424 (Proposed Standard). Internet Engineering Task Force (March 2009). http://www.ietf.org/rfc/rfc5424.txt
Chuvakin, A., Marty, R., et al.: Common Event Expression. White Paper, MITRE (June (2008)
Hewlett-Packard. Implementing ArcSight CEF. 20. Hewlett-Packard (June 2013)
Barnum, S., Martin, R., et al.: The CybOX Language Specification. Draft 1. The MITRE Corporation (April 2012)
Sapegin, A., Jaeger, D., et al.: Hierarchical Object Log Format for Normalisation of Security Events. In: Proceedings of the 9th International Conference on Information Assurance and Security (IAS 2013), Yassmine Hammamet, Tunisia, pp. 25–30 (December 2013)
Friedl, J.E.F.: Mastering Regular Expressions. In: Oram, A. (ed.) 3rd edn. O’Reilly Media (August 2006)
Sparvieri, L.: SAP HANA Text Analysis. SAP (January 2014). http://scn.sap.com/community/developer-center/hana/blog/2013/01/03/sap-hana-text-analysis
Kobayashi, S., Fukuda, K., Esaki, H.: Towards an NLPbased log template generation algorithm for system log analysis. In: Proceedings of The Ninth International Conference on Future Internet Technologies, p. 11 (2014)
Azodi, A., Jaeger, D., et al.: Pushing the limits in event normalisation to improve attack detection in IDS/SIEM systems. In: Proceedings of the First Internation Conference on Advanced Cloud and Big Data (CBD 2013), Nanjing, China (December 2013)
Azodi, A., Jaeger, D., et al.: A new approach to building a multi- tier direct access knowledge base for IDS/SIEM systems. In: Proceedings of the 11th IEEE International Conference on Dependable, Autonomic and Secure Computing (DASC 2013), Chengdu, China (December 2013)
Real-time Event Analysis and Monitoring System (REAMS). http://hpi.de/en/meinel/security-tech/network-security/securityanalytics/reams.html (visited on November 5, 2015)
The Honeynet Project. Honeynet Challenges: Scan of the Month 34. Web Site (2005). http://old.honeynet.org/scans/scan34/ (visited on May 4, 2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 IFIP International Federation for Information Processing
About this paper
Cite this paper
Jaeger, D., Azodi, A., Cheng, F., Meinel, C. (2015). Normalizing Security Events with a Hierarchical Knowledge Base. In: Akram, R., Jajodia, S. (eds) Information Security Theory and Practice. WISTP 2015. Lecture Notes in Computer Science(), vol 9311. Springer, Cham. https://doi.org/10.1007/978-3-319-24018-3_15
Download citation
DOI: https://doi.org/10.1007/978-3-319-24018-3_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-24017-6
Online ISBN: 978-3-319-24018-3
eBook Packages: Computer ScienceComputer Science (R0)