Abstract
In Network Anomaly and Botnet Detection the main source of input for analysis is the network traffic, which has to be transmitted from its capture source to the analysis system. High-volume data sources often generate traffic volumes prohibiting direct pass-through of bulk data into researchers hands.
In this paper we achieve a reduction in volume of transmitted test data from network flow captures by aggregating raw data using extraction of protocol semantics. This is orthogonal to classic bulk compression algorithms. We propose a formalization for this concept called Descriptors and extend it to network flow data.
A comparison with common bulk data file compression formats will be given for full Packet Capture (PCAP) files, giving 4 to 5 orders of magnitude in size reduction using Descriptors.
Our approach aims to be compatible with Internet Protocol Flow Information Export (IPFIX) and other standardized network flow data formats as possible inputs.
Chapter PDF
Similar content being viewed by others
References
Ahmad, R., Ghani, M., Haris, S.H.C., Waleed, G.M.: Anomaly detection of ip header threats (2012)
Bykova, M., Shawn Ostermann, B.T.: Detecting network intrusions via a statistical analysis of network packet characteristics (2001)
Deri, L., Maselli, G., Suin, S.: Design and implementation of an anomaly detection system: An empirical approach (2003)
Rossow, C., Dietrich, C.J., Bos, H., Cavallaro, L., van Steen, M., Freiling, F.C., Pohlmann, N.: Sandnet: Network traffic analysis of malicious software. In: Proceedings of the First Workshop on Building Analysis Datasets and Gathering Experience Returns for Security, pp. 78–88. ACM (2011)
Politopoulos, P.I., Markatos, E.P., Ioannidis, S.: Evaluation of compression of remote network monitoring data streams. In: IEEE Network Operations and Management Symposium Workshops, NOMS Workshops 2008, pp. 109–115. IEEE (2008)
Palmieri, F., Fiore, U., Castiglione, A.: A distributed approach to network anomaly detection based on independent component analysis. Concurrency and Computation: Practice and Experience, n/a–n/a (2013)
Lakhina, A., Crovella, M., Diot, C.: Characterization of network-wide anomalies in traffic flows. In: Proceedings of the 4th ACM SIGCOMM Conference on Internet Measurement, pp. 201–206. ACM (2004)
Bilge, L., Kirda, E., Kruegel, C., Balduzzi, M.: Exposure: Finding malicious domains using passive dns analysis. In: Proceedings of NDSS (2011)
Bilge, L., Balzarotti, D., Robertson, W., Kirda, E., Kruegel, C.: Disclosure: Detecting botnet command and control servers through large-scale netflow analysis. In: Proceedings of the 28th Annual Computer Security Applications Conference, pp. 129–138. ACM (2012)
Tegeler, F., Fu, X., Vigna, G., Krügel, C.: Finding bots in network traffic without deep packet inspection. In: The 8th ACM International Conference on Emerging Networking EXperiments and Technologies (CoNEXT 2012), Nice, France (December 2012)
Claise, E.B.: Cisco systems netflow services export version 9 (2004)
Abt, S., Wener, S., Baier, H.: Performance evaluation of classification and feature selection algorithms for netflow-based protocol recognition. In: INFORMATIK 2013, pp. 2184–2197. Gesellschaft für Informatik (2013)
Network Working Group, Trammell, B.: Bidirectional Flow Export Using IP Flow Information Export (IPFIX), http://www.ietf.org/rfc/rfc5103.txt
Petersen, D., Himmelsbach, K., Bastke, S., Pohlmann, N.: Measuring and warning (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 IFIP International Federation for Information Processing
About this paper
Cite this paper
Fourné, M., Stegemann, K., Petersen, D., Pohlmann, N. (2014). Aggregation of Network Protocol Data Near Its Source. In: Linawati, Mahendra, M.S., Neuhold, E.J., Tjoa, A.M., You, I. (eds) Information and Communication Technology. ICT-EurAsia 2014. Lecture Notes in Computer Science, vol 8407. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-55032-4_49
Download citation
DOI: https://doi.org/10.1007/978-3-642-55032-4_49
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-55031-7
Online ISBN: 978-3-642-55032-4
eBook Packages: Computer ScienceComputer Science (R0)