Tram Truong-Huu
ABSTRACT
Network anomalies can arise due to various causes such as abnormal behaviors from users, malfunctioning network devices, malicious activities performed by attackers, malicious software or botnets. With the emergence of machine learning and especially deep learning, many works in the literature developed learning models that are able to detect network anomalies. However, these models require massive amounts of labeled data for model training and may not be able to detect unknown anomalous traffic or zero-day attacks. Unsupervised learning techniques such as autoencoder and its variants do not require labeled data but their performance is still poor. Generative adversarial networks (GANs) have successfully demonstrated their capability of implicitly learning data distributions of arbitrarily complex dimensions. This motivates us to carry out an empirical study on the capability of GANs in network anomaly detection. We adopt two existing GAN models and develop new neural networks for their components, i.e., generator and discriminator. We carry out extensive experiments to evaluate the performance of GANs and compare with existing unsupervised detection techniques. We use multiple datasets that include both realistic traffic captures (PCAP) and synthetic traffic generated by simulation platforms. We develop a traffic aggregation technique to extract statistical features that are useful for the models to learn traffic behaviors. The experimental results show that GANs outperform the existing techniques with a significant improvement in different performance metrics.
- Martin Arjovsky and Léon Bottou. 2017. Towards Principled Methods for Training Generative Adversarial Networks. In Proc. ICLR 2017. Toulon, France.Google Scholar
- L. Bilge et al. 2012. Disclosure: Detecting Botnet Command and Control Servers Through Large-scale NetFlow Analysis. In Proc. ACSAC'12. Orlando, USA.Google ScholarDigital Library
- B. Claise et al. 2013. Specification of the IP Flow Information Export (IPFIX) Protocol for the Exchange of Flow Information. Technical Report RFC 7011. Cisco Systems.Google Scholar
- A. Creswell et al. 2018. Generative Adversarial Networks: An Overview. IEEE Signal Processing Magazine, Vol. 35, 1 (Jan. 2018), 53--65.Google ScholarCross Ref
- N. Duffield et al. 2009. Rule-Based Anomaly Detection on IP Flows. In IEEE INFOCOM 2009. Rio de Janeiro, Brazil, 424--432.Google Scholar
- Vincent Dumoulin et al. 2017. Adversarially Learned Inference. In Proc. ICLR 2017. Toulon, France.Google Scholar
- Ian Goodfellow et al. 2014. Generative Adversarial Nets. In Proc. NIPS 2014. Montréal, Canada.Google Scholar
- G. Kathareios et al. 2017. Catch It If You Can: Real-Time Network Anomaly Detection with Low False Alarm Rates. In Proc. 16th IEEE International Conference on Machine Learning and Applications. Cancun, Mexico, 924--929.Google ScholarCross Ref
- Tae-Young Kim and Sung-Bae Cho. 2018. Web traffic anomaly detection using C-LSTM neural networks. Expert Systems with Applications, Vol. 106 (Sept. 2018).Google Scholar
- D. Kwon et al. 2018. An Empirical Study on Network Anomaly Detection Using Convolutional Neural Networks. In Proc. IEEE CDCS 2018. Vienna, Austria.Google ScholarCross Ref
- Anukool Lakhina et al. 2004. Characterization of Network-wide Anomalies in Traffic Flows. In Proc. ACM SIGCOMM IMC'04. Taormina, Sicily, Italy, 201--206.Google Scholar
- Hung-yi Lee and Yu Tsao. 2018. Generative Adversarial Network and its Applications to Speech Signal and Natural Language Processing. http://sigport.org/2863Google Scholar
- Chunyuan Li et al. 2017. ALICE: Towards Understanding Adversarial Learning for Joint Distribution Matching. In Proc. NIPS 2017. Long Beach, USA.Google Scholar
- Takeru Miyato et al. 2018. Spectral Normalization for Generative Adversarial Networks. In Proc. ICLR 2018. Vancouver, Canada.Google Scholar
- N. Moustafa and J. Slay. 2015. UNSW-NB15: a comprehensive data set for network intrusion detection systems (UNSW-NB15 network data set). In Proc. 2015 Military Communications and Information Systems Conference. Canberra, Australia.Google Scholar
- S. Naseer et al. 2018. Enhanced Network Anomaly Detection Based on Deep Neural Networks. IEEE Access, Vol. 6 (Aug. 2018), 48231--48246.Google ScholarCross Ref
- Q. P. Nguyen et al. 2019. GEE: A Gradient-based Explainable Variational Autoencoder for Network Anomaly Detection. In Proc. IEEE CNS 2019. Washington.Google ScholarCross Ref
- Alec Radford et al. 2016. Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks. In Proc. International Conference on Learning Representation, Workshop Track. Caribe Hilton, San Juan, Puerto Rico.Google Scholar
- Thomas Schlegl et al. 2017. Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery. In Proc. International Conference on Information Processing in Medical Imaging. Boone, USA, 146--157.Google Scholar
- I. Sharafaldin et al. 2018. Toward Generating a New Intrusion Detection Dataset and Intrusion Traffic Characterization. In Proc. ICISSP 2018. Canberra, Australia.Google ScholarCross Ref
- N. Shone et al. 2018. A Deep Learning Approach to Network Intrusion Detection. IEEE Trans. Emerg. Topics Comput. Intell., Vol. 2, 1 (Feb. 2018), 41--50.Google ScholarCross Ref
- Fernando Silveira et al. 2010. ASTUTE: Detecting a Different Class of Traffic Anomalies. In Proc. ACM SIGCOMM 2010. New Delhi, India, 267--278.Google Scholar
- F. Simmross-Wattenberg et al. 2011. Anomaly Detection in Network Traffic Based on Statistical Inference and α-Stable Modeling. IEEE Trans. Depend. Sec. Comput., Vol. 8, 4 (July 2011), 494--509.Google ScholarDigital Library
- R. Sommer et al. 2010. Outside the Closed World: On Using Machine Learning for Network Intrusion Detection. In Proc. IEEE S&P 2010. Oakland, CA, USA.Google ScholarDigital Library
- Statista Research Department. 2016. Internet of Things - number of connected devices worldwide 2015--2025. White Paper.Google Scholar
- Stratosphere. 2015. Stratosphere Laboratory Datasets. https://www.stratosphereips.org/datasets-overview Retrieved March 13, 2020.Google Scholar
- Mahbod Tavallaee et al. 2009. A Detailed Analysis of the KDD CUP 99 Data Set. In Proc. IEEE CISDA'09. Ottawa, Ontario, Canada, 53--58.Google Scholar
- L.J.P. van der Maaten and G.E. Hinton. 2008. Visualizing High-Dimensional Data Using t-SNE. Journal of Machine Learning Research, Vol. 9 (Nov. 2008), 2579--2605.Google Scholar
- J. Wang et al. 2015. Statistical Traffic Anomaly Detection in Time-Varying Communication Networks. IEEE Trans. Control Netw. Syst., Vol. 2, 2 (June 2015).Google ScholarCross Ref
- M. Wang et al. 2018. Machine Learning for Networking: Workflow, Advances and Opportunities. IEEE Network, Vol. 32, 2 (Mar. 2018), 92--99.Google ScholarCross Ref
- W. Xu et al. 2018. Internet of Vehicles in Big Data Era. IEEE/CAA Journal of Automatica Sinica, Vol. 5, 1 (Jan. 2018), 19--35.Google ScholarCross Ref
- Houssam Zenati et al. 2018. Adversarially Learned Anomaly Detection. In Proc. IEEE ICDM 2018. Singapore, 727--736.Google Scholar
- Shuangfei Zhai et al. 2016. Deep Structured Energy Based Models for Anomaly Detection. In Proc. ICML 2016. New York, NY, USA, 1100--1109.Google Scholar
- Mingyi Zhu et al. 2018. Network Anomaly Detection and Identification Based on Deep Learning Methods. In CLOUD 2018. Seattle, WA, USA, 219--234.Google Scholar
- Bo Zong et al. 2018. Deep Autoencoding Gaussian Mixture Model for Unsupervised Anomaly Detection. In Proc. ICLR 2018. Vancouver, BC, Canada.Google Scholar
Index Terms
- An Empirical Study on Unsupervised Network Anomaly Detection using Generative Adversarial Networks
Recommendations
Unsupervised anomaly detection via generative adversarial networks: poster abstract
IPSN '19: Proceedings of the 18th International Conference on Information Processing in Sensor NetworksUnsupervised anomaly detection is a fundamental problem in various research areas and application domains, namely the discrimination of abnormal samples from normal samples where training data are only composed of one class (normal) while testing data ...
Detecting Anomalies in Videos using Perception Generative Adversarial Network
AbstractThis paper presents a novel end-to-end unsupervised deep learning approach for video anomaly detection. We propose to utilize the Perception Generative Adversarial Net (Perception-GAN), which is trained using the initial portion of the video. The ...
Improved autoencoder for unsupervised anomaly detection
AbstractDeep autoencoder‐based methods are the majority of deep anomaly detection. An autoencoder learning on training data is assumed to produce higher reconstruction error for the anomalous samples than the normal samples and thus can distinguish ...
Comments