Ziming Zhao
Zhaoxuan Li
ABSTRACT
To detect unknown attack traffic, anomaly-based network intrusion detection systems (NIDSs) are widely used in Internet infrastructure. However, the security communities realize some limitations when they put most existing proposals into practice. The challenges are mainly concerned with (i) fine-grained emerging attack detection and (ii) incremental updates/adaptations. To tackle these problems, we propose to decouple the need for model capabilities by transforming known/new class identification issues into multiple independent one-class learning tasks. Based on the above core ideas, we develop Trident, a universal framework for fine-grained unknown encrypted traffic detection. It consists of three main modules, i.e., tSieve, tScissors, and tMagnifier are used for profiling traffic, determining outlier thresholds, and clustering respectively, each of which supports custom configuration. Using four popular datasets of network traces, we show that Trident significantly outperforms 16 state-of-the-art (SOTA) methods. Furthermore, a series of experiments (concept drift, overhead/parameter evaluation) demonstrate the stability, scalability, and practicality of Trident.
Supplemental Material
- August A. Balkema and Laurens De Haan. Residual life time at great age. The Annals of probability, pages 792--804, 1974.Google Scholar
- Federico Barbero, Feargus Pendlebury, Fabio Pierazzi, et al. Transcending TRANSCEND: revisiting malware classification in the presence of concept drift. In IEEE Symposium on Security and Privacy, pages 805--823. IEEE, 2022.Google Scholar
- Diogo Barradas et al. Flowlens: Enabling efficient flow classification for ml-based network security applications. In NDSS. The Internet Society, 2021.Google Scholar
- Junyoung Chung, Çaglar Gülçehre, et al. Empirical evaluation of gated recurrent neural networks on sequence modeling. CoRR, abs/1412.3555, 2014.Google Scholar
- Min Du, Zhi Chen, Chang Liu, Rajvardhan Oak, and Dawn Song. Lifelong anomaly detection through unlearning. In CCS, pages 1283--1297. ACM, 2019.Google Scholar
- Min Du, Feifei Li, et al. Deeplog: Anomaly detection and diagnosis from system logs through deep learning. In CCS, pages 1285--1298. ACM, 2017.Google Scholar
- Martin Ester et al. A density-based algorithm for discovering clusters in large spatial databases with noise. In KDD, pages 226--231. AAAI Press, 1996.Google Scholar
- FireEye. M-trends reports: Insights into today's breaches and cyber attacks. https://content.fireeye.com/m-trends/rpt-m-trends-2020, 2020.Google Scholar
- Canadian Institute for Cybersecurity. Cse-cic-ids2018 on aws. [EB/OL], 2018. https://www.unb.ca/cic/datasets/ids-2018.html Accessed November 27, 2020.Google Scholar
- Canadian Institute for Cybersecurity. Intrusion detection evaluation dataset (cicids2017). [EB/OL], 2018. https://www.unb.ca/cic/datasets/ids-2017.html Accessed November 27, 2020.Google Scholar
- Chuanpu Fu et al. Detecting unknown encrypted malicious traffic in real time via flow interaction graph analysis. In NDSS. The Internet Society, 2023.Google Scholar
- Chuanpu Fu, Qi Li, Meng Shen, and Ke Xu. Realtime robust malicious traffic detection via frequency domain analysis. In CCS, pages 3431--3446. ACM, 2021.Google Scholar
- Hongyang Gao and Shuiwang Ji. Graph u-nets. In ICML, volume 97 of Proceedings of Machine Learning Research, pages 2083--2092. PMLR, 2019.Google Scholar
- Scott D Grimshaw. Computing maximum likelihood estimates for the generalized pareto distribution. Technometrics, pages 185--191, 1993.Google ScholarCross Ref
- Dongqi Han et al. Anomaly detection in the openworld: Normality shift detection, explanation, and adaptation. In NDSS. The Internet Society, 2023.Google Scholar
- Dongqi Han, Zhiliang Wang, et al. DeepAID: Interpreting and Improving Deep Learning-based Anomaly Detection in Security Applications. In CCS, pages 3197--3217. ACM, 2021.Google Scholar
- John A Hartigan and Manchek AWong. A k-means clustering algorithm. Journal of the royal statistical society. series c (applied statistics), 28(1):100--108, 1979.Google Scholar
- Jordan Holland, Paul Schmitt, Nick Feamster, and Prateek Mittal. New directions in automated traffic analysis. In CCS, pages 3366--3383. ACM, 2021.Google Scholar
- Guodong Huang et al. Efficient and low overhead website fingerprinting attacks and defenses based on TCP/IP traffic. In WWW, pages 1991--1999. ACM, 2023.Google Scholar
- James Pickands III. Statistical inference using extreme order statistics. The Annals of Statistics, pages 119--131, 1975.Google Scholar
- Peipei Jiang et al. Building in-the-cloud network functions: Security and privacy challenges. Proc. IEEE, 109(12):1888--1919, 2021.Google ScholarCross Ref
- Roberto Jordaney, Kumar Sharad, et al. Transcend: Detecting concept drift in malware classification models. In USENIX Security Symposium, pages 625--642. USENIX Association, 2017.Google Scholar
- Maciej Korczynski and Andrzej Duda. Markov chain fingerprinting to classify encrypted traffic. In INFOCOM, pages 781--789. IEEE, 2014.Google ScholarCross Ref
- Arash Habibi Lashkari, Gerard Draper-Gil, Mohammad Saiful Islam Mamun, and Ali A. Ghorbani. Characterization of tor traffic using time based features. In ICISSP, pages 253--262. SciTePress, 2017.Google ScholarCross Ref
- Nicole A. Lazar. Statistics of extremes: Theory and applications. Technometrics, 47(3):376--377, 2005.Google ScholarCross Ref
- Hongda Li, Hongxin Hu, Guofei Gu, Gail-Joon Ahn, and Fuqiang Zhang. vnids: Towards elastic security with safe and efficient virtualization of network intrusion detection systems. In CCS, pages 17--34. ACM, 2018.Google Scholar
- Wenhao Li, Xiao-Yu Zhang, et al. Prograph: Robust network traffic identification with graph propagation. IEEE/ACM Trans. Netw., pages 1--15, 2022.Google Scholar
- Junjie Liang et al. FARE: enabling fine-grained attack categorization under low-quality labeled data. In NDSS. The Internet Society, 2021.Google Scholar
- Xinjie Lin, Gang Xiong, Gaopeng Gou, Zhen Li, Junzheng Shi, and Jing Yu. ETBERT: A contextualized datagram representation with pre-training transformers for encrypted traffic classification. In WWW, pages 633--642. ACM, 2022.Google Scholar
- Chang Liu et al. Fs-net: A flow sequence network for encrypted traffic classification. In INFOCOM, pages 1171--1179. IEEE, 2019.Google Scholar
- Fei Tony Liu, Kai Ming Ting, and Zhi-Hua Zhou. Isolation forest. In ICDM, pages 413--422. IEEE Computer Society, 2008.Google ScholarDigital Library
- Zaoxing Liu et al. Jaqen: A high-performance switch-native approach for detecting and mitigating volumetric ddos attacks with programmable switches. In USENIX Security Symposium, pages 3829--3846. USENIX Association, 2021.Google Scholar
- Pierre-Francois Marteau. Random partitioning forest for point-wise and collective anomaly detection - application to network intrusion detection. IEEE Trans. Inf. Forensics Secur., 16:2157--2172, 2021.Google ScholarCross Ref
- Dongyu Meng and Hao Chen. Magnet: A two-pronged defense against adversarial examples. In CCS, pages 135--147. ACM, 2017.Google Scholar
- Yisroel Mirsky et al. Kitsune: An ensemble of autoencoders for online network intrusion detection. In NDSS. The Internet Society, 2018.Google Scholar
- Xin Mu, Kai Ming Ting, and Zhi-Hua Zhou. Classification under streaming emerging new classes: A solution using completely-random trees. IEEE Trans. Knowl. Data Eng., 29(8):1605--1618, 2017.Google ScholarDigital Library
- Sanghak Oh et al. Appsniffer: Towards robust mobile app fingerprinting against VPN. In WWW, pages 2318--2328. ACM, 2023.Google Scholar
- Jorge Luis Rivero Pérez et al. A grassmannian approach to zero-shot learning for network intrusion detection. In ICONIP (1), volume 10634 of Lecture Notes in Computer Science, pages 565--575. Springer, 2017.Google ScholarDigital Library
- Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In MICCAI (3), volume 9351 of Lecture Notes in Computer Science, pages 234--241. Springer, 2015.Google ScholarCross Ref
- Christian Rossow. Amplification hell: Revisiting network protocols for ddos abuse. In NDSS. The Internet Society, 2014.Google Scholar
- Peter J Rousseeuw. Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. Journal of computational and applied mathematics, 20:53--65, 1987.Google Scholar
- Tal Shapira et al. Flowpic: Encrypted internet traffic classification is as easy as image recognition. In INFOCOM Workshops, pages 680--687. IEEE, 2019.Google Scholar
- Alban Siffer, Pierre-Alain Fouque, et al. Anomaly detection in streams with extreme value theory. In KDD, pages 1067--1075. ACM, 2017.Google ScholarDigital Library
- Robin Sommer and Vern Paxson. Outside the closed world: On using machine learning for network intrusion detection. In IEEE Symposium on Security and Privacy, pages 305--316. IEEE Computer Society, 2010.Google ScholarDigital Library
- Zhuoxue Song, Ziming Zhao, et al. I2RNN: An Incremental and Interpretable Recurrent Neural Network for Encrypted Traffic Classification. IEEE Transactions on Dependable and Secure Computing, 2023.Google Scholar
- Alexander Strehl and Joydeep Ghosh. Cluster ensembles - A knowledge reuse framework for combining multiple partitions. J. Mach. Learn. Res., 3:583--617, 2002.Google ScholarDigital Library
- Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. Sequence to sequence learning with neural networks. In NIPS, pages 3104--3112, 2014.Google ScholarDigital Library
- Thijs van Ede et al. FlowPrint: Semi-Supervised Mobile-App Fingerprinting on Encrypted Network Traffic. In NDSS. The Internet Society, 2020.Google Scholar
- WeiWang et al. Malware traffic classification using convolutional neural network for representation learning. In ICOIN, pages 712--717. IEEE, 2017.Google ScholarCross Ref
- Zhongjie Wang et al. Symtcp: Eluding stateful deep packet inspection with automated discrepancy discovery. In NDSS. The Internet Society, 2020.Google Scholar
- Zhongjie Wang et al. Themis: Ambiguity-aware network intrusion detection based on symbolic model comparison. In CCS, pages 3384--3399. ACM, 2021.Google Scholar
- Junyuan Xie, Ross B. Girshick, and Ali Farhadi. Unsupervised deep embedding for clustering analysis. In ICML, volume 48 of JMLR Workshop and Conference Proceedings, pages 478--487. JMLR.org, 2016.Google Scholar
- Fengli Xu et al. Understanding Mobile Traffic Patterns of Large Scale Cellular Towers in Urban Environment. IEEE/ACM Trans. Netw., 25(2):1147--1161, 2017.Google ScholarDigital Library
- Weilin Xu, David Evans, and Yanjun Qi. Feature squeezing: Detecting adversarial examples in deep neural networks. In NDSS. The Internet Society, 2018.Google Scholar
- Jian Yang et al. Conditional variational auto-encoder and extreme value theory aided two-stage learning approach for intelligent fine-grained known/unknown intrusion detection. IEEE Trans. Inf. Forensics Secur., 16:3538--3553, 2021.Google ScholarCross Ref
- Limin Yang, Wenbo Guo, et al. CADE: detecting and explaining concept drift samples for security applications. In USENIX Security Symposium, pages 2327-- 2344. USENIX Association, 2021.Google Scholar
- Yijun Yang, Ruiyuan Gao, Yu Li, Qiuxia Lai, and Qiang Xu. What you see is not what the network infers: Detecting adversarial examples based on semantic contradiction. In NDSS. The Internet Society, 2022.Google Scholar
- Haozhen Zhang, Le Yu, et al. TFE-GNN: A temporal fusion encoder using graph neural networks for fine-grained encrypted traffic classification. InWWW, pages 2066--2075. ACM, 2023.Google ScholarDigital Library
- Menghao Zhang, Guanyu Li, et al. Poseidon: Mitigating Volumetric DDoS Attacks with Programmable Switches. In NDSS. The Internet Society, 2020.Google Scholar
- Ziming Zhao et al. CMD: Co-analyzed IoT Malware Detection Beyond the Network Traffic Domain. IEEE Transactions on Mobile Computing, 2023.Google Scholar
- Ziming Zhao et al. Effective DDoS Mitigation via ML-Driven In-network Traffic Shaping. IEEE Transactions on Dependable and Secure Computing, 2024.Google Scholar
- Ziming Zhao, Zhaoxuan Li, et al. DDoS Family: A Novel Perspective for Massive Types of DDoS Attacks. Comput. Secur., 2023.Google Scholar
- Ziming Zhao, Zhaoxuan Li, et al. ERNN: Error-Resilient RNN for Encrypted Traffic Detection towards Network-Induced Phenomena. IEEE Trans. Dependable Secur. Comput., 2023.Google Scholar
- Ziming Zhao, Zhaoxuan Li, Tingting Li, et al. Poster: Detecting adversarial examples hidden under watermark perturbation via usable information theory. In CCS, pages 3636--3638. ACM, 2023.Google Scholar
- Ziming Zhao, Zhaoxuan Li, Zhuoxue Song, and Fan Zhang. Work-in-progress: Towards real-time IDS via RNN and programmable switches co-designed approach. In RTSS, pages 431--434. IEEE, 2023.Google ScholarCross Ref
- Shitong Zhu, Shasha Li, et al. You do (not) belong here: detecting DPI evasion attacks with context learning. In CoNEXT, pages 183--197. ACM, 2020.Google ScholarDigital Library
Index Terms
- Trident: A Universal Framework for Fine-Grained and Class-Incremental Unknown Traffic Detection
Recommendations
Class-Incremental Novel Class Discovery
Computer Vision – ECCV 2022AbstractWe study the new task of class-incremental Novel Class Discovery (class-iNCD), which refers to the problem of discovering novel categories in an unlabelled data set by leveraging a pre-trained model that has been trained on a labelled data set ...
TRIDENT: an automated approach to traffic engineering in IP/MPLS over ASON/GMPLS networks
This paper addresses the problem of designing a capacity management/traffic engineering procedure for an IP/MPLS over ASON/GMPLS scenario. We suggest TRIDENT, a procedure whose main goal is to dynamically provide the bandwidth required to transport ...
Multi-view class incremental learning
AbstractMulti-view learning (MVL) has gained great success in integrating information from multiple perspectives of a dataset to improve downstream task performance. To make MVL methods more practical in an open-ended environment, this paper investigates ...
Highlights- A novel multi-view class incremental learning paradigm is investigated.
- Randomized representation learning ensures view-optimal working states.
- Orthogonality fusion progressively integrates each newly collected view.
- Selective ...
Comments