Distributed real-time SlowDoS attacks detection over encrypted traffic using Artificial Intelligence

https://doi.org/10.1016/j.jnca.2020.102871Get rights and content

Abstract

SlowDoS attacks exploit slow transmissions on application-level protocols like HTTP to carry out denial of service against web-servers. These attacks are difficult to be detected with traditional signature-based intrusion detection approaches, even more when the HTTP traffic is encrypted. To cope with this challenge, this paper describes and AI-based anomaly detection system for real-time detection of SlowDoS attacks over application-level encrypted traffic. Our system monitors in real-time the network traffic, analyzing, processing and aggregating packets into conversation flows, getting valuable features and statistics that are dynamically analyzed in streaming for AI-based anomaly detection. The distributed AI model running in Apache Spark-streaming, combines clustering analysis for anomaly detection, along with deep learning techniques to increase detection accuracy in those cases where clustering obtains ambiguous probabilities. The proposal has been implemented and validated in a real testbed, showing its feasibility, performance and accuracy for detecting in real-time different kinds of SlowDoS attacks over encrypted traffic. The achieved results are close to the optimal precision value with a success rate 98%, while the false negative rate takes a value below 0.5%.

Introduction

Denial-of-Service-(DoS) based attacks are continuously evolving increasing its complexity and range, thereby making more and more difficult to perform a timely and accuracy detection. Traditional DoS attacks, that aims to incapacitate a resource from serving its genuine clients, have been intensively studied in the literature through different schemes intended to protect network infrastructures (Yuan and Mills, 2005). DoS attacks, concretely application-layer DoS attacks are recently getting research attraction (Zeebaree et al., 2018), since they are able to compromise a web-server through other means beyond traditional ones such as network flooding or exhausting server's resources such as sockets, memory, CPU, and I/O bandwidth. In particular, SlowDoS attacks (Cambiaso et al., 2013) are a type of application-layer DoS using low-rate packet transmission (Aiello et al., 2014). Namely, most of the SlowDoS attacks, such as Slowris or SlowPost, exploits HTTP protocol, widely adopted in application-layer services, by sending incomplete http requests, or keeping the connection with the server busy through sending the HTTP posts using slow ratio and without reaching content-length values.

Furthermore, the prominence of the Internet of Things (Perwej et al., 2019) is introducing millions of interconnected network devices which suggest new scenarios and connectivity models that are increasing the attack surface. Since neither high-performance systems nor immense network bandwidth are needed to perform a SlowDoS attack, simple constrained IoT devices are suitable to carry out a denial. Indeed, the biggest advantage of SlowDoS attacks relies on the scarce bandwidth needed, meaning few IoT attackers –i.e bots– using a low-rate packet transmission can overwhelm their victim.

SlowDoS attacks are arduous to be detected using traditional signature-based intrusion detection systems, as it becomes hard to inspect and match an attack signature (Khan et al., 2017) affecting legitimate application-level HTTP requests. Anomaly-based Network Intrusion Detection Systems (NIDS) analyze incoming network traffic in order to detect any invasive or illegitimate behaviour. This approach performs properly for detecting not only known attacks, but also unknown or “zero-day” attacks (Khan et al., 2017; Thakare and Kaur, 2017). There are some research works that focus on detecting SlowDoS attacks using probability distributions (Tripathi et al., 2016) or (Zhou et al., 2018) that analyzes HTTP messages. However, those proposals are not applicable over encrypted application traffic or have been not properly validated in a real testbed dynamically monitoring and detecting SlowDoS in real-time.

To fill this gap, this paper proposes a real-time, Artificial Intelligence (AI)-based system for dynamic anomaly-based detection of application-level SlowDoS attacks over encrypted traffic. Our system monitors in real-time the network traffic, processing and aggregating packets in conversation flows, extracting additional meaningful network features as statistics that are dynamically analyzed in streaming for distributed AI-based anomaly detection. To this aim, our AI model, running in a distributed way in Apache Spark-streaming, combines clustering analysis and deep learning techniques to increase accuracy in attack detection. This system consists of two algorithms aimed to capture normal behaviour patterns and detect anomalies. The first one is based on clustering models and the second on exploits Deep Learning techniques intended to detect attacks with ambiguous probabilities in the clustering.

The proposed anomaly-detection system – and its associated AI model – has been implemented and extensively validated in a real testbed by running different kinds of SlowDoS attacks. These tests show its performance to monitor, aggregate and analyze real-time network traffic in a distributed way, as well as the capability of detecting in streaming the attacks run over application-level encrypted traffic with high precision average accuracy 0.98 for different SlowDoS attacks.

The rest of this paper is structured as follows. Section 2 describes the related work and Section 3 the background in Intrusion Detection System (IDS) and SlowDos attacks. Section 4 presents the AI-Based security framework. Section 5 delves into the AI-based model. Extensive performance and accuracy evaluation is done in section 6. Finally, section 7 concludes the paper.

Section snippets

Related work

There are several recent surveys that analyze the application of artificial intelligence mechanisms (Hatcher and Yu, 2018), on the security of machine learning in malware C&C detection (Gardiner and Nagaraja, 2016; Kedziora et al., 2019; Gibert et al., 2020), for anomaly detection (Chalapathy and Chawla, 2019) and specific surveys (Khalaf et al., 2019) that focuses on detecting DoS attacks and enrich defense mechanisms using AI. Another survey (Khan et al., 2017) introduces a heuristic-based

Intrusion Detection systems (IDS)

Traditional Intrusion Detection Systems (IDS) follow a signature-based approach to detect DoS, monitoring ongoing network traffic in real-time and looking for sequences or patterns that may match an attack signature previously introduced. Signatures can be identified based on packet headers and network addresses, that contain sequences of data that are known to be a particular familiar attack. Signature-based IDS like Snort a widely-used solution since it is based on open source software and

AI-based cyber-attacks detection framework for application-level encrypted traffic

This section describes the AI-based framework aimed to detect in real-time SlowDoS attacks over encrypted application-level traffic. The infrastructure is an anomaly-based Network Intrusion Detection System (NIDS) that continuously monitors the network traffic, inspecting and aggregating traffic in real-time and following an AI-based approach for detecting DoS-based attacks over encrypted traffic. Furthermore, our framework, unlike traditional signature-based IDS, has the ability to detect

Artificial intelligence for SlowDoS attacks detection

This section presents the developed AI model devised and optimized to detect SlowDoS attack. Our model consists of an algorithm in five phases: preprocessing, clustering, histogram matrix, Deep Learning and detection. The AI-model combines clustering with deep learning techniques to increase overall accuracy. The clustering technique alone did not reach the desired level of precision, since this does not take into account aspects supported by DL module, such as the correlation between variables.

Implementation and evaluation

This section introduces the tools used to implement the 3-layer framework presented previously, the process for obtaining our data-set and the evaluation. The data-set has been obtained using specific monitoring of diverse SlowDoS attacks in our streaming platform. Notice that actual security datasets public available (e.g. NSL-KDD), do not consider SlowDoS attacks. In addition, this section also presents the quality results focused on evaluating the accuracy of the AI-based model using the

Conclusions and future work

This paper has described a distributed AI-based anomaly detection system aimed to detect SlowDoS attacks at application-level over encrypted traffic. The layered framework deeps inspect the network traffic in real-time, aggregating network flows and conversations, extracting valuable features used by the distributed AI-model to detect dynamically attacks.

The solution has been implemented, and extensively validated in a real testbed running different kind of SlowDoS attacks. The paper has shown

Credit author statement

Norberto Garcia: Investigation, Software, Data curation, Writing- Original draft preparation. Tomas Alcaniz: Data curation, Software, Writing- Original draft preparation. Aurora Gonzalez-Vidal: Investigation, Methodology, Writing- Original draft preparation, Supervision. Jorge Bernal Bernabe: Conceptualization, Investigation, Methodology, Writing- Original draft preparation, Supervision. Diego Rivera: Software, Writing- Original draft preparation. Antonio Skarmeta: Conceptualization, Writing-

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgements

This work has been sponsored by the European Commission through H2020 CyberSec4Europe project (contract 830929), H2020 INSPIRE-5Gplus project (contract 871808) and H2020 IoTCrawler project (contract 779852). It has been also partially funded by AXA Postdoctoral Scholarship awarded by the AXA Research Fund, ERDF funds of project UMU-CAMPUS LIVING LAB EQC2019-006176-P. It was also co-financed by the European Social Fund (ESF) and the Youth European Initiative (YEI) under the Spanish Seneca

Norberto García Marín received the B.Sc degree in Computer Science from the University of Murcia. He is currently pursuing the M.Sc in New Technologies in Computer Science in the University of Murcia, specialized in networks and telematics. He has been working in projects such as CyberSec4Europe. His main interests include security, privacy and artificial intelligence.

References (47)

  • E. Cambiaso et al.

    Slow dos attacks: definition and categorisation

    Int. J. Trust Manag. Comput. Commun.

    (2013)
  • R. Chalapathy et al.

    Deep Learning for Anomaly Detection: A Survey

    (2019)
  • B. Cusack et al.

    Detecting slow DDos attacks on mobile devices

  • J. Dromard et al.

    Online and scalable unsupervised network anomaly detection method

    IEEE Trans. Netw. Serv. Manag.

    (2016)
  • Enea qosmos ixengine datasheet

  • E. Eskin et al.

    A geometric framework for unsupervised anomaly detection

  • F. Falco et al.

    Quantitative comparison of unsupervised anomaly detection algorithms for intrusion detection

  • J. Gardiner et al.

    On the security of machine learning in malware C&C detection: a survey

    ACM Comput. Surv.

    (2016)
  • M. Goldstein et al.

    A comparative evaluation of unsupervised anomaly detection algorithms for multivariate data

    PloS One

    (2016)
  • A. Gonzlez-Vidal et al.

    Iot for water management: towards intelligent anomaly detection

  • W.G. Hatcher et al.

    A survey of deep learning: platforms, applications and emerging research trends

    IEEE Access

    (2018)
  • J. Heaton

    Introduction to Neural Networks with Java

    (2008)
  • M. Kedziora et al.

    Malware detection using machine learning algorithms and reverse engineering of android java code

    Int. J. Netw. Secur. Appl. (IJNSA)

    (2019)
  • Cited by (36)

    • Intrusion detection system in distributed cloud computing: Hybrid clustering and classification methods

      2023, Measurement: Sensors
      Citation Excerpt :

      However, these models may struggle in modern multi-cloud environments, which features dynamic attacks and closely related attacks. The KDDcup99 dataset is also getting old, so it may not accurately interpret how networks are used today [20]. According to Ref. [21], SVM is a method that is used in data mining to extract predicted data.

    • Automatic, verifiable and optimized policy-based security enforcement for SDN-aware IoT networks

      2022, Computer Networks
      Citation Excerpt :

      In the workflow of the proposed approach, by providing and deploying different types of monitoring and analysis tools, the infrastructure is able to detect a big amount of different kinds of attacks as well as notify the system for strange behaviors. For this detection phase, the proposal can rely on different existing monitoring agents and tools [8,45], such as Montimage Monitoring Tool (MMT)-probe [46], IoT brokers, Intrusion Detection System (IDS) instances (e.g., Snort), multiple incident detectors (e.g., MMT-security), event-based security tools such as XL-SIEM or Apache Storm, and AI-based attack detection modules such as [45]. In particular, the attacks that can be identified for this approach are the attacks that are carried out throughout the network.

    • A new detection method for LDoS attacks based on data mining

      2022, Future Generation Computer Systems
    • Evaluating Federated Learning for intrusion detection in Internet of Things: Review and challenges

      2022, Computer Networks
      Citation Excerpt :

      In the context of IoT, recent efforts have been proposed by considering specific IoT devices and technologies [3]. Indeed, the use of Deep Learning (DL) techniques has been recently evaluated through different types of neural networks for the detection of different attacks in such scenarios [29–31]. Despite these efforts, most of the proposed IDS approaches for IoT are based on centralized approaches in which devices send their local data to data centers in the cloud or servers with considerable computing capabilities to be analyzed through ML/DL techniques [11].

    • Adversarial Deep Learning approach detection and defense against DDoS attacks in SDN environments

      2021, Future Generation Computer Systems
      Citation Excerpt :

      Security mechanisms are applied for detecting and preventing network systems from the actions of malicious agents. For this purpose, Network Intrusion Detection System (NIDS) is a widely used technique against Internet-based attacks [9,10]. NIDS provides a set of tools able to recognize abnormal network behaviors automatically.

    View all citing articles on Scopus

    Norberto García Marín received the B.Sc degree in Computer Science from the University of Murcia. He is currently pursuing the M.Sc in New Technologies in Computer Science in the University of Murcia, specialized in networks and telematics. He has been working in projects such as CyberSec4Europe. His main interests include security, privacy and artificial intelligence.

    Tomás Alcañiz Cascales graduated in Physics from the University of Murcia in 2017 and Master's Degree in Big Data Analysis Technologies. Currently he is collaborating on the European H2020 research project IoTCrawler. His main research interests are related to data science, artificial intelligence and machine learning.

    Dra. Aurora Gonzalez Vidal graduated in Mathematics from the University of Murcia in 2014. In 2015 she got a fellowship to work in the Statistical Division of the Research Support Service, where she specialized in Statistics and Data Analysis. Afterwards, she studied a Big Data Master. In 2019, she got a Ph.D. in Computer Science, focusing her research on Data Analysis for Energy Efficiency. Currently, she is a postdoctoral researcher at the University of Murcia. She has collaborated in several national and European projects such as ENTROPY, IoTCrawler, and DEMETER. Her research covers machine learning in IoT-based smart environments, missing values imputation, and time series segmentation. She is the president of the R Users Association UMUR

    Dr. Jorge Bernal Bernabe received the MSc, Master and PhD in Computer Science from the University of Murcia. Currently, he is a postdoctoral researcher in the University of Murcia. He has published over 50 papers in international conferences and journals. He has been involved in the scientific committee of numerous conferences. During the last years, he has been working in several European research projects such as Inter-Trust, SocIoTal, ARIES, OLYMPUS, ANASTACIA, INSPIRE-5G, CyberSec4Europe. His scientific activity is mainly devoted to the security, trust and privacy management in distributed systems and IoT.

    Dr. Diego Rivera received the bachelor of computer sciences and computing engineering degree from the University of Chile, Santiago, Chile, in 2011 and 2013, respectively, and the Ph.D. degree focused on the analysis of the quality of experience for OTT services from the Université Paris-Saclay, Paris, France, in 2016. From 2012 to 2013, he was a Research Assistant with NIC Chile Research Labs, University of Chile, where he was involved with research on concurrency issues in Linux UDP sockets. He is currently a Research and Project Engineer with Montimage, Paris, where he is involved in the development of security monitoring solutions for Internet of Things-based cyber-physical systems.

    Dr Antonio Skarmeta received the M.S. degree in Computer Science from the University of Granada and B.S. (Hons.) and the Ph.D. degrees in Computer Science from the University of Murcia Spain. Since 2009 he is Full Professor at the same department and University. Antonio F. Skarmeta has worked on different research projects in the national and international area in the networking, security and IoT area, like Euro6IX, ENABLE, DAIDALOS, SWIFT, SEMIRAMIS, SMARTIE, SOCIOTAL, IoT6 ANASTACIA, CyberSec4Europe. His main interested is in the integration of security services, identity, IoT and Smart Cities. He has been heading of the research group ANTS since its creation on 1995. He has published over 200 international papers and being member of several program committees.

    View full text