Elsevier

Computers & Security

Volume 30, Issue 8, November 2011, Pages 613-624
Computers & Security

Analysis of update delays in signature-based network intrusion detection systems

https://doi.org/10.1016/j.cose.2011.08.010Get rights and content

Abstract

Network Intrusion Detection Systems (NIDS) play a fundamental role on security policy deployment and help organizations in protecting their assets from network attacks. Signature-based NIDS rely on a set of known patterns to match malicious traffic. Accordingly, they are unable to detect a specific attack until a specific signature for the corresponding vulnerability is created, tested, released and deployed. Although vital, the delay in the updating process of these systems has not been studied in depth. This paper presents a comprehensive statistical analysis of this delay in relation to the vulnerability disclosure time, the updates of vulnerability detection systems (VDS), the software patching releases and the publication of exploits. The widely deployed NIDS Snort and its detection signatures release dates have been used. Results show that signature updates are typically available later than software patching releases. Moreover, Snort rules are generally released within the first 100 days from the vulnerability disclosure and most of the times exploits and the corresponding NIDS rules are published with little difference. Implications of these results are drawn in the context of security policy definition. This study can be easily kept up to date due to the methodology used.

Introduction

It is well known that software development process is by far not perfect. The failure to follow secure coding practices along the lack of adequate and effective tools for the testing phase of the software life-cycle can lead to uncontrolled failures in running systems. In many occasions, these errors can be used by malicious users to modify the expected behavior of the original code, thus surpassing the limitations imposed by the programmer for their own benefit. Programming errors turn then into security vulnerabilities. The risk of these vulnerabilities being remotely exploited has dramatically increased over the last years due to the great development of communication networks. Former romantic hackers have been replaced by a crowd of economically driven attackers, whose efforts in breaking into systems only focus on achieving some sort of revenue. Therefore, the release of patches by software vendors as soon as new vulnerabilities are discovered is critical to ensure the availability of resources and to avoid loss of data integrity or information disclosure. Nevertheless, the inherent difficulties of the patch development process and the incapacity or unwillingness shown by vendors to release on time solutions to minimize system exposure, have triggered the security problem of windows of vulnerability, namely, the period of time vulnerabilities are disclosed but unpatched.

Vulnerability Detection Systems (VDS) are software tools used to discover vulnerable network services at risk of being exploited. The information obtained is managed by security administrators who should be willing to take actions to mitigate this risk while software updates are released. In spite of that, and not in few cases, the deployment of new patches in large network infrastructures involves such an effort that services are kept vulnerable for long periods of time.

Network Intrusion Detection Systems (NIDS) are introduced as a solution to monitor and detect attacks on vulnerable services. Although intrusion detection has become an extensive and promising research field, where anomaly detection techniques have been developed to deal with unknown vulnerabilities, misuse detection approaches based on signatures – rules written from intrusion trails – are the current standard in real scenarios. Commercial NIDS have evolved into Network Intrusion Prevention Systems (NIPS), which are capable of blocking ongoing detected attacks and introduce non-signature detection capabilities based on heuristics and behavior analysis. Nevertheless, administrators do not go beyond the use of vendor-recommended signatures in approximately 65% of new deployments. Neither are blocking capabilities used in more than 25% of deployments and approximately only 10% of enterprises make an advanced use of detection engines, developing custom signatures and using anomaly detection techniques in order to identify unknown attacks (Young and Pescatore, 2010). The main reason is the high number of false alarms that anomaly detectors present, which can cause undesired traffic blocking and increase the difficulty to keep the normal behavior of a system up to date. As a consequence, every commercial network prevention system deployed in a corporative environment is to a large extent based on the signature detection paradigm and their performance rely thus, on the development of detection rules by security researchers. As this task requires considerable effort and extensive previous testing to avoid false alarms and inconsistencies, performance of signature-based NIDS depends not only on high detection ratios, but also on the time it takes developers to release a new detection rule when a new vulnerability is disclosed. Nowadays, every corporate security program takes into account the need of an intrusion detection system to increase visibility of events in networks but, not in few cases, the mere fact of deploying the system causes network administrators to become overconfident about the level of protection. If new detection rules are not released on time and the security perception is strongly based on the NIDS performance, the risk of missing a successful attack highly increases.

Some research have been conducted in measuring and comparing the patch development process of vendors (Frei and Tellenbach, 2008; Schryen, 2009; Liu et al., 2009) and numerous studies have examined different approaches for evaluating NIDS effectiveness (Ulvila, 2004; Gu et al., 2006; Orfila et al., 2008; Orfila et al.,; Chen et al., 2010). However, vulnerable time windows caused by delays in the updating process of signature-based NIDS have not been yet explored and quantified as a performance metric. The goal of this research is to fill this gap and apply a formal methodology to evaluate a widely deployed open source signature-based NIDS (i.e. Snort (Snort intrusion detection system, 2010)) by means of measuring the update delays of its detection rules, namely, the time interval between the release of a signature and the related security event. Accordingly, a time-span is statistically modeled first from the existing delay between vulnerability disclosures and specific rule releases. Then, the release of software patches is compared to NIDS updates. Following, the confrontation is done against the updates of a popular vulnerability scanner (i.e. Nessus (Nessus vulnerability scanner, 2010)) and finally a comparison is made between the publication of exploits and the corresponding NIDS rules in order to measure the corresponding NIDS update delay. This comprehensive work allows us to draw some conclusions, such as answering the question of how useful signature-based NIDS can be to mitigate risks. Fig. 1 depicts a general arrangement of the mentioned events. The relationships between their occurrence dates are quantified and statistically estimated in this work.

The rest of this paper is organized as follows. In Section 2 we establish the research context of this study and describe the related work in the field of NIDS evaluation. The goal pursued, the analyzed variables and the methodology followed to obtain valuable data is exposed in Section 3. Formal modeling of this data and the obtained numerical results are presented and discussed in Section 4. We gather the conclusions regarding the performance of the NIDS under study in Section 5. Finally, future work is introduced in Section 6.

Section snippets

Related work

In 2002, Lippmann and Webster presented one of the first attempts to analyze the interaction between software patches, VDS and signature-based NIDS (Lippmann and Webster, 2002). They introduced concepts such as “window of vulnerability” and “window of visibility”, namely the time interval when a compromised system can be detected by an IDS. Their work concludes that software patches, used to prevent vulnerabilities from being exploited, are available before or simultaneously with NIDS

Goal and methodology

First, this section presents the goal pursued by this research, then exposes the different variables that have been used as reference for calculations and finally establishes the experimental setup designed in order to gather correlated data from different sources.

Results & data modeling

In this Section, the update delays calculated from each of the analyzed variables are statically modeled and numerical results are presented. The measures directly obtained by the scripts have been depicted in several histograms allowing us to estimate the best fitting probability density function and its parameters using the Kolmogórov–Smirnov test (Massey, 1951). Table 2 shows how measures are calculated. The titem – with item {CVE, Bugtraq, patch, plugin, exploit, snort } – indicates the

Conclusions

In this paper we have characterized the time response of the Snort rule release process. This task has been done through the comparison of its update time versus several related security events such as vulnerability disclosures, software security patch releases, Nessus plugin releases and exploit publications. The time interval between each of these events and the release of NIDS rule updates have been defined as different NIDS update delays. To the best of our knowledge, this is the first

Future work

Lippmann and Webster (Lippmann and Webster, 2002) tried in 2002 to determine the role of the NIDS and its performance against the VDS and the patching process timing. Although our findings agree with some of their conclusions, the size of the analyzed sample and the statistical model we present represent a quantitative and a qualitative improvement. According to our results and their implications, we encourage to build a security system that gathers information from the services the

Hugo Gascon is an associate Security Researcher at the Computer Science Department of Carlos III University of Madrid. He holds a M.Sc. in Telecommunication Engineering from the same university and has previously conducted research in routing and network infrastructure. He currently combines research at the Information Technology Security Group with his work as a security consultant, designing secure network architectures and evaluating network security by means of penetration testing. His

References (35)

  • E. Biermann

    A comparison of intrusion detection systems

    Computers & Security

    (2001)
  • D. Nizovtsev et al.

    To disclose or not? An analysis of software user behavior

    Information Economics and Policy

    (2007)
  • A. Orfila et al.

    Autonomous decision on intrusion detection with trained BDI agents

    Computer Communications

    (2008)
  • U. Aickelin et al.

    Rule generalisation in intrusion detection systems using snort

    International Journal of Electronic Security and Digital Forensics

    (2007)
  • A. Arora et al.

    Does information security attack frequency increase with vulnerability disclosure? An empirical analysis

    Information Systems Frontiers

    (2006)
  • Bugtraq vulnerability database

    (2010)
  • H. Cavusoglu et al.

    Efficiency of vulnerability disclosure mechanisms to disseminate vulnerability knowledge

    IEEE Transactions on Software Engineering

    (2007)
  • Z. Chen et al.

    A pragmatic methodology for testing intrusion prevention systems

    The Computer Journal

    (2008)
  • Z. Chen et al.

    Research on evaluation method of intrusion detection system

  • Easyfit distribution fitting software

    (2010)
  • J. Flowers

    US Patent 7,509,681: interoperability of vulnerability and intrusion detection systems

    (2009)
  • Frei S, Tellenbach, B. 0-Day patch-Exposing vendors (In)security performance. In: BlackHat Europe;...
  • Frei S, May M, Fiedler U. Large-scale vulnerability analysis. In: SIGCOMM workshop on Large-scale attack defense; 2006....
  • Gu G, Fogla P, Dagon D, Lee W, Škorić B. Measuring intrusion detection capability: an information-theoretic approach....
  • R.J. Gula et al.

    US Patent 7,761,918: system and method for scanning a network

    (2010)
  • International electrotechnical commission

    (2011)
  • International organization for standardization

    (2011)
  • Cited by (0)

    Hugo Gascon is an associate Security Researcher at the Computer Science Department of Carlos III University of Madrid. He holds a M.Sc. in Telecommunication Engineering from the same university and has previously conducted research in routing and network infrastructure. He currently combines research at the Information Technology Security Group with his work as a security consultant, designing secure network architectures and evaluating network security by means of penetration testing. His research is focused on intrusion detection systems, evasion techniques and vulnerability analysis and exploitation.

    Agustín Orfila is Associate Professor at the Computer Science Department at Carlos III University of Madrid. He received his B.S. and M.S. in Physics at Complutense University of Madrid (Spain) and his Ph.D in Computer Science at Universidad Carlos III de Madrid (Spain). Dr. Orfila's main research interests lie in the area of network and computer security, focusing on Intrusion Detection Systems and Radio Frequency Identification Systems. He has several international publications and he has been serving as a reviewer for several ISI journals. He is also active in different research projects related to Security on Information Technologies.

    Jorge Blasco is Teaching Assistant at the Computer Science Department of Carlos III University of Madrid. He holds a B.Sc. in Computer Engineering (2007) and a M.Sc. in Computer Science and Technology (2008) from the same university. Currently, he belongs to the Information Technology Security Group, where he is performing his Ph.D. studies. His research is focused on steganography, steganalysis and information leakage protection mechanisms.

    View full text