Skip to main content

Advertisement

SpringerLink
Log in

Intrusion detection with evolutionary learning classifier systems

  • Published:
Natural Computing Aims and scope Submit manuscript

Abstract

Evolutionary Learning Classifier Systems (LCSs) combine reinforcement learning or supervised learning with effective genetics-based search techniques. Together these two mechanisms enable LCSs to evolve solutions to decision problems in the form of easy to interpret rules called classifiers. Although LCSs have shown excellent performance on some data mining tasks, many enhancements are still needed to tackle features like high dimensionality, huge data sizes, non-uniform distribution of classes, etc. Intrusion detection is a real world problem where such challenges exist and to which LCSs have not previously been applied. An intrusion detection problem is characterised by huge network traffic volumes, difficult to realize decision boundaries between attacks and normal activities and highly imbalanced attack class distribution. Moreover, it demands high accuracy, fast processing times and adaptability to a changing environment. We present the results and analysis of two classifier systems (XCS and UCS) on a subset of a publicly available benchmark intrusion detection dataset which features serious class imbalances and two very rare classes. We introduce a better approach for handling the situation when no rules match an input on the test set and recommend this be adopted as a standard part of XCS and UCS. We detect little sign of overfitting in XCS but somewhat more in UCS. However, both systems tend to reach near-best performance in very few passes over the training data. We improve the accuracy of these systems with several modifications and point out aspects that can further enhance their performance. We also compare their performance with other machine learning algorithms and conclude that LCSs are a competitive approach to intrusion detection.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. XCS stands for eXtended Classifier System and UCS stands for Supervised learning Classifier System.

  2. an action in XCS assigns a class to its inputs. We will use class and action interchangeably hereon.

  3. another parameter, incremented every time a classifier participates in an actionset.

  4. a parameter which indicates how many virtual copies of the classifier are represented by the data structure.

  5. analogous to the actionset in XCS.

  6. See (Brown et al. 2007) on the division by summed numerosity. Informally, we found that it made no difference to our results.

  7. note that we iterate through the training set rather than sampling at random.

  8. DM = Distance Metric, BAM = Best Action Map, NGA = No GA, NMUT = No Mutation, NXO = No Crossover, FC = Fixed Covering.

References

  • Bacardit J, Butz MV (2004) Data mining in learning classifier systems: comparing XCS with GAssist. Illinois Genetic Algorithms Laboratory, University of Illinois at Urbana-Champaign. IlliGAL Report No. 2004030

  • Bernadó E (2002) Contributions to Genetic Based Classifier Systems. PhD thesis, Enginyeria i Arquitectura La Salle, Ramon Llull University, Barcelona, Spain

  • Bernadó E, Garrell JM (2003) Accuracy-based learning classifier systems: models, analysis and applications to classification tasks. Evol Comput 11(3):209–238

    Article  Google Scholar 

  • Bernadó E, Llorà X, Garrell JM (2002) XCS and GALE: a comparative study of two learning classifier systems on data mining In: IWLCS ‘01: Revised papers from the 4th International Workshop on Advances in Learning Classifier Systems, Springer-Verlag, London, UK, pp 115–132

  • Booker L (1985) Improving the performance of genetic algorithms in classifier systems. In: Grefenstette JJ (ed) Proceedings of the 1st Int Conf on genetic algorithms and their applications (ICGA). Lawrence Erlbaum Associates, pp 80–92

  • Brown G, Kovacs T, Marshall JAR (2007) UCSpv: Principled Voting in UCS Rule Populations. In: Lipson H et al (eds) To appear in the proceedings of the 2007 Genetic and Evolutionary Computation Conference (GECCO’07). ACM

  • Butz MV (2004) Rule-based evolutionary online learning systems: learning bounds, classification, and prediction. PhD thesis, University of Illinois at Urbana-Champaign

  • Butz MV, Pelikan M (2001) Analyzing the evolutionary pressures in XCS. In: Spector L et al (eds) Proceedings of the genetic and evolutionary computation conference (GECCO-2001). Morgan Kaufmann, pp 935–942

  • Butz MV, Kovacs T, Lanzi PL, Wilson SW (2001) How XCS evolves accurate classifiers. In: Spector L et al (eds) Proceedings of the Genetic and evolutionary computation conference (GECCO-2001). Morgan Kaufmann, pp 927–934

  • Butz M, Kovacs T, Lanzi PL, Wilson SW (2004) Toward a theory of generalization and learning in XCS. IEEE Trans Evol Comput 8(1):28–46

    Article  Google Scholar 

  • Dam HH, Abbass HA, Lokan C (2005) Be real! XCS with continuous valued inputs. In: Proceedings of eighth international workshop on learning classifier systems. Washington DC

  • Dawson D (2003) Improving performance in size-constrained extended classifier systems. In: Cant’u-Paz E, Foster JA, Deb K, Davis D, Roy R, O’Reilly U-M, Beyer H-G, Standish R, Kendall G, Wilson S, Harman M, Wegener J, Dasgupta D, Potter MA, Schultz AC, Dowsland K, Jonoska N, Miller J (eds) Genetic and evolutionary computation—GECCO-2003. Springer-Verlag, Berlin, pp 1870–1881

    Chapter  Google Scholar 

  • Elkan C (2000) Results of the KDD’99 classifier learning. SIGKDD Explorat Newslett 1(2):63–64

    Article  Google Scholar 

  • Ertoz L, Eilertson E, Lazarevic A, Tan PN, Kumar V, Srivastava J, Dokas P (2004) Minds-minnesota intrusion detection system. Next Generation Data Mining

  • Greenyer A (2000) Coil challenge 2000. The use of a learning classifier system JXCS. Technical Report LIACS Technical Report 2000-2009, Sentient Machine Research, Amsterdam and Leiden Institute of Advanced Computer Science, Leiden

  • Hettich S, Bay SD (1999) The UCI KDD Archive. http://www.kdd.ics.uci.edu/databases/kddcup99/kddcup99.html

  • Holland JH (1975) Adaptation in natural and artificial systems. University of Michigan Press, Ann Arbor, Republished by the MIT press, 1992

    Google Scholar 

  • Hurst J, Bull L (2003) Self-adaptation in classifier system controllers. Artif Life Robotics 5(2):109–119

    Article  Google Scholar 

  • Kovacs T (1997) XCS classifier system reliably evolves accurate, complete, and minimal representations for Boolean functions. In: Roy, Chawdhry, Pant (eds) Soft computing in engineering design and manufacturing. Springer-Verlag, London, pp 59–68

  • Kovacs T (2000) Strength or accuracy? Fitness calculation in learning classifier systems. In: Lanzi PL, Stolzmann W, Wilson SW (eds) Learning classifier systems, from foundations to applications. Springer, pp 143–160

  • Kovacs T (2006) A study of structural and parametric learning in XCS. Evol Comput 14(1):1–19

    Article  Google Scholar 

  • Lee W, Stolfo SJ (2001) A framework for constructing features and models for intrusion detection systems. ACM Trans Inf Syst Secur 3(4):227–261

    Article  Google Scholar 

  • Lee W, Stolfo S, Mok K (1999) A data mining framework for building intrusion detection models. In: Proceedings of the 1999 IEEE Symposium on Security and Privacy, May 1999, Oakland, CA, pp 120–132

  • Lippmann RP, Zissman MA (1998) 1998 DARPA/AFRL off-line intrusion detection evaluation. Dataset available at http://www.ll.mit.edu/IST/ideval/data/data_index.html

  • Mahoney MV, Chan PK (2003) Learning rules for anomaly detection of hostile network traffic. In: Third IEEE international conference on data mining, ICDM 2003, 19–22 Nov 2003, Melborne, Florida, USA, pp 601–604

  • Ning P, Xu D, Healey C, Amant R (2004) Building attack scenarios through integration of complementary alert correlation methods. In: Proceedings of the 11th annual network and distributed system security symposium (NDSS’04), 5–6 February 2004, San Diego, California, USA, pp 97–111

  • Noel S, Robertson E, Jajodia S (2004) Correlating intrusion events and building attack scenarios through attack graph distances. Computer Security Applications Conference, 2004. 20th Annual, 6–10 December 2004, Tucson, Arizona, USA, pp 350–359

  • Orriols A, Bernado-Mansilla E (2006) Class imbalance problem in UCS classifier system: fitness adaptation. The 2005 IEEE Congress on Evolutionary Computation, 8–12 July 2006, Seatle, WA, USA, pp 604–611

  • Orriols-Puig A, Bernadó-Mansilla E (2006) Bounding XCS’s parameters for unbalanced datasets. In: Proceedings of the 8th annual conference on Genetic and evolutionary computation, pp 1561–1568

  • Ramesh A, Mahesh JV (2001) PNrule: a new framework for learning classifier models in data mining (a case-study in network intrusion detection). In: Grossman R, Kumar V (eds) 1st SIAM Conference on Data Mining, 5–7 April 2001, Chicago, IL, USA

  • Sabhnani M, Serpen G (2003) Application of machine learning algorithms to KDD intrusion detection dataset within misuse detection context. In: MLMTA, 23–26 June 2006, Las Vegas, Nevada, USA, pp 209–215

  • Sampson P (1982) Fitting conic sections to “very scattered” data: an iterative refinement of the Bookstein algorithm. Comp Graphics Image Proce 18:97–108

    Article  Google Scholar 

  • Shafi K, Abbass HA, Zhu W (2006) An adaptive rule-based intrusion detection architecture. In: Proceedings of the 2006 RNSA security technology conference. Canberra, Australia, pp 307–319

  • Shafi K, Abbass HA, Zhu W (2006) The role of early stopping and population size in XCS for intrusion detection. In: Proceedings of the 6th international conference on simulated evolution and learning. Lecture Notes in Computer Science, pp 50–57

  • Stone C, Bull L (2003) For real! XCS with continuous-valued inputs. Evol Comput 11(3):299–336

    Article  Google Scholar 

  • Wada A, Takadama K, Shimohara K, Katai O (2007) Analyzing parameter sensitivity and classifier representations for real-valued XCS. In: Kovacs T et al (eds) Learning classifier systems. International Workshops, IWLCS 2003–2005. Revised selected papers, LNAI 4399, Springer, pp 1–16

  • Wilson SW (2000) Get real! XCS with continuous-valued inputs. In: Lanzi P, Stolzmann W, Wilson S (eds) Learning classifier systems, from foundations to applications, LNAI-1813. Berlin, pp 209–219

  • Wilson SW (1995) Classifier fitness based on accuracy. Evol Comput 3(2):149–175

    Article  Google Scholar 

  • Wilson SW (2001) Mining oblique data with XCS. In: Lanzi PL, Stolzmann W, Wilson SW (eds) Proceedings of the third international workshop (IWLCS-2000). Lecture Notes in Artificial Intelligence, pp 158–174

  • Witten IH, Frank E (2000) Data mining: practical machine learning tools and techniques with java implementations. Morgan Kaufmann

Download references

Acknowledgements

This work is partly funded by a University College Postgraduate Research Scholarship (UCPRS). Most of these experiments were run on AC3 supercomputing facilities.

Author information

Authors and Affiliations

  1. School of Information Technology and Electrical Engineering, UNSW@ADFA, Canberra, Australia

    Kamran Shafi, Hussein A. Abbass & Weiping Zhu

  2. Department of Computer Science, University of Bristol, Bristol, BS8 1UB, UK

    Tim Kovacs

Authors
  1. Kamran Shafi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tim Kovacs
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hussein A. Abbass
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Weiping Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kamran Shafi.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Shafi, K., Kovacs, T., Abbass, H.A. et al. Intrusion detection with evolutionary learning classifier systems. Nat Comput 8, 3–27 (2009). https://doi.org/10.1007/s11047-007-9053-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11047-007-9053-9

Keywords

Search

Navigation