Skip to main content
SpringerLink
Log in

A survey on artificial intelligence techniques for security event correlation: models, challenges, and opportunities

  • Published:
Artificial Intelligence Review Aims and scope Submit manuscript

Abstract

Information systems need to process a large amount of event monitoring data. The process of finding the relationships between events is called correlation, which creates a context between independent events and previously collected information in real time and normalizes it for subsequent processing. In cybersecurity, events can determine the steps of attackers and can be analyzed as part of a specific attack strategy. In this survey, we present the systematization of security event correlation models in terms of their representation in AI-based monitoring systems as: rule-based, semantic, graphical and machine learning based-models. We define the main directions of current research in the field of AI-based security event correlation and the methods used for the correlation of both single events and their sequences in attack scenarios. We also describe the prospects for the development of hybrid correlation models. In conclusion, we identify the existing problems in the field and possible ways to overcome them.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  • Abdullayeva FJ (2021) Advanced persistent threat attack detection method in cloud computing based on autoencoder and softmax regression algorithm. Array 10(100):067

    Google Scholar 

  • Ahmad F (2017) Web application firewall. https://github.com/faizann24/Fwaf-Machine-Learning-driven-Web-Application-Firewall (Accessed 03-July-2022)

  • Albasheer H, Md Siraj M, Mubarakali A et al (2022) Cyber-attack prediction based on network intrusion detection systems for alert correlation techniques: a survey. Sensors 22(4):1494

    Google Scholar 

  • Almseidin M, Piller I, Al-Kasassbeh M et al (2019) Fuzzy automaton as a detection mechanism for the multi-step attack. Int J Adv Sci Eng Inf Technol 9(2):575–586

    Google Scholar 

  • Alshamrani A, Myneni S, Chowdhary A et al (2019) A survey on advanced persistent threats: techniques, solutions, challenges, and research opportunities. IEEE Commun Surve Tutor 21(2):1851–1877

    Google Scholar 

  • Amza C, Cecchet E, Chanda A, et al (2002) Specification and implementation of dynamic web site benchmarks. In: 2002 IEEE international workshop on workload characterization, pp 3–13

  • Asuncion A, Newman D (2007) Uci machine learning repository. http://archive.ics.uci.edu/ml/index.php, (accessed 03-July-2022)

  • Bajtoš T, Sokol P, Mézešová T (2020) Multi-stage cyber-attacks detection in the industrial control systems. Recent developments on industrial control systems resilience. Springer, Cham, pp 151–173

    Google Scholar 

  • Barrett M (2018) Framework for improving critical infrastructure cybersecurity. Version 1.1. NIST Special Publication

  • Barzegar M, Shajari M (2018) Attack scenario reconstruction using intrusion semantics. Expert Syst Appl 108:119–133

    Google Scholar 

  • Beer F, Bühler U (2017) Feature selection for flow-based intrusion detection using rough set theory. In: 2017 IEEE 14th international conference on networking. Sensing and Control (ICNSC), IEEE, pp 617–624

  • Bénard C, Biau G, Da Veiga S et al (2021) SIRIUS: stable and interpretable rule set for classification. Electron J Stat 15(1):427–505

    MathSciNet  MATH  Google Scholar 

  • Bhattacharjya D, Shanmugam K, Gao T, et al (2020) Event-driven continuous time Bayesian networks. In: Proceedings of the AAAI conference on artificial intelligence, pp 3259–3266

  • Bojanowski P, Grave E, Joulin A et al (2017) Enriching word vectors with subword information. Trans Assoc Comput Linguist 5:135–146

    Google Scholar 

  • Bollacker K, Evans C, Paritosh P, et al (2008) Freebase: a collaboratively created graph database for structuring human knowledge. In: Proceedings of the 2008 ACM SIGMOD international conference on Management of data, pp 1247–1250

  • Chang YC, Wang SD (2016) The concept of attack scenarios and its applications in Android malware detection. In: 2016 IEEE 18th international conference on high performance computing and communications, IEEE, pp 1485–1492

  • chen h, xiao r, jin s (2020a) real-time detection of cloud tenant malicious behavior based on CNN. In: 2020 IEEE international conference on parallel & distributed processing with applications, big data & cloud computing, sustainable computing & communications, social computing & networking (ISPA/BDCloud/SocialCom/SustainCom), IEEE, pp 998–1005

  • Chen R, Zhang S, Li D, et al (2020b) Logtransfer: Cross-system log anomaly detection for software systems with transfer learning. In: 2020 IEEE 31st international symposium on software reliability engineering (ISSRE), IEEE, pp 37–47

  • Cheng H, Xie Z, Shi Y et al (2019) Multi-step data prediction in wireless sensor networks based on one-dimensional CNN and bidirectional LSTM. IEEE Access 7:117,883-117,896

    Google Scholar 

  • Cichonski P, Millar T, Grance T et al (2012) Computer security incident handling guide. Special Publication 800-61. NIST Spec Publ 800(61):1–147

    Google Scholar 

  • Cinque M, Della Corte R, Pecchia A (2020) Contextual filtering and prioritization of computer application logs for security situational awareness. Future Gener Comput Syst 111:668–680

    Google Scholar 

  • CNSS (2022) Committee on National Security Systems (CNSS) Glossary. Committee on National Security Systems Instruction (CNSSI) No. 4009. Committee on National Security Systems

  • Contagio Mobile (2011) Contagiodump. mobile malware sample. http://contagiominidump.blogspot.com/. (accessed 03-July-2022)

  • Cook K, Grinstein G, Whiting M, et al (2012) Vast challenge 2012: visual analytics for big data. In: 2012 IEEE conference on visual analytics science and technology (VAST), IEEE, pp 251–255

  • Creech G, Hu J (2013) A semantic approach to host-based intrusion detection systems using contiguous and discontiguous system call patterns. IEEE Trans Comput 63(4):807–819

    MathSciNet  MATH  Google Scholar 

  • DARPA TC (2020) Transparent computing engagement 3 data release. https://github.com/darpa-i2o/Transparent-Computing, (accessed 03-July-2022)

  • DEF CON Communications, Inc. (2021) Defcon21 ctf dataset. https://media.defcon.org/DEF%20CON%2021/. (accessed 03-July-2022)

  • Dempster AP (2008) Upper and lower probabilities induced by a multivalued mapping. Classic works of the Dempster–Shafer theory of belief functions. Springer, Berlin, Heidelberg, pp 57–72

    Google Scholar 

  • Deng A, Hooi B (2021) Graph neural network-based anomaly detection in multivariate time series. Proc. AAAI Conf. Artif. Intell. 35(5):4027–4035

    Google Scholar 

  • Deng S, Zhang N, Li L, et al (2021) OntoED: Low-resource event detection with ontology embedding. In: Proceedings of the 59th annual meeting of the association for computational linguistics, pp 2828–2839

  • Devlin J, Chang MW, Lee K, et al (2019) BERT: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 conference of the north american chapter of the association for computational linguistics: human language technologies (NAACL-HLT), pp 4171–4186

  • Dhaou A, Bertoncello A, Gourvénec S, et al (2021) Causal and interpretable rules for time series analysis. In: Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining, pp 2764–2772

  • Do Xuan C, Dao MH (2021) A novel approach for APT attack detection based on combined deep learning model. Neural Comput Appl 33(20):13,251-13,264

    Google Scholar 

  • Du M, Li F, Zheng G, et al (2017) Deeplog: Anomaly detection and diagnosis from system logs through deep learning. In: Proceedings of the 2017 ACM SIGSAC conference on computer and communications security, pp 1285–1298

  • Dwivedi N, Tripathi A (2015) Event correlation for intrusion detection systems. In: 2015 IEEE international conference on computational intelligence & communication technology, IEEE, pp 133–139

  • Eckmann ST, Vigna G, Kemmerer RA (2002) Statl: an attack language for state-based intrusion detection. J Comput Secur 10(1–2):71–103

    Google Scholar 

  • Force JT (2018) Risk management framework for information systems and organizations. Special publication 800–37 rev. 2. NIST Spec Publ 800:1–37

    Google Scholar 

  • Garcia S, Grill M, Stiborek J et al (2014) An empirical comparison of botnet detection methods. Comput Secur 45:100–123

    Google Scholar 

  • Ghafouri A, Vorobeychik Y, Koutsoukos X (2018) Adversarial regression for detecting attacks in cyber-physical systems. In: Proceedings of the 27th International joint conference on artificial intelligence. AAAI Press, Stockholm, IJCAI’18, pp 3769–3775

  • Giménez CT, Villegas AP, Marañón GÁ (2010) Http data set csic 2010. https://www.isi.csic.es/dataset/. (accessed 03-July-2022)

  • Glasser J, Lindauer B (2013) Bridging the gap: A pragmatic approach to generating insider threat data. In: 2013 IEEE security and privacy workshops, IEEE, pp 98–104

  • Goh J, Adepu S, Junejo KN, et al (2016) A dataset to support research in the design of secure water treatment systems. In: International conference on critical information infrastructures security. Springer, Cham, pp 88–99

  • Guan S, Jin X, Wang Y, et al (2019) Link prediction on n-ary relational data. In: Proceedings of the 28th International Conference on World Wide Web (WWW’19), pp 583–593

  • Guo H, Yuan S, Wu X (2021) LogBERT: Log anomaly detection via BERT. In: 2021 international joint conference on neural networks (IJCNN), pp 1–8

  • Haas S, Fischer M (2019) On the alert correlation process for the detection of multi-step attacks and a graph-based realization. ACM SIGAPP Appl Comput Rev 19(1):5–19

    Google Scholar 

  • Hamed T, Ernst JB, Kremer SC (2018) A survey and taxonomy of classifiers of intrusion detection systems. Computer and network security essentials, pp 21–39

  • Han X, Pasquier T, Bates A, et al (2020) UNICORN: Runtime provenance-based detector for advanced persistent threats. In: Network and distributed system security symposium, pp 1–18

  • Hassan WU, Guo S, Li D, et al (2019) Nodoze: Combatting threat alert fatigue with automated provenance triage. In: Network and distributed systems security symposium, pp 1–15

  • Hassan WU, Noureddine MA, Datta P, et al (2020) OmegaLog: High-fidelity attack investigation via transparent multi-layer log analysis. In: Network and distributed system security symposium, pp 1–16

  • Heigl M, Weigelt E, Urmann A et al (2021) Exploiting the outcome of outlier detection for novel attack pattern recognition on streaming data. Electronics 10(17):2160

    Google Scholar 

  • Holgado P, Villagrá VA, Vazquez L (2017) Real-time multistep attack prediction based on hidden Markov models. IEEE Trans Depend Secure Comput 17(1):134–147

    Google Scholar 

  • Hossain M, Xie J (2020) Third eye: context-aware detection for hidden terminal emulation attacks in cognitive radio-enabled IoT networks. IEEE Trans Cogn Commun Netw 6(1):214–228

    Google Scholar 

  • Hostiadi DP, Susila MD, Huizen RR (2019) A new alert correlation model based on similarity approach. In: 2019 1st international conference on cybernetics and intelligent system (ICORIS), IEEE, pp 133–137

  • Huang L, Ji H, Cho K, et al (2018) Zero-shot transfer learning for event extraction. In: Proceedings of the 56th annual meeting of the association for computational linguistics, pp 2160–2170

  • Huang Y, Sun H, Xu K et al (2021) CoRelatE: learning the correlation in multi-fold relations for knowledge graph embedding. Knowl-Based Syst 213(106):601

    Google Scholar 

  • Husák M, Komárková J, Bou-Harb E et al (2018) Survey of attack projection, prediction, and forecasting in cyber security. IEEE Commun Surv Tutor 21(1):640–660

    Google Scholar 

  • ISO (2015a) CISO/IEC 27039:2015. Information technology - Security techniques - Selection, deployment and operations of intrusion detection systems (IDPS). ISO/IEC International Standards Organization

  • ISO (2015b) ISO/IEC/IEEE 15288:2015. Software Life Cycle Processes. ISO/IEC International Standards Organization, Systems and Software Engineering

  • ISO (2022) ISO/IEC 27001:2022. Information technology-Security techniques—Information security management systems—Requirements, ISO/IEC International Standards Organization

  • Jaeger D, Ussath M, Cheng F, et al (2015) Multi-step attack pattern detection on normalized event logs. In: 2015 IEEE 2nd international conference on cyber security and cloud computing, IEEE, pp 390–398

  • Johnson C, Badger L, Waltermire D, et al (2016) Guide to cyber threat information sharing. Special Publication 800-150. NIST special publication 800 (150)

  • Joloudari JH, Haderbadi M, Mashmool A et al (2020) Early detection of the advanced persistent threat attack using performance analysis of deep learning. IEEE Access 8:186,125-186,137

    Google Scholar 

  • Katipally R, Yang L, Liu A (2011) Attacker behavior analysis in multi-stage attack detection system. In: Proceedings of the seventh annual workshop on cyber security and information intelligence research, pp 1–4

  • Kent AD (2016) Cyber security data sources for dynamic network research. In: Dynamic networks and cyber-security. WSPC (Europe), pp 37–65

  • Kent KA, Souppaya M (2006) Guide to Computer Security Log Management: Recommendations of the National Institute of Standards and Technology. Special Publication 800-92. NIST special publication pp 1–72

  • Khan MA, Abuhasel KA (2021) An evolutionary multi-hidden Markov model for intelligent threat sensing in industrial Internet of things. J Supercomput 77(6):6236–6250

    Google Scholar 

  • Khosravi M, Ladani BT (2020) Alerts correlation and causal analysis for APT based cyber attack detection. IEEE Access 8:162,642-162,656

    Google Scholar 

  • Kim M, Park Y, Han I, et al (2020) A fast alert correlation method with Bayesian feature constraints. In: International conference on cyber warfare and security, academic conferences international limited, pp 277–285

  • Kotenko I, Fedorchenko A, Saenko I et al (2018a) Parallelization of security event correlation based on accounting of event type links. In: 2018 26th euromicro international conference on parallel. distributed and network-based processing (PDP), IEEE, pp 462–469

  • Kotenko I, Saenko I, Branitskiy A (2018) Framework for mobile Internet of things security monitoring based on Big Data processing and machine learning. IEEE Access 6(72):714–723

    Google Scholar 

  • Kotenko I, Saenko I, Ageev S (2019) Hierarchical fuzzy situational networks for online decision-making: application to telecommunication systems. Knowl-Based Syst 185(104):935

    Google Scholar 

  • Kotenko I, Fedorchenko A, Doynikova E (2020) Data analytics for security management of complex heterogeneous systems: event correlation and security assessment tasks. Advances in cyber security analytics and decision systems. Springer, Cham, pp 79–116

    Google Scholar 

  • Kotenko I, Gaifulina D, Zelichenok I (2022) Systematic literature review of security event correlation methods. IEEE Access 10:43,387-43,420

    Google Scholar 

  • Kovačević I, Groš S, Slovenec K (2020) Systematic review and quantitative comparison of cyberattack scenario detection and projection. Electronics 9(10):1722

    Google Scholar 

  • Kushwah D, Singh RR, Tomar DS (2019) An approach to meta-alert generation for anomalous tcp traffic. In: International conference on security & privacy. Springer, Cham, pp 193–216

  • Lallie HS, Debattista K, Bal J (2020) A review of attack graph and attack tree visual syntax in cyber security. Comput Sci Rev 35(100):219

    MathSciNet  MATH  Google Scholar 

  • Lanoe D, Hurfin M, Totel E (2018) A scalable and efficient correlation engine to detect multi-step attacks in distributed systems. In: 2018 IEEE 37th symposium on reliable distributed systems (SRDS). IEEE, pp 31–40

  • Le Q, Mikolov T (2014) Distributed representations of sentences and documents. In: International conference on machine learning, PMLR, pp 1188–1196

  • Lee Y, Kim J, Kang P (2021) LAnoBERT: system log anomaly detection based on BERT masked language model. arXiv preprint arXiv:2111.09564 pp 1–15

  • Li G, Nguyen TH, Jung JJ (2021a) Traffic incident detection based on dynamic graph embedding in vehicular edge computing. Appl Sci 11(13):5861

    Google Scholar 

  • Li S, Zhang Q, Wu X et al (2021b) Attribution classification method of APT malware in IoT using machine learning techniques. Secur Commun Netw 2021:1–12

    Google Scholar 

  • Limmer T, Dressler F (2008) Survey of event correlation techniques for attack detection in early warning systems. University of Erlangen, Dept of Computer Science, Technical Report pp 1–37

  • Liu F, Wen Y, Zhang D, et al (2019a) Log2vec: A heterogeneous graph embedding based approach for detecting cyber threats within enterprise. In: Proceedings of the 2019 ACM SIGSAC conference on computer and communications security, pp 1777–1794

  • Liu J, Chen Y, Liu K (2019b) Exploiting the ground-truth: an adversarial imitation based knowledge distillation approach for event detection. Proc AAAI Conf Artif Intell 33(01):6754–6761

    Google Scholar 

  • Liu L, Chen C, Zhang J et al (2020) Doc2vec-based insider threat detection through behaviour analysis of multi-source security logs. In: 2020 IEEE 19th international conference on trust. security and privacy in computing and communications (TrustCom), IEEE, pp 301–309

  • Liu X (2020) A network attack path prediction method using attack graph. J Ambient Intell Hum Comput 2020:1–8

    Google Scholar 

  • LL-MIT (1998) 1998 darpa intrusion detection evaluation dataset. https://www.ll.mit.edu/r-d/datasets/1998-darpa-intrusion-detection-evaluation-dataset (accessed 03-July-2022)

  • LL-MIT (2000) 2000 darpa intrusion detection scenario specific dataset. https://www.ll.mit.edu/r-d/datasets/2000-darpa-intrusion-detection-scenario-specific-datasets (accessed 03-July-2022)

  • Luo W, Zhang H, Yang X, et al (2020) Dynamic heterogeneous graph neural network for real-time event prediction. In: Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pp 3213-3223

  • Lv S, Qian W, Huang L, et al (2019) Sam-net: Integrating event-level and chain-level attentions to predict what happens next. In: Proceedings of the AAAI conference on artificial intelligence, pp 6802–6809

  • Ma Y, Wu Y, Yu D et al (2022) Vulnerability association evaluation of internet of thing devices based on attack graph. Int J Distrib Sens Netw 18(5):15501329221097,817

    Google Scholar 

  • Mahdavi E, Fanian A, Amini F (2020) A real-time alert correlation method based on code-books for intrusion detection systems. Comput Secur 89(101):661

    Google Scholar 

  • Mao B, Liu J, Lai Y et al (2021) Mif: a multi-step attack scenario reconstruction and attack chains extraction method based on multi-information fusion. Comput Netw 198(108):340

    Google Scholar 

  • Meidan Y, Bohadana M, Mathov Y et al (2018) N-baiot-network-based detection of IoT botnet attacks using deep autoencoders. IEEE Pervasive Comput 17(3):12–22

    Google Scholar 

  • Meier M, Bischof N, Holz T (2002) SHEDEL: a simple hierarchical event description language for specifying attack signatures. Security in the Information Society. Springer, Boston, pp 559–571

    Google Scholar 

  • Microsoft (2021) Microsoft Windows App Development. Event Types. https://learn.microsoft.com/en-us/windows/win32/eventlog/event-types (accessed 12-November-2022)

  • Mikolov T, Chen K, Corrado G, et al (2013a) Efficient estimation of word representations in vector space. In: 1st International Conference on Learning Representations. ICLR 2013, Scottsdale, Arizona, May 2–4, pp 1–12

  • Mikolov T, Sutskever I, Chen K et al (2013b) Distributed representations of words and phrases and their compositionality. Adv Neural Inf Process Syst 26:1–9

    Google Scholar 

  • Milajerdi SM, Gjomemo R, Eshete B, et al (2019) Holmes: real-time apt detection through correlation of suspicious information flows. In: 2019 IEEE symposium on security and privacy (SP), IEEE, pp 1137–1152

  • Miller GA (1995) Wordnet: a lexical database for English. Commun ACM 38(11):39–41

    Google Scholar 

  • Min B, Yoo J, Kim S et al (2021) Network anomaly detection using memory-augmented deep autoencoder. IEEE Access 9:104,695-104,706

    Google Scholar 

  • Mirheidari SA, Arshad S, Jalili R (2013) Alert correlation algorithms: a survey and taxonomy. International symposium on cyberspace safety and security. Springer, Cham, pp 183–197

    Google Scholar 

  • Morzeux (2020) Httpparamsdataset. https://github.com/Morzeux/HttpParamsDataset (accessed 03-July-2022)

  • Moustafa N, Slay J (2015) UNSW-NB15: a comprehensive data set for network intrusion detection systems (UNSW-NB15 network data set). In: 2015 Military communications and information systems conference (MilCIS), pp 1–6

  • Nasir M, Muhammad K, Bellavista P et al (2020) Prioritization and alert fusion in distributed IoT sensors using Kademlia based distributed hash tables. IEEE Access 8:175,194-175,204

    Google Scholar 

  • Nasr M, Bahramali A, Houmansadr A (2018) Deepcorr: strong flow correlation attacks on tor using deep learning. In: Proceedings of the 2018 ACM SIGSAC conference on computer and communications security, pp 1962–1976

  • Navarro J, Deruyver A, Parrend P (2018) A systematic survey on multi-step attack detection. Comput Secur 76:214–249

    Google Scholar 

  • NETRESEC (2000) Mid-atlantic collegiate cyber defense competition. https://www.netresec.com/?page=MACCDC (accessed 03-July-2022)

  • NETRESEC (2015) 4sics geek lounge. https://www.netresec.com/?page=PCAP4SICS (accessed 03-July-2022)

  • Nguyen T, Grishman R (2018) Graph convolutional networks with argument-aware pooling for event detection. Proc AAAI Conf Artif Intell 32(1):5900–5907

    Google Scholar 

  • Oki M, Takeuchi K, Uematsu Y (2018) Mobile network failure event detection and forecasting with multiple user activity data sets. Proc AAAI Conf Artif Intell 32(1):7786–7792

    Google Scholar 

  • Oliner A, Stearley J (2007) What supercomputers say: A study of five system logs. In: 37th annual IEEE/IFIP international conference on dependable systems and networks (DSN’07), IEEE, pp 575–584

  • Pennington J, Socher R, Manning CD (2014) Glove: Global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 1532–1543

  • Peters ME, Neumann M, Iyyer M, et al (2018) Deep contextualized word representations. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. ACL, New Orleans, pp 2227–2237

  • Quintero-Bonilla S, Martín del Rey A (2020) A new proposal on the advanced persistent threat: a survey. Appl Sci 10(11):3874

    Google Scholar 

  • Ramaki AA, Rasoolzadegan A, Bafghi AG (2018) A systematic mapping study on intrusion alert analysis in intrusion detection systems. ACM Comput Surv (CSUR) 51(3):1–41

    Google Scholar 

  • Ramilli M (2016) Malware training sets: a machine learning dataset for everyone. https://marcoramilli.com/2016/12/16/malware-training-sets-a-machine-learning-dataset-for-everyone/ (accessed 03-July-2022)

  • Ring M, Schlör D, Wunderlich S et al (2021) Malware detection on windows audit logs using LSTMs. Comput Secur 109(102):389

    Google Scholar 

  • Ross R, Stoneburner G, Fabius-Greene J, et al (2011) Managing information security risk: organization, mission, and information system view. Special Publication 800-39. NIST special publication pp 1–88

  • Ryciak P, Wasielewska K, Janicki A (2022) Anomaly detection in log files using selected natural language processing methods. Appl Sci 12(10):5089

    Google Scholar 

  • Salah S, Maciá-Fernández G, Díaz-Verdejo JE (2013) A model-based survey of alert correlation techniques. Comput Netw 57(5):1289–1317

    Google Scholar 

  • Sarker IH, Furhad MH, Nowrozy R (2021) AI-driven cybersecurity: an overview, security intelligence modeling and research directions. SN Comput Sci 2(3):1–18

    Google Scholar 

  • Scarfone K, Souppaya M, Cody A et al (2008) Technical guide to information security testing and assessment. NIST Spec Publ 800(115):2–25

    Google Scholar 

  • Sen Ö, van der Velde D, Wehrmeister KA et al (2022) On using contextual correlation to detect multi-stage cyber attacks in smart grids. Sustain Energy Grids Netw 32(100):821

    Google Scholar 

  • Seyyar YE, Yavuz AG, Unver HM (2022) An attack detection framework based on BERT and deep learning. IEEE Access Early Access, pp 1–13

  • Sharafaldin I, Lashkari AH, Ghorbani AA (2018) Toward generating a new intrusion detection dataset and intrusion traffic characterization. In: Proceedings of the 4th international conference on information systems security and privacy (ICISSP 2018), pp 108–116

  • Shawly T, Elghariani A, Kobes J et al (2019) Architectures for detecting interleaved multi-stage network attacks using hidden Markov models. IEEE Trans Depend Secure Comput 18(5):2316–2330

    Google Scholar 

  • Shen Y, Mariconti E, Vervier PA, et al (2018) Tiresias: predicting security events through deep learning. In: Proceedings of the 2018 ACM SIGSAC conference on computer and communications security, pp 592–605

  • Shiravi A, Shiravi H, Tavallaee M et al (2012) Toward developing a systematic approach to generate benchmark datasets for intrusion detection. Comput Secur 31(3):357–374

    Google Scholar 

  • Shittu R, Healing A, Ghanea-Hercock R et al (2015) Intrusion alert prioritisation and attack detection using post-correlation analysis. Comput Secur 50:1–15

    Google Scholar 

  • Siddiqui AJ, Boukerche A (2021) TempoCode-IoT: temporal codebook-based encoding of flow features for intrusion detection in internet of things. Clust Comput 24(1):17–35

    Google Scholar 

  • Sikos LF (2021) Ai in digital forensics: ontology engineering for cybercrime investigations. Wiley Interdiscip Rev 3(3):e1394

    Google Scholar 

  • Stephan G, Pascal H, Andreas A et al (2007) Knowledge representation and ontologies. Semantic web services: concepts, technologies, and applications. Springer, Berlin, Heidelberg, pp 51–105

    Google Scholar 

  • Stouffer K, Stouffer K, Zimmerman T, et al (2017) Cybersecurity framework manufacturing profile. NISTIR 8183 Rev. 1. US Department of Commerce, National Institute of Standards and Technology

  • Sun J, Gu L, Chen K (2020) An efficient alert aggregation method based on conditional rough entropy and knowledge granularity. Entropy 22(3):324

    MathSciNet  Google Scholar 

  • Sun X, Dai J, Liu P et al (2018) Using Bayesian networks for probabilistic identification of zero-day attack paths. IEEE Trans Inf Forensics Secur 13(10):2506–2521

    Google Scholar 

  • Tanwar P, Prasad T, Aswal MS (2010) Comparative study of three declarative knowledge representation techniques. Int J Comput Sci Eng 2(07):2274–2281

    Google Scholar 

  • Xl Tao, Shi L, Zhao F et al (2021) A hybrid alarm association method based on AP clustering and causality. Wirel Commun Mobile Comput 2021(5):1–10

    Google Scholar 

  • Tavallaee M, Bagheri E, Lu W, et al (2009) A detailed analysis of the KDD CUP 99 data set. In: 2009 IEEE symposium on computational intelligence for security and defense applications, IEEE, pp 1–6

  • Tidjon LN, Frappier M, Mammar A (2020) Intrusion detection using ASTDs. International conference on advanced information networking and applications. Springer, Cham, pp 1397–1411

    Google Scholar 

  • Viterbi A (1967) Error bounds for convolutional codes and an asymptotically optimum decoding algorithm. IEEE Trans Inf Theory 13(2):260–269

    MATH  Google Scholar 

  • Vlahakis G, Apostolou D, Kopanaki E (2018) Enabling situation awareness with supply chain event management. Expert Syst Appl 93:86–103

    Google Scholar 

  • Walker C, Strassel S, Medero J et al (2006) Ace 2005 multilingual training corpus. Linguist Data Consort 57:45

    Google Scholar 

  • Wang J, Tang Y, He S et al (2020) LogEvent2vec: logevent-to-vector based anomaly detection for large-scale logs in internet of things. Sensors 20(9):2451

    Google Scholar 

  • Wang J, Zhao C, He S et al (2022) LogUAD: log unsupervised anomaly detection based on Word2Vec. Comput Syst Sci Eng 41(3):1207–1222

    Google Scholar 

  • Wang Q, Jiang J, Shi Z, et al (2018) A novel multi-source fusion model for known and unknown attack scenarios. In: 2018 17th IEEE international conference on trust, security and privacy in computing and communications/12th IEEE international conference on big data science and engineering (TrustCom/BigDataSE), IEEE, pp 727–736

  • Wang X, Gong X, Yu L et al (2021a) MAAC: Novel alert correlation method to detect multi-step attack. In: 2021 IEEE 20th international conference on trust. Security and privacy in computing and communications (TrustCom), IEEE, pp 726–733

  • Wang Z, Chen Z, Ni J, et al (2021b) Multi-scale one-class recurrent neural networks for discrete event sequence anomaly detection. In: Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining, pp 3726–3734

  • Welch LR (2003) Hidden Markov models and the Baum–Welch algorithm. IEEE Inf Theory Soc Newsl 53(4):10–13

    Google Scholar 

  • Wen J, Li J, Mao Y, et al (2016) On the representation and embedding of knowledge bases beyond binary relations. In: Proceedings of the twenty-fifth international joint conference on artificial intelligence, pp 1300–1307

  • Wood M, Erlinger M (2007) Intrusion detection message exchange requirements. IETF Request for Comment (RFC) 4766

  • Xie T, Zheng Q, Zhang W (2018) Mining temporal characteristics of behaviors from interval events in e-learning. Inf Sci 447:169–185

    Google Scholar 

  • Xu W, Huang L, Fox A, et al (2009) Online system problem detection by mining patterns of console logs. In: 2009 ninth IEEE international conference on data mining, IEEE, pp 588–597

  • Yu Beng L, Ramadass S, Manickam S et al (2014) A survey of intrusion alert correlation and its design considerations. IETE Tech Rev 31(3):233–240

    Google Scholar 

  • Zegeye WK, Dean RA, Moazzami F (2018) Multi-layer hidden Markov model based intrusion detection system. Mach Learn Knowl Extr 1(1):265–286

    Google Scholar 

  • Zeng J, Wu S, Chen Y et al (2019) Survey of attack graph analysis methods from the perspective of data and knowledge processing. Secur Commun Netw 2019:1–16

    Google Scholar 

  • Zeng J, Chua ZL, Chen Y, et al (2021) Watson: Abstracting behaviors from audit logs via aggregation of contextual semantics. In: Proceedings of the 28th annual network and distributed system security symposium, NDSS, pp 1–18

  • Zhan Y, Haddadi H (2019) Towards automating smart homes: Contextual and temporal dynamics of activity prediction. In: adjunct proceedings of the 2019 ACM international joint conference on pervasive and ubiquitous computing and Proceedings of the 2019 ACM international symposium on wearable computers, pp 413–417

  • Zhang H, Jin X, Li Y et al (2019) A multi-step attack detection model based on alerts of smart grid monitoring system. IEEE Access 8:1031–1047

    Google Scholar 

  • Zhang X, Wu T, Zheng Q et al (2022a) Multi-step attack detection based on pre-trained hidden Markov models. Sensors 22(8):2874

    Google Scholar 

  • Zhang Y, Zhao S, Zhang J (2019b) RTMA: Real time mining algorithm for multi-step attack scenarios reconstruction. In: 2019 IEEE 21st international conference on high performance computing and communications, IEEE, pp 2103–2110

  • Zheng H, Wang Y, Han C et al (2018) Learning and applying ontology for machine learning in cyber attack detection. In: 2018 17th IEEE international conference on trust. Security and privacy in computing and communications, IEEE, pp 1309–1315

  • Zimba A, Chen H, Wang Z (2019) Bayesian network based weighted APT attack paths modeling in cloud computing. Future Gener Comput Syst 96:525–537

    Google Scholar 

Download references

Acknowledgements

This work was supported by the Analytical Center for the Government of the Russian Federation (IGK 000000D730321P5Q0002), agreement No. 70-2021-00141.

Author information

Author notes

  1. Diana Levshun and Igor Kotenko have contributed equally to this work.

Authors and Affiliations

  1. St. Petersburg Federal Research Center of the Russian Academy of Sciences (SPC RAS), St. Petersburg, 199178, Russia

    Diana Levshun

  2. ITMO University, St. Petersburg, 197101, Russia

    Igor Kotenko

Authors
  1. Diana Levshun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Igor Kotenko
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Igor Kotenko.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Levshun, D., Kotenko, I. A survey on artificial intelligence techniques for security event correlation: models, challenges, and opportunities. Artif Intell Rev 56, 8547–8590 (2023). https://doi.org/10.1007/s10462-022-10381-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10462-022-10381-4

Keywords

Search

Navigation