Abstract
Traditional rule-based approaches to system monitoring have many areas for improvement. Rules are time-consuming to maintain, and their ability to detect unforeseen future incidents is limited. Online log anomaly detection workflows have the potential to improve upon rule-based methods by providing fine-grained, automated detection of abnormal behavior. However, system and process logs are not static. Code and configuration changes may alter the sequences of log entries produced by these processes, impacting the models trained on their previous behavior. These changes result in false positive signals that can overwhelm production services engineers and drown out alerts for real issues. For this reason, log drift is a significant obstacle to utilizing online log anomaly detection approaches for monitoring in industrial settings. This study explores the different types of log drift and classifies them using a newly introduced taxonomy. It then evaluates the impact these types of drift have on online anomaly detection workflows. Several potential mitigation methods are presented and evaluated based on synthetic and real-world log data. Finally, possible directions for future research are provided and discussed.
Supported by organization Nomura Securities Co., Ltd.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
References
Ahmad, S., Lavin, A., Purdy, S., Agha, Z.: Unsupervised real-time anomaly detection for streaming data. Neurocomputing 262, 134–147 (2017). https://doi.org/10.1016/j.neucom.2017.04.070, https://www.sciencedirect.com/science/article/pii/S0925231217309864, online Real-Time Learning Strategies for Data Streams
Chen, Y., Luktarhan, N., Lv, D.: LogLS: research on system log anomaly detection method based on dual LSTM. Symmetry 14(3), 454 (2022). https://www.mdpi.com/2073-8994/14/3/454
Du, M., Li, F., Zheng, G., Srikumar, V.: Deeplog: anomaly detection and diagnosis from system logs through deep learning. In: Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, pp. 1285–1298. CCS ’17, Association for Computing Machinery, New York, NY, USA (2017). https://doi.org/10.1145/3133956.3134015
Du, Q., Zhao, L., Xu, J., Han, Y., Zhang, S.: Log-based anomaly detection with multi-head scaled dot-product attention mechanism. In: Strauss, C., Kotsis, G., Tjoa, A.M., Khalil, I. (eds.) DEXA 2021. LNCS, vol. 12923, pp. 335–347. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-86472-9_31
Duan, X., Ying, S., Yuan, W., Cheng, H., Yin, X.: QLLog: a log anomaly detection method based on Q-learning algorithm. Inf. Process. Manag. 58(3), 102540 (2021). https://doi.org/10.1016/j.ipm.2021.102540, https://www.sciencedirect.com/science/article/pii/S0306457321000479
Gama, J.A., Žliobaitundefined, I., Bifet, A., Pechenizkiy, M., Bouchachia, A.: A survey on concept drift adaptation. ACM Comput. Surv. 46(4) (2014). https://doi.org/10.1145/2523813, https://doi-org.waseda.idm.oclc.org/10.1145/2523813
Hershey, J.R., Olsen, P.A.: Approximating the Kullback Leibler divergence between gaussian mixture models. In: 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP ’07, vol. 4, pp. IV-317-IV-320 (2007). https://doi.org/10.1109/ICASSP.2007.366913
Iglesias Vázquez, F., Hartl, A., Zseby, T., Zimek, A.: Anomaly detection in streaming data: a comparison and evaluation study. Expert Syst. Appl. 233, 120994 (2023). https://doi.org/10.1016/j.eswa.2023.120994, https://www.sciencedirect.com/science/article/pii/S0957417423014963
Kabinna, S., Shang, W., Bezemer, C.P., Hassan, A.E.: Examining the stability of logging statements. In: 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER), vol. 1, pp. 326–337 (2016). https://doi.org/10.1109/SANER.2016.29
Lupton, S., Yu, L., Washizaki, H., Yoshioka, N., Fukazawa, Y.: Assessment of real-world incident detection through a component-based online log anomaly detection pipeline framework. In: The 10th International Conference on Dependable Systems and Their Applications (DSA 2023), pp. 1–2. Tokyo, Japan (2023)
Pal, A., Kumar, M.: DLME: distributed log mining using ensemble learning for fault prediction. IEEE Syst. J. 13(4), 3639–3650 (2019). https://doi.org/10.1109/JSYST.2019.2904513
Zhang, B., Zhang, H., Moscato, P., Zhang, A.: Anomaly detection via mining numerical workflow relations from logs. In: 2020 International Symposium on Reliable Distributed Systems (SRDS), pp. 195–204 (2020). https://doi.org/10.1109/SRDS51746.2020.00027
Zhang, X., et al.: Robust log-based anomaly detection on unstable log data. In: Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pp. 807–817. ESEC/FSE 2019, Association for Computing Machinery, New York, NY, USA (2019). https://doi.org/10.1145/3338906.3338931, https://doi-org.waseda.idm.oclc.org/10.1145/3338906.3338931
Zhu, B., Li, J., Gu, R., Wang, L.: An approach to cloud platform log anomaly detection based on natural language processing and LSTM. In: 2020 3rd International Conference on Algorithms, Computing and Artificial Intelligence. ACAI 2020, Association for Computing Machinery, New York, NY, USA (2020). https://doi.org/10.1145/3446132.3446415
Acknowledgments
The authors thank Xingfang Wu from the Polytechnique Montréal and the Data Science Initiative (DSI), Nomura Securities Co., Ltd. members for their support in preparing and completing this research. This work was partially supported by the JST-Mirai Program (Grant Number JPMJMI20B8).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Lupton, S., Washizaki, H., Yoshioka, N., Fukazawa, Y. (2024). Log Drift Impact on Online Anomaly Detection Workflows. In: Kadgien, R., Jedlitschka, A., Janes, A., Lenarduzzi, V., Li, X. (eds) Product-Focused Software Process Improvement. PROFES 2023. Lecture Notes in Computer Science, vol 14483. Springer, Cham. https://doi.org/10.1007/978-3-031-49266-2_19
Download citation
DOI: https://doi.org/10.1007/978-3-031-49266-2_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-49265-5
Online ISBN: 978-3-031-49266-2
eBook Packages: Computer ScienceComputer Science (R0)