Abstract
In today’s dynamically evolving data landscapes, detecting and adapting to concept drifts in streaming data is imperative. Concept drift occurs when there’s a shift in the statistical characteristics of input features, like their mean or variance, or when the relationship between these features and the target label changes over time. This drift can decrease a model’s accuracy because the model is trained on older data. As the data evolves, the model becomes outdated, which can lead to incorrect predictions and reduced performance. This paper introduces the Incremental Restricted Boltzmann Machine (IRBM), an approach designed to address these challenges. The IRBM adapts the traditional architecture and learning paradigms of Restricted Boltzmann Machines (RBMs) to incrementally process and learn from evolving data streams, ensuring model efficacy and accuracy over time. Through extensive experiments, we demonstrate the IRBM’s ability to swiftly detect concept drifts, adapt its internal representations, and maintain robust performance even when confronted with significant data evolutions. The proposed approach outperforms existing methods with an accuracy of 77.42%, 75.32%, 92.12% and 89.21% for electricity, phishing, weather, and rotating hyperplane respectively. Our findings suggest that the IRBM not only offers an effective approach to understanding and adapting to changing patterns in streaming data but also outperforms the other state-of-the-art techniques.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Aggarwal, C.C.: Chapter 9 A survey of stream classification algorithms (2015)
Domingos, P., Hulten, G.: Mining high-speed data streams. In: Proceedings of the 6th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 71–80 (2000)
Tsymbal, A.: The problem of concept drift: definitions and related work. Computer Science Department, Trinity College, Dublin, Ireland, Technical report, vol. 106 (2004)
Žliobaitė, I., Pechenizkiy, M., Gama, J.: An overview of concept drift applications. In: Japkowicz, N., Stefanowski, J. (eds.) Big Data Analysis: New Algorithms for a New Society. SBD, vol. 16, pp. 91–114. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-26989-4_4
Hesse, G., Lorenz, M.: Conceptual survey on data stream processing systems, pp. 798–803 (2015)
Mehta, S.: Concept drift in streaming data classification algorithms, platforms and issues. Procedia Comput. Sci. 122, 804–811 (2017)
Ayad, O.: Learning under concept drift with support vector machines. In: Wermter, S., Weber, C., Duch, W., Honkela, T., Koprinkova-Hristova, P., Magg, S., Palm, G., Villa, A.E.P. (eds.) ICANN 2014. LNCS (LNAI and LNB), vol. 8681, pp. 587–594. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-11179-7_74
Gama, J., Žliobaite, I., Bifet, A., Pechenizkiy, M., Bouchachia, A.: A survey on concept drift adaptation. ACM Comput. Surv. 46(4), 44 (2014)
Cohen, L., Avrahami-Bakish, G., Last, M., Kandel, A., Kipersztok, O.: Real-time data mining of non-stationary data streams from sensor networks. Inform. Fusion 9(3), 344–353 (2008)
Lu, J., Liu, A., Dong, F., Gu, F., Gama, J., Zhang, G.: Learning under concept drift: a review. IEEE Trans. Knowl. Data Eng. 31, 2346–2363 (2018)
Ditzler, G., Roveri, M., Alippi, C., Polikar, R.: Learning in nonstationary environments: a survey. IEEE Comput. Intell. Mag. 10(4), 12–25 (2015). https://doi.org/10.1109/MCI.2015.2471196
Fischer, A., Igel, C.: An introduction to restricted Boltzmann machines. In: Alvarez, L., Mejail, M., Gomez, L., Jacobo, J. (eds.) CIARP 2012. LNCS, vol. 7441, pp. 14–36. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33275-3_2. ISBN 978-3-642-33274-6
Jaworski, M., Duda, P., Rutkowski, L.: On applying the restricted Boltzmann machine to active concept drift detection (2017). https://doi.org/10.1109/SSCI.2017.8285409
Korycki, Ł., Krawczyk, B.: Concept drift detection from multi-class imbalanced data streams. arXiv Learning (2021)
Jaworski, M., Duda, P., Rutkowska, D., Rutkowski, L.: On handling missing values in data stream mining algorithms based on the restricted Boltzmann machine. In: Gedeon, T., Wong, K.W., Lee, M. (eds.) ICONIP 2019. CCIS, vol. 1143, pp. 347–354. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-36802-9_37
Vu, H., Nguyen, T.D., Phung, D.: Detection of unknown anomalies in streaming videos with generative energy-based Boltzmann models. arXiv Computer Vision and Pattern Recognition (2018)
Zhu, Q., Zhou, J., Wang, W.: Concept drift detection and update algorithm based on online restricted Boltzmann machine. In: Liang, Q., Wang, W., Mu, J., Liu, X., Na, Z. (eds.) AIC 2022. LNEE, vol. 871, pp. 305–311. Springer, Singapore (2023). https://doi.org/10.1007/978-981-99-1256-8_36
Xu, S., Wang, J.: Dynamic extreme learning machine for data stream classification. Neurocomputing 238, 433–449 (2017)
Xiulin, Z., Peipei, L., Xindong, W.: Data stream classification based on extreme learning machine: a review. Big Data Res. 30, 100356 (2022). https://doi.org/10.1016/j.bdr.2022.100356
Neto, Á.C.L., Coelho, R.A., de Castro, C.L.: An incremental learning approach using long short-term memory neural networks. J. Control Autom. Electr. Syst. 33, 1457–1465 (2020)
Oza, N.C., Russell, S.J.: Online bagging and boosting. In: Richardson, T.S., Jaakkola, T.S. (eds.) Proceedings of the Eighth International Workshop on Artificial Intelligence and Statistics, pp. 229–236. PMLR (2001). https://proceedings.mlr.press/r3/oza01a.html
Dataset. Electricity dataset (2014). https://github.com/scikit-multiflow/streaming-datasets/blob/master/elec.csv. Accessed September 2023
Dataset. Rotating hyperplane dataset (2014). https://github.com/scikit-multiflow/streaming-datasets/blob/master/hyperplane.csv. Accessed September 2023
Dataset. Weather dataset (2014). https://github.com/scikit-multiflow/streaming-datasets/blob/master/weather.csv. Accessed September 2023
Dataset. Phishing dataset (2014). https://github.com/ogozuacik/concept-drift-datasets-scikit-multiflow/tree/master/real-world/phishing.csv. Accessed September 2023
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Suryawanshi, S., Goswami, A., Patil, P. (2024). IRBM: Incremental Restricted Boltzmann Machines for Concept Drift Detection and Adaption in Evolving Data Streams. In: Garg, D., Rodrigues, J.J.P.C., Gupta, S.K., Cheng, X., Sarao, P., Patel, G.S. (eds) Advanced Computing. IACC 2023. Communications in Computer and Information Science, vol 2053. Springer, Cham. https://doi.org/10.1007/978-3-031-56700-1_37
Download citation
DOI: https://doi.org/10.1007/978-3-031-56700-1_37
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-56699-8
Online ISBN: 978-3-031-56700-1
eBook Packages: Computer ScienceComputer Science (R0)