Abstract
Energy is crucial to economic and social development. Accurate energy consumption forecasting is essential to effective energy management, reasonable energy layout planning, and ensuring the sustainable and healthy development of the energy industry. Nevertheless, precisely and efficiently forecasting energy consumption remains to be a challenge. Previous studies have proposed solutions mainly from traditional machine learning and mathematical statistics, which can effectively forecast short-term energy consumption in small-scale data. However, it is still challenging to explore the characteristics of high-dimensional and large-scale energy data and predict medium- to long-term energy consumption and its fluctuation trends. In this paper, a new time series energy consumption prediction model is proposed, which combines the attention mechanism of the Transformer model with the natural language processing ability of LSTM, based on a dataset of Spain’s energy production and climate change from 2015 to 2018. Compared with the state-of-the-art models such as RNN, GRU, and LSTM, our model achieves better performance in the 7-day energy consumption prediction task (RMSE = 0.7, MAE = 0.5).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ewa, C., Joanicjusz, N., Łukasz, N.: ARIMA models in electrical load forecasting and their robustness to noise. Energies 14(23), 7952 (2021)
Liu, Y., Wang, Y., Yu, F.Y., et al.: Prediction and analysis of electricity demand in Jilin province 14th five-year plan based on GM (1, 1) method. J. Green Sci. Technol. 24(18), 232–236 (2022)
Peng, L.L., Fan, G.F., Yu, M., et al.: Electric load forecasting based on wavelet transform and random forest. Adv. Theory Simul. 4(12) (2021)
Wan, Q., Wang, Q.L., Wang, R.H., et al.: Short-term load forecasting of a regional power grid based on support vector machine. Power Syst. Clean Energy 32(12), 14–20 (2016)
Cao, H.Z., Wang, T.L., Chen, P.D., et al.: Solar energy forecasting in short term based on the ASO-BPNN model. Front. Energy Res. (2022)
Lai, C.S., Mo, Z.Y., Wang, T., et al.: Load forecasting based on deep neural network and historical data augmentation. IET Gener. Transm. Distrib. 14(24), 5927–5934 (2020)
Ibrahim, N.M., Megahed, A.I., Abbasy, N.H.: Short-term individual household load forecasting framework using LSTM deep learning approach. In: 2021 5th International Symposium on Multidisciplinary Studies and Innovative Technologies (ISMSIT), pp. 257–262. IEEE (2021)
Lu, C., Li, J., Zhang, G., et al.: A GRU-based short-term multi-energy loads forecast approach for integrated energy system. In: 2022 4th Asia Energy and Electrical Engineering Symposium (AEEES), pp. 209–213. IEEE (2022)
Jin, N.Y., Jo, H.H.: Prediction of weekly load using stacked bidirectional LSTM and stacked unidirectional LSTM. J. Korean Inst. Inf. Technol. 18, 9–17 (2020)
Dorado Rueda, F., Durán Suárez, J., Del Real, T.A.: Short-term load forecasting using encoder-decoder WaveNet: application to the French grid. Energies 14(9), 2524 (2021)
Vaswani, A., Shazeer, N., Parmar, N., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Chowdhary, K.R., Chowdhary, K.R.: Natural language processing. Fundam. Artif. Intell., 603–649 (2020)
Duong-Ngoc, H., Nguyen-Thanh, H., Nguyen-Minh, T.: Short term load forecast using deep learning. In: 2019 Innovations in Power and Advanced Computing Technologies (i-PACT), vol. 1, pp. 1–5. IEEE (2019)
Huang, L., Qin, J., Zhou, Y., et al.: Normalization techniques in training DNNs: methodology, analysis and application. IEEE Trans. Pattern Anal. Mach. Intell. (2023)
Voita, E., Talbot, D., Moiseev, F., et al.: Analyzing multi-head self-attention: specialized heads do the heavy lifting, the rest can be pruned. arXiv preprint arXiv:1905.09418 (2019)
He, K., Zhang, X., Ren, S., et al.: Identity mappings in deep residual networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, 11–14 October 2016, Proceedings, Part IV 14, vol. 9908, pp. 630–645. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46493-0_38
Cilimkovic, M.: Neural networks and back propagation algorithm. Institute of Technology Blanchardstown, Blanchardstown Road North Dublin, 15(1) (2015)
Hodson, T.O.: Root-mean-square error (RMSE) or mean absolute error (MAE): when to use them or not. Geosci. Mod. Dev. 15(14), 5481–5487 (2022)
Zhang, Z.: Improved Adam optimizer for deep neural networks. In: 2018 IEEE/ACM 26th International Symposium on Quality of Service (IWQoS), pp. 1–2. IEEE (2018)
Popoola, S.I., Adetiba, E., Atayero, A.A., et al.: Optimal model for path loss predictions using feed-forward neural networks. Cogent Eng. 5(1), 1444345 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Wang, H., Li, J., Chang, L. (2024). Predicting Time Series Energy Consumption Based on Transformer and LSTM. In: Li, J., Zhang, B., Ying, Y. (eds) 6GN for Future Wireless Networks. 6GN 2023. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 553. Springer, Cham. https://doi.org/10.1007/978-3-031-53401-0_27
Download citation
DOI: https://doi.org/10.1007/978-3-031-53401-0_27
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-53400-3
Online ISBN: 978-3-031-53401-0
eBook Packages: Computer ScienceComputer Science (R0)