Skip to main content
SpringerLink
Log in

Oriented transformer for infectious disease case prediction

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Accurate prediction of infectious disease cases plays a crucial role in achieving effective infection prevention and control. However, the inherent variability of incubation periods and progression dynamics of infectious diseases pose significant challenges to the accuracy of predicting multiple diseases. Multiple representation fusion (MRF) methods would improve the performance of prediction models, due to their capability to capture diverse temporal dependencies that reflect potential disease transmission patterns. But the traditional fusion approach for infectious disease prediction still faces many challenges, including the requirement of auxiliary data, vulnerability to disease evolution, and lack of intuitive explanation. To address these challenges, this paper proposes an oriented transformer (ORIT) for infectious diseases case prediction. Contrary to traditional MRF structures that integrate representations from multiple data sources, the MRF in the proposed ORIT combines multi-orientation context vectors solely by capturing multi-dimensional temporal relationships within disease case data. Furthermore, this paper considers the heterogeneity of the incubation period in the prediction of different infectious disease cases. Lastly, this paper conducts comprehensive experiments to evaluate the proposed method using two real datasets of infectious diseases, and compares it with 21 well-known prediction methods. The experimental results verify the effectiveness of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Algorithm 1
Algorithm 2
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Data Availibility

The data used to support the findings of this study are available from the corresponding author upon request.

References

  1. Abbaszadeh Shahri A, Asheghi R, Khorsand Zak M (2021) A hybridized intelligence model to improve the predictability level of strength index parameters of rocks. Neural Comput Appl 33:3841–3854. https://doi.org/10.1007/s00521-020-05223-9

    Article  Google Scholar 

  2. Abbaszadeh Shahri A, Shan C, Zäll E, Larsson S (2021) Spatial distribution modeling of subsurface bedrock using a developed automated intelligence deep learning procedure: A case study in sweden. J Rock Mech Geotechnical Eng 13(6):1300–1310. https://doi.org/10.1016/j.jrmge.2021.07.006

    Article  Google Scholar 

  3. Abbaszadeh Shahri A, Shan C, Larsson S (2022) A novel approach to uncertainty quantification in groundwater table modeling by automated predictive deep learning. Nat Resour Res 31(3):1351–1373. https://doi.org/10.1007/s11053-022-10051-w

    Article  Google Scholar 

  4. Alamo T, Reina DG, Gata PM, Preciado VM, Giordano G (2021) Data-driven methods for present and future pandemics: Monitoring, modelling and managing. Annu Rev Control 52:448–464. https://doi.org/10.1016/j.arcontrol.2021.05.003

  5. Asheghi R, Hosseini SA, Saneie M, Shahri AA (2020) Updating the neural network sediment load models using different sensitivity analysis methods: a regional application. J Hydroinf 22(3):562–577. https://doi.org/10.2166/hydro.2020.098

    Article  Google Scholar 

  6. Bai S, Kolter JZ, Koltun V (2018) An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv:1803.01271

  7. Bandara K, Bergmeir C, Hewamalage H (2020) LSTM-MSNet: Leveraging Forecasts on Sets of Related Time Series With Multiple Seasonal Patterns. IEEE Trans Neural Netw Learning Syst pp 1–14, https://doi.org/10.1109/TNNLS.2020.2985720

  8. Bracher J, Held L (2022) Endemic-epidemic models with discrete-time serial interval distributions for infectious disease prediction. Int J Forecast 38(3):1221–1233

    Article  Google Scholar 

  9. Brauwers G, Frasincar F (2023) A general survey on attention mechanisms in deep learning. IEEE Trans Knowl Data Eng 35(4):3279–3298. https://doi.org/10.1109/TKDE.2021.3126456

    Article  Google Scholar 

  10. Chang SL, Piraveenan M, Pattison P, Prokopenko M (2020) Game theoretic modelling of infectious disease dynamics and intervention methods: a review. J Biol Dyn 14(1):57–89. https://doi.org/10.1080/17513758.2020.1720322

    Article  MathSciNet  Google Scholar 

  11. Chaudhari S, Mithal V, Polatkan G, Ramanath R (2021a) An attentive survey of attention models. ACM Trans Intell Syst Technol 12(5):53:1 – 53:32, https://doi.org/10.1145/3465055

  12. Chaudhari S, Mithal V, Polatkan G, Ramanath R (2021) An attentive survey of attention models. ACM Trans Intell Syst Technol 12(5):1–32. https://doi.org/10.1145/3465055

    Article  Google Scholar 

  13. Cho K, van Merrienboer B, Gülçehre Ç, Bahdanau D, Bougares F, Schwenk H, Bengio Y (2014) Learning phrase representations using RNN encoder-decoder for statistical machine translation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, ACL, Doha, Qatar, pp 1724–1734, https://doi.org/10.3115/v1/d14-1179

  14. Djenouri Y, Belhadi A, Srivastava G, Lin JCW (2021) Secure collaborative augmented reality framework for biomedical informatics. IEEE J Biomed Health Inform 26(6):2417–2424. https://doi.org/10.1109/JBHI.2021.3139575

    Article  Google Scholar 

  15. Du L, Gao R, Suganthan PN, Wang DZ (2022) Bayesian optimization based dynamic ensemble for time series forecasting. Inf Sci 591:155–175. https://doi.org/10.1016/j.ins.2022.01.010

    Article  Google Scholar 

  16. Efimov D, Ushirobira R (2021) On an interval prediction of covid-19 development based on a seir epidemic model. Annu Rev Control 51:477–487. https://doi.org/10.1016/j.arcontrol.2021.01.006

    Article  MathSciNet  Google Scholar 

  17. Hewamalage H, Bergmeir C, Bandara K (2021) Recurrent neural networks for time series forecasting: Current status and future directions. Int J Forecast 37(1):388–427. https://doi.org/10.1016/j.ijforecast.2020.06.008

    Article  Google Scholar 

  18. Hochreiter S, Schmidhuber J (1997) Long Short-term Memory. Neural Comput 9:1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735

  19. Hong J, Liu F, Qi H, Tu W, Ward MP, Ren M, Zhao Z, Su Q, Huang J, Chen X, Le J, Ren X, Hu Y, Cowling B, Li Z, Chang Z, Zhang Z (2022) Changing epidemiology of hand, foot, and mouth disease in china, 2013 2019: a population-based study. The Lancet Regional Health - Western Pacific 20:100370. https://doi.org/10.1016/j.lanwpc.2021.100370

  20. Huang S, Wang D, Wu X, Tang A (2019) Dsanet: Dual self-attention network for multivariate time series forecasting. In: Proceedings of the 28th International Conference on Information and Knowledge Management, ACM, Beijing, China, pp 2129 – 2132, https://doi.org/10.1145/3357384.3358132

  21. Huang Y, Zhang P, Wang Z, Lu Z, Wang Z (2023) HFMD cases prediction using transfer one-step-ahead learning. Neural Process Lett 55(3):2321–2339. https://doi.org/10.1007/s11063-022-10795-9

    Article  Google Scholar 

  22. Kingma DP, Ba J (2015) Adam: A method for stochastic optimization. In: Proceedings of the 3rd International Conference on Learning Representations, OpenReview.net, San Diego, CA, USA

  23. Lai G, Chang W, Yang Y, Liu H (2018) Modeling long- and short-term temporal patterns with deep neural networks. In: Proceedings of the 41st International Conference on Research and Development in Information Retrieval, ACM, Ann Arbor, MI, USA, pp 95 – 104, https://doi.org/10.1145/3209978.3210006

  24. Lim B, Zohren S (2021) Time-series forecasting with deep learning: a survey. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 379(2194):20200209. https://doi.org/10.1098/rsta.2020.0209

    Article  MathSciNet  Google Scholar 

  25. Lin JCW, Djenouri Y, Srivastava G, Fourier-Viger P (2022) Efficient evolutionary computation model of closed high-utility itemset mining. Appl Intell 52(9):10604–10616. https://doi.org/10.1007/s10489-021-03134-3

    Article  Google Scholar 

  26. Mabrouk AB, Abdallah NB, Dhifaoui Z (2008) Wavelet decomposition and autoregressive model for time series prediction. Appl Math Comput 199(1):334–340. https://doi.org/10.1016/j.amc.2007.09.067

    Article  MathSciNet  Google Scholar 

  27. Mora C, McKenzie T, Gaw IM, Dean JM, von Hammerstein H, Knudson TA, Setter RO, Smith CZ, Webster KM, Patz JA et al (2022) Over half of known human pathogenic diseases can be aggravated by climate change. Nat Clim Chang 12(9):869–875. https://doi.org/10.1038/s41558-022-01426-1

    Article  Google Scholar 

  28. Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, Killeen T, Lin Z, Gimelshein N, Antiga L, Desmaison A, Köpf A, Yang EZ, DeVito Z, Raison M, Tejani A, Chilamkurthy S, Steiner B, Fang L, Bai J, Chintala S (2019) Pytorch: An imperative style, high-performance deep learning library. In: Proceddings of the 33rd Annual Conference on Neural Information Processing Systems, Vancouver, BC, Canada, vol 32, pp 8024–8035

  29. Qin Y, Song D, Chen H, Cheng W, Jiang G, Cottrell GW (2017) A dual-stage attention-based recurrent neural network for time series prediction. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence, ijcai.org, Melbourne, Australia, pp 2627 – 2633, https://doi.org/10.24963/ijcai.2017/366

  30. Srivastava G, Lin JCW, Pirouz M, Li Y, Yun U (2020) A pre-large weighted-fusion system of sensed high-utility patterns. IEEE Sens J 21(14):15626–15634. https://doi.org/10.1109/JSEN.2020.2991045

    Article  Google Scholar 

  31. Stockdale JE, Liu P, Colijn C (2022) The potential of genomics for infectious disease forecasting. Nat Microbiol 7(11):1736–1743

    Article  Google Scholar 

  32. Subissi L, von Gottberg A, Thukral L, Worp N, Oude Munnink BB, Rathore S, Abu-Raddad LJ, Aguilera X, Alm E, Archer BN et al (2022) An early warning system for emerging sars-cov-2 variants. Nat Med 28(6):1110–1115. https://doi.org/10.1038/s41591-022-01836-w

    Article  Google Scholar 

  33. Sutskever I, Vinyals O, Le QV (2014) Sequence to sequence learning with neural networks. In: Proceedings of the 28th Annual Conference on Neural Information Processing Systems, MIT Press, Montreal, Quebec, Canada, vol 27, pp 3104 – 3112

  34. Sweileh WM (2022) Global research activity on mathematical modeling of transmission and control of 23 selected infectious disease outbreak. Glob Health 18(1):1–14

    Article  Google Scholar 

  35. Tadić B, Melnik R, (2021) Microscopic dynamics modeling unravels the role of asymptomatic virus carriers in sars-cov-2 epidemics at the interplay between biological and social factors. Comput Biol Med 133:104422. https://doi.org/10.1016/j.compbiomed.2021.104422

  36. Thurner S, Klimek P, Hanel R (2020) A network-based explanation of why most covid-19 infection curves are linear. Proc Natl Acad Sci 117(37):22684–22689. https://doi.org/10.1073/pnas.2010398117

    Article  Google Scholar 

  37. Tiwari S, Chanak P, Singh SK (2023) A review of the machine learning algorithms for covid-19 case analysis. IEEE Transactions on Artificial Intelligence 4(1):44–59. https://doi.org/10.1109/TAI.2022.3142241

    Article  Google Scholar 

  38. Tsukuda S, Watashi K (2020) Hepatitis b virus biology and life cycle. Antiviral Res 182:104925. https://doi.org/10.1016/j.antiviral.2020.104925

  39. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need. In: Proceedings of the 31st International conference on Neural Information Processing Systems, Red Hook, NY, USA, pp 5998–6008

  40. Wang Z, Cai B (2022) COVID-19 cases prediction in multiple areas via shapelet learning. Appl Intell 52(1):595–606. https://doi.org/10.1007/s10489-021-02391-6

    Article  Google Scholar 

  41. Wang Z, Huang Y, He B, Luo T, Wang Y, Lin Y (2019) TDDF: HFMD outpatients prediction based on time series decomposition and heterogenous data fusion in Xiamen, China. In: Proceedings of the 15th International Conference Advanced Data Miningand Applications, Springer, Dalian, China, pp 658 – 667, https://doi.org/10.1007/978-3-030-35231-8_48

  42. Wang Z, Huang Y, He B (2021) Dual-grained representation for hand, foot, and mouth disease prediction within public health cyber-physical systems. Software: Practice and Experience 51(11):2290 – 2305, https://doi.org/10.1002/spe.2940

  43. Wang Z, Su Q, Chao G, Cai B, Huang Y, Fu Y (2022) A multi-view time series model for share turnover prediction. Appl Intell 52:14595–14606. https://doi.org/10.1007/s10489-021-02979-y

    Article  Google Scholar 

  44. Wu Y, Yang Y, Nishiura H, Saitoh M (2018) Deep learning for epidemiological predictions. In: Proceedings of the 41st International Conference on Research and Development in Information Retrieval, ACM, Ann Arbor, MI, USA, pp 1085 – 1088, https://doi.org/10.1145/3209978.3210077

  45. Xiao J, Zhu Q, Yang F, Zeng S, Zhu Z, Gong D, Li Y, Zhang L, Li B, Zeng W, Li X, Rong Z, Hu J, He G, Sun J, Lu J, Liu T, Ma W, Sun L (2022) The impact of enterovirus a71 vaccination program on hand, foot, and mouth disease in guangdong, china: A longitudinal surveillance study. J Infect 85(4):428–435. https://doi.org/10.1016/j.jinf.2022.06.020

    Article  Google Scholar 

  46. Xu X, Ren W (2022) A hybrid model of stacked autoencoder and modified particle swarm optimization for multivariate chaotic time series forecasting. Appl Soft Comput 116:108321. https://doi.org/10.1016/j.asoc.2021.108321

    Article  Google Scholar 

  47. Yang Z, Zhang Q, Cowling BJ, Lau EH (2017) Estimating the incubation period of hand, foot and mouth disease for children in different age groups. Sci Rep 7(1):16464

    Article  Google Scholar 

  48. Zhang P, Wang Z, Chao G, Huang Y, Yan J (2022a) An oriented attention model for infectious disease cases prediction. In: Proceedings of the 34th International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, Springer, Kitakyushu, Japan, vol 13343, https://doi.org/10.1007/978-3-031-08530-7_11

  49. Zhang P, Wang Z, Huang Y, Wang M (2022) Dual-grained directional representation for infectious disease case prediction. Knowl-Based Syst 256:109806. https://doi.org/10.1016/j.knosys.2022.109806

  50. Zhou H, Zhang S, Peng J, Zhang S, Li J, Xiong H, Zhang W (2021) Informer: Beyond efficient transformer for long sequence time-series forecasting. In: Proceedings of the 35th AAAI Conference on Artificial Intelligence, AAAI Press

Download references

Acknowledgements

This work was supported in part by the Natural Science Foundation of Fujian Province (CN) (nos. 2021J01857, 2021J01859, and 2022J01335). Thanks to the Xiamen City Center for Disease Control and Prevention (XMCDC) for sharing the data. We gratefully appreciate the editor and anonymous reviewers for their valuable insights and suggestions which enormously benefited this paper.

Author information

Authors and Affiliations

  1. College of Computer Engineering, Jimei University, Yinjiang Road 185, Xiamen, 361021, China

    Zhijin Wang & Yonggang Fu

  2. School of Science, Jimei University, Yinjiang Road 185, Xiamen, 361021, China

    Pesiong Zhang

  3. College of Electronic Information, Guangxi Minzu University, Daxue East Road 188, Nanning, 530006, China

    Yaohui Huang

  4. School of Computer Science and Technology, Harbin Institute of Technology, 2 West Culture Road, Weihai, 264209, People’s Republic of China

    Guoqing Chao

  5. School of Information Science and Engineering, Ningbo University, Fenghua Road 818, Ningbo, 315211, China

    Xijiong Xie

Authors
  1. Zhijin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pesiong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yaohui Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Guoqing Chao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xijiong Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yonggang Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhijin Wang.

Ethics declarations

Conflict of interest

None.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Z., Zhang, P., Huang, Y. et al. Oriented transformer for infectious disease case prediction. Appl Intell 53, 30097–30112 (2023). https://doi.org/10.1007/s10489-023-05101-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-023-05101-6

Keywords

Search

Navigation