Abstract
Event sequences with spatiotemporal characteristics have been rapidly produced in various domains, such as earthquakes in seismology, electronic medical records in health care, and transactions in the financial market. These data are discrete events that are continuous and often continue for weeks, months, or years, and past events may trigger subsequent events. In this context, modeling spatiotemporal event sequences and forecasting the occurrence time and marker of the next event has become a hot topic. However, existing models either fail to capture the long-term temporal dependencies or ignore the essential spatial information between sequences. Moreover, existing models learn the influence of past events and predicted future events in Euclidean space, which has been shown to cause a significant distortion in hierarchical structure data. To correctly predict future events from historical events and design interventions and controls to guide the event dynamics to the desired results and inspired by the high capacity of modeling data in hyperbolic space, we proposed a novel hyperbolic graph transformer Hawkes process (HGTHP) model to capture the long-term temporal dependencies and spatial information from historical events with a hierarchical structure. The core concept of the HGTHP is to integrate the learned spatial information into the event embedding as auxiliary information and capture long-short term temporal dependencies from event sequences in non-Euclidean space by a hyperbolic self-attention mechanism. Numerous experiments on synthetic and real-world datasets proved that the proposed model obtains spatiotemporal information from hyperbolic space, and its predictions outperform those of state-of-the-art baselines in both time and marker, proving the proposed model’s effectiveness.
Similar content being viewed by others
References
Zhang L, Liu J, Zuo X (2023) Doubly time-dependent hawkes process and applications in failure sequence analysis. Comput Stat 38(2):1057–1093
Okawa M, Iwata T, Tanaka Y, Kurashima T, Toda H, Kashima H (2022) Context-aware spatio-temporal event prediction via convolutional hawkes processes. Mach Learn 111(8):2929–2950
Cui Z, Sun X, Pan L, Liu S, Xu G (2023) Event-based incremental recommendation via factors mixed hawkes process. Inf Sci 639:119007
Jiang Y, Porter M.D (2022) Simulating fake news dissemination on twitter with multivariate hawkes processes. In: IEEE international conference on big data, big data 2022, Osaka, Japan, December 17-20, 2022, pp 3597–3606
Shi Z, Cartlidge J (2022) State dependent parallel neural hawkes process for limit order book event stream prediction and simulation. In: KDD ’22: The 28th ACM SIGKDD conference on knowledge discovery and data mining, Washington, DC, USA, August 14 - 18, 2022, pp 1607–1615
Ye F, Hu Z, Yilmaz E (2023) Modeling user satisfaction dynamics in dialogue via hawkes process. In: Proceedings of the 61st annual meeting of the association for computational linguistics (Volume 1: Long Papers), ACL 2023, Toronto, Canada, July 9-14, 2023, pp 8875–8889
Bronstein MM, Bruna J, LeCun Y, Szlam A, Vandergheynst P (2017) Geometric deep learning: going beyond euclidean data. IEEE Signal Process Mag 34(4):18–42
Khrulkov V, Mirvakhabova L, Ustinova E, Oseledets IV, Lempitsky VS (2020) Hyperbolic image embeddings. In: 2020 IEEE/CVF conference on computer vision and pattern recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020, pp 6417–6427
Sala F, Sa CD, Gu A, Ré C (2018) Representation tradeoffs for hyperbolic embeddings. In: Proceedings of the 35th international conference on machine learning, ICML 2018, vol 80. Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018, pp 4457–4466
Gülçehre Ç Denil M, Malinowski M, Razavi A, Pascanu R, Hermann KM, Battaglia PW, Bapst V, Raposo D, Santoro A, Freitas N (2019) Hyperbolic attention networks. In: 7th international conference on learning representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019
Dhingra B, Shallue CJ, Norouzi M, Dai AM, Dahl GE (2018) Embedding text in hyperbolic spaces. In: Proceedings of the Twelfth Workshop on Graph-Based Methods for Natural Language Processing, TextGraphs@NAACL-HLT 2018, New Orleans, Louisiana, USA, June 6, 2018, pp 59–69
Peng W, Varanka T, Mostafa A, Shi H, Zhao G (2022) Hyperbolic deep neural networks: a survey. IEEE Trans Pattern Anal Mach Intell 44(12):10023–10044
Sarkar R (2011) Low distortion delaunay embedding of trees in hyperbolic plane. In: Graph drawing - 19th international symposium, GD 2011, vol 7034. Eindhoven, The Netherlands, September 21-23, 2011, Revised Selected Papers, pp 355–366
Hawkes AG (1971) Spectra of some self-exciting and mutually exciting point processes. Biometrika 58(1):83–90
Ogata Y (1988) Statistical models for earthquake occurrences and residual analysis for point processes. J Am Stat Assoc 83(401):9–27
Li L, Zha H (2013) Dyadic event attribution in social networks with mixtures of hawkes processes. In: 22nd ACM international conference on information and knowledge management, CIKM’13, San Francisco, CA, USA, October 27 - November 1, 2013, pp 1667–1672
Kobayashi R, Lambiotte R (2016) Tideh: time-dependent hawkes process for predicting retweet dynamics. In: Proceedings of the 10th international conference on web and social media, Cologne, Germany, May 17-20, 2016, pp 191–200
Li L, Zha H (2016) Household structure analysis via hawkes processes for enhancing energy disaggregation. In: Kambhampati S (ed), Proceedings of the twenty-fifth international joint conference on artificial intelligence, IJCAI 2016, New York, NY, USA, 9-15 July 2016, pp 2553–2559
Salehi F, Trouleau W, Grossglauser M, Thiran P (2019) Learning hawkes processes from a handful of events. In: Advances in neural information processing systems 32: annual conference on neural information processing systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp 12694 –12704
Du N, Dai H, Trivedi R, Upadhyay U, Gomez-Rodriguez M, Song L (2016) Recurrent marked temporal point processes: embedding event history to vector. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp 1555–1564
Xiao S, Yan J, Yang X, Zha H, Chu SM (2017) Modeling the intensity function of point process via recurrent neural networks. In: Proceedings of the Thirty-First AAAI conference on artificial intelligence, February 4-9, 2017, San Francisco, California, USA, pp 1597–1603
Mei H, Eisner J (2017) The neural hawkes process: a neurally self-modulating multivariate point process. In: Advances in neural information processing systems 30: annual conference on neural information processing systems 2017, December 4-9, 2017, Long Beach, CA, USA, pp 6754–6764
Zhang Q, Lipani A, Kirnap Ö, Yilmaz E (2020) Self-attentive hawkes process. In: Proceedings of the 37th international conference on machine learning, ICML 2020, 13-18 July 2020, Virtual Event, vol 119, pp 11183–11193
Zuo S, Jiang H, Li Z, Zhao T, Zha H (2020) Transformer hawkes process. In: Proceedings of the 37th international conference on machine learning, ICML 2020, 13-18 July 2020, Virtual Event, vol 119, pp 11692–11702
Liu Y, Yan T, Chen H (2018) Exploiting graph regularized multi-dimensional hawkes processes for modeling events with spatio-temporal characteristics. In: Lang J (ed) Proceedings of the twenty-seventh international joint conference on artificial intelligence, IJCAI 2018, July 13-19, 2018, Stockholm, Sweden, pp 2475–2482
Yu L, Xu X, Trajcevski G, Zhou F (2022) Transformer-enhanced hawkes process with decoupling training for information cascade prediction. Knowl Based Syst 255:109740
Shang J, Sun M (2019) Geometric hawkes processes with graph convolutional recurrent neural networks. In: The Thirty-Third AAAI conference on artificial intelligence, AAAI 2019, The thirty-first innovative applications of artificial intelligence conference, IAAI 2019, The Ninth AAAI symposium on educational advances in artificial intelligence, EAAI 2019, Honolulu, Hawaii, USA, January 27 - February 1, 2019, pp 4878–4885
Du H, Zhou Y, Ma Y, Wang S (2021) Astrologer: exploiting graph neural hawkes process for event propagation prediction with spatio-temporal characteristics. Knowl Based Syst 228:107247
Du H, Zhou Y (2022) Nostradamus: A novel event propagation prediction approach with spatio-temporal characteristics in non-euclidean space. Neural Netw 145:386–394
Zhu S, Ding R, Zhang M, Hentenryck PV, Xie Y (2022) Spatio-temporal point processes with attention for traffic congestion event modeling. IEEE Trans Intell Transp Syst 23(7):7298–7309
Chiang W-H, Liu X, Mohler G (2022) Hawkes process modeling of covid-19 with mobility leading indicators and spatial covariates. Int J Forecast 38(2):505–520
Jiang L, Wang P, Cheng K, Liu K, Yin M, Jin B, Fu Y (2021) Eduhawkes: a neural hawkes process approach for online study behavior modeling. In: Proceedings of the 2021 SIAM international conference on data mining, SDM 2021, Virtual Event, April 29 - May 1, 2021, pp 567–575
Gülçehre Ç, Denil M, Malinowski M, Razavi A, Pascanu R, Hermann KM, Battaglia PW, Bapst V, Raposo D, Santoro A, Freitas N (2019) Hyperbolic attention networks. In: 7th international conference on learning representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019
Sawhney R, Agarwal S, Thakkar M, Wadhwa A, Shah RR (2021) Hyperbolic online time stream modeling. In: Proceedings of the 44th International ACM SIGIR conference on research and development in information retrieval, pp 1682–1686
Chami I, Ying Z, Ré C, Leskovec J (2019) Hyperbolic graph convolutional neural networks. In: Advances in neural information processing systems 32: annual conference on neural information processing systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp 4869–4880
Dai J, Wu Y, Gao Z, Jia Y (2021) A hyperbolic-to-hyperbolic graph convolutional network. In: IEEE conference on computer vision and pattern recognition, CVPR 2021, Virtual, June 19-25, 2021, pp 154–163
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need. In: Advances in neural information processing systems 30: annual conference on neural information processing systems 2017, December 4-9, 2017, Long Beach, CA, USA, pp 5998–6008
Li Y, Du N, Bengio S (2018) Time-dependent representation for neural event sequence prediction. In: 6th international conference on learning representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Workshop Track Proceedings
Zhou K, Zha H, Song L (2013) Learning triggering kernels for multi-dimensional hawkes processes. In: Proceedings of the 30th International Conference on Machine Learning, ICML 2013, vol 28. Atlanta, GA, USA, 16-21 June 2013, pp 1301–1309
Grant S, Betts B (2013) Encouraging user behaviour with achievements: an empirical study. In: Proceedings of the 10th working conference on mining software repositories, MSR ’13, San Francisco, CA, USA, May 18-19, 2013, pp 65–68
Monti F, Bronstein MM, Bresson X (2017) Geometric matrix completion with recurrent multi-graph neural networks. In: Advances in neural information processing systems 30: annual conference on neural information processing systems 2017, December 4-9, 2017, Long Beach, CA, USA, pp 3697–3707
Shchur O, Bilos M, Günnemann S (2020) Intensity-free learning of temporal point processes. In: 8th international conference on learning representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Xie, Y., Wu, J. HGTHP: a novel hyperbolic geometric transformer hawkes process for event prediction. Appl Intell 54, 357–374 (2024). https://doi.org/10.1007/s10489-023-05169-0
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-023-05169-0