Skip to main content
SpringerLink
Log in

H\(^2\)-Nets: Hyper-hodge Convolutional Neural Networks for Time-Series Forecasting

  • Conference paper
  • First Online:
Machine Learning and Knowledge Discovery in Databases: Research Track (ECML PKDD 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14173))

Abstract

Hypergraphs recently have emerged as a new promising alternative to describe complex dependencies in spatio-temporal processes, resulting in the newest trend in multivariate time series forecasting, based semi-supervised learning of spatio-temporal data with Hypergraph Convolutional Networks. Nevertheless, such recent approaches are often limited in their capability to accurately describe higher-order interactions among spatio-temporal entities and to learn hidden interrelations among network substructures. Motivated by the emerging results on simplicial convolution, we introduce the concepts of Hodge theory and Hodge Laplacians, that is, a higher-order generalization of the graph Laplacian, to hypergraph learning. Furthermore, we develop a novel framework for hyper-simplex-graph representation learning which describes complex relationships among both graph and hyper-simplex-graph simplices and, as a result, simultaneously extracts latent higher-order spatio-temporal dependencies. We provide theoretical foundations behind the proposed hyper-simplex-graph representation learning and validate our new Hodge-style Hyper-simplex-graph Neural Networks (H\(^2\)-Nets) on 7 real world spatio-temporal benchmark datasets. Our experimental results indicate that H\(^2\)-Nets outperforms the state-of-the-art methods by a significant margin, while demonstrating lower computational costs.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Agarwal, S., Branson, K., Belongie, S.: Higher order learning with graphs. In: Proceedings of the International Conference on Machine Learning, pp. 17–24 (2006)

    Google Scholar 

  2. Arya, D., Gupta, D.K., Rudinac, S., Worring, M.: Hypersage: generalizing inductive representation learning on hypergraphs. arXiv preprint arXiv:2010.04558 (2020)

  3. Atwood, J., Towsley, D.: Diffusion-convolutional neural networks. In: Advances in Neural Information Processing Systems, vol. 29. Curran Associates, Inc. (2016)

    Google Scholar 

  4. Bai, L., Yao, L., Li, C., Wang, X., Wang, C.: Adaptive graph convolutional recurrent network for traffic forecasting. In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M.F., Lin, H. (eds.) Advances in Neural Information Processing Systems, vol. 33, pp. 17804–17815. Curran Associates, Inc. (2020)

    Google Scholar 

  5. Bai, S., Kolter, J.Z., Koltun, V.: An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271 (2018)

  6. Barbarossa, S., Sardellitti, S., Ceci, E.: Learning from signals defined over simplicial complexes. In: IEEE Data Science Workshop. IEEE (2018)

    Google Scholar 

  7. Benson, A.R., Abebe, R., Schaub, M.T., Jadbabaie, A., Kleinberg, J.: Simplicial closure and higher-order link prediction. Proc. Natl. Acad. Sci. USA 115(48), E11221–E11230 (2018)

    Google Scholar 

  8. Bilal, U., Jemmott, J.B., Schnake-Mahl, A., Murphy, K., Momplaisir, F.: Racial/ethnic and neighbourhood social vulnerability disparities in COVID-19 testing positivity, hospitalization, and in-hospital mortality in a large hospital system in Pennsylvania: a prospective study of electronic health records. Lancet Reg. Health-Am. 10, 100220 (2022)

    Google Scholar 

  9. Bodnar, C., et al.: Weisfeiler and lehman go topological: message passing simplicial networks. In: ICLR 2021 Workshop on Geometrical and Topological Representation Learning (2021)

    Google Scholar 

  10. Bolla, M.: Spectra, euclidean representations and clusterings of hypergraphs. Discret. Math. 117(1–3), 19–39 (1993)

    Google Scholar 

  11. Bronstein, M.M., Bruna, J., Cohen, T., Veličković, P.: Geometric deep learning: grids, groups, graphs, geodesics, and gauges (2021)

    Google Scholar 

  12. Bunch, E., You, Q., Fung, G., Singh, V.: Simplicial 2-complex convolutional neural networks. In: NeurIPS 2020 Workshop on Topological Data Analysis and Beyond (2020)

    Google Scholar 

  13. Cao, D., et al.: Spectral temporal graph neural network for multivariate time-series forecasting. In: Advances in Neural Information Processing Systems (2020)

    Google Scholar 

  14. Chen, Y., Jiang, T., Heleno, M., Moreira, A., Gel, Y.R.: Evaluating distribution system reliability with hyperstructures graph convolutional nets. In: IEEE International Conference on Big Data, pp. 1793–1800 (2022)

    Google Scholar 

  15. Chen, Y., Batsakis, S., Poor, H.V.: Higher-order spatio-temporal neural networks for COVID-19 forecasting. In: IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 1–5. IEEE (2023)

    Google Scholar 

  16. Chen, Y., Gel, Y., Poor, H.V.: Time-conditioned dances with simplicial complexes: zigzag filtration curve based supra-hodge convolution networks for time-series forecasting. Adv. Neural. Inf. Process. Syst. 35, 8940–8953 (2022)

    Google Scholar 

  17. Chen, Y., Gel, Y.R., Poor, H.V.: BScNets: block simplicial complex neural networks. In: Proceedings of the AAAI Conference on Artificial Intelligence (2022)

    Google Scholar 

  18. Chen, Y., Heleno, M., Moreira, A., Gel, Y.R.: Topological graph convolutional networks solutions for power distribution grid planning. In: Pacific-Asia Conference on Knowledge Discovery and Data Mining, pp. 123–134 (2023)

    Google Scholar 

  19. Chen, Y., Jacob, R.A., Gel, Y.R., Zhang, J., Poor, H.V.: Learning power grid outages with higher-order topological neural networks. IEEE Trans. Power Syst. (2023)

    Google Scholar 

  20. Chen, Y., Segovia, I., Gel, Y.R.: Z-GCNETs: time zigzags at graph convolutional networks for time series forecasting. In: Proceedings of the International Conference on Machine Learning, pp. 1684–1694 (2021)

    Google Scholar 

  21. Chen, Y., Segovia-Dominguez, I., Coskunuzer, B., Gel, Y.: Tamp-s2gcnets: coupling time-aware multipersistence knowledge representation with spatio-supra graph convolutional networks for time-series forecasting. In: International Conference on Learning Representations (2022)

    Google Scholar 

  22. Ebli, S., Defferrard, M., Spreemann, G.: Simplicial neural networks. In: Topological Data Analysis and Beyond workshop at Advances in Neural Information Processing Systems (2020)

    Google Scholar 

  23. Feng, Y., You, H., Zhang, Z., Ji, R., Gao, Y.: Hypergraph neural networks. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 3558–3565 (2019)

    Google Scholar 

  24. Geng, X., et al.: Spatiotemporal multi-graph convolution network for ride-hailing demand forecasting. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 3656–3663 (2019)

    Google Scholar 

  25. Gilmer, J., Schoenholz, S.S., Riley, P.F., Vinyals, O., Dahl, G.E.: Neural message passing for quantum chemistry. In: Precup, D., Teh, Y.W. (eds.) Proceedings of the International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 70, pp. 1263–1272. PMLR (2017)

    Google Scholar 

  26. Guo, S., Lin, Y., Feng, N., Song, C., Wan, H.: Attention based spatial-temporal graph convolutional networks for traffic flow forecasting. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 922–929 (2019)

    Google Scholar 

  27. Hajij, M., Istvan, K., Zamzmi, G.: Cell complex neural networks. In: NeurIPS 2020 Workshop on Topological Data Analysis and Beyond (2020)

    Google Scholar 

  28. Hamilton, W.L., Ying, R., Leskovec, J.: Inductive representation learning on large graphs. In: Advances in Neural Information Processing Systems (2017)

    Google Scholar 

  29. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  30. Hsiao, W.Y., Liu, J.Y., Yeh, Y.C., Yang, Y.H.: Compound word transformer: learning to compose full-song music over dynamic directed hypergraphs. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 4503–4511 (2021)

    Google Scholar 

  31. Huang, J., Yang, J.: UniGNN: a unified framework for graph and hypergraph neural networks. In: Proceedings of the International Joint Conference on Artificial Intelligence (2021)

    Google Scholar 

  32. Jiang, J., Wei, Y., Feng, Y., Cao, J., Gao, Y.: Dynamic hypergraph neural networks. In: Proceedings of the International Joint Conference on Artificial Intelligence (2019)

    Google Scholar 

  33. Karimi, A.M., Wu, Y., Koyuturk, M., French, R.H.: Spatiotemporal graph neural network for performance prediction of photovoltaic power systems. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 15323–15330 (2021)

    Google Scholar 

  34. Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: Proceedings of the International Conference on Learning Representations (2017)

    Google Scholar 

  35. Lai, G., Chang, W.C., Yang, Y., Liu, H.: Modeling long-and short-term temporal patterns with deep neural networks. In: ACM SIGIR Conference on Research & Development in Information Retrieval, pp. 95–104 (2018)

    Google Scholar 

  36. Li, Y., Yu, R., Shahabi, C., Liu, Y.: Diffusion convolutional recurrent neural network: data-driven traffic forecasting. In: Proceedings of the International Conference on Learning Representations (2018)

    Google Scholar 

  37. Liang, Y., et al.: UrbanFM: inferring fine-grained urban flows. In: Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM (2019)

    Google Scholar 

  38. Lim, L.H.: Hodge laplacians on graphs. SIAM Rev. 62(3), 685–715 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  39. Oreshkin, B.N., Carpov, D., Chapados, N., Bengio, Y.: N-beats: neural basis expansion analysis for interpretable time series forecasting. In: Proceedings of the International Conference on Learning Representations (2019)

    Google Scholar 

  40. Pan, Z., Liang, Y., Wang, W., Yu, Y., Zheng, Y., Zhang, J.: Urban traffic prediction from spatio-temporal data using deep meta learning. In: Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM (2019)

    Google Scholar 

  41. Rangapuram, S.S., Seeger, M.W., Gasthaus, J., Stella, L., Wang, Y., Januschowski, T.: Deep state space models for time series forecasting. In: Advances in Neural Information Processing Systems, vol. 31 (2018)

    Google Scholar 

  42. Reitsma, M.B., et al.: Racial/ethnic disparities in COVID-19 exposure risk, testing, and cases at the subcounty level in California: study examines racial/ethnic disparities in COVID-19 risk, testing, and cases. Health Aff. 40(6), 870–878 (2021)

    Article  MathSciNet  Google Scholar 

  43. Roddenberry, T.M., Glaze, N., Segarra, S.: Principled simplicial neural networks for trajectory prediction. In: Meila, M., Zhang, T. (eds.) Proceedings of the 38th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 139, pp. 9020–9029. PMLR (2021)

    Google Scholar 

  44. Roddenberry, T.M., Segarra, S.: HodgeNet: graph neural networks for edge data. In: The 53rd Asilomar Conference on Signals, Systems, and Computers. IEEE (2019)

    Google Scholar 

  45. Sawhney, R., Agarwal, S., Wadhwa, A., Derr, T., Shah, R.R.: Stock selection via spatiotemporal hypergraph attention network: a learning to rank approach. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 497–504 (2021)

    Google Scholar 

  46. Schaub, M.T., Benson, A.R., Horn, P., Lippner, G., Jadbabaie, A.: Random walks on simplicial complexes and the normalized hodge 1-laplacian. SIAM Rev. 62(2), 353–391 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  47. Schaub, M.T., Zhu, Y., Seby, J.B., Roddenberry, T.M., Segarra, S.: Signal processing on higher-order networks: livin’on the edge... and beyond. Signal Process. 187, 108149 (2021)

    Google Scholar 

  48. Sen, R., Yu, H.F., Dhillon, I.S.: Think globally, act locally: a deep neural network approach to high-dimensional time series forecasting. In: Advances in Neural Information Processing Systems, vol. 32 (2019)

    Google Scholar 

  49. Song, C., Lin, Y., Guo, S., Wan, H.: Spatial-temporal synchronous graph convolutional networks: A new framework for spatial-temporal network data forecasting. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 01, pp. 914–921 (2020)

    Google Scholar 

  50. Srinivasan, D., Chan, C.W., Balaji, P.: Computational intelligence-based congestion prediction for a dynamic urban street network. Neurocomputing 72(10–12), 2710–2716 (2009)

    Article  Google Scholar 

  51. Sun, X., et al.: Heterogeneous hypergraph embedding for graph classification. In: Proceedings of the ACM International Conference on Web Search and Data Mining, pp. 725–733 (2021)

    Google Scholar 

  52. Tran, A., Mathews, A., Ong, C.S., Xie, L.: Radflow: a recurrent, aggregated, and decomposable model for networks of time series. In: Proceedings of the International World Wide Web Conference, pp. 730–742 (2021)

    Google Scholar 

  53. Wang, J., Zhang, Y., Wei, Y., Hu, Y., Piao, X., Yin, B.: Metro passenger flow prediction via dynamic hypergraph convolution networks. IEEE Trans. Intell. Transp. Syst. 22(12), 7891–7903 (2021)

    Article  Google Scholar 

  54. Wu, S., Rizoiu, M.A., Xie, L.: Estimating attention flow in online video networks. Proc. ACM Hum.-Comput. Interact. 3, 1–25 (2019)

    Google Scholar 

  55. Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., Yu, P.S.: A comprehensive survey on graph neural networks. IEEE Trans. Neural Netw. Learn. Syst. 32(1), 4–24 (2021)

    Article  MathSciNet  Google Scholar 

  56. Wu, Z., Pan, S., Long, G., Jiang, J., Zhang, C.: Graph WaveNet for deep spatial-temporal graph modeling. In: Proceedings of the International Joint Conference on Artificial Intelligence. International Joint Conferences on Artificial Intelligence Organization (2019)

    Google Scholar 

  57. Xia, L., Huang, C., Xu, Y., Dai, P., Bo, L., Zhang, X., Chen, T.: Spatial-temporal sequential hypergraph network for crime prediction with dynamic multiplex relation learning. In: Proceedings of the International Joint Conference on Artificial Intelligence (2021)

    Google Scholar 

  58. Yadati, N., Nimishakavi, M., Yadav, P., Nitin, V., Louis, A., Talukdar, P.: Hypergcn: a new method of training graph convolutional networks on hypergraphs. In: Advances in Neural Information Processing Systems, pp. 1511–1522 (2019)

    Google Scholar 

  59. Yan, S., Xiong, Y., Lin, D.: Spatial temporal graph convolutional networks for skeleton-based action recognition. In: Proceedings of the AAAI Conference on Artificial Intelligence (2018)

    Google Scholar 

  60. Yao, H., Tang, X., Wei, H., Zheng, G., Li, Z.: Revisiting spatial-temporal similarity: a deep learning framework for traffic prediction. In: Proceedings of the AAAI Conference on Artificial Intelligence (2019)

    Google Scholar 

  61. Yao, H., et al.: Deep multi-view spatial-temporal network for taxi demand prediction. In: Proceedings of the AAAI Conference on Artificial Intelligence (2018)

    Google Scholar 

  62. Yi, J., Park, J.: Hypergraph convolutional recurrent neural network. In: Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery & Data Mining August 2020. ACM (2020)

    Google Scholar 

  63. Yin, H., Benson, A.R., Leskovec, J.: Higher-order clustering in networks. Phys. Rev. E 97(5), 052306 (2018)

    Article  Google Scholar 

  64. Yin, N., et al.: Dynamic hypergraph convolutional network. In: IEEE International Conference on Data Engineering (2022)

    Google Scholar 

  65. Yu, B., Yin, H., Zhu, Z.: Spatio-temporal graph convolutional networks: a deep learning framework for traffic forecasting. In: Proceedings of the International Joint Conference on Artificial Intelligence. International Joint Conferences on Artificial Intelligence Organization (2018)

    Google Scholar 

  66. Zhang, J., Zheng, Y., Qi, D.: Deep spatio-temporal residual networks for citywide crowd flows prediction. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 1655–1661 (2017)

    Google Scholar 

  67. Zhang, J., Zheng, Y., Qi, D., Li, R., Yi, X.: DNN-based prediction model for spatio-temporal data. In: Proceedings of the ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems. ACM (2016)

    Google Scholar 

  68. Zheng, W., Yan, L., Gou, C., Wang, F.Y.: Two heads are better than one: hypergraph-enhanced graph reasoning for visual event ratiocination. In: Proceedings of the International Conference on Machine Learning, vol. 139, pp. 12747–12760 (2021)

    Google Scholar 

Download references

Acknowledgements

This work was supported by the NASA grant # 21-AIST21_2-0059 and the ONR grant # N00014-21-1-2530. The views expressed in the article do not necessarily represent the views of the NASA and ONR.

Author information

Authors and Affiliations

  1. Temple University, Philadelphia, PA, 19122, USA

    Yuzhou Chen

  2. University of Texas at Dallas, Richardson, TX, 75080, USA

    Tian Jiang & Yulia R. Gel

  3. National Science Foundation, Alexandria, VA, 22314, USA

    Yulia R. Gel

Authors
  1. Yuzhou Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tian Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yulia R. Gel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yuzhou Chen .

Editor information

Editors and Affiliations

  1. University of Michigan, Ann Arbor, MI, USA

    Danai Koutra

  2. University of Vienna, Vienna, Austria

    Claudia Plant

  3. Max Planck Institute for Software Systems, Kaiserslautern, Germany

    Manuel Gomez Rodriguez

  4. Politecnico di Torino, Turin, Italy

    Elena Baralis

  5. CENTAI, Turin, Italy

    Francesco Bonchi

Ethics declarations

Ethical Statement

The H\(^2\)-Nets approach and, more generally, the ideas of Hodge theory open a new pathway for learning the key multi-node interactions in many domains associated with spatio-temporal data analysis where such critical higher-order interactions are typically neglected. Such applications range from wildfire plume tracking in complex terrains to bio-threat surveillance to human mobility monitoring. While we do not anticipate any negative societal impacts of the proposed H\(^2\)-Nets ideas and the concepts of the Hodge theory, it is important to emphasise that we currently lack any formal inferential tools to quantify the uncertainties associated with learning high-order interactions, which limits our abilities in risk quantification as well as interpretability and explainability.

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Chen, Y., Jiang, T., Gel, Y.R. (2023). H\(^2\)-Nets: Hyper-hodge Convolutional Neural Networks for Time-Series Forecasting. In: Koutra, D., Plant, C., Gomez Rodriguez, M., Baralis, E., Bonchi, F. (eds) Machine Learning and Knowledge Discovery in Databases: Research Track. ECML PKDD 2023. Lecture Notes in Computer Science(), vol 14173. Springer, Cham. https://doi.org/10.1007/978-3-031-43424-2_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-43424-2_17

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-43423-5

  • Online ISBN: 978-3-031-43424-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation