Xiaona Li
Bin Guo
Abstract
Recently, dynamic graph representation learning has attracted more and more attention from both academic and industrial communities due to its capabilities of capturing different real-world phenomena. For a dynamic graph represented as a sequence of timestamped events, there are two kinds of evolutionary essences: temporal and repetitive information. At present, the temporal information of interactions (e.g., timestamps) have been deeply explored. However, as another vital nature of dynamic graphs, the repetitive information of interactions between two nodes is neglected, which may lead to inaccurate node representation. To address this issue, we propose a novel continuous-time dynamic graph representation learning model, which consists of a node-level-memory based module, a historical high-order neighborhood based vertical aggregation module and a repetitive-topological information based horizontal aggregation module. In particular, to characterize the evolving pattern of the repetitive information of interactions between a pair of nodes, we put forward a repetitive-interaction based attention mechanism to integrate the two key attributes (i.e., the content and the number of interactions) of repetitive interactions at different moments, based on the insight that the repetitive behaviors of nodes are widespread and essential. We conduct extensive experiments including future link prediction tasks (for transductive and inductive learning) and dynamic node classification task, and results on three real-life dynamic graph datasets demonstrate that the proposed method significantly outperforms state-of-the-art baselines, for both observed nodes and new ones.
- [1] . 2020. Continuous-time dynamic graph learning via neural interaction processes. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 145–154.Google Scholar
Digital Library
- [2] . 2020. Predictive assembling model reveals the self-adaptive elastic properties of lamellipodial actin networks for cell migration. Communications Biology 3, 1 (2020), 1–17.Google ScholarCross Ref
- [3] . 2019. Traffic graph convolutional recurrent neural network: A deep learning framework for network-scale traffic learning and forecasting. IEEE Transactions on Intelligent Transportation Systems 21, 11 (2019), 4883–4894.Google ScholarCross Ref
- [4] . 2019. Graph neural networks for social recommendation. In Proceedings of the World Wide Web Conference. 417–426.Google ScholarDigital Library
- [5] . 2017. Protein interface prediction using graph convolutional networks. In Proceedings of the Advances in Neural Information Processing Systems.Google Scholar
- [6] . 2016. node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 855–864.Google ScholarDigital Library
- [7] . 2017. Inductive representation learning on large graphs. In Proceedings of the Advances in Neural Information Processing Systems. 1024–1034.Google Scholar
- [8] . 2020. English-to-japanese diverse translation by combining forward and backward outputs. In Proceedings of the Fourth Workshop on Neural Generation and Translation. 134.Google Scholar
- [9] . 2008. ROS-generating mitochondrial DNA mutations can regulate tumor cell metastasis. Science 320, 5876 (2008), 661–664.Google ScholarCross Ref
- [10] . 2019. Relational representation learning for dynamic (knowledge) graphs: A survey. arXiv:1905.11485. Retrieved from https://arxiv.org/abs/1905.11485.Google Scholar
- [11] . 2020. Representation learning for dynamic graphs: A survey. Journal of Machine Learning Research 21, 70 (2020), 1–73.Google Scholar
- [12] . 2016. Variational graph auto-encoders. arXiv:1611.07308. Retrieved from https://arxiv.org/abs/1611.07308.Google Scholar
- [13] . 2017. Semi-supervised classification with graph convolutional networks. ICLR (Poster) (2017). arXiv:1609.02907. Retrieved from https://arxiv.org/abs/1609.02907.Google Scholar
- [14] . 2019. Active cell migration is critical for steady-state epithelial turnover in the gut. Science 365, 6454 (2019), 705–710.Google ScholarCross Ref
- [15] . 2020. Habitual behavior is goal-driven. Perspectives on Psychological Science 15, 5 (2020), 1256–1271.Google ScholarCross Ref
- [16] . 2019. Predicting dynamic embedding trajectory in temporal interaction networks. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 1269–1278.Google ScholarDigital Library
- [17] . 2022. Graph neural networks in urban intelligence. In Proceedings of the Graph Neural Networks: Foundations, Frontiers, and Applications. Springer, 579–593.Google ScholarCross Ref
- [18] . 2020. HAMBox: Delving into mining high-quality anchors on face detection. In Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 13043–13051.Google ScholarCross Ref
- [19] . 2019. Temporal network embedding with micro-and macro-dynamics. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 469–478.Google ScholarDigital Library
- [20] . 2020. Streaming graph neural networks. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 719–728.Google ScholarDigital Library
- [21] . 2020. Dynamic graph convolutional networks. Pattern Recognition 97 (2020), 107000.Google ScholarDigital Library
- [22] . 2020. Evolvegcn: Evolving graph convolutional networks for dynamic graphs. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 5363–5370.Google ScholarCross Ref
- [23] . 2017. Cancer cell motility: Lessons from migration in confined spaces. Nature Reviews Cancer 17, 2 (2017), 131.Google ScholarCross Ref
- [24] . 2014. Deepwalk: Online learning of social representations. In Proceedings of the ACM International Conference on Information and Knowledge Management. 701–710.Google ScholarDigital Library
- [25] . 2020. Continuous-time link prediction via temporal dependent graph neural network. In Proceedings of the World Wide Web Conference. 3026–3032.Google ScholarDigital Library
- [26] . 2020. Temporal graph networks for deep learning on dynamic graphs. arXiv:2006.10637. Retrieved from https://arxiv.org/abs/2006.10637.Google Scholar
- [27] . 2020. Dysat: Deep neural representation learning on dynamic graphs via self-attention networks. In Proceedings of the 13th International Conference on Web Search and Data Mining. 519–527.Google ScholarDigital Library
- [28] . 2022. Graph neural networks for communication networks: Context, use cases and opportunities. IEEE Network (2022).Google ScholarCross Ref
- [29] . 2015. Line: Large-scale information network embedding. In Proceedings of the World Wide Web Conference. 1067–1077.Google ScholarDigital Library
- [30] . 2019. Dyrep: Learning representations over dynamic graphs. In Proceedings of the International Conference on Learning Representations (Poster) 2019.Google Scholar
- [31] . 2018. Graph attention networks. Stat 1050, 20 (2017), 10–48550.Google Scholar
- [32] . 2016. Structural deep network embedding. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 1225–1234.Google ScholarDigital Library
- [33] . 2021. A review on graph neural network methods in financial applications. arXiv:2111.15367. Retrieved from https://arxiv.org/abs/2111.15367.Google Scholar
- [34] . 2017. Community preserving network embedding. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 31.Google ScholarCross Ref
- [35] . 2021. Inductive representation learning in temporal networks via causal anonymous walks. arXiv:2101.05974. Retrieved from https://arxiv.org/abs/2101.05974.Google Scholar
- [36] . 2023. Exploring multi-dimension user-item interactions with attentional knowledge graph neural networks for recommendation. IEEE Transactions on Big Data 9, 1 (2023), 212–226.Google ScholarCross Ref
- [37] . 2014. Discovering and profiling overlapping communities in location-based social networks. IEEE Transactions on Systems, Man, and Cybernetics: Systems 44, 4 (2014), 499–509.Google ScholarCross Ref
- [38] . 2009. The habitual consumer. Journal of Consumer Psychology 19, 4 (2009), 579–592.Google ScholarCross Ref
- [39] . 2020. EvoNet: A neural network for predicting the evolution of dynamic graphs. In Proceedings of the Artificial Neural Networks and Machine Learning ICANN 2020: 29th International Conference on Artificial Neural Networks. Springer, 594–606.Google ScholarDigital Library
- [40] . 2020. Inductive representation learning on temporal graphs. In ICLR (2020). 1–19.Google Scholar
- [41] . 2021. Graph neural networks and their current applications in bioinformatics. Frontiers in Genetics 12 (2021), 690049.Google Scholar
- [42] . 2020. Learning temporal interaction graph embedding via coupled memory networks. In Proceedings of the World Wide Web Conference. 3049–3055.Google ScholarDigital Library
- [43] . 2019. Large scale evolving graphs with burst detection. In Proceedings of the IJCAI. 4412–4418.Google ScholarCross Ref
- [44] . 2018. Graph neural networks: A review of methods and applications. arXiv:1812.08434. Retrieved from https://arxiv.org/abs/1812.08434.Google Scholar
- [45] . 2020. Automating botnet detection with graph neural networks. arXiv:2003.06344. Retrieved from https://arxiv.org/abs/2003.06344.Google Scholar
- [46] . 2018. Dynamic network embedding by modeling triadic closure process. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 32.Google ScholarCross Ref
Index Terms
- A Hybrid Continuous-Time Dynamic Graph Representation Learning Model by Exploring Both Temporal and Repetitive Information
Recommendations
Predictive temporal embedding of dynamic graphs
ASONAM '19: Proceedings of the 2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and MiningIn recent years, substantial effort has been devoted to learning to represent the static graphs and their substructures. A few studies explored utilizing temporal information available in a dynamic setting in order to address the node representation ...
Multi-relational dynamic graph representation learning
AbstractIn recent years, many dynamic graph representation learning methods have emerged due to the ubiquity of dynamic graph networks, such as social networks, medical networks, citation networks and traffic networks. Many researchers only consider the ...
DDGK: Learning Graph Representations for Deep Divergence Graph Kernels
WWW '19: The World Wide Web ConferenceCan neural networks learn to compare graphs without feature engineering? In this paper, we show that it is possible to learn representations for graph similarity with neither domain knowledge nor supervision (i.e. feature engineering or labeled graphs). ...
Comments