Abstract
In this paper, we study the problem of learning graph embeddings for dynamic networks and the ability to generalize to unseen nodes called inductive learning. Firstly, we overview the state-of-the-art methods and techniques for constructing graph embeddings and learning algorithms for both transductive and inductive approaches. Secondly, we propose an improved GSM based on GraphSAGE algorithm and set up the experiments on datasets CORA, Reddit, and HSEcite, which is collected from Scopus citation database across the authors with affiliation to NRU HSE in 2011–2017. The results show that our three-layer model with attention-based aggregation function, added normalization layers, regularization (dropout) outperforms suggested by the respective authors’ GraphSAGE models with mean, LSTM, and pool aggregation functions, thus giving more insight into possible ways to improve inducting learning model based on GraphSAGE model.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Mislove, A., Marcon, M., Gummadi, K.P., Druschel, P., Bhattacharjee, B.: Measurement and analysis of online social networks. In: Proceedings of the 7th ACM SIGCOMM Conference on Internet Measurement, pp. 29–42. ACM (2007)
Laptsuev, R., Ananyeva, M., Meinster, D., Makarov, I., Karpov, I., Zhukov, L.: Information propagation strategies in online social networks. In: Large Scale Networks - Computational Aspects and Applications - Computational Aspects and Applications, pp. 1–8 (2018)
Khayrullin, R.M., Makarov, I., Zhukov, L.E.: Predicting psychology attributes of a social network user. In Proceedings of EEML Workshop. Ceur WP, pp. 1–10 (2017)
Kyriakopoulos, F., Thurner, S., Puhr, C., Schmitz, S.W.: Network and eigenvalue analysis of financial transaction networks. Eur. Phys. J. B 71(4), 523 (2009)
Makarov, I., Gerasimova, O., Sulimov, P., Zhukov, L.E.: Recommending co-authorship via network embeddings and feature engineering: the case of national research university higher school of economics. In: Proceedings of the 18th ACM/IEEE on Joint Conference on Digital Libraries, ser. JCDL ’18, pp. 365–366. New York, NY, USA, ACM, (2018). https://doi.org/10.1145/3197026.3203911
Makarov, I., Bulanov, O., Gerasimova, O., Meshcheryakova, N., Karpov, I., Zhukov, L.E.: Scientific matchmaker: collaborator recommender system. In: van der Aalst, W.M., Ignatov, D.I., Khachay, M., Kuznetsov, S.O., Lempitsky, V., Lomazova, I.A., Loukachevitch, N., Napoli, A., Panchenko, A., Pardalos, P.M., Savchenko, A.V., Wasserman, S. (eds.) Analysis of Images, Social Networks and Texts, pp. 404–410. Springer International Publishing, Cham (2018)
Makarov, I., Bulanov, O., Zhukov, L.E.: Co-author recommender system. In: Kalyagin, V.A., Nikolaev, A.I., Pardalos, P.M., Prokopyev, O.A. (eds.) Models, Algorithms, and Technologies for Network Analysis, pp. 251–257. Springer International Publishing, Cham (2017)
Kurmukov, A., Ananyeva, M., Dodonova, Y., Gutman, B., Faskowitz, J., Jahanshad, N., Thompson, P., Zhukov, L.: Classifying phenotypes based on the community structure of human brain networks. In: Graphs in Biomedical Image Analysis, Computational Anatomy and Imaging Genetics, pp. 3–11. Springer, Berlin (2017)
Brohee, S., Van Helden, J.: Evaluation of clustering algorithms for protein-protein interaction networks. BMC Bioinform. 7(1), 488 (2006)
Goyal, P., Ferrara, E.: Graph embedding techniques, applications, and performance: a survey. Knowl.-Based Syst. 151, 78–94 (2018)
Cui, P., Wang, X., Pei, J., Zhu, W.: A survey on network embedding. arXiv:1711.08752 (2017)
Cai, H., Zheng, V.W., Chang, K.: A comprehensive survey of graph embedding: problems, techniques and applications. IIEEE Trans. Knowl. Data Eng (2018)
Leskovec, J.: Deep learning for network biology. part 1: network propagation and node embeddings. Tutorial (2018)
Hamilton, W., Ying, Z., Leskovec, J.: Inductive representation learning on large graphs. In: Advances in Neural Information Processing Systems, pp. 1024–1034 (2017)
Mees, A., Rapp, P., Jennings, L.: Singular-value decomposition and embedding dimension. Phys. Rev. A 36(1), 340 (1987)
Ou, M., Cui, P., Pei, J., Zhang, Z., Zhu, W.: Asymmetric transitivity preserving graph embedding. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1105–1114. ACM (2016)
Wang, X., Cui, P., Wang, J., Pei, J., Zhu, W., Yang, S.: Community preserving network embedding. In: AAAI, pp. 203–209 (2017)
Wang, J.: Locally linear embedding. In: Geometric Structure of High-Dimensional Data and Dimensionality Reduction, pp. 203–220. Springer, Berlin (2012)
Cao, S., Lu, W., Xu, Q.: Grarep: learning graph representations with global structural information. In: Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, pp. 891–900. ACM (2015)
Tang, J., Qu, M., Wang, M., Zhang, M., Yan, J., Mei, Q.: Line: large-scale information network embedding. In: Proceedings of the 24th International Conference on World Wide Web, pp. 1067–1077. International World Wide Web Conferences Steering Committee (2015)
Perozzi, B., Al-Rfou, R., Skiena, S.: Deepwalk: online learning of social representations. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 701–710. ACM (2014)
Tu, C., Zhang, W., Liu, Z., Sun, M., et al.: Max-margin deepwalk: discriminative learning of network representation. In: IJCAI, pp. 3889–3895 (2016)
Grover, A., Leskovec, J.: Node2vec: scalable feature learning for networks. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 855–864. ACM (2016)
Rozemberczki, B., Sarkar, R.: Fast sequence-based embedding with diffusion graphs. In: International Workshop on Complex Networks, pp. 99–107. Springer, Berlin (2018)
Weston, J., Ratle, F., Mobahi, H., Collobert, R.: Deep learning via semi-supervised embedding. In: Neural Networks: Tricks of the Trade, pp. 639–655 (2012)
Wang, D., Cui, P., Zhu, W.: Structural deep network embedding. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1225–1234. ACM (2016)
Zhu, X., Ghahramani, Z., Lafferty, J.D.: Semi-supervised learning using gaussian fields and harmonic functions. In: Proceedings of the 20th International Conference on Machine Learning (ICML-03), pp. 912–919 (2003)
Belkin, M., Niyogi, P., Sindhwani, V.: Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J. Mach. Learn. Res. 7, 2399–2434 (2006)
Ma, J., Cui, P., Zhu, W.: Depthlgp: learning embeddings of out-of-sample nodes in dynamic networks. In: AAAI (2018)
Bojchevski, A., GĂĽnnemann, S.: Deep gaussian embedding of attributed graphs: unsupervised inductive learning via ranking. arXiv:1707.03815 (2017)
Velickovic, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.: Graph attention networks, 1(2). arXiv:1710.10903 (2017)
Acknowledgements
Sections 2–5 were prepared under the support by the Russian Science Foundation under grant 17-11-01294, performed at National Research University Higher School of Economics, Russia. Section 1 was prepared under support by RFBR grant 16-29-09583 “Development of methodology, methods and tools for identifying and countering the proliferation of malicious information campaigns in the Internet”.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Ananyeva, M., Makarov, I., Pendiukhov, M. (2020). GSM: Inductive Learning on Dynamic Graph Embeddings. In: Bychkov, I., Kalyagin, V., Pardalos, P., Prokopyev, O. (eds) Network Algorithms, Data Mining, and Applications. NET 2018. Springer Proceedings in Mathematics & Statistics, vol 315. Springer, Cham. https://doi.org/10.1007/978-3-030-37157-9_6
Download citation
DOI: https://doi.org/10.1007/978-3-030-37157-9_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-37156-2
Online ISBN: 978-3-030-37157-9
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)