Jeroen Bollen
Jan Van Den Bussche
Stijn Vansummeren
ABSTRACT
Graph Neural Networks (GNNs) are a form of deep learning that enable a wide range of machine learning applications on graph-structured data. The learning of GNNs, however, is known to pose challenges for memory-constrained devices such as GPUs. In this paper, we study exact compression as a way to reduce the memory requirements of learning GNNs on large graphs. In particular, we adopt a formal approach to compression and propose a methodology that transforms GNN learning problems into provably equivalent compressed GNN learning problems. In a preliminary experimental evaluation, we give insights into the compression ratios that can be obtained on real-world graphs and apply our methodology to an existing GNN benchmark.
- Christel Baier and Joost-Pieter Katoen. 2008. Principles of Model Checking. MIT Press.Google Scholar
Digital Library
- Pablo Barceló, Egor V. Kostylev, Mikaël Monet, Jorge Pérez, Juan L. Reutter, and Juan-Pablo Silva. 2020. The Expressive Power of Graph Neural Networks as a Query Language. ACM SIGMOD Record 49, 2 (2020), 6--17.Google ScholarDigital Library
- Peter W. Battaglia, Jessica B. Hamrick, Victor Bapst, Alvaro Sanchez-Gonzalez, Vinicius Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, Caglar Gulcehre, Francis Song, Andrew Ballard, Justin Gilmer, George Dahl, Ashish Vaswani, Kelsey Allen, Charles Nash, Victoria Langston, Chris Dyer, Nicolas Heess, Daan Wierstra, Pushmeet Kohli, Matt Botvinick, Oriol Vinyals, Yujia Li, and Razvan Pascanu. 2018. Relational Inductive Biases, Deep Learning, and Graph Networks. arXiv:arXiv:1806.01261 Google ScholarCross Ref
- Jeroen Bollen, Jasper Steegmans, Jan Van den Bussche, and Stijn Vansummeren. 2023. Learning Graph Neural Networks using Exact Compression (Extended Version). Technical Report. http://arxiv.org/abs/2304.14793.Google Scholar
- A. Cardon and M. Crochemore. 1982. Partitioning a Graph in O (|A| log2 |V|). Theoretical Computer Science 19, 1 (July 1982), 85--98.Google ScholarCross Ref
- Jie Chen, Tengfei Ma, and Cao Xiao. 2018. FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling. In ICLR. OpenReview.net.Google Scholar
- Chenhui Deng, Zhiqiang Zhao, Yongyu Wang, Zhiru Zhang, and Zhuo Feng. [n. d.]. GraphZoom: A Multi-level Spectral Approach for Accurate and Scalable Graph Embedding. In ICLR 2020. OpenReview.net.Google Scholar
- Floris Geerts, Jasper Steegmans, and Jan Van den Bussche. 2022. On the Expressive Power of Message-Passing Neural Networks as Global Feature Map Transformers. In Foundations of Information and Knowledge Systems (LNCS). Springer, 20--34.Google Scholar
- Alessandro Generale, Till Blume, and Michael Cochez. 2022. Scaling R-GCN Training with Graph Summarization. In Companion Proceedings of the Web Conference 2022 (WWW '22). ACM, 1073--1082.Google Scholar
- Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. [n. d.]. Neural Message Passing for Quantum Chemistry. In ICML 2017. JMLR.org, 1263--1272.Google Scholar
- Martin Grohe. [n. d.]. The Logic of Graph Neural Networks. In LICS '21. ACM, 1--17.Google Scholar
- William L. Hamilton. 2020. Graph Representation Learning. Morgan & Claypool Publishers.Google Scholar
- William L. Hamilton, Rex Ying, and Jure Leskovec. [n. d.]. Inductive Representation Learning on Large Graphs. In NIPS 2017. Curran Associates Inc., 1025--1035.Google Scholar
- Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. 2020. Open Graph Benchmark: Datasets for Machine Learning on Graphs. In Advances in Neural Information Processing Systems, Vol. 33. Curran Associates, Inc., 22118--22133.Google Scholar
- Wenbing Huang, Tong Zhang, Yu Rong, and Junzhou Huang. [n. d.]. Adaptive Sampling towards Fast Graph Representation Learning. In NIPS 2018. Curran Associates Inc., 4563--4572.Google Scholar
- Jure Leskovec and Andrej Krevl. [n. d.]. SNAP Datasets: Stanford Large Network Dataset Collection.Google Scholar
- Jiongqian Liang, Saket Gurukar, and Srinivasan Parthasarathy. 2021. MILE: A Multi-Level Framework for Scalable Graph Embedding. Proceedings of the International AAAI Conference on Web and Social Media 15 (May 2021), 361--372.Google ScholarCross Ref
- Ningyi Liao, Dingheng Mo, Siqiang Luo, Xiang Li, and Pengcheng Yin. 2022. SCARA: Scalable Graph Neural Networks with Feature-Oriented Optimization. PVLDB 15, 11 (2022), 3240--3248.Google ScholarDigital Library
- Haiyang Lin, Mingyu Yan, Xiaocheng Yang, Mo Zou, Wenming Li, Xiaochun Ye, and Dongrui Fan. 2022. Characterizing and Understanding Distributed GNN Training on GPUs. IEEE Computer Architecture Letters 21, 1 (2022), 21--24. Google ScholarDigital Library
- Christopher Morris, Martin Ritzert, Matthias Fey, William L. Hamilton, Jan Eric Lenssen, Gaurav Rattan, and Martin Grohe. 2019. Weisfeiler and Leman Go Neural: Higher-Order Graph Neural Networks. AAAI 33, 01 (2019), 4602--4609. Google ScholarDigital Library
- Jingshu Peng, Zhao Chen, Yingxia Shao, Yanyan Shen, Lei Chen, and Jiannong Cao. 2022. Sancus: Staleness-Aware Communication-Avoiding Full-Graph Decentralized Training in Large-Scale Graph Neural Networks. PVLDB 15, 9 (2022), 1937--1950. Google ScholarDigital Library
- Guillaume Salha, Romain Hennequin, Viet-Anh Tran, and Michalis Vazirgiannis. 2019. A Degeneracy Framework for Scalable Graph Autoencoders. In IJCAI. ijcai.org, 3353--3359.Google Scholar
- Qiange Wang, Yanfeng Zhang, Hao Wang, Chaoyi Chen, Xiaodong Zhang, and Ge Yu. [n. d.]. NeutronStar: Distributed GNN Training with Hybrid Dependency Management. In SIGMOD 2022. ACM, 1301--1315.Google Scholar
- Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and Philip S. Yu. 2021. A Comprehensive Survey on Graph Neural Networks. IEEE Transactions on Neural Networks and Learning Systems 32, 1 (2021), 4--24.Google ScholarCross Ref
- Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. [n. d.]. How Powerful Are Graph Neural Networks?. In ICLR 2019. OpenReview.net.Google Scholar
- Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L. Hamilton, and Jure Leskovec. [n. d.]. Graph Convolutional Neural Networks for Web-Scale Recommender Systems. In KDD 2018. ACM, 974--983. Google ScholarDigital Library
- Binhang Yuan, Cameron R. Wolfe, Chen Dun, Yuxin Tang, Anastasios Kyrillidis, and Chris Jermaine. 2022. Distributed Learning of Fully Connected Neural Networks Using Independent Subnet Training. PVLDB 15, 8 (2022), 1581--1590.Google ScholarDigital Library
- Chenguang Zheng, Hongzhi Chen, Yuxuan Cheng, Zhezheng Song, Yifan Wu, Changji Li, James Cheng, Hao Yang, and Shuai Zhang. 2022. ByteGNN: Efficient Graph Neural Network Training at Large Scale. PVLDB 15, 6 (2022), 1228--1242.Google ScholarDigital Library
- Zulun Zhu, Jiaying Peng, Jintang Li, Liang Chen, Qi Yu, and Siqiang Luo. 2022. Spiking Graph Convolutional Networks. In IJCAI, Vol. 3. 2434--2440.Google Scholar
Index Terms
- Learning Graph Neural Networks using Exact Compression
Recommendations
On Graph Learning with Neural Networks
Machine Learning, Optimization, and Data ScienceAbstractGraphs are ideal for modeling natural systems where relations may be intrinsic among data objects. With massive data available, learning graph models from data has become potentially feasible as well as necessary. Yet from the traditional machine ...
Generating a Graph Colouring Heuristic with Deep Q-Learning and Graph Neural Networks
Learning and Intelligent OptimizationAbstractThe graph colouring problem consists of assigning labels, or colours, to the vertices of a graph such that no two adjacent vertices share the same colour. In this work we investigate whether deep reinforcement learning can be used to discover a ...
Toward the analysis of graph neural networks
ICSE-NIER '22: Proceedings of the ACM/IEEE 44th International Conference on Software Engineering: New Ideas and Emerging ResultsGraph Neural Networks (GNNs) have recently emerged as an effective framework for representing and analyzing graph-structured data. GNNs have been applied to many real-world problems such as knowledge graph analysis, social networks recommendation, and ...
Comments