En Meng
ABSTRACT
The complexity of the graph structure poses a challenge for graph representation learning. Contrastive learning offers a straightforward and efficient unsupervised framework for graph representation learning. It achieves unsupervised learning by augmenting the original views and comparing them with the augmented views. Several methods based on this framework have achieved significant progress in the field of graph representation learning. Despite its success, the factors contributing to good augmented views in graph contrast learning have received less attention. In order to address this issue, we introduce the graph info-min principle. We investigate the relationship between mutual information (MI) and good augmented views through experimental and theoretical analysis. Additionally, we present a new contrastive learning method called Info-min Contrastive Learning (IMCL). Specifically, The method comprises an adaptive graph augmentation generator and a pseudo-label generator. The graph augmentation generator ensures sufficient differentiation between the augmented and original views. The pseudo-label generator generates pseudo-labels as supervision signals, ensuring consistency between the classification results of augmented views and original views. Our method demonstrates excellent performance through extensive experimental results on various datasets.
- Karsten M Borgwardt, Cheng Soon Ong, Stefan Schönauer, SVN Vishwanathan, Alex J Smola, and Hans-Peter Kriegel. 2005. Protein function prediction via graph kernels. Bioinformatics, Vol. 21, suppl_1 (2005), i47--i56.Google Scholar
Digital Library
- Asim Kumar Debnath, Rosa L Lopez de Compadre, Gargi Debnath, Alan J Shusterman, and Corwin Hansch. 1991. Structure-activity relationship of mutagenic aromatic and heteroaromatic nitro compounds. correlation with molecular orbital energies and hydrophobicity. Journal of medicinal chemistry , Vol. 34, 2 (1991), 786--797.Google ScholarCross Ref
- Paul D Dobson and Andrew J Doig. 2003. Distinguishing enzyme structures from non-enzymes without alignments. Journal of molecular biology , Vol. 330, 4 (2003), 771--783.Google ScholarCross Ref
- Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining. 855--864.Google ScholarDigital Library
- Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. Advances in neural information processing systems , Vol. 30 (2017).Google Scholar
- Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. 2020. Open graph benchmark: Datasets for machine learning on graphs. Advances in neural information processing systems , Vol. 33 (2020), 22118--22133.Google Scholar
- Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016).Google Scholar
- Ke Liang, Yue Liu, Sihang Zhou, Wenxuan Tu, Yi Wen, Xihong Yang, Xiangjun Dong, and Xinwang Liu. 2023 a. Knowledge Graph Contrastive Learning Based on Relation-Symmetrical Structure. IEEE Transactions on Knowledge and Data Engineering (2023).Google Scholar
- Ke Liang, Lingyuan Meng, Meng Liu, Yue Liu, Wenxuan Tu, Siwei Wang, Sihang Zhou, and Xinwang Liu. 2023 b. Learn from Relational Correlations and Periodic Events for Temporal Knowledge Graph Reasoning. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. 1559--1568.Google ScholarDigital Library
- Ke Liang, Lingyuan Meng, Meng Liu, Yue Liu, Wenxuan Tu, Siwei Wang, Sihang Zhou, Xinwang Liu, and Fuchun Sun. 2022. Reasoning over different types of knowledge graphs: Static, temporal and multi-modal. arXiv preprint arXiv:2212.05767 (2022).Google Scholar
- Ke Liang, Lingyuan Meng, Sihang Zhou, Siwei Wang, Wenxuan Tu, Yue Liu, Meng Liu, and Xinwang Liu. 2023 c. Message Intercommunication for Inductive Relation Reasoning. arXiv preprint arXiv:2305.14074 (2023).Google Scholar
- Ke Liang, Sihang Zhou, Yue Liu, Lingyuan Meng, Meng Liu, and Xinwang Liu. 2023 d. Structure Guided Multi-modal Pre-trained Transformer for Knowledge Graph Reasoning. arXiv preprint arXiv:2307.03591 (2023).Google Scholar
- Meng Liu, Ke Liang, Dayu Hu, Hao Yu, Yue Liu, Lingyuan Meng, Wenxuan Tu, Sihang Zhou, and Xinwang Liu. 2023 a. TMac: Temporal Multi-Modal Graph Learning for Acoustic Event Classification. In Proceedings of the 31st ACM International Conference on Multimedia.Google Scholar
- Meng Liu, Ke Liang, Bin Xiao, Sihang Zhou, Wenxuan Tu, Yue Liu, Xihong Yang, and Xinwang Liu. 2023 b. Self-Supervised Temporal Graph learning with Temporal and Structural Intensity Alignment. arXiv preprint arXiv:2302.07491 (2023).Google Scholar
- Meng Liu, Yue Liu, Ke Liang, Siwei Wang, Sihang Zhou, and Xinwang Liu. 2023 c. Deep Temporal Graph Clustering. arXiv preprint arXiv:2305.10738 (2023).Google Scholar
- Christopher Morris, Nils M Kriege, Franka Bause, Kristian Kersting, Petra Mutzel, and Marion Neumann. 2020. Tudataset: A collection of benchmark datasets for learning with graphs. arXiv preprint arXiv:2007.08663 (2020).Google Scholar
- Nino Shervashidze, Pascal Schweitzer, Erik Jan Van Leeuwen, Kurt Mehlhorn, and Karsten M Borgwardt. 2011. Weisfeiler-lehman graph kernels. Journal of Machine Learning Research , Vol. 12, 9 (2011).Google Scholar
- Nino Shervashidze, SVN Vishwanathan, Tobias Petri, Kurt Mehlhorn, and Karsten Borgwardt. 2009. Efficient graphlet kernels for large graph comparison. In Artificial intelligence and statistics. PMLR, 488--495.Google Scholar
- Kihyuk Sohn. 2016. Improved deep metric learning with multi-class n-pair loss objective. Advances in neural information processing systems , Vol. 29 (2016).Google Scholar
- Susheel Suresh, Pan Li, Cong Hao, and Jennifer Neville. 2021. Adversarial graph augmentation to improve graph contrastive learning. Advances in Neural Information Processing Systems , Vol. 34 (2021), 15920--15933.Google Scholar
- Petar Velivc ković , Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. arXiv preprint arXiv:1710.10903 (2017).Google Scholar
- Petar Velickovic, William Fedus, William L Hamilton, Pietro Liò, Yoshua Bengio, and R Devon Hjelm. 2019. Deep Graph Infomax. ICLR (Poster), Vol. 2, 3 (2019), 4.Google Scholar
- Nikil Wale, Ian A Watson, and George Karypis. 2008. Comparison of descriptor spaces for chemical compound retrieval and classification. Knowledge and Information Systems , Vol. 14, 3 (2008), 347--375.Google ScholarDigital Library
- Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. 2018. How powerful are graph neural networks? arXiv preprint arXiv:1810.00826 (2018).Google Scholar
- Pinar Yanardag and SVN Vishwanathan. 2015. Deep graph kernels. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining. 1365--1374.Google ScholarDigital Library
- Yihang Yin, Qingzhong Wang, Siyu Huang, Haoyi Xiong, and Xiang Zhang. 2022. Autogcl: Automated graph contrastive learning via learnable view generators. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 8892--8900.Google ScholarCross Ref
- Yuning You, Tianlong Chen, Yang Shen, and Zhangyang Wang. 2021. Graph contrastive learning automated. In International Conference on Machine Learning. PMLR, 12121--12132.Google Scholar
- Hengrui Zhang, Qitian Wu, Junchi Yan, David Wipf, and Philip S Yu. 2021. From canonical correlation analysis to self-supervised graph neural networks. Advances in Neural Information Processing Systems , Vol. 34 (2021), 76--89.Google Scholar
- Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang. 2021. Graph contrastive learning with adaptive augmentation. In Proceedings of the Web Conference 2021. 2069--2080.Google ScholarDigital Library
Index Terms
- Graph Contrastive Learning with Graph Info-Min
Recommendations
Hierarchical Graph Contrastive Learning
Machine Learning and Knowledge Discovery in Databases: Research TrackAbstractUnsupervised graph representation learning with GNNs is critically important due to the difficulty of obtaining graph labels in many real applications. Graph contrastive learning (GCL), a recently popular method for unsupervised learning on graphs,...
Graph prototypical contrastive learning
AbstractUnsupervised graph representation learning methods based on contrastive learning have drawn increasing attention and achieved promising performance. Most of these methods only model instance-level feature similarity while ignoring the ...
Weight-Aware Graph Contrastive Learning
Artificial Neural Networks and Machine Learning – ICANN 2022AbstractIn contrastive learning, samples usually have different contributions to optimization. This inference applies to the specific downstream tasks of applying contrastive learning to graph learning. Nevertheless, the nodes, i.e., samples, are equally ...
Comments