Shaosheng Cao
ABSTRACT
In this paper, we present {GraRep}, a novel model for learning vertex representations of weighted graphs. This model learns low dimensional vectors to represent vertices appearing in a graph and, unlike existing work, integrates global structural information of the graph into the learning process. We also formally analyze the connections between our work and several previous research efforts, including the DeepWalk model of Perozzi et al. as well as the skip-gram model with negative sampling of Mikolov et al.
We conduct experiments on a language network, a social network as well as a citation network and show that our learned global representations can be effectively used as features in tasks such as clustering, classification and visualization. Empirical results demonstrate that our representation significantly outperforms other state-of-the-art methods in such tasks.
- A. Ahmed, N. Shervashidze, S. Narayanamurthy, V. Josifovski, and A. J. Smola. Distributed large-scale natural graph factorization. In WWW, pages 37--48. International World Wide Web Conferences Steering Committee, 2013. Google Scholar
Digital Library
- D. Arthur and S. Vassilvitskii. k-mean: The advantages of careful seeding. In SODA, pages 1027--1035. Society for Industrial and Applied Mathematics, 2007. Google ScholarDigital Library
- M. Belkin and P. Niyogi. Laplacian eigenmaps and spectral techniques for embedding and clustering. In NIPS, volume 14, pages 585--591, 2001.Google ScholarDigital Library
- J. A. Bullinaria and J. P. Levy. Extracting semantic representations from word co-occurrence statistics: A computational study. BRM, 39(3):510--526, 2007.Google ScholarCross Ref
- J. A. Bullinaria and J. P. Levy. Extracting semantic representations from word co-occurrence statistics: stop-lists, stemming, and svd. BRM, 44(3):890--907, 2012.Google ScholarCross Ref
- J. Caron. Experiments with lsa scoring: Optimal rank and basis. In CIR, pages 157--169, 2001. Google ScholarDigital Library
- P. Comon. Independent component analysis, a new concept? Signal processing, 36(3):287--314, 1994. Google ScholarDigital Library
- T. F. Cox and M. A. Cox. Multidimensional scaling. CRC Press, 2000.Google ScholarCross Ref
- C. Eckart and G. Young. The approximation of one matrix by another of lower rank. Psychometrika, 1(3):211--218, 1936.Google ScholarCross Ref
- R.-E. Fan, K.-W. Chang, C.-J. Hsieh, X.-R. Wang, and C.-J. Lin. Liblinear: A library for large linear classification. JMLR, 9:1871--1874, 2008. Google ScholarDigital Library
- M. U. Gutmann and A. Hyvärinen. Noise-contrastive estimation of unnormalized statistical models, with applications to natural image statistics. JMLR, 13(1):307--361, 2012. Google ScholarDigital Library
- G. E. Hinton and R. R. Salakhutdinov. Reducing the dimensionality of data with neural networks. Science, 313(5786):504--507, 2006.Google ScholarCross Ref
- C. Jutten and J. Herault. Blind separation of sources, part i: An adaptive algorithm based on neuromimetic architecture. Signal processing, 24(1):1--10, 1991. Google ScholarDigital Library
- V. Klema and A. J. Laub. The singular value decomposition: Its computation and some applications. Automatic Control, 25(2):164--176, 1980.Google ScholarCross Ref
- T. K. Landauer, P. W. Foltz, and D. Laham. An introduction to latent semantic analysis. Discourse processes, 25(2--3):259--284, 1998.Google Scholar
- O. Levy and Y. Goldberg. Neural word embedding as implicit matrix factorization. In NIPS, pages 2177--2185, 2014.Google ScholarDigital Library
- K. Lund and C. Burgess. Producing high-dimensional semantic spaces from lexical co-occurrence. BRMIC, 28(2):203--208, 1996.Google ScholarCross Ref
- T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean. Distributed representations of words and phrases and their compositionality. In NIPS, pages 3111--3119, 2013.Google ScholarDigital Library
- J. Pennington, R. Socher, and C. D. Manning. Glove: Global vectors for word representation. EMNLP, 12, 2014.Google Scholar
- B. Perozzi, R. Al-Rfou, and S. Skiena. Deepwalk: Online learning of social representations. In SIGKDD, pages 701--710. ACM, 2014. Google ScholarDigital Library
- S. T. Roweis and L. K. Saul. Nonlinear dimensionality reduction by locally linear embedding. Science, 290(5500):2323--2326, 2000.Google ScholarCross Ref
- B. Sarwar, G. Karypis, J. Konstan, and J. Riedl. Incremental singular value decomposition algorithms for highly scalable recommender systems. In ICIS, pages 27--28. Citeseer, 2002.Google Scholar
- J. Shi and J. Malik. Normalized cuts and image segmentation. PAMI, 22(8):888--905, 2000. Google ScholarDigital Library
- A. Strehl, J. Ghosh, and R. Mooney. Impact of similarity measures on web-page clustering. In Workshop on Artificial Intelligence for Web Search (AAAI 2000), pages 58--64, 2000.Google Scholar
- J. Tang, M. Qu, M. Wang, M. Zhang, J. Yan, and Q. Mei. Line: Large-scale information network embedding. In WWW. ACM, 2015. Google ScholarDigital Library
- J. Tang, J. Zhang, L. Yao, J. Li, L. Zhang, and Z. Su. Arnetminer: extraction and mining of academic social networks. In SIGKDD, pages 990--998. ACM, 2008. Google ScholarDigital Library
- L. Tang and H. Liu. Relational learning via latent social dimensions. In SIGKDD, pages 817--826. ACM, 2009. Google ScholarDigital Library
- J. B. Tenenbaum, V. De Silva, and J. C. Langford. A global geometric framework for nonlinear dimensionality reduction. Science, 290(5500):2319--2323, 2000.Google ScholarCross Ref
- F. Tian, B. Gao, Q. Cui, E. Chen, and T.-Y. Liu. Learning deep representations for graph clustering. In AAAI, 2014.Google ScholarDigital Library
- P. D. Turney. Domain and function: A dual-space model of semantic relations and compositions. JAIR, pages 533--585, 2012. Google ScholarDigital Library
- L. Van der Maaten and G. Hinton. Visualizing data using t-sne. JMLR, 9(2579--2605):85, 2008.Google Scholar
Index Terms
- GraRep: Learning Graph Representations with Global Structural Information
Recommendations
k-tuple domination in graphs
In a graph G, a vertex is said to dominate itself and all of its neighbors. For a fixed positive integer k, the k-tuple domination problem is to find a minimum sized vertex subset in a graph such that every vertex in the graph is dominated by at least k ...
Improved algorithms for recognizing p-Helly and hereditary p-Helly hypergraphs
A hypergraph H is set of vertices V together with a collection of nonempty subsets of it, called the hyperedges of H. A partial hypergraph of H is a hypergraph whose hyperedges are all hyperedges of H, whereas for V^'@?V the subhypergraph (induced by V^'...
The clique operator on circular-arc graphs
A circular-arc graphG is the intersection graph of a collection of arcs on the circle and such a collection is called a model of G. Say that the model is proper when no arc of the collection contains another one, it is Helly when the arcs satisfy the ...
Comments