Abstract
Spectral analysis approaches have been actively studied in machine learning and data mining areas, due to their generality, efficiency, and rich theoretical foundations. As a natural non-linear generalization of Graph Laplacian, p-Laplacian has recently been proposed, which interpolates between a relaxation of normalized cut and the Cheeger cut. However, the relaxation can only be applied to two-class cases. In this paper, we propose full eigenvector analysis of p-Laplacian and obtain a natural global embedding for multi-class clustering problems, instead of using greedy search strategy implemented by previous researchers. An efficient gradient descend optimization approach is introduced to obtain the p-Laplacian embedding space, which is guaranteed to converge to feasible local solutions. Empirical results suggest that the greedy search method often fails in many real-world applications with non-trivial data structures, but our approach consistently gets robust clustering results. Visualizations of experimental results also indicate our embedding space preserves the local smooth manifold structures existing in real-world data.
Article PDF
Similar content being viewed by others
References
Allegretto, W., & Huang, Y. X. (1998). A picone’s identity for the p-Laplacian and applications. Nonlinear Analysis, 32, 819–830.
Amari, S. (1998). Natural gradient works efficiently in learning. Neural Computation, 10, 251–276.
Amghibech, S. (2003). Eigenvalues of the discrete p-Laplacian for graphs. Ars Comb, 67, 283–302.
Amghibech, S. (2006). Bounds for the largest p-Laplacian eigenvalue for graphs. Discrete Mathematics, 306, 2762–2771.
Anastasakos, T., Hillard, D., Kshetramade, S., & Raghavan, H. (2009). A collaborative filtering approach to ad recommendation using the query-ad click graph. In D. W. L. Cheung, I. Y. Song, W. W. Chu, Hu, X., & J. J. Lin (Eds.), CIKM (pp. 1927–1930). New York: ACM.
Asuncion, A., & Newman, D. (2007). UCI machine learning repository.
Bach, F. R., & Jordan, M. I. (2006). Learning spectral clustering, with application to speech separation. Journal of Machine Learning Research, 7, 1963–2001.
Belkin, M., & Niyogi (2001). Laplacian eigenmaps and spectral techniques for embedding and clustering. In NIPS (Vol. 14, pp. 585–591). Cambridge: MIT Press.
Belkin, Matveeva, & Niyogi (2004). Regularization and semi-supervised learning on large graphs. In COLT: Proceedings of the workshop on computational learning theory. San Mateo: Morgan Kaufmann.
Bouchala, J. (2003). Resonance problems for p-Laplacian. Mathematics and Computers in Simulation, 61, 599–604.
Bühler, T., & Hein, M. (2009). Spectral clustering based on the graph p-Laplacian. In ICML (Vol. 382, pp. 81–88). New York: ACM.
Chen, G., & Lerman, G. (2009). Spectral curvature clustering (SCC). International Journal of Computer Vision, 81, 317–330.
Cheng, H., Tan, P. N., Sticklen, J., & Punch, W. F. (2007). Recommendation via query centered random walk on K-partite graph. In ICDM (pp. 457–462). New York: IEEE Computer Society.
Chung, F. (1997). Spectral graph theory. Providence: AMS.
Cun, Y. L. L., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proc. IEEE, 86, 2278–2324.
Ding, C. H. Q., & He, X. (2005). On the equivalence of nonnegative matrix factorization and spectral clustering. In SDM.
Georghiades, A., Belhumeur, P., & Kriegman, D. (2001). From few to many: illumination cone models for face recognition under variable lighting and pose. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23, 643–660.
Guattery, Miller (1998). On the quality of spectral separators. SIAM Journal on Matrix Analysis and Applications, 19.
Hein, Audibert, & von Luxburg (2005). From graphs to manifolds—weak and strong pointwise consistency of graph Laplacians. In Auer, P., Meir, R. (Eds.), Proc. of the 18th conf. on learning theory (COLT) (pp. 486–500). Berlin: Springer.
Jain, V., & Zang, H. (2007). A spectral approach to shape-based retrieval of articulated 3D models. Computer-Aided Design, 39, 398–407.
Jin, R., Ding, C. H. Q., & Kang, F. (2005). A probabilistic approach for optimizing spectral clustering.
Kulis, B., Basu, S., Dhillon, I. S., & Mooney, R. J. (2009). Semi-supervised graph clustering: a kernel approach. Machine Learning, 74, 1–22.
Liu, Y., Eyal, E., & Bahar, I. (2008). Analysis of correlated mutations in HIV-1 protease using spectral clustering. Bioinformatics, 24, 1243–1250.
Robles-Kelly, A., & Hancock, E. R. (2007). A Riemannian approach to graph embedding. Pattern Recognition, 40.
Shi, J., & Malik, J. (2000). Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22, 888–905.
White, S., & Smyth, P. (2005). A spectral clustering approach to finding communities in graph. In SDM.
Zhou, D. B. O., Lal, T. N., Weston, J., & Schölkopf, B. (2003). Learning with local and global consistency. In NIPS (Vol. 16, pp. 321–328). Cambridge: MIT Press.
Author information
Authors and Affiliations
Corresponding author
Additional information
Editors: José L. Balcázar, Francesco Bonchi, Aristides Gionis, and Michèle Sebag.
Rights and permissions
About this article
Cite this article
Luo, D., Huang, H., Ding, C. et al. On the eigenvectors of p-Laplacian. Mach Learn 81, 37–51 (2010). https://doi.org/10.1007/s10994-010-5201-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10994-010-5201-z