Abstract
We present an algorithm for learning a mixture of distributions based on spectral projection. We prove a general property of spectral projection for arbitrary mixtures and show that the resulting algorithm is efficient when the components of the mixture are logconcave distributions in \(\Re^{n}\) whose means are separated. The separation required grows with k, the number of components, and log n. This is the first result demonstrating the benefit of spectral projection for general Gaussians and widens the scope of this method. It improves substantially on previous results, which focus either on the special case of spherical Gaussians or require a separation that has a considerably larger dependence on n.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Achlioptas, D., McSherry, F.: On Spectral Learning of Mixtures of Distributions, this proceedings
Arora, S., Kannan, R.: Learning mixtures of arbitrary Gaussians. In: Proc. 33st ACM STOC (2001)
DasGupta, S.: Learning mixtures of Gaussians. In: Proc. of FOCS (1999)
DasGupta, S., Schulman, L.: A two-round variant of EM for Gaussian mixtures. Uncertainty in Artificial Intelligence (2000)
Dinghas, A.: Uber eine Klasse superadditiver Mengenfunktionale von Brunn–Minkowski–Lusternik-schem Typus. Math. Zeitschr. 68, 111–125 (1957)
Giannopoulos, A.A., Milman, V.D.: Concentration property on probability spaces. Adv. Math. 156(1), 77–106 (2000)
Golub, G., Van Loan, C.: Matrix Computations. Johns Hopkins University Press, Baltimore (1989)
Kannan, R., Salmasian, H., Vempala, S.: The Spectral Method for Mixture Models. ECCC Tech. Rep. 067 (2004)
Kearns, M., Vazirani, U.: An Introduction to Computational Learning Theory. MIT Press, Cambridge (1994)
Leindler, L.: On a certain converse of Hölder’s Inequality II. Acta Sci. Math. Szeged 33, 217–223 (1972)
Lindsay, B.: Mixture models: theory, geometry and applications. American Statistical Association, Virginia (1995)
Lovász, L., Vempala, S.: Logconcave functions: Geometry and Efficient Sampling Algorithms. In: Proc. of FOCS (2003)
Lovász, L., Vempala, S.: The Geometry of Logconcave Functions and an O*(n3) sampling algorithm, Microsoft Research Tech. Report MSR-TR-2003-04
Motwani, R., Raghavan, P.: Randomized Algorithms. Cambridge University Press, Cambridge (1995)
Prékopa, A.: Logarithmic concave measures and functions. Acta Sci. Math. Szeged 34, 335–343 (1973)
Rudelson, M.: Random vectors in the isotropic position. J. Funct. Anal. 164, 60–72 (1999)
Titterington, D.M., Smith, A.F.M., Makov, U.E.: Statistical analysis of finite mixture distributions. Wiley, Chichester (1985)
Vempala, S.: On the Spectral Method for Mixture Models, IMA workshop on Data Analysis and Optimization (2003), http://www.ima.umn.edu/talks/workshops/5-6-9.2003/vempala/vempala.html
Vempala, S., Wang, G.: A spectral algorithm for learning mixtures of distributions. In: Proc. of FOCS (2002); J. Comput. System Sci. 68(4), 841–860 (2004)
Vempala, S., Wang, G.: The benefit of spectral projection for document clustering. In: Proc. of the 3rd Workshop on Clustering High Dimensional Data and its Applications. SIAM International Conference on Data Mining (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kannan, R., Salmasian, H., Vempala, S. (2005). The Spectral Method for General Mixture Models. In: Auer, P., Meir, R. (eds) Learning Theory. COLT 2005. Lecture Notes in Computer Science(), vol 3559. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11503415_30
Download citation
DOI: https://doi.org/10.1007/11503415_30
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-26556-6
Online ISBN: 978-3-540-31892-7
eBook Packages: Computer ScienceComputer Science (R0)