Haim Avron
Abstract
We analyze the convergence of randomized trace estimators. Starting at 1989, several algorithms have been proposed for estimating the trace of a matrix by 1/MΣi=1M ziT Azi, where the zi are random vectors; different estimators use different distributions for the zis, all of which lead to E(1/MΣi=1M ziT Azi) = trace(A). These algorithms are useful in applications in which there is no explicit representation of A but rather an efficient method compute zTAz given z. Existing results only analyze the variance of the different estimators. In contrast, we analyze the number of samples M required to guarantee that with probability at least 1-δ, the relative error in the estimate is at most ϵ. We argue that such bounds are much more useful in applications than the variance. We found that these bounds rank the estimators differently than the variance; this suggests that minimum-variance estimators may not be the best.
We also make two additional contributions to this area. The first is a specialized bound for projection matrices, whose trace (rank) needs to be computed in electronic structure calculations. The second is a new estimator that uses less randomness than all the existing estimators.
- Achlioptas, D. 2001. Database-friendly random projections. In PODS '01: Proceedings of the 20th ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems. ACM, New York, 274--281. Google Scholar
Digital Library
- Ailon, N., and Chazelle, B. 2006. Approximate nearest neighbors and the fast Johnson-Lindenstrauss transform. In Proceedings of the 38th Annual ACM Symposium on Theory of Computing (STOC'06). ACM, New York, 557--563. Google ScholarDigital Library
- Avron, H., Maymounkov, P., and Toledo, S. 2010. Blendenpik: Supercharging LAPACK's least-squares solver. SIAM J. Sci. Comput. 32, 3, 1217--1236.Google ScholarDigital Library
- Bai, Z., Fahey, M., and Golub, G. 1996. Some large scale matrix computation problems. J. Comput. Appl. Math 74, 71--89. Google ScholarDigital Library
- Bai, Z., Fahey, M., Golub, G., Menon, M., and Richter, E. 1998. Computing partial eigenvalue sum in electronic structure calculations. Tech. rep. SCCM-98-03, Stanford University.Google Scholar
- Bekas, C., Kokiopoulou, E., and Saad, Y. 2007. An estimator for the diagonal of a matrix. Appl. Numer. Math. 57, 11-12, 1214--1229. Google ScholarDigital Library
- Box, G. E. P., Hunter, W. G., and Hunter, J. S. 1978. Statistics for Experimenters: An Introduction to Design, Data Analysis, and Model Building. Wiley & Sons.Google Scholar
- D'Elia, M., Haber, H., and Horesh, L. 2011. Design of proper orthogonal decomposition bases by means of stochastic optimization. To be submitted.Google Scholar
- Drabold, D. A., and Sankey, O. F. 1993. Maximum entropy approach for linear scaling in the electronic structure problem. Phys. Rev. Lett. 70, 3631--3634.Google ScholarCross Ref
- Feller, W. 1971. An Introduction to Probability Theory and Its Applications, Vol. 2, 3rd. Ed. Wiley.Google Scholar
- Gudmundsson, T., Kenney, C. S., and Laub, A. J. 1995. Small-sample statistical estimates for matrix norms. SIAM J. Matrix Anal. Appl. 16, 3, 776--792. Google ScholarDigital Library
- Hutchinson, M. F. 1989. A stochastic estimator of the trace of the influence matrix for Laplacian smoothing splines. Comm. Stat. Simulat. Comput. 18, 1059--1076.Google ScholarCross Ref
- Iitaka, T., and Ebisuzaki, T. 2004. Random phase vector for calculating the trace of a large matrix. Phys. Rev. E 69, 057701--1--057701--4.Google ScholarCross Ref
- Janssen, A. J. E. M., van Leeuwaarden, J. S. H., and Zwart, B. 2008. Gaussian expansions and bounds for the Poisson distribution applied to the Erlang B formula. Adv. App. Prob. 40, 1, 122--143.Google ScholarCross Ref
- Kenney, C. S., Laub, A. J., and Reese, M. S. 1998. Statistical condition estimation for linear systems. SIAM J. Sci. Comput. 19, 2, 566--583. Google ScholarDigital Library
- Li, P., Hastie, T., and Church, K. 2007. Nonlinear estimators and tail bounds for dimension reduction in l<sub>1</sub> using Cauchy random projections. In Learning Theory, Lecture Notes in Computer Science Series, Vol. 4539. Springer, Berlin, Chap. 37, 514--529. Google ScholarDigital Library
- Silver, R. N., and R&#246;der, H. 1997. Calculation of densities of states and spectral functions by Chebychev recursion and maximum entropy. Phys. Rev. E 56, 4822--4829.Google ScholarCross Ref
- Tsourakakis, C. E. 2008. Fast counting of triangles in large real networks without counting: Algorithms and laws. In Proceedings of the IEEE International Conference on Data Mining (ICDM'08), 608--617. Google ScholarDigital Library
- Wallace, D. L. 1959. Bounds on normal approximations to student's and the chi-square distributions. Ann. Math. Stat. 30, 4, 1121--1130.Google ScholarCross Ref
- Wang, L. W. 1994. Calculating the density of states and optical-absorption spectra of large quantum systems by the plane-wave moments method. Phys. Rev. B 49, 10154--10158.Google ScholarCross Ref
- Wheeler, J. C., and Blumstein, C. 1972. Modified moments for harmonic solids. Phys. Rev. B 6, 4380--4382.Google ScholarCross Ref
- Wong, M. N., Hickernell, F. J., and Liu, K. I. 2004. Computing the trace of a function of a sparse matrix via Hadamard-like sampling. Tech rep. 377(7/04), Hong Kong Baptist University.Google Scholar
Index Terms
- Randomized algorithms for estimating the trace of an implicit symmetric positive semi-definite matrix
Recommendations
Improved Bounds on Sample Size for Implicit Matrix Trace Estimators
This article is concerned with Monte Carlo methods for the estimation of the trace of an implicitly given matrix $$A$$A whose information is only available through matrix-vector products. Such a method approximates the trace by an average of $$N$$N ...
On Randomized Trace Estimates for Indefinite Matrices with an Application to Determinants
AbstractRandomized trace estimation is a popular and well-studied technique that approximates the trace of a large-scale matrix B by computing the average of for many samples of a random vector X. Often, B is symmetric positive definite (SPD) but a ...
Estimating Positive Definite Matrices using Frechet Mean
BIOSTEC 2015: Proceedings of the International Joint Conference on Biomedical Engineering Systems and Technologies - Volume 4Estimation of covariance matrices is a common problem in signal processing applications. Commonly applied
techniques based on the cost optimization (e.g. maximum likelihood estimation) result in an unconstrained
estimation in which the positive definite ...
Comments