Grey Ballard
Alex Druinsky
ABSTRACT
The performance of parallel algorithms for sparse matrix-matrix multiplication is typically determined by the amount of interprocessor communication performed, which in turn depends on the nonzero structure of the input matrices. In this paper, we characterize the communication cost of a sparse matrix-matrix multiplication algorithm in terms of the size of a cut of an associated hypergraph that encodes the computation for a given input nonzero structure. Obtaining an optimal algorithm corresponds to solving a hypergraph partitioning problem. Our hypergraph model generalizes several existing models for sparse matrix-vector multiplication, and we can leverage hypergraph partitioners developed for that computation to improve application-specific algorithms for multiplying sparse matrices.
- K. Akbudak and C. Aykanat. Simultaneous input and output matrix partitioning for outer-product-parallel sparse matrix-matrix multiplication. SISC, 36(5):C568--C590, 2014.Google Scholar
Cross Ref
- G. Ballard, A. Buluç, J. Demmel, L. Grigori, B. Lipshitz, O. Schwartz, and S. Toledo. Communication optimal parallel multiplication of sparse random matrices. In SPAA '13, pages 222--231. ACM, 2013. Google ScholarDigital Library
- G. Ballard, J. Hu, and C. Siefert. Reducing communication costs for sparse matrix multiplication within algebraic multigrid. Technical report SAND2015-3275, Sandia Natl. Labs., 2015.Google Scholar
- E. Boman, K. Devine, L. Fisk, R. Heaphy, B. Hendrickson, C. Vaughan, Ü.c Catalyürek, D. Bozdag, W. Mitchell, and J. Teresco. Zoltan 3.0: parallel partitioning, boad-balancing, and data management services; user's guide. Technical Report SAND2007--4748W, Sandia Natl. Labs., 2007.Google Scholar
- A. Buluć and J. R. Gilbert. Parallel sparse matrix-matrix multiplication and indexing: implementation and experiments. SISC, 34(4):C170--C191, 2012.Google ScholarCross Ref
- Ü.c Catalyürek and C. Aykanat. PaToH: a multilevel hypergraph partitioning tool, version 3.0. Technical report, Dept. of Computer Engineering, Bilkent Univ., 1999.Google Scholar
- Ü.c Catalyürek and C. Aykanat. A fine-grain hypergraph model for 2D decomposition of sparse matrices. In IPDPS '01, pages 118--123, 2001. Google ScholarDigital Library
- Ü.c Catalyürek, C. Aykanat, and B. Uçar. On two-dimensional sparse matrix partitioning: models, methods, and a recipe. SISC, 32(2):656--683, 2010. Google ScholarDigital Library
- M. Gee, C. Siefert, J. Hu, R. Tuminaro, and M. Sala. ML 5.0 Smoothed Aggregation User's Guide. Technical Report SAND2006-2649, Sandia Natl. Labs., 2006.Google Scholar
- S. Krishnamoorthy," U.c Catalyürek, J. Nieplocha, A. Rountev, and P. Sadayappan. Hypergraph partitioning for automatic memory hierarchy management. In SC '06, pages 34--46, 2006. Google ScholarDigital Library
- T. Lengauer. Combinatorial Algorithms for Integrated Circuit Layout. Wiley, 1990. Google ScholarDigital Library
Index Terms
- Hypergraph Partitioning for Parallel Sparse Matrix-Matrix Multiplication
Recommendations
Hypergraph Partitioning for Sparse Matrix-Matrix Multiplication
We propose a fine-grained hypergraph model for sparse matrix-matrix multiplication (SpGEMM), a key computational kernel in scientific computing and data analysis whose performance is often communication bound. This model correctly describes both the ...
Partitioning Models for Scaling Parallel Sparse Matrix-Matrix Multiplication
We investigate outer-product--parallel, inner-product--parallel, and row-by-row-product--parallel formulations of sparse matrix-matrix multiplication (SpGEMM) on distributed memory architectures. For each of these three formulations, we propose a ...
Parallel Sparse Matrix-Matrix Multiplication and Indexing: Implementation and Experiments
Generalized sparse matrix-matrix multiplication (or SpGEMM) is a key primitive for many high performance graph algorithms as well as for some linear solvers, such as algebraic multigrid. Here we show that SpGEMM also yields efficient algorithms for general ...
Comments