Abstract
In this paper, we propose cuSLINK, a novel and state-of-the-art reformulation of the SLINK algorithm on the GPU which requires only O(Nk) space and uses a parameter k to trade off space and time. We also propose a set of novel and reusable building blocks that compose cuSLINK. These building blocks include highly optimized computational patterns for k-NN graph construction, spanning trees, and dendrogram cluster extraction. We show how we used our primitives to implement cuSLINK end-to-end on the GPU, further enabling a wide range of real-world data mining and machine learning applications that were once intractable. In addition to being a primary computational bottleneck in the popular HDBSCAN algorithm, the impact of our end-to-end cuSLINK algorithm spans a large range of important applications, including cluster analysis in social and computer networks, natural language processing, and computer vision.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Anibal, J., et al.: HAL-X: scalable hierarchical clustering for rapid and tunable single-cell analysis. PLoS Comput. Biol. 18(10), e1010349 (2022)
Arefin, A.S., Riveros, C., Berretta, R., Moscato, P.: kNN-Borůvka-GPU: a fast and scalable MST construction from kNN graphs on GPU. In: Murgante, B., et al. (eds.) ICCSA 2012. LNCS, vol. 7333, pp. 71–86. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-31125-3_6. http://www.newcastle.edu.au/research-centre/cibm/
Arefin, A.S., Riveros, C., Berretta, R., Moscato, P.: KNN-MST-agglomerative: a fast and scalable graph-based data clustering approach on GPU. In: 2012 7th International Conference on Computer Science & Education (ICCSE), pp. 585–590. IEEE (2012)
Bakunas-Milanowski, D., Rego, V., Sang, J., Chansu, Y.: Efficient algorithms for stream compaction on GPUS. Int. J. Netw. Comput. 7(2), 208–226 (2017)
Cayton, L.: Accelerating nearest neighbor search on manycore systems. In: 2012 IEEE 26th International Parallel and Distributed Processing Symposium, pp. 402–413. IEEE (2012)
Chan, D.M., Rao, R., Huang, F., Canny, J.F.: T-SNE-CUDA: GPU-accelerated t-SNE and its applications to modern data. In: 2018 30th International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD), pp. 330–338 (2018). https://doi.org/10.1109/CAHPC.2018.8645912
Chang, D.J., Kantardzic, M.M., Ouyang, M.: Hierarchical clustering with CUDA/GPU. In: ISCA PDCCS, pp. 7–12. Citeseer (2009)
Clarkson, K.L.: Fast algorithms for the all nearest neighbors problem. In: 24th Annual Symposium on Foundations of Computer Science (SFCS 1983), pp. 226–232 (1983). https://doi.org/10.1109/SFCS.1983.16
Da Silva Sousa, C., Mariano, A., Proença, A.: A generic and highly efficient parallel variant of Boråvka’s algorithm. In: Proceedings - 23rd Euromicro International Conference on Parallel, Distributed, and Network-Based Processing, PDP 2015, pp. 610–617 (2015). https://doi.org/10.1109/PDP.2015.72. http://cudpp.github.io/https://www.semanticscholar.org/paper/A-Generic-and-Highly-Efficient-Parallel-Variant-of-Sousa-Mariano/e9106835936711b416189cd5917dd61704510ee4
Dash, M., Petrutiu, S., Scheuermann, P.: pPOP: fast yet accurate parallel hierarchical clustering using partitioning. Data Knowl. Eng. 61(3), 563–578 (2007)
Gasperini, M., et al.: A genome-wide framework for mapping gene regulation via cellular genetic screens. Cell 176(1–2), 377–390 (2019)
Harish, P., Vineet, V., Narayanan, P.J.: Large graph algorithms for massively multithreaded architectures. Iiit/Tr(74), 1–20 (2009). 10.1.1.417.2999
Hendrix, W., Palsetia, D., Patwary, M.M.A., Agrawal, A., Liao, W.K., Choudhary, A.: A scalable algorithm for single-linkage hierarchical clustering on distributed-memory architectures. In: 2013 IEEE Symposium on Large-Scale Data Analysis and Visualization (LDAV), pp. 7–13. IEEE (2013)
Hendrix, W., Patwary, M.M.A., Agrawal, A., Liao, W.K., Choudhary, A.: Parallel hierarchical clustering on shared memory platforms. In: 2012 19th International Conference on High Performance Computing, pp. 1–9. IEEE (2012)
Jegou, H., Douze, M., Schmid, C.: Product quantization for nearest neighbor search. IEEE Trans. Pattern Anal. Mach. Intell. 33(1), 117–128 (2010)
Johnson, J., Douze, M., Jégou, H.: Billion-scale similarity search with GPUS (2017). https://doi.org/10.48550/ARXIV.1702.08734. https://arxiv.org/abs/1702.08734
McInnes, L., Healy, J.: Accelerated hierarchical density based clustering. In: IEEE International Conference on Data Mining Workshops, ICDMW 2017-November, pp. 33–42 (2017). https://doi.org/10.1109/ICDMW.2017.12
Naumov, M., et al.: AMGX: a library for GPU accelerated algebraic multigrid and preconditioned iterative methods. SIAM J. Sci. Comput. 37(5), S602–S626 (2015). https://doi.org/10.1137/140980260
Nolet, C.J., et al.: Bringing UMAP closer to the speed of light with GPU acceleration (2020). https://doi.org/10.48550/ARXIV.2008.00325. https://arxiv.org/abs/2008.00325
Olson, C.F.: Parallel algorithms for hierarchical clustering. Parallel Comput. 21(8), 1313–1325 (1995)
Pan, J., Manocha, D.: Fast GPU-based locality sensitive hashing for K-nearest neighbor computation. In: Proceedings of the 19th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, pp. 211–220 (2011)
Raff, E.: JSAT: Java statistical analysis tool, a library for machine learning. J. Mach. Learn. Res. 18(23), 1–5 (2017)
Sankaranarayanan, J., Samet, H., Varshney, A.: A fast all nearest neighbor algorithm for applications involving large point-clouds. Comput. Graph. 31(2), 157–174 (2007)
Shalom, S.A.A., Dash, M., Tue, M.: An approach for fast hierarchical agglomerative clustering using graphics processors with CUDA. In: Zaki, M.J., Yu, J.X., Ravindran, B., Pudi, V. (eds.) PAKDD 2010. LNCS (LNAI), vol. 6119, pp. 35–42. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-13672-6_4. https://link.springer.com/chapter/10.1007/978-3-642-13672-6_4
Shalom, S.A., Dash, M.: Efficient hierarchical agglomerative clustering algorithms on GPU using data partitioning. In: 2011 12th International Conference on Parallel and Distributed Computing, Applications and Technologies, pp. 134–139. IEEE (2011)
Shalom, S.A., Dash, M., Tue, M., Wilson, N.: Hierarchical agglomerative clustering using graphics processor with compute unified device architecture. In: 2009 International Conference on Signal Processing Systems, pp. 556–561. IEEE (2009)
Sibson, R.: SLINK: an optimally efficient algorithm for the single-link cluster method. Comput. J. 16(1), 30–34 (1973). https://doi.org/10.1093/comjnl/16.1.30
Tarjan, R.E.: Efficiency of a good but not linear set union algorithm. J. ACM (JACM) 22(2), 215–225 (1975)
Vineet, V., Harish, P., Patidar, S., Narayanan, P.J.: Fast minimum spanning tree for large graphs on the GPU. In: Proceedings of the SIGGRAPH/Eurographics Workshop on Graphics Hardware, pp. 167–172 (2009). https://cvit.iiit.ac.in/resources
Yengo, L., et al.: A saturated map of common genetic variants associated with human height. Nature 610(7933), 704–712 (2022)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Nolet, C.J. et al. (2023). cuSLINK: Single-Linkage Agglomerative Clustering on the GPU. In: Koutra, D., Plant, C., Gomez Rodriguez, M., Baralis, E., Bonchi, F. (eds) Machine Learning and Knowledge Discovery in Databases: Research Track. ECML PKDD 2023. Lecture Notes in Computer Science(), vol 14169. Springer, Cham. https://doi.org/10.1007/978-3-031-43412-9_42
Download citation
DOI: https://doi.org/10.1007/978-3-031-43412-9_42
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43411-2
Online ISBN: 978-3-031-43412-9
eBook Packages: Computer ScienceComputer Science (R0)
