Abstract
The problem of data distribution in NoSQL databases is particularly difficult in the case of graph databases since the data often represent a large, highly connected graph. We face this task with monitoring of user queries, for which we created a logging module providing information serving as an input to a redistribution algorithm which bases on a lightweight method of Adaptive Partitioning but incorporates our enhancements overcoming its present drawbacks (local optima, balancing, edge weights). The results of our experiments show 70% – 80% reduction of communication between cluster nodes which is a comparable result to other methods, which, however, are more computationally demanding or suffer from other shortcomings.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Barabasi, A.L., Bonabeau, E.: Scale-free networks. Sci. Am. 288(5), 50–59 (2003). https://doi.org/10.1038/scientificamerican0503-60
Bichot, C.E., Siarry, P.: Graph Partitioning, 1st edn. John Wiley, Incorporated, Hoboken (2013). https://ebookcentral.proquest.com. Accessed 21 Nov 2018
Buluç, A., Meyerhenke, H., Safro, I., Sanders, P., Schulz, C.: Recent advances in graph partitioning. In: Kliemann, L., Sanders, P. (eds.) Algorithm Engineering. LNCS, vol. 9220, pp. 117–158. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-49487-6_4
Dong, F., Zhang, J., Luo, J., Shen, D., Jin, J.: Enabling application-aware flexible graph partition mechanism for parallel graph processing systems: superblock an application-aware graph partition mechanism. Concurr. Comput.: Pract. Exp. 29(6), e3849 (2016). https://doi.org/10.1002/cpe.3849
Karypis, G., Kumar, V.: A fast and high quality multilevel scheme for partitioning irregular graphs. SIAM J. Sci. Comput. 20(1), 359–392 (1998). 10.1.1.39.3415
Leskovec, J., Lang, K., Dasgupta, A., Mahoney, M.: Community structure in large networks: natural cluster sizes and the absence of large well-defined clusters. Internet Math. 6(1), 29–123 (2009). https://snap.stanford.edu/data/roadNet-PA.html. Accessed 11 Oct 2019
Malewicz, G., et al.: Pregel: a system for large-scale graph processing. In: Proceedings of the 2010 ACM SIGMOD International Conference on Management of data, pp. 135–146. Indianapolis, Indiana, USA (2010). https://doi.org/10.1145/1807167.1807184
Martella, C., Logothetis, D., Loukas, A., Siganos, G.: Spinner: Scalable graph partitioning in the cloud. In: 2017 IEEE 33rd International Conference on Data Engineering, pp. 1083–1094 (04 2017)
McAuley, J., Leskovec, J.: Learning to discover social circles in ego networks. In: Advances in Neural Information Processing Systems (2012). https://snap.stanford.edu/data/ego-Twitter.html. Accessed 11 Oct 2019
Raghavan, U.N., Albert, R., Kumara, S.: Near linear time algorithm to detect community structures in large-scale networks. Phys. Rev. E 76(3), 036106 (2007). https://doi.org/10.1103/PhysRevE.76.036106
Rahimian, F., Payberah, A.H., Girdzijauskas, S., Jelasity, M., Haridi, S.: A distributed algorithm for large-scale graph partitioning. ACM Trans. Auton. Adapt. Syst. 10(2), 1–24 (2015). https://doi.org/10.1145/2714568
The Apache Software Foundation: Apache tinkerpop, August 2019. http://tinkerpop.apache.org/. Accessed 11 Oct 2019
Ugander, J., Backstrom, L.: Balanced label propagation for partitioning massive graphs. In: Proceedings of the Sixth ACM International Conference on Web Search and Data Mining, Rome, Italy, pp. 507–516 (2013). https://doi.org/10.1145/2433396.2433461
Vaquero, L.M., Cuadrado, F., Logothetis, D., Martella, C.: Adaptive partitioning for large-scale dynamic graphs. In: 2014 IEEE 34th International Conference on Distributed Computing Systems, Madrid, Spain, pp. 144–153 (2014). https://doi.org/10.1109/ICDCS.2014.23
Acknowledgements
The research described in this paper was supported by the internal CTU grant “Advanced Research in Software Engineering”, 2020.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Svitáková, L., Valenta, M., Pokorný, J. (2020). Effective Data Redistribution Based on User Queries in a Distributed Graph Database. In: Nguyen, N., Jearanaitanakij, K., Selamat, A., Trawiński, B., Chittayasothorn, S. (eds) Intelligent Information and Database Systems. ACIIDS 2020. Lecture Notes in Computer Science(), vol 12034. Springer, Cham. https://doi.org/10.1007/978-3-030-42058-1_18
Download citation
DOI: https://doi.org/10.1007/978-3-030-42058-1_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-42057-4
Online ISBN: 978-3-030-42058-1
eBook Packages: Computer ScienceComputer Science (R0)