Skip to main content
SpringerLink
Log in

VSEP: A Distributed Algorithm for Graph Edge Partitioning

  • Conference paper
  • First Online:
Algorithms and Architectures for Parallel Processing (ICA3PP 2015)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 9532))

Abstract

With the exponential growth of graph structured data in recent years, parallel distributed techniques play an increasingly important role in processing large-scale graphs. Since strong connections exist between vertices in graph data, the high communication cost for transforming boundary data is unavoidable in the distributed techniques. How to partition a large graph into several partitions with low coupling and balanced scale becomes a critical problem. Most of research in the literature studies vertex partitioning methods, which leads us to reconsider an alternative approach for edge partitioning. In this paper, we propose a distributed algorithm for graph partition based on edge partitioning, named as VSEP. A novel vertex permutation method is used to partition the large graphs iteratively. Experimental results indicate that VSEP reduces the number of times vertices are cut by about \(10\,\%\sim 20\,\%\) comparing with a state-of-the-art algorithm while retains the scale balance.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Malewicz, G., Austern, M.H., Bik, A.J.C., et al.: Pregel: a system for large-scale graph processing. In: Proceedings of the 2010 ACM SIGMOD International Conference on Management of data, pp. 135–146. ACM (2010)

    Google Scholar 

  2. Low, Y., Gonzalez, J.E., Kyrola, A., et al.: Graphlab: a new framework for parallel machine learning (2014). arXiv preprint arXiv:1408.2041

  3. Gonzalez, J.E., Low, Y., Gu, H., et al.: PowerGraph: distributed graph-parallel computation on natural graphs. In: OSDI, vol. 12(1), p. 2 (2012)

    Google Scholar 

  4. Xin, R.S., Gonzalez, J.E., Franklin, M.J., et al.: Graphx: a resilient distributed graph system on spark. In: First International Workshop on Graph Data Management Experiences and Systems, p. 2. ACM (2013)

    Google Scholar 

  5. Garey, M.R., Johnson, D.S., Stockmeyer, L.: Some simplified NP-complete graph problems. Theoret. Comput. Sci. 1(3), 237–267 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  6. Abou-Rjeili, A., Karypis, G.: Multilevel algorithms for partitioning power-law graphs. In: 2006 20th International Parallel and Distributed Processing Symposium, IPDPS 2006, p. 10. IEEE (2006)

    Google Scholar 

  7. Lang, K.: Finding good nearly balanced cuts in power law graphs (2004). Preprint

    Google Scholar 

  8. Leskovec, J., Lang, K.J., Dasgupta, A., et al.: Community structure in large networks: natural cluster sizes and the absence of large well-defined clusters. Internet Math. 6(1), 29–123 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  9. Kim, M., Candan, K.S.: SBV-Cut: vertex-cut based graph partitioning using structural balance vertices. Data Knowl. Eng. 72, 285–303 (2012)

    Article  Google Scholar 

  10. Guerrieri, A., Montresor, A.: Distributed edge partitioning for graph processing (2014). arXiv preprint arXiv:1403.6270

  11. Rahimian, F., Payberah, A.H., Girdzijauskas, S., Haridi, S.: Distributed vertex-cut partitioning. In: Magoutis, K., Pietzuch, P. (eds.) DAIS 2014. LNCS, vol. 8460, pp. 186–200. Springer, Heidelberg (2014)

    Chapter  Google Scholar 

  12. Albert, R., Jeong, H., Barabsi, A.L.: Error and attack tolerance of complex networks. Nature 406(6794), 378–382 (2000)

    Article  Google Scholar 

  13. Rahimian, F., Payberah, A.H., Girdzijauskas, S., et al.: Ja-be-ja: a distributed algorithm for balanced graph partitioning. In: 2013 IEEE 7th International Conference on Self-Adaptive and Self-Organizing Systems (SASO), pp. 51–60. IEEE (2013)

    Google Scholar 

  14. Baños, R., Gil, C., Ortega, J., Montoya, F.G.: Multilevel heuristic algorithm for graph partitioning. In: Raidl, G.R., et al. (eds.) EvoIASP 2003, EvoWorkshops 2003, EvoSTIM 2003, EvoROB/EvoRobot 2003, EvoCOP 2003, EvoBIO 2003, and EvoMUSART 2003. LNCS, vol. 2611, pp. 143–153. Springer, Heidelberg (2003)

    Chapter  Google Scholar 

  15. Bui, T.N., Moon, B.R.: Genetic algorithm and graph partitioning. IEEE Trans. Comput. 45(7), 841–855 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  16. Hendrickson, B., Leland, R.: A multi-level algorithm for partitioning graphs (1995)

    Google Scholar 

  17. Karypis, G., Kumar, V.: A fast and high quality multilevel scheme for partitioning irregular graphs. SIAM J. Sci. Comput. 20(1), 359–392 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  18. Karypis, G., Kumar, V.: Parallel multilevel series k-way partitioning scheme for irregular graphs. SIAM Rev. 41(2), 278–300 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  19. Walshaw, C., Cross, M.: Mesh partitioning: a multilevel balancing and refinement algorithm. SIAM J. Sci. Comput. 22(1), 63–80 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  20. Sanders, P., Schulz, C.: Engineering multilevel graph partitioning algorithms. In: Demetrescu, C., Halldórsson, M.M. (eds.) ESA 2011. LNCS, vol. 6942, pp. 469–480. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  21. Soper, A.J., Walshaw, C., Cross, M.: A combined evolutionary search and multilevel optimisation approach to graph-partitioning. J. Global Optim. 29(2), 225–241 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  22. Chardaire, P., Barake, M., McKeown, G.P.: A PROBE-based heuristic for graph partitioning. IEEE Trans. Comput. 56(12), 1707–1720 (2007)

    Article  MathSciNet  Google Scholar 

  23. Sanders, P., Schulz, C.: Distributed evolutionary graph partitioning. In: ALENEX, pp. 16–29 (2012)

    Google Scholar 

  24. Talbi, E.G.: Metaheuristics: From Design to Implementation. Wiley, New York (2009)

    Book  MATH  Google Scholar 

  25. Carlini, E., Dazzi, P., Esposito, A., Lulli, A., Ricci, L.: Balanced graph partitioning with apache spark. In: Žilinskas, J., et al. (eds.) Euro-Par 2014, Part I. LNCS, vol. 8805, pp. 129–140. Springer, Heidelberg (2014)

    Google Scholar 

  26. Zaharia, M., Chowdhury, M., Franklin, M.J., et al.: Spark: cluster computing with working sets. In: Proceedings of the 2nd USENIX Conference on Hot Topics in Cloud Computing, p. 10 (2010)

    Google Scholar 

  27. The graph partitioning archive. http://staffweb.cms.gre.ac.uk/~wc06/partition

  28. Stanford large network dataset collection. http://snap.stanford.edu/data/index.html

Download references

Acknowledgment

This research was supported by the National Natural Science Foundation of China (No. 61202477 and No. 61272427); the Strategic Priority Research Program of the Chinese Academy of Sciences (No. XDA06031000).

Author information

Authors and Affiliations

  1. Institute of Information Engineering, Chinese Academy of Sciences, Beijing, 100093, China

    Yu Zhang, Yanbing Liu, Jing Yu, Ping Liu & Li Guo

  2. University of Chinese Academy of Sciences, Beijing, 100049, China

    Yu Zhang

Authors
  1. Yu Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yanbing Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jing Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ping Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Li Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yu Zhang .

Editor information

Editors and Affiliations

  1. Central South University, Changsha, China

    Guojin Wang

  2. Sch of Information Technologies, Sydney University, Sydney, New South Wales, Australia

    Albert Zomaya

  3. Department of Information and Commu, University of Murcia, Murcia, Murcia, Spain

    Gregorio Martinez

  4. Changsha, China

    Kenli Li

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Zhang, Y., Liu, Y., Yu, J., Liu, P., Guo, L. (2015). VSEP: A Distributed Algorithm for Graph Edge Partitioning. In: Wang, G., Zomaya, A., Martinez, G., Li, K. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2015. Lecture Notes in Computer Science(), vol 9532. Springer, Cham. https://doi.org/10.1007/978-3-319-27161-3_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-27161-3_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-27160-6

  • Online ISBN: 978-3-319-27161-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation