Yannis Thomas
Nikos Fotiou
George Xylomenos
George C. Polyzos
ABSTRACT
Distributed Hash Tables (DHT) are once again attracting research interest, 20 years after their inception, as a promising solution for scalable and decentralized content storage. A prominent example is the InterPlanetary File System (IPFS), a distributed peer-to-peer storage system with more than 20K online peers, which uses the Kademlia DHT. We design and implement H-Kademlia, a hierarchical version of Kademlia for IPFS, where peers are distributed into disjoint sets of users, or clusters. Peer clustering can offer resilience to network partition, privacy of in-cluster content lookups, as well as improved caching, content filtering and access control. We assess the performance of IPFS over H-Kademlia via simulations that use real traces from the IPFS network. Our findings show that our design delivers the benefits of peer clustering, without significant performance penalties.
- Marc Sanchez Artigas, Pedro Garcia Lopez, and Antonio F. Skarmeta. 2007. A Comparative Study of Hierarchical DHT Systems. In Proc. of the IEEE Conference on Local Computer Networks (LCN). IEEE, New York, NY, USA, 325--333.Google Scholar
- Michael Backes, Ian Goldberg, Aniket Kate, and Tomas Toft. 2012. Adding query privacy to robust DHTs. In Proc. of the ACM Symposium on Information, Computer and Communications Security. ACM, New York, NY, USA, 30--31.Google ScholarDigital Library
- Leonhard Balduf, Sebastian Henningsen, Martin Florian, Sebastian Rust, and BjÃűrn Scheuermann. 2022. Monitoring Data Requests in Decentralized Data Storage Systems: A Case Study of IPFS. arXiv:2104.09202 [cs.NI]Google Scholar
- Ingmar Baumgart and Sebastian Mies. 2007. S/Kademlia: A practicable approach towards secure key-based routing. In Proc. of the International Conference on Parallel and Distributed Systems (IPDPS). IEEE, New York, NY, USA, 1--8.Google ScholarDigital Library
- Juan Benet. 2014. IPFS - Content Addressed, Versioned, P2P File System. arXiv:1407.3561 [cs.NI]Google Scholar
- Miguel Castro, Peter Druschel, Ayalvadi Ganesh, Antony Rowstron, and Dan S Wallach. 200f2. Secure routing for structured peer-to-peer overlay networks. ACM SIGOPS Operating Systems Review 36, SI (200f2), 299--314.Google Scholar
- Nikos Fotiou, Konstantinos V. Katsaros, George Xylomenos, and George C. Polyzos. 2015. H-Pastry: An inter-domain topology aware overlay for the support of name-resolution services in the future Internet. Computer Communications 62 (2015), 13--22.Google ScholarDigital Library
- Michael J. Freedman, Eric Freudenthal, and David Mazières. 2004. Democratizing Content Publication with Coral. In Proc. of the USENIX Symposium on Networked Systems Design and Implementation (NSDI). USENIX Association, Berkeley, CA, USA, 18.Google Scholar
- Prasanna Ganesan, Krishna Gummadi, and H. Garcia-Molina. 2004. Canon in G major: designing DHTs with hierarchical structure. In Proc. of the 24th International Conference on Distributed Computing Systems (ICDCS). IEEE, New York, NY, USA, 263--272.Google Scholar
- Nicholas J. A. Harvey, Michael B. Jones, Stefan Saroiu, Marvin Theimer, and Alec Wolman. 2003. SkipNet: A Scalable Overlay Network with Practical Locality Properties. In Proc. of the USENIX Symposium on Internet Technologies and Systems (USITS). USENIX Association, Berkeley, CA, USA, 14 pages.Google Scholar
- Yuh-Jzer Joung and Jiaw-Chang Wang. 2007. Chord2: A two-layer Chord for reducing maintenance overhead via heterogeneity. Computer Networks 51, 3 (2007), 712--731.Google ScholarDigital Library
- Isaias Martinez-Yelmo, Ruben Cuevas, Carmen Guerrero, and Andreas Mauthe. 2008. Routing Performance in a Hierarchical DHT-based Overlay Network. In Proc. of the Euromicro Conference on Parallel, Distributed and Network-Based Processing (PDP). IEEE, New York, NY, USA, 508--515.Google ScholarDigital Library
- Petar Maymounkov and David Mazieres. 2002. Kademlia: A peer-to-peer information system based on the XOR metric. In Proc. of the International Workshop on Peer-to-Peer Systems. Springer, Berlin, Heidelberg, 53--65.Google ScholarCross Ref
- Josyl Mariela B. Rocamora and Jhoanna Rhodette I. Pedrasa. 2016. Evaluation of hierarchical DHTs to mitigate churn effects in mobile networks. Computer Communications 85 (2016), 41--57.Google ScholarDigital Library
- Antony Rowstron and Peter Druschel.2001. Pastry: Scalable, decentralized object location, and routing for large-scale peer-to-peer systems. In Proc. of the IFIP/ACM International Conference on Distributed Systems Platforms and Open Distributed Processing. Springer, Berlin, Heidelberg, 329--350.Google Scholar
- Zhenyang Shi. 2021. Nebula Crawlings. https://github.com/wcgcyx/nebula-crawler/tree/812f33515342321461e39b17ac02a91858926e14.Google Scholar
- Zhenyang Shi. 2021. Nebula Crawlings. https://github.com/wcgcyx/nebula-crawler#6-correlation-between- uptime-and-geolocation.Google Scholar
- Emil Sit and Robert Morris. 2002. Security considerations for peer-to-peer distributed hash tables. In Proc. of the International Workshop on Peer-to-Peer Systems. Springer, Berlin, Heidelberg, 261--269.Google ScholarCross Ref
- Dennis Trautwein. 2021. Nebula Crawlings. https://github.com/dennis-tra/nebula-crawler-reports/blob/main/2021/calendar-week-44/ipfs/README.md.Google Scholar
- Dennis Trautwein, Aravindh Raman, Gareth Tyson, Ignacio Castro, Will Scott, Moritz Schubotz, Bela Gipp, and Yiannis Psaras. 2022. Design and evaluation of IPFS: a storage layer for the decentralized web. In Proc. of the ACM SIGCOMM Conference. ACM, New York, NY, USA, 739--752.Google ScholarDigital Library
- Peng Wang, Julong Lan, Yuxiang Hu, and Shuqiao Chen. 2015. Towards locality-aware DHT for fast mapping service in future Internet. Computer Communications 66 (2015), 14--24.Google ScholarDigital Library
Index Terms
- Peer Clustering for the InterPlanetary File System
Recommendations
Next-Generation Distributed Hash Tables
CoNEXT-SW '23: Proceedings of the on CoNEXT Student Workshop 2023Distributed Hash Tables (DHTs) serve as the backbone of numerous modern decentralized systems like the InterPlanetary File System (IPFS) and Ethereum. As these systems evolve and expand, there is a growing need to enhance and optimize their underlying ...
Locality in structured peer-to-peer networks
Distributed hash tables (DHTs), used in a number of structured peer-to-peer (P2P) systems, provide efficient mechanisms for resource placement and location. A key distinguishing feature of current DHT systems, such as Chord, Pastry, CAN and Tapestry, is ...
A survey on resource discovery mechanisms, peer-to-peer and service discovery frameworks
Service and resource discovery has become an integral part of modern networked systems. In this survey we give an overview of the existing solutions for service and resource discovery for a wide variety of network types. We cover techniques used in ...
Comments