Skip to main content
SpringerLink
Log in

Peer counting and sampling in overlay networks based on random walks

  • Published:
Distributed Computing Aims and scope Submit manuscript

Abstract

In this article, we address the problem of counting the number of peers in a peer-to-peer system. This functionality has proven useful in the design of several peer-to-peer applications. However, it is delicate to achieve when nodes are organised in an overlay network, and each node has only a limited, local knowledge of the whole system. In this paper, we propose a generic technique, called the Sample&Collide method, to solve this problem. It relies on a sampling sub-routine which returns randomly chosen peers. Such a sampling sub-routine is of independent interest. It can be used for instance for neighbour selection by new nodes joining the system. We use a continuous time random walk to obtain such samples. The core of the method consists in gathering random samples until a target number of redundant samples are obtained. This method is inspired by the “birthday paradox” technique of Bawa et al. (Estimating aggregates on a peer-to-peer network, Technical Report, Department of Computer Science, Stanford University), upon which it improves by achieving a target variance with fewer samples. We analyse the complexity and accuracy of the proposed method. We illustrate in particular how expansion properties of the overlay affect its performance. We use simulations to evaluate its performance in both static and dynamic environments with sudden changes in peer populations, and verify that it tracks varying system sizes accurately.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Albert R. and Barabási A.-L (2002). Statistical mechanics of complex networks. Rev. Mod. Phys. 74: 47

    Article  Google Scholar 

  2. Aldous, D., Fill, J.: Reversible markov chains and random walks on graphs. Monograph in preparation, http://stat-www.berkeley. edu/users/aldous/RWG/book.html

  3. Alon N. and Spencer J. (2002). The Probabilistic Method. Wiley, London

    Google Scholar 

  4. Alouf S., Altman E., Nain, P.: Optimal on-line estimation of the size of a dynamic multicast group. IEEE INFOCOM ’02 (2002)

  5. Bar-Yossef, Z., Friedman, R., Kliot, G.: Rawms—random walk based lightweight membership service for wireless ad-hoc networks. In: Mobihoc. Florence, Italy (2006)

  6. Bawa, M., Garcia-Molina, H., Gionis, A., Motwani, R.: Estimating aggregates on a peer-to-peer network. Technical Report, Dept. of computer science, Stanford University (2003)

  7. Billingsley P. (1999). Convergence of Probability Measures. Wiley, London

    MATH  Google Scholar 

  8. Bolot, J.-C., Turletti, T., Wakeman, I.: Scalable feedback control for multicast video distribution in the internet. ACM SIGCOMM (1994)

  9. Boyd, S., Ghosh, A., Prabhakar, B., Shah, D.: Gossip algorithms: design, analysis and applications. IEEE INFOCOM (2005)

  10. Castro, M., Druschel, P., Ganesh, A., Rowstron, A., wallach, D.: Security for structured peer-to-peer overlay networks. OSDI (2002)

  11. Cover T.M. and Thomas J.A. (1991). Elements of Information Theory. Wiley, London

    MATH  Google Scholar 

  12. Dolev, D., Mokryn, O., Shavitt, Y.: On multicast trees: structure and size estimation. IEEE INFOCOM (2003)

  13. Dolev, S., Schiller, E., Welsh, J.: Random walk for self-stabilizing group communication in ad-hoc networks. PODC (2002)

  14. Eugster, P., Handurukande, S., Guerraoui, R., Kermarrec, A.-M., Kouznetsov, P.: Lightweight probabilistic broadcast. ACM Trans. Comput. Syst. 21(4), (2003)

  15. Friedman, T., Towsley, D.: Multicast session membership size estimation. IEEE INFOCOM (1999)

  16. Ganesh, A.J., Kermarrec, A.-M., Massoulié, L.: Peer-to-peer membership management for gossip-based protocols. IEEE Trans Comput. 52(2), 2003

  17. Ganesh, A.J., Massoulié, L., Towsley, D.: The effect of network topology on the spread of epidemics. IEEE INFOCOM (2005)

  18. Ganesh, A.J., Xue, F.: Expansion properties of k-out random graphs. Preprint (2004)

  19. Horowitz K. and Malkhi D (2003). Estimating network size from local information. Inf. Process. Lett. 88(5): 237–243

    Article  MathSciNet  Google Scholar 

  20. Jelasity, M., Montresor, A.: Epidemic-style proactive aggregation in large overlay networks. ICDCS (2004)

  21. Jelasity M., Preuß, M.: On obtaining global information in a peer-to-peer fully distributed environment, vol. 2400. LNCS. Springer, Heidelberg, pp. 573–577 (2002)

  22. Jelasity, M., Voulgaris, S., Guerraoui, R., Kermarrec, A-M., van Steen, M.: Gossip-based peer sampling (submitted) (2007)

  23. Kostoulas, D., Psaltoulis, D., Gupta, I., Birman, K., Demers, A.: Decentralized schemes for size estimation in large and dynamic groups. IEEE NCA ’05 (2005)

  24. Le Merrer, E., Kermarrec, A.-M., Massoulié, L.: Peer to peer size estimation in large and dynamic networks: a comparative study. In: The proceedings of HPDC-15, June (2006)

  25. Li, J., Lim, D.-Y.: A robust aggregation tree on distributed hash tables. MIT student oxygen workshop 2004, (2004)

  26. Lindvall T. (2002). Lectures on the Coupling Method. Dover, New york

    MATH  Google Scholar 

  27. Linial, N., Wigderson, A.: Expander graphs and their applications. Lecture Notes, http://www.cs.huji.ac.il/~nati/ (2002)

  28. Malkhi, D., Naor, M., Ratajczak, D.: Viceroy: a scalable and dynamic emulation of the butterfly. In: Proceedings of principles of distributed computing (PODC 2002), (2002)

  29. Manku, G.S.: Routing networks for distributed hash tables. PODC (2003)

  30. Massoulié, L., Le Merrer, E., Kermarrec, A.-M., Ganesh, A.J.: Peer counting and sampling in overlay networks: random walk methods. PODC (2006)

  31. Mohar, B.: Some applications of laplace eigenvalues of graphs, in graph symmetry: algebric methods and applications. In: Hahn, G., Sabidussi, G.(eds.) NATO ASI Series C, vol. 497. Kluwer, Dordrecht, pp. 225-275 (1997)

  32. Mosk-Aoyama, D., Shah, D.: Computing separable functions via gossip. ACM PODC (2006)

  33. Nonnenmacher, J., Biersack, E.W.: Optimal multicast feedback. IEEE INFOCOMM ’98 (1998)

  34. Ntarmos, N., Triantafillou, P., Weikum, G.: Counting at large: efficient cardinality estimation in internet-scale data networks. Proc. ICDE ’06 (2006)

  35. http://www.peersim.sourceforge.net

  36. Psaltoulis, D., Kostoulas, D., Gupta, I., Birman, K., Demers, A.: Practical algorithms for size estimation in large and dynamic groups. PODC (2004)

  37. Ross S. (2001). Simulation. Elsevier, Amsterdam

    Google Scholar 

  38. Zhang, X., Liu, J., Li, B., Yum, T.-S.P.: Donet/coolstreaming: A data-driven overlay network for live media streaming. IEEE INFOCOM (2005)

Download references

Author information

Authors and Affiliations

  1. Microsoft Research, Cambridge, UK

    A. J. Ganesh

  2. INRIA, Rennes, France

    A. -M. Kermarrec

  3. IRISA/INRIA and France Telecom R&D, Lannion, France

    E. Le Merrer

  4. Thomson Research, Paris, France

    L. Massoulié

Authors
  1. A. J. Ganesh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. A. -M. Kermarrec
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. E. Le Merrer
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. L. Massoulié
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to A. J. Ganesh.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ganesh, A.J., Kermarrec, A.M., Le Merrer, E. et al. Peer counting and sampling in overlay networks based on random walks. Distrib. Comput. 20, 267–278 (2007). https://doi.org/10.1007/s00446-007-0027-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00446-007-0027-z

Keywords

Search

Navigation