Márk Jelasity
Skip Abstract Section
Abstract
Gossip-based communication protocols are appealing in large-scale distributed applications such as information dissemination, aggregation, and overlay topology management. This paper factors out a fundamental mechanism at the heart of all these protocols: the peer-sampling service. In short, this service provides every node with peers to gossip with. We promote this service to the level of a first-class abstraction of a large-scale distributed system, similar to a name service being a first-class abstraction of a local-area system. We present a generic framework to implement a peer-sampling service in a decentralized manner by constructing and maintaining dynamic unstructured overlays through gossiping membership information itself. Our framework generalizes existing approaches and makes it easy to discover new ones. We use this framework to empirically explore and compare several implementations of the peer-sampling service. Through extensive simulation experiments we show that---although all protocols provide a good quality uniform random stream of peers to each node locally---traditional theoretical assumptions about the randomness of the unstructured overlays as a whole do not hold in any of the instances. We also show that different design decisions result in severe differences from the point of view of two crucial aspects: load balancing and fault tolerance. Our simulations are validated by means of a wide-area implementation.
- Albert, R., Jeong, H., and Barabási, A.-L. 2000. Error and attack tolerance of complex networks. Nature 406, 378--382.Google Scholar
Cross Ref
- Allavena, A., Demers, A., and Hopcroft, J. E. 2005. Correctness of a gossip based membership protocol. In Proceedings of the 24th annual ACM symposium on principles of distributed computing (PODC'05). ACM Press. Google ScholarDigital Library
- Barabási, A.-L. 2002. Linked: The New Science of Networks. Perseus, Cambridge, MA.Google Scholar
- Bhagwan, R., Savage, S., and Voelker, G. 2003. Understanding Availability. In 2nd International Workshop on Peer-to-Peer Systems. Lecture Notes in Computer Science. Springer-Verlag, Berlin, Germany.Google Scholar
- Birman, K. P., Hayden, M., Ozkasap, O., Xiao, Z., Budiu, M., and Minsky, Y. 1999. Bimodal multicast. ACM Trans. Comput. Syst. 17, 2 (May), 41--88. Google ScholarDigital Library
- Dabek, F., Zhao, B., Druschel, P., Kubiatowicz, J., and Stoica, I. 2003. Towards a common API for structured peer-to-peer overlays. In Proceedings of the 2nd International Workshop on Peer-to-Peer Systems (IPTPS'03). Berkeley, CA.Google Scholar
- DAS2. The distributed ASCI supercomputer 2 (DAS-2). http://www.cs.vu.nl/das2/.Google Scholar
- Demers, A., Greene, D., Hauser, C., Irish, W., Larson, J., Shenker, S., Sturgis, H., Swinehart, D., and Terry, D. 1987. Epidemic algorithms for replicated database maintenance. In Proceedings of the 6th Annual ACM Symposium on Principles of Distributed Computing (PODC'87). ACM Press, 1--12. Google ScholarDigital Library
- Dorogovtsev, S. N. and Mendes, J. F. F. 2002. Evolution of networks. Advan. Phys. 51, 1079--1187.Google ScholarCross Ref
- Eugster, P. T., Guerraoui, R., Handurukande, S. B., Kermarrec, A.-M., and Kouznetsov, P. 2003. Lightweight probabilistic broadcast. ACM Trans. Comput. Syst. 21, 4, 341--374. Google ScholarDigital Library
- Eugster, P. T., Guerraoui, R., Kermarrec, A.-M., and Massoulié, L. 2004. Epidemic information dissemination in distributed systems. IEEE Comput. 37, 5 (May), 60--67. Google ScholarDigital Library
- Ganesh, A. J., Kermarrec, A.-M., and Massoulié, L. 2003. Peer-to-peer membership management for gossip-based protocols. IEEE Trans. Comput. 52, 2 (Feb.). Google ScholarDigital Library
- Gupta, I., Birman, K. P., and van Renesse, R. 2002. Fighting fire with fire: using randomized gossip to combat stochastic scalability limits. Qual. Reliabi. Engin. Int. 18, 3, 165--184.Google ScholarCross Ref
- Jelasity, M., Guerraoui, R., Kermarrec, A.-M., and van Steen, M. 2004. The peer sampling service: Experimental evaluation of unstructured gossip-based implementations. In Middleware 2004, H.-A. Jacobsen, Ed. Lecture Notes in Computer Science, vol. 3231. Springer-Verlag, 79--98. Google ScholarDigital Library
- Jelasity, M., Kowalczyk, W., and van Steen, M. 2003. Newscast computing. Tech. rep. IR-CS-006 (Nov.), Department of Computer Science, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands.Google Scholar
- Jelasity, M., Kowalczyk, W., and van Steen, M. 2004. An approach to massively distributed aggregate computing on peer-to-peer networks. In Proceedings of the 12th Euromicro Conference on Parallel, Distributed and Network-Based Processing (PDP'04). IEEE Computer Society, 200--207.Google ScholarCross Ref
- Jelasity, M., Montresor, A., and Babaoglu, O. 2004. A modular paradigm for building self-organizing peer-to-peer applications. In Engineering Self-Organising Systems, G. Di Marzo Serugendo, A. Karageorgos, O. F. Rana, and F. Zambonelli, Eds. Lecture Notes in Artificial Intelligence, vol. 2977. Springer, 265--282.Google Scholar
- Jelasity, M., Montresor, A., and Babaoglu, O. 2005. Gossip-based aggregation in large dynamic networks. ACM Trans. Comput. Syst. 23, 2 (May). Google ScholarDigital Library
- Karp, R. M., Schindelhauer, C., Shenker, S., and Vöcking, B. 2000. Randomized Rumor Spreading. In 14th Symposium on the Foundations of Computer Science. IEEE Computer Society Press, Los Alamitos, CA., 565--574. Google ScholarDigital Library
- Kempe, D., Dobra, A., and Gehrke, J. 2003. Gossip-based computation of aggregate information. In Proceedings of the 44th Annual IEEE Symposium on Foundations of Computer Science (FOCS'03). IEEE Computer Society, 482--491. Google ScholarDigital Library
- Kermarrec, A.-M., Massoulié, L., and Ganesh, A. J. 2003. Probablistic reliable dissemination in large-scale systems. IEEE Trans. Parall. Distrib. Syst. 14, 3 (March). Google ScholarDigital Library
- King, V. and Saia, J. 2004. Choosing a random peer. In Proceedings of the 23rd annual ACM Symposium on Principles of Distributed Computing (PODC'04). ACM Press, 125--130. Google ScholarDigital Library
- Kostić, D., Rodriguez, A., Albrecht, J., Bhirud, A., and Vahdat, A. 2003. Using random subsets to build scalable network services. In Proceedings of the USENIX Symposium on Internet Technologies and Systems (USITS'03). Google ScholarDigital Library
- Kowalczyk, W. and Vlassis, N. 2005. Newscast EM. In 17th Advances in Neural Information Processing Systems (NIPS). L. K. Saul, Y. Weiss, and L. Bottou, Eds. MIT Press, Cambridge, MA, 713--720.Google Scholar
- Law, C. and Siu, K.-Y. 2003. Distributed construction of random expander graphs. In Proceedings of The 22nd Annual Joint Conference of the IEEE Computer and Communications Societies (INFOCOM'03). San Francisco, CA.Google Scholar
- Li, M. and Vitányi, P. 1997. An Introduction to Kolmogorov Complexity and its Applications 2nd Ed. Springer Verlag. Google ScholarDigital Library
- Loguinov, D., Kumar, A., Rai, V., and Ganesh, S. 2003. Graph-theoretic analysis of structured peer-to-peer systems: Routing distances and fault resilience. In Proceedings of ACM SIGCOMM 2003. ACM Press, 395--406. Google ScholarDigital Library
- Marsaglia, G. 1995. The Marsaglia random number CDROM including the Diehard battery of tests of randomness. Florida State University. http://www.stat.fsu.edu/pub/diehard.Google Scholar
- Marsaglia, G. and Tsang, W. W. 2002. Some difficult-to-pass tests of randomness. J. Statis. Softw. 7, 3, 1--8.Google ScholarCross Ref
- Montresor, A., Jelasity, M., and Babaoglu, O. 2004. Robust aggregation protocols for large-scale overlay networks. In Proceedings of the International Conference on Dependable Systems and Networks (DSN). IEEE Computer Society, 19--28. Google ScholarDigital Library
- Newman, M. E. J. 2002. Random graphs as models of networks. In Handbook of Graphs and Networks: From the Genome to the Internet, S. Bornholdt and H. G. Schuster Eds., John Wiley, New York, NY (Chapter 2)Google Scholar
- Pandurangan, G., Raghavan, P., and Upfal, E. 2003. Building low-diameter peer-to-peer networks. IEEE J. Selec. Areas Comm. 21, 6 (Aug.) 995--1002. Google ScholarDigital Library
- Pastor-Satorras, R. and Vespignani, A. 2001. Epidemic dynamics and endemic states in complex networks. Phys. Rev. E 63, 066117.Google ScholarCross Ref
- PeerSim. PeerSim. http://peersim.sourceforge.net/.Google Scholar
- Pittel, B. 1987. On spreading a rumor. SIAM J. Appl. Math. 47, 1 (Feb.) 213--223. Google ScholarDigital Library
- Ratnasamy, S., Francis, P., Handley, M., Karp, R., and Schenker, S. 2001. A scalable content-addressable network. In Proceedings of the Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications (SIGCOMM). ACM Press, 161--172. Google ScholarDigital Library
- Rowstron, A. and Druschel, P. 2001. Pastry: Scalable, distributed object location and routing for large-scale peer-to-peer systems. In Middleware 2001. R. Guerraoui Ed. Lecture Notes in Computer Science, vol. 2218. Springer-Verlag, 329--350. Google ScholarDigital Library
- Saroiu, S., Gummadi, P. K., and Gribble, S. D. 2003. Measuring and analyzing the characteristics of Napster and Gnutella hosts. Multim. Syst. J. 9, 2 (Aug.) 170--184. Google ScholarDigital Library
- Sen, S. and Wang, J. 2004. Analyzing Peer-to-Peer Traffic Across Large Networks. IEEE/ACM Trans. Network. 12, 2 (April) 219--232. Google ScholarDigital Library
- Stavrou, A., Rubenstein, D., and Sahu, S. 2004. A Lightweight, Robust P2P System to Handle Flash Crowds. IEEE J. Select. Areas Comm. 22, 1 (Jan.) 6--17. Google ScholarDigital Library
- Stoica, I., Morris, R., Karger, D., Kaashoek, M. F., and Balakrishnan, H. 2001. Chord: A scalable peer-to-peer lookup service for internet applications. In Proceedings of the Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications (SIGCOMM). ACM Press, 149--160. Google ScholarDigital Library
- Sun, Q. and Sturman, D. 2000. A Gossip-Based Reliable Multicast for Large-Scale High-Throughput Applications. In International Conference on Dependable Systems and Networks. IEEE Computer Society Press, Los Alamitos, CA., 347--358. Google ScholarDigital Library
- van Renesse, R., Birman, K. P., and Vogels, W. 2003. Astrolabe: A robust and scalable technology for distributed system monitoring, management, and data mining. ACM Trans. Comput. Syst. 21, 2 (May) 164--206. Google ScholarDigital Library
- van Renesse, R., Minsky, Y., and Hayden, M. 1998. A gossip-style failure detection service. In Middleware 1998. 55--70. Google ScholarDigital Library
- Voulgaris, S., Gavidia, D., and van Steen., M. 2005. CYCLON: Inexpensive membership management for unstructured P2P overlays. J. Netw. Syst. Manag. 13, 2, 197--217.Google ScholarCross Ref
- Voulgaris, S. and van Steen, M. 2003. An Epidemic Protocol for Managing Routing Tables in very large Peer-to-Peer Networks. In 14th Workshop on Distributed Systems: Operations and Management. Lecture Notes in Computer Science, vol. 2867. Springer-Verlag, Berlin, 41--54.Google Scholar
- Watts, D. J. and Strogatz, S. H. 1998. Collective dynamics of 'small-world' networks. Nature 393, 440--442.Google ScholarCross Ref
- Zhong, M., Shen, K., and Seiferas, J. 2005. Non-uniform random membership management in peer-to-peer networks. In Proceedings of the IEEE INFOCOM.Google Scholar
Index Terms
- Gossip-based peer sampling
Recommendations
NAT-resilient Gossip Peer Sampling
ICDCS '09: Proceedings of the 2009 29th IEEE International Conference on Distributed Computing SystemsGossip peer sampling protocols now represent a solid basis to build and maintain peer to peer (p2p) overlay networks. They provide peers with a random sample of the network and maintain connectivity in highly dynamic settings. They rely on the ...
A Probabilistic Gossip-based Secure Protocol for Unstructured P2P Networks
Gossip based communication is an efficient tool to manage unstructured decentralized Peer-to-Peer (P2P) overlay connectivity. Such unstructured overlay connectivity is prone to security risks. Few malicious nodes are able to isolate all peer of the ...
Proactive gossip-based management of semantic overlay networks: Research Articles
Parallel and Distributed Computing (EuroPar 2005)Much research on content-based P2P searching for file-sharing applications has focused on exploiting semantic relations between peers to facilitate searching. Current methods suggest reactive ways to manage semantic relations: they rely on the usage of ...
Comments