Márk Jelasity
Alberto Montresor
Ozalp Babaoglu
Abstract
As computer networks increase in size, become more heterogeneous and span greater geographic distances, applications must be designed to cope with the very large scale, poor reliability, and often, with the extreme dynamism of the underlying network. Aggregation is a key functional building block for such applications: it refers to a set of functions that provide components of a distributed system access to global information including network size, average load, average uptime, location and description of hotspots, and so on. Local access to global information is often very useful, if not indispensable for building applications that are robust and adaptive. For example, in an industrial control application, some aggregate value reaching a threshold may trigger the execution of certain actions; a distributed storage system will want to know the total available free space; load-balancing protocols may benefit from knowing the target average load so as to minimize the load they transfer. We propose a gossip-based protocol for computing aggregate values over network components in a fully decentralized fashion. The class of aggregate functions we can compute is very broad and includes many useful special cases such as counting, averages, sums, products, and extremal values. The protocol is suitable for extremely large and highly dynamic systems due to its proactive structure---all nodes receive the aggregate value continuously, thus being able to track any changes in the system. The protocol is also extremely lightweight, making it suitable for many distributed applications including peer-to-peer and grid computing systems. We demonstrate the efficiency and robustness of our gossip-based protocol both theoretically and experimentally under a variety of scenarios including node and communication failures.
- Barabási, A.-L. 2002. Linked: the new science of networks. Perseus, Cambridge, Mass.Google Scholar
- Bavier, A., Bowman, M., Chun, B., Culler, D., Karlin, S., Muir, S., Peterson, L., Roscoe, T., Spalink, T., and Wawrzoniak, M. 2004. Operating system support for planetary-scale services. In Proceedings of the First Symposium on Network Systems Design and Implementation (NSDI'04). USENIX, 253--266. Google Scholar
- Demers, A., Greene, D., Hauser, C., Irish, W., Larson, J., Shenker, S., Sturgis, H., Swinehart, D., and Terry, D. 1987. Epidemic algorithms for replicated database maintenance. In Proceedings of the 6th Annual ACM Symposium on Principles of Distributed Computing (PODC'87). ACM Press, Vancouver, British Columbia, Canada, 1--12. Google Scholar
- Dolev, D., Lynch, N., Pinter, S., Stark, E., and Weihl, W. 1986. Reaching approximate agreement in the presence of faults. JACM 33, 3 (July), 499--516. Google Scholar
- Eugster, P. T., Guerraoui, R., Kermarrec, A.-M., and Massoulié, L. 2004. Epidemic information dissemination in distributed systems. IEEE Comput. 37, 5 (May), 60--67. Google Scholar
- Fekete, A. 1994. Asynchronous approximate agreement. Information and Computation 115, 1 (November), 95--124. Google Scholar
- Ghosh, B. and Muthukrishnan, S. 1996. Dynamic load balancing by random matchings. J. Comput. Syst. Sci. 53, 3 (December), 357--370. Google Scholar
- Gupta, I., van Renesse, R., and Birman, K. P. 2001. Scalable fault-tolerant aggregation in large process groups. In Proceedings of the International Conference on Dependable Systems and Networks (DSN'01). IEEE Computer Society, Göteborg, Sweden. Google Scholar
- Horowitz, K. and Malkhi, D. 2003. Estimating network size from local information. Information Processing Letters 88, 5, 237--243. Google Scholar
- Jelasity, M., Guerraoui, R., Kermarrec, A.-M., and van Steen, M. 2004. The peer sampling service: Experimental evaluation of unstructured gossip-based implementations. In Middleware 2004, H.-A. Jacobsen, Ed. Lecture Notes in Computer Science, vol. 3231. Springer-Verlag. Google Scholar
- Jelasity, M., Montresor, A., and Babaoglu, O. 2004a. Detection and removal of malicious peers in gossip-based protocols. In FuDiCo II: S.O.S. Bertinoro, Italy. http://www.cs.utexas.edu/users/lorenzo/sos/.Google Scholar
- Jelasity, M., Montresor, A., and Babaoglu, O. 2004b. A modular paradigm for building self-organizing peer-to-peer applications. In Engineering Self-Organising Systems, G. Di Marzo Serugendo, A. Karageorgos, O. F. Rana, and F. Zambonelli, Eds. Lecture Notes in Artificial Intelligence, vol. 2977. Springer, 265--282.Google Scholar
- Jelasity, M. and van Steen, M. 2002. Large-scale newscast computing on the Internet. Tech. Rep. IR-503, Vrije Universiteit Amsterdam, Department of Computer Science, Amsterdam, The Netherlands. October.Google Scholar
- Joseph, J. and Fellenstein, C. 2003. Grid Computing. Prentice Hall. Google Scholar
- Kempe, D., Dobra, A., and Gehrke, J. 2003. Gossip-based computation of aggregate information. In Proceedings of the 44th Annual IEEE Symposium on Foundations of Computer Science (FOCS'03). IEEE Computer Society, 482--491. Google Scholar
- Kutylowski, M. and Letkiewicz, D. 2003. Computing average value in ad hoc networks. In Mathematical Foundations of Computer Science (MFCS'2003), B. Rovan and P. Vojtáš, Eds. Number 2747 in Lecture Notes in Computer Science. Springer, 511--520.Google Scholar
- Madden, S., Szewczyk, R., Franklin, M. J., and Culler, D. 2002. Supporting aggregate queries over ad-hoc wireless sensor networks. In Fourth IEEE Workshop on Mobile Computing Systems and Applications (WMCSA'02). IEEE Computer Society, Callicoon, New York, 49--58. Google Scholar
- Milojicic, D. S., Kalogeraki, V., Lukose, R., Nagaraja, K., Pruyne, J., Richard, B., Rollins, S., and Xu, Z. 2002. Peer-to-peer computing. Tech. Rep. HPL-2002-57, HP Labs, Palo Alto.Google Scholar
- Montresor, A., Jelasity, M., and Babaoglu, O. 2004. Decentralized ranking in large-scale overlay networks. Tech. Rep. UBLCS-2004-18, University of Bologna, Dept. of Computer Science, Bologna, Italy. December. http://www.cs.unibo.it/pub/TR/UBLCS/2004/2004-18.pdf.Google Scholar
- Nekovee, M., Soppera, A., and Burbridge, T. 2003. An adaptive method for dynamic audience size estimation in multicast. In Group Communications and Charges: Technology and Business Models, B. Stiller, G. Carle, M. Karsten, and P. Reichl, Eds. Number 2816 in Lecture Notes in Computer Science. Springer, 23--33.Google Scholar
- Pease, M., Shostak, R., and Lamport, L. 1980. Reaching agreement in the presence of faults. JACM 27, 2, 228--234. Google Scholar
- PeerSim. http://peersim.sourceforge.net/.Google Scholar
- Ripeanu, M., Iamnitchi, A., and Foster, I. 2002. Mapping the gnutella network. IEEE Internet Computing 6, 1, 50--57. Google Scholar
- Saroiu, S., Gummadi, P. K., and Gribble, S. D. 2003. Measuring and analyzing the characteristics of Napster and Gnutella hosts. Multimedia Systems Journal 9, 2 (August), 170--184. Google Scholar
- van Renesse, R. 2003. The importance of aggregation. In Future Directions in Distributed Computing, A. Schiper, A. A. Shvartsman, H. Weatherspoon, and B. Y. Zhao, Eds. Number 2584 in Lecture Notes in Computer Science. Springer, 87--92. Google Scholar
- van Renesse, R., Birman, K. P., and Vogels, W. 2003. Astrolabe: A robust and scalable technology for distributed system monitoring, management, and data mining. ACM Trans. Comput. Syst. 21, 2 (May), 164--206. Google Scholar
- van Renesse, R., Minsky, Y., and Hayden, M. 1998. A gossip-style failure detection service. In Middleware '98, N. Davies, K. Raymond, and J. Seitz, Eds. Springer, 55--70. Google Scholar
- Watts, D. J. 1999. Small Worlds: The Dynamics of Networks Between Order and Randomness. Princeton University Press. Google Scholar
- Watts, D. J. and Strogatz, S. H. 1998. Collective dynamics of ‘small-world’ networks. Nature 393, 440--442.Google Scholar
- Yalagandula, P. and Dahlin, M. 2004. A scalable distributed information management system. In Proceedings of ACM SIGCOMM 2004. ACM Press, Portland, Oregon, USA, 379--390. Google Scholar
Index Terms
- Gossip-based aggregation in large dynamic networks
Recommendations
Gossip-based peer sampling
Gossip-based communication protocols are appealing in large-scale distributed applications such as information dissemination, aggregation, and overlay topology management. This paper factors out a fundamental mechanism at the heart of all these ...
A Probabilistic Gossip-based Secure Protocol for Unstructured P2P Networks
Gossip based communication is an efficient tool to manage unstructured decentralized Peer-to-Peer (P2P) overlay connectivity. Such unstructured overlay connectivity is prone to security risks. Few malicious nodes are able to isolate all peer of the ...
Decentralized Aggregation Protocols in Peer-to-Peer Networks: A Survey
MACE '09: Proceedings of the 4th IEEE International Workshop on Modelling Autonomic Communications EnvironmentsIn large scale decentralized and dynamic networks such as Peer-to-Peer ones, being able to deal with quality of service requires the establishment of a decentralized, autonomous and efficient management strategy. In this context, there is a need to know ...
Reviews
Petcu Dana
A new robust and adaptive protocol for computing aggregate values over network components is presented and studied. It is suitable for large and dynamic systems, including peer-to-peer or grid computing systems. The core of the protocol is a decentralized proactive push-pull gossip-based communication scheme. All of the stages of building a protocol are addressed in detail: the conceptual framework, a theoretical analysis to prove its correctness, cost and performance estimations, simulation results, usage examples, comparisons with similar approaches, and experimental evidence of the protocol's robustness and adaptivity to network dynamics. An interesting conclusion is that the very fast decrease of the variance of the average approximation provided by the protocol is independent of the network size. This is very important for the protocol's scalability in the context of usage in very large systems. Moreover, the average convergence factor is independent of the distribution of the node values. A churn scenario, in which nodes continuously join and leave the network, is taken into account. Two sources of random failures, node crashes and link failures, are examined with regard to their effects on the accuracy of the estimates provided by the protocol. Simulations were performed for these types of failures, sustaining the protocol's robustness. Efficiency is proven for several network topologies with small diameters. The protocol was implemented on the well-known PlanetLab platform, and the theoretical and simulation results are confirmed by the tests performed using this implementation. Online Computing Reviews Service
Access critical reviews of Computing literature here
Become a reviewer for Computing Reviews.
Comments