John Augustine
ABSTRACT
We study robust and efficient distributed algorithms for searching, storing, and maintaining data in dynamic Peer-to-Peer (P2P) networks. P2P networks are highly dynamic networks that experience heavy node churn (i.e., nodes join and leave the network continuously over time). Our goal is to guarantee, despite high node churn rate, that a large number of nodes in the network can store, retrieve, and maintain a large number of data items. Our main contributions are fast randomized distributed algorithms that guarantee the above with high probability even under high adversarial churn. In particular, we present the following main results:
1. A randomized distributed search algorithm that with high probability guarantees that searches from as many as n - o(n) nodes (n is the stable network size) succeed in O(log n )-rounds despite O(n/log1+δn) churn, for any small constant δ > 0, per round. We assume that the churn is controlled by an oblivious adversary (that has complete knowledge and control of what nodes join and leave and at what time and has unlimited computational power, but is oblivious to the random choices made by the algorithm).
2. A storage and maintenance algorithm that guarantees, with high probability, data items can be efficiently stored (with only θ(log n) copies of each data item) and maintained in a dynamic P2P network with churn rate up to O(n/log1+δn) per round. Our search algorithm together with our storage and maintenance algorithm guarantees that as many as n - o(n) nodes can efficiently store, maintain, and search even under O(n/log1+δn) churn per round. Our algorithms require only polylogarithmic in n bits to be processed and sent (per round) by each node.
To the best of our knowledge, our algorithms are the first-known, fully-distributed storage and search algorithms that provably work under highly dynamic settings (i.e., high churn rates per step). Furthermore, they are localized (i.e., do not require any global topological knowledge) and scalable. A technical contribution of this paper, which may be of independent interest, is showing how random walks can be provably used to derive scalable distributed algorithms in dynamic networks with adversarial node churn.
- John Augustine, Anisur Rahaman Molla, Ehab Morsy, Gopal Pandurangan, Peter Robinson, and Eli Upfal. Storage and search in dynamic peer-to-peer networks. CoRR, abs/1305.1121, 2013. Google Scholar
Digital Library
- John Augustine, Gopal Pandurangan, Peter Robinson, and Eli Upfal. Towards robust and efficient computation in dynamic peer-to-peer networks. In ACM-SIAM, SODA 2012, pages 551--569. SIAM, 2012. Google ScholarDigital Library
- C. Avin, M. Koucký, and Z. Lotker. How to explore a fast-changing world (cover time of a simple random walk on evolving graphs). In Proc. of 35th Coll. on Automata, Languages and Programming (ICALP), pages 121--132, 2008. Google ScholarDigital Library
- Baruch Awerbuch and Christian Scheideler. Towards a scalable and robust DHT. Theory of Computing Systems, 45:234--260, 2009. Google ScholarDigital Library
- Ozalp Babaoglu, Moreno Merzolla, and Michele Tamburini. Design and implementation of a p2p cloud system. SAC, March, 2012. Google ScholarDigital Library
- Amitabha Bagchi, Ankur Bhargava, Amitabh Chaudhary, David Eppstein, and Christian Scheideler. The effect of faults on network expansion. Theory Comput. Syst., 39(6):903--928, 2006. Google ScholarDigital Library
- Edward Bortnikov, Maxim Gurevich, Idit Keidar, Gabriel Kliot, and Alexander Shraer. Brahms: Byzantine resilient random membership sampling. Computer Networks, 53, March, 2009. Google ScholarDigital Library
- John F. Canny. Collaborative filtering with privacy. In IEEE Symposium on Security and Privacy, pages 45--57, 2002. Google ScholarDigital Library
- Arnaud Casteigts, Paola Flocchini, Walter Quattrociocchi, and Nicola Santoro. Time-varying graphs and dynamic networks. CoRR, abs/1012.0009, 2010. Short version in ADHOC-NOW 2011. Google ScholarDigital Library
- Arnaud Casteigts, Paola Flocchini, Walter Quattrociocchi, and Nicola Santoro. Time-varying graphs and dynamic networks. CoRR, abs/1012.0009, 2010. Google ScholarDigital Library
- Yu-Wei Chan, Tsung-Hsuan Ho, Po-Chi Shih, and Yeh-Ching Chung. Malugo: A peer-to-peer storage system. Int. J. ad hoc and ubiquitous computing, 5(4), 2010. Google ScholarDigital Library
- Website of Cloudmark Inc. http://cloudmark.com/.Google Scholar
- Website of Crashplan Inc. http://www.crashplan.com/.Google Scholar
- Atish Das Sarma, Anisur Rahaman Molla, and Gopal Pandurangan. Fast distributed computation in dynamic networks via random walks. In DISC, pages 136--150, 2012. Google ScholarDigital Library
- Souptik Datta, Kanishka Bhaduri, Chris Giannella, Ran Wolff, and Hillol Kargupta. Distributed data mining in peer-to-peer networks. IEEE Internet Computing, 10(4):18--26, 2006. Google ScholarDigital Library
- P. Druschel and A. Rowstron. Past: A large-scale, persistent peer-to-peer storage utility. In HotOS VIII, pages 75--80, 2001. Google ScholarDigital Library
- P. Druschel and A. Rowstron. Storage management and caching in past, a large-scale, persistent peer-to-peer storage utility. In Proc. of ACM SOSP, 2001. Google ScholarDigital Library
- Cynthia Dwork, David Peleg, Nicholas Pippenger, and Eli Upfal. Fault tolerance in networks of bounded degree. SIAM J. Comput., 17(5):975--988, 1988. Google ScholarDigital Library
- Jarret Falkner, Michael Piatek, John P. John, Arvind Krishnamurthy, and Thomas E. Anderson. Profiling a million user dht. In Internet Measurement Comference, pages 129--134, 2007. Google ScholarDigital Library
- Amos Fiat and Jared Saia. Censorship resistant peer-to-peer content addressable networks. In SODA, pages 94--103, 2002. Google ScholarDigital Library
- A. J. Ganesh, A.-M. Kermarrec, E. Le Merrer, and L. Massoulié. Peer counting and sampling in overlay networks based on random walks. Distributed Computing, 20:267--278, 2007.Google ScholarDigital Library
- Roxana Geambasu, Tadayoshi Kohno, Amit A. Levy, and Henry M. Levy. Vanish: Increasing data privacy with self-destructing data. In USENIX Security Symposium, pages 299--316, 2009. Google ScholarDigital Library
- Christos Gkantsidis, Milena Mihail, and Amin Saberi. Hybrid search schemes for unstructured peer-to-peer networks. In IEEE INFOCOM, 2005.Google ScholarCross Ref
- P. Krishna Gummadi, Stefan Saroiu, and Steven D. Gribble. A measurement study of napster and gnutella as examples of peer-to-peer file sharing systems. Computer Communication Review, 32(1):82, 2002. Google ScholarDigital Library
- Ragib Hasan, Zahid Anwar, William Yurcik, Larry Brumbaugh, and Roy Campbell. A survey of peer-to-peer storage techniques for distributed file systems. In Proc of ITCC, pages 205--213. IEEE Computer Society, 2005. Google ScholarDigital Library
- Kirsten Hildrum and John Kubiatowicz. Asymptotically efficient approaches to fault-tolerance in peer-to-peer networks. In DISC, volume 2848 of Lecture Notes in Computer Science, pages 321--336. Springer, 2003.Google Scholar
- T. Jacobs and G. Pandurangan. Stochastic analysis of a churn-tolerant structured peer-to-peer scheme. Peer-to-Peer Networking and Applications, 2012.Google Scholar
- Bruce M. Kapron, David Kempe, Valerie King, Jared Saia, and Vishal Sanwalani. Fast asynchronous byzantine agreement and leader election with full information. ACM Transactions on Algorithms, 6(4), 2010. Google ScholarDigital Library
- Valerie King, Jared Saia, Vishal Sanwalani, and Erik Vee. Towards secure and scalable computation in peer-to-peer networks. In FOCS, pages 87--98, 2006. Google ScholarDigital Library
- F. Kuhn and R. Oshman. Dynamic networks: Models and algorithms. SIGACT News, 42(1):82--96, 2011. Google ScholarDigital Library
- Fabian Kuhn, Nancy Lynch, and Rotem Oshman. Distributed computation in dynamic networks. In ACM STOC, pages 513--522, 2010. Google ScholarDigital Library
- Fabian Kuhn, Stefan Schmid, and Roger Wattenhofer. Towards worst-case churn resistant peer-to-peer systems. Distributed Computing, 22(4):249--267, 2010.Google ScholarDigital Library
- C. Law and K.-Y. Siu. Distributed construction of random expander networks. In INFOCOM 2003. Twenty-Second Annual Joint Conference of the IEEE Computer and Communications. IEEE Societies, volume 3, pages 2133--2143 vol.3, march-3 april 2003.Google ScholarCross Ref
- Eng Keong Lua, Jon Crowcroft, Marcelo Pias, Ravi Sharma, and Steven Lim. A survey and comparison of peer-to-peer overlay network schemes. IEEE communications survey and tutorial, 2004.Google Scholar
- Q. Lv, P. Cao, E. Cohen, K. Li, and S. Shenker. Search and replication in unstructured peer-to-peer networks. In Proceedings of the 16th international conference on Supercomputing, pages 84--95, 2002. Google ScholarDigital Library
- David J. Malan and Michael D. Smith. Host-based detection of worms through peer-to-peer cooperation. In Vijay Atluri and Angelos D. Keromytis, editors, WORM, pages 72--80. ACM Press, 2005. Google ScholarDigital Library
- Ruggero Morselli, Bobby Bhattacharjee, Michael A. Marsh, and Aravind Srinivasan. Efficient lookup on unstructured topologies. IEEE Journal on selected areas in communications, 15(1), January 2007. Google ScholarDigital Library
- Moni Naor and Udi Wieder. A simple fault tolerant distributed hash table. In IPTPS, pages 88--97, 2003.Google ScholarCross Ref
- Gopal Pandurangan, Prabhakar Raghavan, and Eli Upfal. Building low-diameter P2P networks. In FOCS, pages 492--499, 2001. Google ScholarDigital Library
- Arjan Peddemors. Cloud storage and peer-to-peer storage - end-user considerations and product overview. http://www.novay.nl/okb/publications/152, 2010.Google Scholar
- J. Saia, A. Fiat, S. Gribble, A. Karlin, and S. Saroiu. Dynamically fault-tolerant content addressable networks. In the Proceedings of the 1st International Workshop on Peer-to-Peer Systems, March 2002. Google ScholarDigital Library
- Christian Scheideler. How to spread adversarial nodes? rotate! In STOC, pages 704--713, 2005. Google ScholarDigital Library
- Subhabrata Sen and Jia Wang. Analyzing peer-to-peer traffic across large networks. In Proceedings of the 2nd ACM SIGCOMM Workshop on Internet measurment, IMW '02, pages 137--150, New York, NY, USA, 2002. ACM. Google ScholarDigital Library
- Moshe Shaked and J. George Shanthikumar. Stochastic Orders. Springer, 2007.Google ScholarCross Ref
- Daniel Stutzbach and Reza Rejaie. Understanding churn in peer-to-peer networks. In Proceedings of the 6th ACM SIGCOMM conference on Internet measurement, IMC '06, pages 189--202, New York, NY, USA, 2006. ACM. Google ScholarDigital Library
- Website of Symform:. http://www.symform.com/.Google Scholar
- Eli Upfal. Tolerating a linear number of faults in networks of bounded degree. Inf. Comput., 115(2):312--320, 1994. Google ScholarDigital Library
- Vasileios Vlachos, Stephanos Androutsellis-Theotokis, and Diomidis Spinellis. Security applications of peer-to-peer networks. Comput. Netw., 45:195--205, June 2004. Google ScholarDigital Library
- Ming Zhong and Kai Sheng. Popularity biased random walks for peer to peer search under the squareroot principle. In IPTPS, 2006.Google Scholar
Index Terms
- Storage and search in dynamic peer-to-peer networks
Recommendations
Sublinear Message Bounds for Randomized Agreement
PODC '18: Proceedings of the 2018 ACM Symposium on Principles of Distributed ComputingThis paper focuses on understanding the message complexity of randomized agreement in synchronous distributed networks. We focus on the so-called implicit agreement problem where each node starts with an input value (0 or 1) and at the end one or more ...
Fast byzantine agreement in dynamic networks
PODC '13: Proceedings of the 2013 ACM symposium on Principles of distributed computingWe study Byzantine agreement in dynamic networks where topology can change from round to round and nodes can also experience heavy churn (i.e., nodes can join and leave the network continuously over time). Our main contributions are randomized ...
Towards robust and efficient computation in dynamic peer-to-peer networks
SODA '12: Proceedings of the twenty-third annual ACM-SIAM symposium on Discrete algorithmsMotivated by the need for robust and fast distributed computation in highly dynamic Peer-to-Peer (P2P) networks, we study algorithms for the fundamental distributed agreement problem. P2P networks are highly dynamic networks that experience heavy node ...
Comments