Alan Mislove
Andreas Haeberlen
ABSTRACT
Peer-to-peer (p2p) technology can potentially be used to build highly reliable applications without a single point of failure. However, most of the existing applications, such as file sharing or web caching, have only moderate reliability demands. Without a challenging proving ground, it remains unclear whether the full potential of p2p systems can be realized.To provide such a proving ground, we have designed, deployed and operated a p2p-based email system. We chose email because users depend on it for their daily work and therefore place high demands on the availability and reliability of the service, as well as the durability, integrity, authenticity and privacy of their email. Our system, ePOST, has been actively used by a small group of participants for over two years.In this paper, we report the problems and pitfalls we encountered in this process. We were able to address some of them by applying known principles of system design, while others turned out to be novel and fundamental, requiring us to devise new solutions. Our findings can be used to guide the design of future reliable p2p systems and provide interesting new directions for future research.
- D. G. Andersen, H. Balakrishnan, M. F. Kaashoek, and R. Morris. Resilient overlay networks. In Proceedings of SOSP'01, Banff, Canada, Oct. 2001. Google Scholar
Digital Library
- Bamboo DHT. http://bamboo-dht.org/.Google Scholar
- BitTorrent protocol. http://www.bittorrent.com/protocol.Google Scholar
- B. Bloom. Space-time tradeoffs in hash coding with allowable errors. Communications of the ACM, 13(7):422--426, July 1970. Google ScholarDigital Library
- W. J. Bolosky, J. R. Douceur, D. Ely, and M. Theimer. Feasibility of a serverless distributed file system deployed on an existing set of desktop PCs. In Proceedings of SIGMETRICS'00, Santa Clara, CA, 2000. Google ScholarDigital Library
- M. Castro, M. Costa, and A. Rowstron. Performance and dependability of structured peer-to-peer overlays. In Proceedings of DSN'04, Florence, Italy, June 2004. Google ScholarDigital Library
- M. Castro, P. Druschel, A. Ganesh, A. Rowstron, and D. Wallach. Security for structured peer-to-peer overlay networks. In Proceedings of OSDI'02, Boston, MA, December 2002. Google ScholarDigital Library
- M. Castro, P. Druschel, A.-M. Kermarrec, and A. Rowstron. SCRIBE: A large-scale and decentralized application-level multicast infrastructure. IEEE JSAC, 20(8), Oct. 2002. Google ScholarDigital Library
- W. Chen and X. Liu. Enforcing routing consistency in structured peer-to-peer overlays: Should we and could we? In Proceedings of IPTPS'06, Santa Barbara, CA, February 2006.Google Scholar
- Y.-H. Chu, S. G. Rao, S. Seshan, and H. Zhang. A case for end system multicast. IEEE Journal on Selected Areas in Communication (JSAC), Special Issue on Networking Support for Multicast, 20(8), 2002. Google ScholarDigital Library
- Coral content distribution network. http://www.scs.cs.nyu.edu/coral/.Google Scholar
- L. P. Cox, C. D. Murray, and B. D. Noble. Pastiche: Making backup cheap and easy. In Proceedings of OSDI'02, Boston, MA, December 2002. Google ScholarDigital Library
- F. Dabek, M. F. Kaashoek, D. Karger, R. Morris, and I. Stoica. Wide-area cooperative storage with CFS. In Proceedings of SOSP'01, Banff, Canada, Oct. 2001. Google ScholarDigital Library
- J. R. Douceur. The Sybil attack. In Proceedings of IPTPS'02, Cambridge, MA, Mar. 2002. Google ScholarDigital Library
- eDonkey file sharing network. http://www.edonkey2000.com/.Google Scholar
- M. J. Freedman, E. Freudenthal, and D. Mazières. Democratizing content publication with Coral. In Proceedings of NSDI'04, San Francisco, CA, 2004. Google ScholarDigital Library
- M. J. Freedman, K. Lakshminarayanan, S. Rhea, and I. Stoica. Non-transitive connectivity and DHTs. In Proceedings of WORLDS'05, San Francisco, CA, Dec 2005. Google ScholarDigital Library
- Freepastry. http://freepastry.rice.edu/.Google Scholar
- R. Friedman and K. Birman. Trading consistency for availability in distributed systems. Technical Report TR95--1579, Cornell University, Apr 1996. Google ScholarDigital Library
- The Gnutella protocol specification, 2000. http://dss.clip2.com/GnutellaProtocol04.pdf.Google Scholar
- S. Guha, N. Daswani, and R. Jain. An experimental study of the skype peer-to-peer voip system. In Proceedings of IPTPS'06, Santa Barbara, CA, February 2006.Google Scholar
- K. P. Gummadi, H. V. Madhyastha, S. D. Gribble, and H. M. Levy. Improving the reliability of internet paths with one-hop source routing. In Proceedings of OSDI'04, San Francisco, CA, Dec. 2004. Google ScholarDigital Library
- A. Haeberlen, A. Mislove, and P. Druschel. Glacier: Highly durable, decentralized storage despite massive correlated failures. In Proceedings of NSDI'05, Boston, MA, May 2005. Google ScholarDigital Library
- M. Harren, J. M. Hellerstein, R. Huebsch, B. T. Loo, S. Shenker, and I. Stoica. Complex queries in DHT-based peer-to-peer networks. In Proceedings of IPTPS'02, Cambridge, MA, Mar. 2002. Google ScholarDigital Library
- Y. hua Chu, A. Ganjam, T. S. E. Ng, S. G. Rao, K. Sripanidkulchai, J. Zhan, and H. Zhang. Early experience with an internet broadcast system based on overlay multicast. In Proceedings of USENIX'04, Boston, MA, June 2004. Google ScholarDigital Library
- D. B. Johnson, D. A. Maltz, and J. Broch. The dynamic source routing protocol for multihop wireless ad hoc networks. In Ad Hoc Networking, edited by Charles E. Perkins, Chapter 5, pages 139--172. Addison-Wesley, 2001. Google ScholarDigital Library
- F. Junqueira, R. Bhagwan, A. Hevia, K. Marzullo, and G. M. Voelker. Surviving internet catastrophes. In Proceedings of USENIX'05, Anaheim, CA, Apr 2005. Google ScholarDigital Library
- B. Karp, S. Ratnasamy, S. Rhea, and S. Shenker. Spurring adoption of DHTs with OpenHash, a public DHT service. In Proceedings of IPTPS'04, San Diego, CA, 2004. Google ScholarDigital Library
- Kazaa. http://www.kazaa.com/us/index.htm.Google Scholar
- J. Kubiatowicz, D. Bindel, Y. Chen, S. Czerwinski, P. Eaton, D. Geels, R. Gummadi, S. Rhea, H. Weatherspoon, W. Weimer, C. Wells, and B. Zhao. OceanStore: An architecture for global-scale persistent storage. In Proceedings of ASPLOS'00, Cambridge, MA, Nov 2000. Google ScholarDigital Library
- Q. Lian, Z. Zhang, M. Yang, B. Y. Zhao, Y. Dai, and X. Li. An empirical study of collusion behavior in the Maze p2p file-sharing system. Technical Report MSR-TR-2006-14, Microsoft Research, February 2006.Google Scholar
- R. C. Merkle. Protocols for public key cryptosystems. In Proceedings of the 1980 IEEE Symposium on Security and Privacy, page 122, 1980.Google ScholarCross Ref
- A. Mislove. "POST: A Decentralized Platform for Reliable Collaborative Applications". Master's thesis, Rice University, Houston, TX, 2004.Google Scholar
- A. Mislove and P. Druschel. Providing administrative control and autonomy in structured overlays. In Proceedings of IPTPS'04, San Diego, CA, February 2004. Google ScholarDigital Library
- A. Mislove, A. Post, C. Reis, P. Willmann, P. Druschel, D. S. Wallach, X. Bonnaire, P. Sens, J.-M. Busca, and L. Arantes-Bezerra. POST: A secure, resilient, cooperative messaging system. In Proceedings of HotOS'03, Lihue, HI, May 2003. Google ScholarDigital Library
- D. Moore, V. Paxson, S. Savage, C. Shannon, S. Staniford, and N. Weaver. Inside the Slammer worm. IEEE Security and Privacy, 1(4):33--39, 2003. Google ScholarDigital Library
- E. T. Mueller, J. D. Moore, and G. J. Popek. A nested transaction mechanism for LOCUS. In Proceedings of SOSP'83, Breton Woods, NH, Oct 1983. Google ScholarDigital Library
- P. Murray. The Anubis service. Technical Report HPL-2005-72, HP Labs, June 2005.Google Scholar
- M. A. Olson, K. Bostic, and M. Seltzer. Berkeley DB. In Proceedings of the FREENIX Track of USENIX'99, Monterey, CA, Jun 1999. Google ScholarDigital Library
- eDonkey 2000 - OverNet. http://www.edonkey2000.com.Google Scholar
- K. Park, V. S. Pai, L. Peterson, and Z. Wang. CoDNS: Improving DNS performance and reliability via cooperative lookups. In Proceedings of OSDI'04, San Francisco, CA, Dec. 2004. Google ScholarDigital Library
- V. Paxson. End-to-end Internet packet dynamics. In Proceedings of SIGCOMM'97, Cannes, France, September 1997. Google ScholarDigital Library
- PlanetLab. http://www.planet-lab.org/.Google Scholar
- V. Ramasubramanian and E. Sirer. The design and implementation of a next generation name service for the Internet. In Proceedings of SIGCOMM'04, Portland, OR, Aug 2004. Google ScholarDigital Library
- S. Rhea, B. Godfrey, B. Karp, J. Kubiatowicz, S. Ratnasamy, S. Shenker, I. Stoica, and H. Yu. OpenDHT: A public DHT service and its uses. In Proceedings of SIGCOMM'05, Philadelphia, PA, Aug. 2005. Google ScholarDigital Library
- A. Rowstron and P. Druschel. Pastry: Scalable, distributed object location and routing for large-scale peer-to-peer systems. In Proceedings of Middleware'01, Heidelberg, Germany, Nov. 2001. Google ScholarDigital Library
- A. Rowstron and P. Druschel. Storage management and caching in PAST, a large-scale, persistent peer-to-peer storage utility. In Proceedings of SOSP'01, Banff, Canada, Oct. 2001. Google ScholarDigital Library
- M. Satyanarayanan, J. J. Kistler, P. Kumar, M. E. Okasaki, E. H. Siegel, and D. C. Steere. Coda: A highly available file system for a distributed workstation environment. IEEE Transactions on Computers, 39(4):447--459, 1990. Google ScholarDigital Library
- K. Sripanidkulchai, A. Ganjam, B. Maggsand, and H. Zhang. The feasability of supporting large-scale live streaming applications with dynamic application end-points. In Proceedings of SIGCOMM'04, Portland, OR, Aug. 2004. Google ScholarDigital Library
- I. Stoica, R. Morris, D. Karger, M. F. Kaashoek, and H. Balakrishnan. Chord: A scalable peer-to-peer lookup service for Internet applications. In Proceedings of SIGCOMM'01, San Diego, CA, Aug. 2001. Google ScholarDigital Library
- J. Stribling. All pairs ping results for PlanetLab. http://www.pdos.lcs.mit.edu/~strib/pl_app/.Google Scholar
- J. Stribling, I. Councill, J. Li, M. F. Kaashoek, D. R. Karger, R. Morris, and S. Shenker. OverCite: A cooperative digital research library. In Proceedings of IPTPS'04, Ithaca, NY, Feb. 2005. Google ScholarDigital Library
- R. van Renesse, K. P. Birman, D. Dumitriu, and W. Vogel. Scalable management and data mining using Astrolabe. In Proceedings of IPTPS'02, Cambridge, MA, Mar. 2002. Google ScholarDigital Library
- M. Walfish, H. Balakrishnan, and S. Shenker. Untangling the web from DNS. In Proceedings of NSDI'04, San Francisco, CA, 2004. Google ScholarDigital Library
- L. Wang, K. Park, R. Pang, V. S. Pai, and L. Peterson. Reliability and security in the CoDeeN content distribution network. In Proceedings of USENIX'04, Boston, MA, June 2004. Google ScholarDigital Library
- P. Yalagandula and M. Dahlin. A scalable distributed information management system. In Proceedings of SIGCOMM'04, Portland, OR, Aug. 2004. Google ScholarDigital Library
- B. Zhao, L. Huang, J. Stribling, S. Rhea, A. Joseph, and J. Kubiatowicz. Tapestry: A resilient global-scale overlay for service deployment. IEEE Journal on Selected Areas in Communications, 22(1), January 2001. Google ScholarDigital Library
Index Terms
- Experiences in building and operating ePOST, a reliable peer-to-peer application
Recommendations
Experiences in building and operating ePOST, a reliable peer-to-peer application
Proceedings of the 2006 EuroSys conferencePeer-to-peer (p2p) technology can potentially be used to build highly reliable applications without a single point of failure. However, most of the existing applications, such as file sharing or web caching, have only moderate reliability demands. ...
Peer-to-peer multimedia applications
MM '06: Proceedings of the 14th ACM international conference on MultimediaIn both academia and industry, peer-to-peer (P2P) applications have attracted great attention. Peer-to-peer file sharing applications, such as Napster, Gnutella, Kazaa, BitTorrent, Skype and PPLive, have witnessed tremendous success among end users. And ...
Understanding churn in peer-to-peer networks
IMC '06: Proceedings of the 6th ACM SIGCOMM conference on Internet measurementThe dynamics of peer participation, or churn, are an inherent property of Peer-to-Peer (P2P) systems and critical for design and evaluation. Accurately characterizing churn requires precise and unbiased information about the arrival and departure of ...
Comments