Brian F. Cooper
Hector Garcia-Molina
Abstract
Data archiving systems rely on replication to preserve information. This paper discusses how a network of autonomous archiving sites can trade data to achieve the most reliable replication. A series of binary trades among sites produces a peer-to-peer archiving network. Two trading algorithms are examined, one based on trading collections (even if they are different sizes) and another based on trading equal sized blocks of space (which can then store collections). The concept of deeds is introduced; deeds track the blocks of space owned by one site at another. Policies for tuning these algorithms to provide the highest reliability, for example by changing the order in which sites are contacted and offered trades, are discussed. Finally, simulation results are presented that reveal which policies are best. The experiments indicate that a digital archive can achieve the best reliability by trading blocks of space (deeds), and that following certain policies will allow that site to maximize its reliability.
- Bastani, F. B. and Yen, I.-L. 1987. A fault tolerant replicated storage system. In Proceedings of the ICDE.]] Google Scholar
- Beagrie, N. 1998. Developing a policy framework for digital preservation. In Proceedings of the 6th DELOS Workshop on Preservation of Digital Information.]]Google Scholar
- Borr, A. 1981. Transaction monitoring in Encompass {TM}: Reliable distributed transaction processing. In Proceedings of the 7th VLDB.]]Google Scholar
- Chen, Y., Edler, J., Goldberg, A. V., Gottlieb, A., Sobti, S., and Yianilos, P. N. 1999. A prototype implementation of archival intermemory. In Proceedings of the ACM International Conference on Digital Libraries.]] Google Scholar
- Chu, W. W. 1969. Multiple file allocation in a multiple computer system. IEEE Trans. Comput. C-18, 10 (Oct.), 885--889.]]Google Scholar
- Cooper, B., Crespo, A., and Garcia-Molina, H. 2000. Implementing a reliable digital object archive. In Proceedings of the European Conference on Digital Libraries (ECDL). In LNCS (Springer-Verlag) volume 1923.]] Google Scholar
- Cooper, B. and Garcia-Molina, H. 2001. Creating trading networks of digital archives. In Proceedings of the 1st Joint ACM/IEEE Conference on Digital Libraries (JCDL).]] Google Scholar
- Du, X. and Maryanski, F. 1988. Data allocation in a dynamically reconfigurable environment. In Proceedings of the ICDE.]] Google Scholar
- Fre. 2000. The Freenet Project. http://freenet.sourceforge.net/.]]Google Scholar
- Garrett, J. and Waters, D. 1996. Preserving digital information: Report of the Task Force on Archiving of Digital Information. Accessible at http://www.rlg.org/ArchTF/.]]Google Scholar
- Gnu. 2001. Gnutella. http://gnutella.wego.com.]]Google Scholar
- Goldberg, A. and Yianilos, P. 1998. Towards an archival intermemory. In Advances in Digital Libraries.]] Google Scholar
- Gray, J., Helland, P., O'Neal, P., and Shasha, D. 1996. The dangers of replication and a solution. In Proceedings of the SIGMOD.]] Google Scholar
- Heminger, A. and Robertson, S. 1998. Digital Rosetta Stone: A conceptual model for maintaining long-term access to digital documents. In Proceedings of the 6th DELOS Workshop on Preservation of Digital Information.]] Google Scholar
- Hindel, R. 1990. Image storage organization. In Proceedings of the 10th Symposium on Computer Applications in Radiology (SCAR).]]Google Scholar
- Hsiao, H. and DeWitt, D. 1990. Chained declustering: A new availability strategy for multiprocessor database machines. In Proceedings of the 6th ICDE.]] Google Scholar
- Kistler, J. J. and Satyanarayanan, M. 1992. Disconnected operation in the Coda file system. ACM Trans. Comput. Syst. 10, 1 (Feb.), 3--25.]] Google Scholar
- Kubiatowicz, J., Bindel, D., Chen, Y., Czerwinski, S. E., Eaton, P. R., Geels, D., Gummadi, R., Rhea, S., Weatherspoon, H., Weimer, W., Wells, C., and Zhao, B. Y. 2000. OceanStore: An architecture for global-scale persistent storage. In Proceedings of the ASPLOS.]] Google Scholar
- Lee, E. and Thekkath, C. 1996. Petal: Distributed virtual disks. In Proceedings of the 7th ASPLOS.]] Google Scholar
- Liskov, B., Ghemawat, S., Gruber, R., Johnson, P., Shrira, L., and Williams, M. 1991. Replication in the Harp file system. In Proceedings of the 13th SOSP.]] Google Scholar
- Loc. 2001. LOCKSS status. http://lockss.stanford.edu/projectstatus.htm.]]Google Scholar
- Maria, N., Gaspar, P., Ferreira, A., and Silva, M. 1998. Information preservation in ARIADNE. In Proceedings of the 6th DELOS Workshop on Preservation of Digital Information.]]Google Scholar
- Martello, S. and Toth, P. 1990. Knapsack Problems: Algorithms and Computer Implementations. J. Wiley and Sons, Chichester, New York.]] Google Scholar
- Morris, J. H., Satyanarayanan, M., Conner, M. H., Howard, J. H., Rosenthal, D. S. H., and Smith, F. D. 1986. Andrew: A distributed personal computing environment. Commun. ACM 29, 3 (March), 184--201.]] Google Scholar
- Patterson, D., Gibson, G., and Katz, R. H. 1988. A case for redundant arrays of inexpensive disks (RAID). SIGMOD Record 17, 3 (Sept.), 109--116.]] Google Scholar
- Rajasekar, A., Marciano, R., and Moore, R. 2000. Collection-based persistent archives. http://www.sdsc.edu/NARA/Publications/OTHER/Persistent/Persistent.html.]]Google Scholar
- Rosenthal, D. S. H. and Reich, V. 2000. Permanent web publishing. In Proceedings 2000 USENIX Annual Technical Conference.]] Google Scholar
- Rothenberg, J. 1995. Ensuring the longevity of digital documents. Scientific American 272, 1 (Jan.), 24--29.]]Google Scholar
- Sandhu, H. and Zhou, S. 1992. Cluster-based file replication in large-scale distributed systems. In Proceedings of SIGMETRICS.]] Google Scholar
- Taaffe, J., Kaldis, M., and Gahm, J. 1990. Q-RSTAR digital image management and transmission. In Proceedings of the 10th Symposium on Computer Applications in Radiology (SCAR).]]Google Scholar
- Wolfson, O., Jajodia, S., and Huang, Y. 1997. An adaptive data replication algorithm. ACM Trans. Database Syst. 2, 2 (June), 255--314.]] Google Scholar
Index Terms
- Peer-to-peer data trading to preserve information
Recommendations
Creating trading networks of digital archives
JCDL '01: Proceedings of the 1st ACM/IEEE-CS joint conference on Digital librariesDigital archives can best survive failures if they have made several c opies of their collections at remote sites. In this paper, we discuss how autonomous sites can cooperate to provide preservation by trading data. We examine the decisions that an ...
Preservation research and sustainable digital libraries
The National Science Foundation and DELOS , the European Commission sponsored Network for Digital Libraries, supported a working group to define a research agenda for digital archiving and preservation (DAP-WG) within the context of digital libraries. ...
The Florida Digital Archive and DAITSS: a working preservation repository based on format migration
The Florida Digital Archive is a long-term digital preservation repository for the use of the libraries of the public universities of Florida. It is managed by the Florida Center for Library Automation (FCLA) and based on Dark Archive in the Sunshine ...
Comments