skip to main content
10.1145/2535838.2535848acmconferencesArticle/Chapter ViewAbstractPublication PagespoplConference Proceedingsconference-collections
research-article

Replicated data types: specification, verification, optimality

Published:08 January 2014Publication History

ABSTRACT

Geographically distributed systems often rely on replicated eventually consistent data stores to achieve availability and performance. To resolve conflicting updates at different replicas, researchers and practitioners have proposed specialized consistency protocols, called replicated data types, that implement objects such as registers, counters, sets or lists. Reasoning about replicated data types has however not been on par with comparable work on abstract data types and concurrent data types, lacking specifications, correctness proofs, and optimality results.

To fill in this gap, we propose a framework for specifying replicated data types using relations over events and verifying their implementations using replication-aware simulations. We apply it to 7 existing implementations of 4 data types with nontrivial conflict-resolution strategies and optimizations (last-writer-wins register, counter, multi-value register and observed-remove set). We also present a novel technique for obtaining lower bounds on the worst-case space overhead of data type implementations and use it to prove optimality of 4 implementations. Finally, we show how to specify consistency of replicated stores with multiple objects axiomatically, in analogy to prior work on weak memory models. Overall, our work provides foundational reasoning tools to support research on replicated eventually consistent stores.

Skip Supplemental Material Section

Supplemental Material

d2_right_t2.mp4

mp4

365 MB

References

  1. Riak key-value store. http://basho.com/products/riak-overview/.Google ScholarGoogle Scholar
  2. S. V. Adve and K. Gharachorloo. Shared memory consistency models: A tutorial. Computer, 29(12), 1996. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. J. Alglave, D. Kroening, V. Nimal, and M. Tautschnig. Software verification for weak memory via program transformation. In ESOP, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. M. Batty, M. Dodds, and A. Gotsman. Library abstraction for C/C++ concurrency. In POPL, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. M. Batty, S. Owens, S. Sarkar, P. Sewell, and T.Weber. Mathematizing C++ concurrency. In POPL, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. A. Bieniusa, M. Zawirski, N. Preguiça, M. Shapiro, C. Baquero, V. Balegas, and S. Duarte. An optimized conflict-free replicated set. Technical Report 8083, INRIA, 2012.Google ScholarGoogle Scholar
  7. A. Bieniusa, M. Zawirski, N. M. Preguiça, M. Shapiro, C. Baquero, V. Balegas, and S. Duarte. Brief announcement: Semantics of eventually consistent replicated sets. In DISC, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. A.-M. Bosneag and M. Brockmeyer. A formal model for eventual consistency semantics. In IASTED PDCS, 2002.Google ScholarGoogle Scholar
  9. S. Burckhardt, R. Alur, and M. M. K. Martin. Checkfence: checking consistency of concurrent data types on relaxed memory models. In PLDI, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. S. Burckhardt, M. Fähndrich, D. Leijen, and B. P. Wood. Cloud types for eventual consistency. In ECOOP, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. S. Burckhardt, A. Gotsman, and H. Yang. Understanding eventual consistency. Technical Report MSR-TR-2013-39, Microsoft Research, 2013.Google ScholarGoogle Scholar
  12. S. Burckhardt, A. Gotsman, H. Yang, and M. Zawirski. Replicated data types: specification, verification, optimality (extended version), 2013. http://research.microsoft.com/apps/pubs/?id=201602. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. S. Burckhardt, D. Leijen, M. Fähndrich, and M. Sagiv. Eventually consistent transactions. In ESOP, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. B. Charron-Bost. Concerning the size of logical clocks in distributed systems. Information Processing Letters, 39(1), 1991. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. J.-Y. Chen and G. Pandurangan. Optimal gossip-based aggregate computation. In SPAA, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. N. Conway, R. Marczak, P. Alvaro, J. M. Hellerstein, and D. Maier. Logic and lattices for distributed programming. In SOCC, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. A. Fekete, D. Gupta, V. Luchangco, N. Lynch, and A. Shvartsman. Eventually-serializable data services. In PODC, 1996. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. G. DeCandia et al. Dynamo: Amazon's highly available key-value store. In SOSP, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. S. Gilbert and N. Lynch. Brewer's conjecture and the feasibility of consistent, available, partition-tolerant web services. SIGACT News, 33(2), 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. M. Helmi, L. Higham, E. Pacheco, and P.Woelfel. The space complexity of long-lived and one-shot timestamp implementations. In PODC, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. C. A. R. Hoare. Proof of correctness of data representations. Acta Inf., 1, 1972.Google ScholarGoogle Scholar
  22. L. Lamport. Time, clocks, and the ordering of events in a distributed system. Commun. ACM, 21(7), 1978. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. C. Li, D. Porto, A. Clement, R. Rodrigues, N. Preguiça, and J. Gehrke. Making geo-replicated systems fast if possible, consistent when necessary. In OSDI, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. H. Liang, X. Feng, and M. Fu. A rely-guarantee-based simulation for verifying concurrent program transformations. In POPL, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. B. Liskov and S. Zilles. Programming with abstract data types. In ACM Symposium on Very High Level Languages, 1974. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. F. Liu, N. Nedev, N. Prisadnikov, M. T. Vechev, and E. Yahav. Dynamic synthesis for relaxed memory models. In PLDI, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. W. Lloyd, M. J. Freedman, M. Kaminsky, and D. G. Andersen. Don't settle for eventual: scalable causal consistency for wide-area storage with COPS. In SOSP, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. J. Manson, W. Pugh, and S. V. Adve. The Java memory model. In POPL, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. F. Mattern. Virtual time and global states of distributed systems. Parallel and Distributed Algorithms, 1989.Google ScholarGoogle Scholar
  30. S. Moran, G. Taubenfeld, and I. Yadin. Concurrent counting. In PODC, 1992. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. H.-G. Roh, M. Jeon, J.-S. Kim, and J. Lee. Replicated abstract data types: Building blocks for collaborative applications. J. Parallel Distrib. Comput., 71(3), 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. M. Shapiro, N. Preguiça, C. Baquero, and M. Zawirski. A comprehensive study of Convergent and Commutative Replicated Data Types. Technical Report 7506, INRIA, 2011.Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. M. Shapiro, N. M. Preguiça, C. Baquero, and M. Zawirski. Conflictfree replicated data types. In SSS, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Y. Sovran, R. Power, M. K. Aguilera, and J. Li. Transactional storage for geo-replicated systems. In SOSP, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. D. B. Terry, A. J. Demers, K. Petersen, M. Spreitzer, M. Theimer, and B. W. Welch. Session guarantees for weakly consistent replicated data. In PDIS, 1994. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. D. B. Terry, M. M. Theimer, K. Petersen, A. J. Demers, M. J. Spreitzer, and C. H. Hauser. Managing update conflicts in Bayou, a weakly connected replicated storage system. In SOSP, 1995. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. M. Zawirski, A. Bieniusa, V. Balegas, S. Duarte, C. Baquero, M. Shapiro, and N. Preguiça. SwiftCloud: Fault-tolerant georeplication integrated all the way to the client machine. Technical Report 8347, INRIA, 2013.Google ScholarGoogle Scholar

Index Terms

  1. Replicated data types: specification, verification, optimality

              Recommendations

              Comments

              Login options

              Check if you have access through your login credentials or your institution to get full access on this article.

              Sign in
              • Published in

                cover image ACM Conferences
                POPL '14: Proceedings of the 41st ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages
                January 2014
                702 pages
                ISBN:9781450325448
                DOI:10.1145/2535838

                Copyright © 2014 ACM

                Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

                Publisher

                Association for Computing Machinery

                New York, NY, United States

                Publication History

                • Published: 8 January 2014

                Permissions

                Request permissions about this article.

                Request Permissions

                Check for updates

                Qualifiers

                • research-article

                Acceptance Rates

                POPL '14 Paper Acceptance Rate51of220submissions,23%Overall Acceptance Rate824of4,130submissions,20%

                Upcoming Conference

                POPL '25

              PDF Format

              View or Download as a PDF file.

              PDF

              eReader

              View online with eReader.

              eReader