Aaron J. Elmore
Divyakant Agrawal
Amr El Abbadi
ABSTRACT
For data-intensive applications with many concurrent users, modern distributed main memory database management systems (DBMS) provide the necessary scale-out support beyond what is possible with single-node systems. These DBMSs are optimized for the short-lived transactions that are common in on-line transaction processing (OLTP) workloads. One way that they achieve this is to partition the database into disjoint subsets and use a single-threaded transaction manager per partition that executes transactions one-at-a-time in serial order. This minimizes the overhead of concurrency control mechanisms, but requires careful partitioning to limit distributed transactions that span multiple partitions. Previous methods used off-line analysis to determine how to partition data, but the dynamic nature of these applications means that they are prone to hotspots. In these situations, the DBMS needs to reconfigure how data is partitioned in real-time to maintain performance objectives. Bringing the system off-line to reorganize the database is unacceptable for on-line applications.
To overcome this problem, we introduce the Squall technique for supporting live reconfiguration in partitioned, main memory DBMSs. Squall supports fine-grained repartitioning of databases in the presence of distributed transactions, high throughput client workloads, and replicated data. An evaluation of our approach on a distributed DBMS shows that Squall can reconfigure a database with no downtime and minimal overhead on transaction latency.
- H-Store. http://hstore.cs.brown.edu.Google Scholar
- MemSQL. http://www.memsql.com.Google Scholar
- MongoDB. http://mongodb.org.Google Scholar
- NuoDB. http://www.nuodb.com.Google Scholar
- VMware vFabric SQLFire. http://www.vmware.com/go/sqlfire.Google Scholar
- VoltDB. http://www.voltdb.com.Google Scholar
- M. Armbrust, A. Fox, R. Griffith, A. D. Joseph, R. Katz, A. Konwinski, G. Lee, D. Patterson, A. Rabkin, I. Stoica, and M. Zaharia. A view of cloud computing. Commun. ACM, 53(4):50--58, Apr. 2010. Google ScholarDigital Library
- S. K. Barker, Y. Chi, H. J. Moon, H. Hacigümüs, and P. J. Shenoy. "Cut me some slack": latency-aware live migration for databases. In EDBT, pages 432--443, 2012. Google ScholarDigital Library
- P. A. Bernstein and N. Goodman. Timestamp-based algorithms for concurrency control in distributed database systems. In VLDB, pages 285--300, 1980. Google ScholarDigital Library
- R. Cattell. Scalable sql and nosql data stores. SIGMOD Rec., 39:12--27, 2011. Google ScholarDigital Library
- C. Clark et al. Live migration of virtual machines. In NSDI, pages 273--286, 2005. Google ScholarDigital Library
- B. F. Cooper, A. Silberstein, E. Tam, R. Ramakrishnan, and R. Sears. Benchmarking Cloud Serving Systems with YCSB. In SoCC, pages 143--154, 2010. Google ScholarDigital Library
- J. Cowling and B. Liskov. Granola: low-overhead distributed transaction coordination. In USENIX ATC, pages 21--34, June 2012. Google ScholarDigital Library
- C. Curino, Y. Zhang, E. P. C. Jones, and S. Madden. Schism: a workload-driven approach to database replication and partitioning. PVLDB, 3(1):48--57, 2010. Google ScholarDigital Library
- S. Das, S. Nishimura, D. Agrawal, and A. El Abbadi. Albatross: Lightweight Elasticity in Shared Storage Databases for the Cloud using Live Data Migration. PVLDB, 4(8):494--505, May 2011. Google ScholarDigital Library
- D. DeWitt and J. Gray. Parallel database systems: the future of high performance database systems. Commun. ACM, 35(6):85--98, 1992. Google ScholarDigital Library
- C. Diaconu, C. Freedman, E. Ismert, P.-A. Larson, P. Mittal, R. Stonecipher, N. Verma, and M. Zwilling. Hekaton: Sql server's memory-optimized oltp engine. In SIGMOD, pages 1243--1254, 2013. Google ScholarDigital Library
- A. J. Elmore, S. Das, D. Agrawal, and A. El Abbadi. Towards an elastic and autonomic multitenant database. NetDB, 2011.Google Scholar
- A. J. Elmore, S. Das, D. Agrawal, and A. El Abbadi. Zephyr: Live Migration in Shared Nothing Databases for Elastic Cloud Platforms. In SIGMOD, pages 301--312, 2011. Google ScholarDigital Library
- N. Folkman. So, that was a bummer. https://web.archive.org/web/20101104120513/http://blog.foursquare.com/2010/10/05/so-that-was-a-bummer/, October 2010.Google Scholar
- T. Haerder and A. Reuter. Principles of transaction-oriented database recovery. ACM Comput. Surv., 15(4):287--317, Dec. 1983. Google ScholarDigital Library
- S. Harizopoulos, D. J. Abadi, S. Madden, and M. Stonebraker. OLTP through the looking glass, and what we found there. In SIGMOD, pages 981--992, 2008. Google ScholarDigital Library
- E. P. Jones. Fault-Tolerant Distributed Transactions for Partitioned OLTP Databases. PhD thesis, MIT, 2011. Google ScholarDigital Library
- R. Kallman, H. Kimura, J. Natkins, A. Pavlo, A. Rasin, S. B. Zdonik, E. P. C. Jones, S. Madden, M. Stonebraker, Y. Zhang, J. Hugg, and D. J. Abadi. H-store: a high-performance, distributed main memory transaction processing system. PVLDB, 1(2):1496--1499, 2008. Google ScholarDigital Library
- K. Li and J. F. Naughton. Multiprocessor main memory transaction processing. DPDS, pages 177--187, 1988. Google ScholarDigital Library
- D. B. Lomet, S. Sengupta, and J. J. Levandoski. The bw-tree: A b-tree for new hardware platforms. In ICDE, pages 302--313, 2013. Google ScholarDigital Library
- N. Malviya, A. Weisberg, S. Madden, and M. Stonebraker. Rethinking main memory oltp recovery. In Data Engineering (ICDE), 2014 IEEE 30th International Conference on, pages 604--615, March 2014.Google ScholarCross Ref
- U. F. Minhas, R. Liu, A. Aboulnaga, K. Salem, J. Ng, and S. Robertson. Elastic scale-out for partition-based database systems. In ICDE Workshops, pages 281--288, 2012. Google ScholarDigital Library
- C. N. Nikolaou, M. Marazakis, and G. Georgiannakis. Transaction routing for distributed OLTP systems: survey and recent results. Inf. Sci., 97:45--82, 1997. Google ScholarDigital Library
- NuoDB LLC. NuoDB Emergent Architecture -- A 21st Century Transactional Relational Database Founded On Partial, On-Demand Replication, Jan. 2013.Google Scholar
- I. Pandis, P. Tözün, R. Johnson, and A. Ailamaki. Plp: Page latch-free shared-everything oltp. In PVLDB, volume 4, pages 610--621, 2011. Google ScholarDigital Library
- A. Pavlo, C. Curino, and S. Zdonik. Skew-aware automatic database partitioning in shared-nothing, parallel OLTP systems. In SIGMOD, pages 61--72, 2012. Google ScholarDigital Library
- A. Pavlo, E. P. Jones, and S. Zdonik. On predictive modeling for optimizing transaction execution in parallel oltp systems. Proc. VLDB Endow., 5:85--96, October 2011. Google ScholarDigital Library
- T. Rafiq. Elasca: Workload-aware elastic scalability for partition based database systems. Master's thesis, University of Waterloo, 2013.Google Scholar
- O. Schiller, N. Cipriani, and B. Mitschang. Prorea: live database migration for multi-tenant rdbms with snapshot isolation. In EDBT, pages 53--64, 2013. Google ScholarDigital Library
- R. Stoica, J. J. Levandoski, and P.-A. Larson. Identifying hot and cold data in main-memory databases. In ICDE, pages 26--37, 2013. Google ScholarDigital Library
- M. Stonebraker, S. Madden, D. J. Abadi, S. Harizopoulos, N. Hachem, and P. Helland. The End of an Architectural Era (It's Time for a Complete Rewrite). In VLDB, pages 1150--1160, 2007. Google ScholarDigital Library
- R. Taft, E. Mansour, M. Serafini, J. Duggan, A. J. Elmore, A. Aboulnaga, A. Pavlo, and M. Stonebraker. E-store: Fine-grained elastic partitioning for distributed transaction processing. Proc. VLDB Endow., 8:245--256, November 2014. Google ScholarDigital Library
- The Transaction Processing Performance Council. TPC-C benchmark (Version 5.10.1), 2009.Google Scholar
- S. Tu, W. Zheng, E. Kohler, B. Liskov, and S. Madden. Speedy transactions in multicore in-memory databases. In SOSP, pages 18--32, 2013. Google ScholarDigital Library
- A. Whitney, D. Shasha, and S. Apter. High Volume Transaction Processing Without Concurrency Control, Two Phase Commit, SQL or CGoogle Scholar
- . In HPTS, 1997.Google Scholar
Index Terms
- Squall: Fine-Grained Live Reconfiguration for Partitioned Main Memory Databases
Recommendations
A parallel migration scheme for fast virtual machine relocation on a cloud cluster
The paper proposes a method for parallelizing migrations to reduce the time required for virtual machine (VM) relocation. During VM relocation, VMs need to wait for their migrations in a chain due to the limited resource of the physical machines (PMs), ...
Nomad: migrating OS-bypass networks in virtual machines
VEE '07: Proceedings of the 3rd international conference on Virtual execution environmentsVirtual machine (VM) technology is experiencing a resurgence due to various benefits including ease of management, security and resource consolidation. Live migration of virtual machines allows transparent movement of OS instances and hosted ...
Urgent Virtual Machine Eviction with Enlightened Post-Copy
VEE '16Virtual machine (VM) migration demands distinct properties under resource oversubscription and workload surges. We present enlightened post-copy, a new mechanism for VMs under contention that evicts the target VM with fast execution transfer and short ...
Comments