Eddy Truyen
ABSTRACT
The most preferred approach in the literature on service-level objectives for multi-tenant databases is to group tenants according to their SLA class in separate database processes and find optimal co-placement of tenants across a cluster of nodes. To implement performance isolation between co-located database processes, request scheduling is preferred over hypervisor-based virtualization that introduces a significant performance overhead. A relevant question is whether the more light-weight container technology such as Docker is a viable alternative for running high-end performance database workloads. Moreover, the recent uprise and industry adoption of container orchestration (CO) frameworks for the purpose of automated placement of cloud-based applications raises the question what is the additional performance overhead of CO frameworks in this context. In this paper, we evaluate the performance overhead introduced by Docker engine and two representative CO frameworks, Docker Swarm and Kubernetes, when running and managing a CPU-bound Cassandra workload in OpenStack. Firstly, we have found that Docker engine deployments that run in host mode exhibit negligible performance overhead in comparison to native OpenStack deployments. Secondly, we have found that virtual IP networking introduces a substantial overhead in Docker Swarm and Kubernetes due to virtual network bridges when compared to Docker engine deployments. This demands for service networking approaches that run in true host mode but offer support for network isolation between containers. Thirdly, volume plugins for persistent storage have a large impact on the overall resource model of a database workload; more specifically, we show that a CPU-bound Cassandra workload changes into an I/O-bound workload in both Docker Swarm and Kubernetes because their local volume plugins introduce a disk I/O performance bottleneck that does not appear in Docker engine deployments. These findings imply that solved placement decisions for native or Docker engine deployments cannot be reused for Docker Swarm and Kubernetes.
- Brendan Burns, Brian Grant, David Oppenheimer, Eric Brewer, and John Wilkes. 2016. Borg, Omega, and Kubernetes. Commun. ACM 59, 5 (2016), 50--57. Google Scholar
Digital Library
- Apache Cassandra. 2018. Understanding the architecture. URL: http://docs.datastax.com/en/cassandra/3.0/cassandra/architecture/archTOC.html, accessed 2018-01-29. (2018).Google Scholar
- Brian F. Cooper, Adam Silberstein, Erwin Tam, Raghu Ramakrishnan, and Russell Sears. 2010. Benchmarking cloud serving systems with YCSB. Proceedings of the 1st ACM symposium on Cloud computing - SoCC '10 (2010), 143--154. Google ScholarDigital Library
- Docker. 2018. Swarm mode overview. https://docs.docker.com/engine/swarm/. (2018). https://docs.docker.com/engine/swarm/ Accessed: February 14 2018.Google Scholar
- Eddy Truyen et al. 2018. Performance overhead of container orchestration frameworks for multi-tenant database deployments. (2018). https://goo.gl/PWwR9f Accessed: December 12 2018.Google Scholar
- Miguel G. Xavier et al. 2015. A Performance Isolation Analysis of Disk-Intensive Workloads on Container-Based Clouds. 2015 23rd Euromicro International Conference on Parallel, Distributed, and Network-Based Processing (2015), 253--260. Google ScholarDigital Library
- Wes Felter, Alexandre Ferreira, Ram Rajamony, and Juan Rubio. 2015. An updated performance comparison of virtual machines and Linux containers. 2015 IEEE International Symposium on Performance Analysis of Systems and Software (IS-PASS) (2015). http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=7095802Google ScholarCross Ref
- D. Gehberger, D. Balla, M. Maliosz, and C. Simon. 2018. Performance Evaluation of Low Latency Communication Alternatives in a Containerized Cloud Environment. In 2018 IEEE 11th International Conference on Cloud Computing (CLOUD). IEEE.Google Scholar
- Dean Jacobs and Stefan Aulbach. 2007. Ruminations on Multi-Tenant Databases. BTW Proceedings 103 (2007).Google Scholar
- Nane Kratzke. 2014. A Lightweight Virtualization Cluster Reference Architecture Derived from Open Source PaaS Platforms. Open Journal of Mobile Computing and Cloud Computing 1, 2 (2014), 17--30.Google Scholar
- Nane Kratzke. 2015. About Microservices, Containers and their Underestimated Impact on Network Performance. In Proceedings of CLOUD COMPUTING 2015 (6th. International Conference on Cloud Computing, GRIDS and Virtualization). 165--169.Google Scholar
- Rouven Krebs, Christof Momm, and Samuel Kounev. 2014. Metrics and techniques for quantifying performance isolation in cloud environments. Science of Computer Programming 90 (2014), 116--134.Google ScholarCross Ref
- Kubernetes. 2018. Production-Grade Container Orchestration. URL: https://kubernetes.io/, accessed 2018-01-23. (2018).Google Scholar
- Kubernetes. 2018. Using kubeadm to Create a Cluster. URL: https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/, accessed 2018-01-29. (2018).Google Scholar
- Willis Lang, Srinath Shankar, Jignesh M. Patel, and Ajay Kalhan. 2014. Towards multi-tenant performance SLOs. IEEE Transactions on Knowledge and Data Engineering 26, 6 (2014), 1447--1463. Google ScholarDigital Library
- Jacob Leverich and Christos Kozyrakis. 2014. Reconciling High Server Utilization and Sub-millisecond Quality-of-service. In Proceedings of the Ninth European Conference on Computer Systems (EuroSys '14). ACM. Google ScholarDigital Library
- Prateek Sharma, Lucas Chaufournier, Prashant Shenoy, and Y C Tay. 2016. Containers and Virtual Machines at Scale: A Comparative Study. In Proceedings of the 17th International Middleware Conference (Middleware '16). ACM, New York, NY, USA. Google ScholarDigital Library
- David Shue, Michael J Freedman, and Anees Shaikh. 2012. Performance Isolation and Fairness for Multi-Tenant Cloud Storage. 10th USENIX Symposium on Operating Systems Design and Implementation (2012), 1--14. Google ScholarDigital Library
- Rebecca Taft, Willis Lang, Jennie Duggan, Aaron J Elmore, Michael Stone-braker, and David Dewitt. 2016. STeP : Scalable Tenant Placement for Managing Database-as-a-Service Deployments. Proceedings of the Seventh AMC Symposium on Cloud Computing (2016), 388--400. Google ScholarDigital Library
- Eddy Truyen, Matt Bruzek, Dimitri Van Landuyt, Bert Lagaisse, and Wouter Joosen. 2018. Evaluation of container orchestration systems for deploying and managing NoSQL database clusters. In Cloud Computing (CLOUD), 2018 IEEE 11th International Conference on. IEEE.Google ScholarCross Ref
- Eddy Truyen, Dimitri Van Landuyt, Vincent Reniers, Ansar Rafique, Bert Lagaisse, and Wouter Joosen. 2016. Towards a container-based architecture for multi-tenant SaaS applications. In ARM 2016 Proceedings of the 15th International Workshop on Adaptive and Reflective Middleware. ACM. Google ScholarDigital Library
- Abhishek Verma, Luis Pedrosa, Madhukar Korupolu, David Oppenheimer, Eric Tune, and John Wilkes. 2015. Large-scale cluster management at Google with Borg. Eurosys (2015). Google ScholarDigital Library
- weaveworks. 2018. Weave Net. https://www.weave.works/oss/net/. (2018). https://www.weave.works/oss/net/ Accessed: February 14 2018.Google Scholar
- Dag Wiers. 2009. Dstat: Versatile resource statistics tool. URL: http://dag.wiee.rs/home-made/dstat/, accessed 2018-01-29. (2009).Google Scholar
- Miguel Gomes Xavier, Marcelo Veiga Neves, and Cesar Augusto Fonticielha De Rose. 2014. A Performance Comparison of Container-Based Virtualization Systems for MapReduce Clusters. 2014 22nd Euromicro International Conference on Parallel, Distributed, and Network-Based Processing (2014), 299--306. Google ScholarDigital Library
Recommendations
Feasibility of container orchestration for adaptive performance isolation in multi-tenant SaaS applications
SAC '20: Proceedings of the 35th Annual ACM Symposium on Applied ComputingSaaS application instances typically serve multiple tenants to improve cost-efficiency. This results in the need for adaptive performance isolation between tenants in order to guarantee custom service level objectives (SLOs) about request latency or ...
Performance Overhead among Three Hypervisors: An Experimental Study Using Hadoop Benchmarks
BIGDATACONGRESS '13: Proceedings of the 2013 IEEE International Congress on Big DataHyper visors are widely used in cloud environments and their impact on application performance has been a topic of significant research and practical interest. We conducted experimental measurements of several benchmarks using Hadoop MapReduce to ...
Performance implications of multi-tier application deployments on Infrastructure-as-a-Service clouds: Towards performance modeling
Hosting a multi-tier application using an Infrastructure-as-a-Service (IaaS) cloud requires deploying components of the application stack across virtual machines (VMs) to provide the application's infrastructure while considering factors such as ...
Comments