Viyom Mittal
ABSTRACT
Serverless computing platforms simplify development, deployment, and automated management of modular software functions. However, existing serverless platforms typically assume an over-provisioned cloud, making them a poor fit for Edge Computing environments where resources are scarce. In this paper we propose a redesigned serverless platform that comprehensively tackles the key challenges for serverless functions in a resource constrained Edge Cloud.
Our Mu platform cleanly integrates the core resource management components of a serverless platform: autoscaling, load balancing, and placement. Each worker node in Mu transparently propagates metrics such as service rate and queue length in response headers, feeding this information to the load balancing system so that it can better route requests, and to our autoscaler to anticipate workload fluctuations and proactively meet SLOs. Data from the Autoscaler is then used by the placement engine to account for heterogeneity and fairness across competing functions, ensuring overall resource efficiency, and minimizing resource fragmentation. We implement our design as a set of extensions to the Knative serverless platform and demonstrate its improvements in terms of resource efficiency, fairness, and response time.
Evaluating Mu, shows that it improves fairness by more than 2x over the default Kubernetes placement engine, improves 99th percentile response times by 62% through better load balancing, reduces SLO violations and resource consumption by pro-active and precise autoscaling. Mu reduces the average number of pods required by more than ~15% for a set of real Azure workloads.
Supplemental Material
- 2021. 503 Service Unavailable. https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/503. [online].Google Scholar
- 2021. Borg: The Predecessor to Kubernetes. https://kubernetes.io/blog/2015/04/borg-predecessor-to-kubernetes/ [online].Google Scholar
- 2021. Horizontal Pod Autoscaler (HPA). https://knative.dev/docs/serving/autoscaling/autoscaling-concepts/#horizontal-pod-autoscaler-hpa [online].Google Scholar
- 2021. Knative. https://knative.dev/ [online].Google Scholar
- 2021. Knative Pod Autoscaler (KPA). https://knative.dev/docs/serving/autoscaling/autoscaling-concepts/#knative-pod-autoscaler-kpa [online].Google Scholar
- 2021. Kubernetes. https://kubernetes.io/ [online].Google Scholar
- Dimitri P Bertsekas, Robert G Gallager, and Pierre Humblet. 1992. Data networks. Vol. 2. Prentice-Hall International New Jersey.Google ScholarDigital Library
- Flavio Bonomi, Rodolfo A. Milito, Jiang Zhu, and Sateesh Addepalli. 2012. Fog computing and its role in the internet of things. In Proceedings of the first edition of the MCC workshop on Mobile cloud computing, MCC@SIGCOMM 2012, Helsinki, Finland, August 17, 2012, Mario Gerla and Dijiang Huang (Eds.). ACM, 13--16. https://doi.org/10.1145/2342509.2342513Google ScholarDigital Library
- Anshul Gandhi, Xi Zhang, and Naman Mittal. 2015. HALO: Heterogeneity-Aware Load Balancing. In 2015 IEEE 23rd International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems. 242--251. https://doi.org/10.1109/MASCOTS.2015.14 ISSN: 1526--7539.Google ScholarDigital Library
- Ali Ghodsi, Matei Zaharia, Benjamin Hindman, Andy Konwinski, Scott Shenker, and Ion Stoica. 2011. Dominant Resource Fairness: Fair Allocation of Multiple Resource Types.. In NSDI. 24--24.Google Scholar
- Katja Gilly, Carlos Juiz, and Ramon Puigjaner. 2011. An up-to-date survey in web load balancing. World Wide Web 14, 2 (March 2011), 105--131. https://doi.org/10.1007/s11280-010-0101-5Google ScholarDigital Library
- Robert Grandl, Ganesh Ananthanarayanan, Srikanth Kandula, Sriram Rao, and Aditya Akella. 2014. Multi-resource packing for cluster schedulers. ACM SIGCOMM Computer Communication Review 44, 4 (2014), 455--466.Google ScholarDigital Library
- Varun Gupta, Mor Harchol Balter, Karl Sigman, and Ward Whitt. 2007. Analysis of join-the-shortest-queue routing for web server farms. Performance Evaluation 64, 9 (Oct. 2007), 1062--1081. https://doi.org/10.1016/j.peva. 2007.06.012Google ScholarCross Ref
- LLC Gurobi Optimization. 2021. Gurobi Optimizer Reference Manual. http://www.gurobi.com [online].Google Scholar
- Pawan Kumar and Rakesh Kumar. 2019. Issues and Challenges of Load Balancing Techniques in Cloud Computing: A Survey. Comput. Surveys 51, 6 (Feb. 2019), 120:1--120:35. https://doi.org/10.1145/3281010Google ScholarDigital Library
- Junfeng Li, Sameer G Kulkarni, K. K. Ramakrishnan, and Dan Li. 2019. Understanding open source serverless platforms: Design considerations and performance. In Proceedings of the 5th International Workshop on Serverless Computing. 37--42.Google ScholarDigital Library
- Wes Lloyd and et al. 2018. Serverless computing: An investigation of factors influencing microservice performance. In 2018 IEEE International Conference on Cloud Engineering (IC2E). IEEE, 159--169.Google Scholar
- Garrett McGrath and Paul R Brenner. 2017. Serverless computing: Design, implementation, and performance. In 2017 IEEE 37th International Conference on Distributed Computing Systems Workshops (ICDCSW). IEEE, 405--410.Google ScholarCross Ref
- M. Mitzenmacher. 2001. The power of two choices in randomized load balancing. IEEE Transactions on Parallel and Distributed Systems 12, 10 (Oct. 2001), 1094--1104. https://doi.org/10.1109/71.963420 Conference Name: IEEE Transactions on Parallel and Distributed Systems.Google ScholarDigital Library
- S. K. Mohanty, G. Premsankar, and M. di Francesco. 2018. An Evaluation of Open Source Serverless Computing Frameworks. In 2018 IEEE International Conference on Cloud Computing Technology and Science (CloudCom). 115--120.Google Scholar
- Andrei Palade, Aqeel Kazmi, and Siobhán Clarke. 2019. An Evaluation of Open Source Serverless Computing Frameworks Support at the Edge. In 2019 IEEE World Congress on Services (SERVICES), Vol. 2642. IEEE, 206--211.Google ScholarCross Ref
- Christos-Alexandros Psomas and Jarett Schwartz. 2013. Beyond beyond dominant resource fairness: Indivisible resource allocation in clusters. Tech Report Berkeley, Tech. Rep. (2013).Google Scholar
- Robert Ricci, Eric Eide, and CloudLab Team. 2014. Introducing Cloud-Lab: Scientific infrastructure for advancing cloud architectures and applications. The magazine of USENIX & SAGE 39, 6 (2014), 36--38.Google Scholar
- Mahadev Satyanarayanan, Zhuo Chen, Kiryong Ha, Wenlu Hu, Wolfgang Richter, and Padmanabhan Pillai. 2014. Cloudlets: at the Leading Edge of Mobile-Cloud Convergence. In Proceedings of the 6th International Conference on Mobile Computing, Applications and Services. ICST, Austin, United States. https://doi.org/10.4108/icst.mobicase.2014.257757Google ScholarCross Ref
- Johann Schleier-Smith, Vikram Sreekanti, Anurag Khandelwal, Joao Carreira, Neeraja J. Yadwadkar, Raluca Ada Popa, Joseph E. Gonzalez, Ion Stoica, and David A. Patterson. 2021. What serverless computing is and should become: the next phase of cloud computing. Commun. ACM 64, 5 (April 2021), 76--84. https://doi.org/10.1145/3406011Google ScholarDigital Library
- Mohammad Shahrad, Rodrigo Fonseca, Íñigo Goiri, Gohar Chaudhry, Paul Batum, Jason Cooke, Eduardo Laureano, Colby Tresness, Mark Russinovich, and Ricardo Bianchini. 2020. Serverless in the wild: Characterizing and optimizing the serverless workload at a large cloud provider. In 2020 {USENIX} Annual Technical Conference ({USENIX} {ATC} 20). 205--218.Google Scholar
- Tarik Taleb, Konstantinos Samdanis, Badr Mada, Hannu Flinck, Sunny Dutta, and Dario Sabella. 2017. On Multi-Access Edge Computing: A Survey of the Emerging 5G Network Edge Cloud Architecture and Orchestration. IEEE Communications Surveys Tutorials 19, 3 (2017), 1657--1681. https://doi.org/10.1109/COMST.2017.2705720Google ScholarDigital Library
- Liang Wang and et al. 2018. Peeking behind the curtains of serverless platforms. In 2018 USENIX Annual Technical Conference (USENIX ATC 18). 133--146.Google Scholar
- Cui Yan. 2017. How does language, memory and package size affect cold starts of AWS Lambda? https://read.acloud.guru/does-coding-language-memory-or-package-size-affect-cold-starts-of-aws-lambda-a15e26d12c76. [online].Google Scholar
Index Terms
- Mu: An Efficient, Fair and Responsive Serverless Framework for Resource-Constrained Edge Clouds
Recommendations
Multi-Tenant Cloud Data Services: State-of-the-Art, Challenges and Opportunities
SIGMOD '22: Proceedings of the 2022 International Conference on Management of DataEnterprises are moving their business-critical workloads to public clouds at an accelerating pace. Multi-tenancy is a crucial tenet for cloud data service providers allowing them to provide services in cost-effective manner by sharing of resources among ...
Recent advancements in resource allocation techniques for cloud computing environment: a systematic review
There are two actors in cloud computing environment cloud providers and cloud users. On one hand cloud providers hold enormous computing resources in the cloud large data centers that rent the resources out to the cloud users on a pay-per-use basis to ...
All but one: FaaS platform elasticity revisited
Serverless computing through Function-as-a-Service (FaaS) platforms is a popular new cloud computing model, offering high elasticity while being fully-managed. However, FaaS platforms do not quantify elasticity; thus, application developers are unaware ...
Comments