H. Andrés Lagar-Cavilla
M. Satyanarayanan
Skip Abstract Section
Abstract
A basic building block of cloud computing is virtualization. Virtual machines (VMs) encapsulate a user’s computing environment and efficiently isolate it from that of other users. VMs, however, are large entities, and no clear APIs exist yet to provide users with programatic, fine-grained control on short time scales.
We present SnowFlock, a paradigm and system for cloud computing that introduces VM cloning as a first-class cloud abstraction. VM cloning exploits the well-understood and effective semantics of UNIX fork. We demonstrate multiple usage models of VM cloning: users can incorporate the primitive in their code, can wrap around existing toolchains via scripting, can encapsulate the API within a parallel programming framework, or can use it to load-balance and self-scale clustered servers.
VM cloning needs to be efficient to be usable. It must efficiently transmit VM state in order to avoid cloud I/O bottlenecks. We demonstrate how the semantics of cloning aid us in realizing its efficiency: state is propagated in parallel to multiple VM clones, and is transmitted during runtime, allowing for optimizations that substantially reduce the I/O load. We show detailed microbenchmark results highlighting the efficiency of our optimizations, and macrobenchmark numbers demonstrating the effectiveness of the different usage models of SnowFlock.
- Altschul, S. F., Madden, T. L., Schaffer, A. A., Zhang, J., Zhang, Z., Miller, W., and Lipman, D. J. 1997. Gapped BLAST and PSI--BLAST: A new generation of protein database search programs. Nucleic Acids Res. 25, 17, 3389--3402.Google Scholar
Cross Ref
- Alvarez, G. A., Borowsky, E., Go, S., Romer, T. R., Szendy, R. B. B., Golding, R., Merchant, A., Spasojevic, M., Veitch, A., and Wilkes, J. 2001. Minerva: An automated resource provisioning tool for large-scale storage systems. ACM Trans. Computer Syst. 19, 4, 483--518. Google ScholarDigital Library
- Amazon.com. Amazon Elastic Compute Cloud (Amazon EC2). http://aws.amazon.com/ec2/.Google Scholar
- ANL. Argonne National Laboratory MPICH2. http://www.mcs.anl.gov/research/projects/mpich2/.Google Scholar
- Apache A. The Apache Software Foundation -- Hadoop. http://hadoop.apache.org/core/.Google Scholar
- Apache B. The Apache Software Foundation -- Hadoop Distributed File System: Architecture and Design. http://hadoop.apache.org/core/docs/current/hdfs_design.html.Google Scholar
- Apodaka, A. A. and Gritz, L. 2000. Advanced RenderMan: Creating CGI for Motion Pictures. Academic Press. Google ScholarDigital Library
- Aqsis.org. Aqsis Renderer -- Freedom to Dream. http://aqsis.org/.Google Scholar
- Barham, P., Dragovic, B., Fraser, K., Hand, S., Harris, T., Ho, A., Neugebauer, R., Pratt, I., and Warfield, A. 2003. Xen and the art of virtualization. In Proceedings of the 17th Symposium on Operating Systems Principles (SOSP’03). 164--177. Google ScholarDigital Library
- Bartolini, N., Bongiovanni, G., and Silvestri, S. 2009. Self-* through self-learning: Overload control for distributed Web systems. Comput. Netw. 53, 5, 727--743. Google ScholarDigital Library
- Bugnion, E., Devine, S., and Rosenblum, M. 1997. Disco: Running commodity operating systems on scalable multiprocessors. In Proceedings of the 16th Symposium on Operating Systems Principles (SOSP’97). 143--156. Google ScholarDigital Library
- Carns, P. H., Ligon III, W. B., Ross, R. B., and Thakur, R. 2000. PVFS: A parallel file system for Linux clusters. In Proceedings of the 4th Annual Linux Showcase and Conference. 317--327. Google ScholarDigital Library
- Chandra, R., Menon, R., Dagum, L., Kohr, D., Maydan, D., and McDonald, J. 2000. Parallel Programming in OpenMP. Elsevier. Google ScholarDigital Library
- Chase, J. S., Irwin, D. E., Grit, L. E., Moore, J. D., and Sprenkle, S. E. 2003. Dynamic virtual clusters in a grid site manager. In Proceedings of the 12th IEEE International Symposium on High Performance Distributed Computing (HPDC’03). 90--103. Google ScholarDigital Library
- Clark, C., Fraser, K., Hand, S., Hansen, J. G., Jul, E., Limpach, C., Pratt, I., and Warfield, A. 2005. Live migration of virtual machines. In Proceedings of the 2nd Symposium on Networked Systems Design and Implementation (NSDI’05). Google ScholarDigital Library
- Cluster Resources. Moab Cluster Suite. http://www.clusterresources.com/pages/products/moab-cluster-suite.php.Google Scholar
- Cully, B., Lefebvre, G., Meyer, D., Feeley, M., Hutchinson, N., and Warfield, A. 2008. Remus: High availability via asynchronous virtual machine replication. In Proceedings of the 5th Symposium on Networked Systems Design and Implementation (NSDI’08). Google ScholarDigital Library
- Dean, J. and Ghemawat, S. 2004. MapReduce: Simplified data processing on large clusters. In Proceedings of the 6th Symposium on Operating Systems Design and Implementation (OSDI’04). 137--150. Google ScholarDigital Library
- Dunlap, G. W., King, S. T., Cinar, S., Basrai, M. A., and Chen, P. M. 2002. ReVirt: Enabling intrusion analysis through virtual-machine logging and replay. In Proceedings of the 5th Symposium on Operating Systems Design and Implementation (OSDI). 211--224. Google ScholarDigital Library
- EBI. European Bioinformatics Institute -- ClustalW2. http://www.ebi.ac.uk/Tools/clustalw2/ index.html.Google Scholar
- el Malek, M. A., Wachs, M., Cipar, J., Sanghi, K., Ganger, G. R., Gibson, G. A., and Reiter, M. K. 2009. File system virtual appliances: Portable file system implementations. Tech. rep. CMU-PDL-09-102, Carnegie Mellon University.Google Scholar
- Elnikety, S., Nahum, E., Tracey, J., and Zwaenepoel, W. 2004. A method for transparent admission control and request scheduling in e-commerce web sites. In Proceedings of the 13th International Conference on World Wide Web (WWW’04). 276--286. Google ScholarDigital Library
- Emeneker, W. and Stanzione, D. 2007. Dynamic virtual clustering. In Proceedings of the IEEE International Conference on Cluster Computing (Cluster). 84--90. Google ScholarDigital Library
- Foster, I., Freeman, T., Keahey, K., Scheftner, D., Sotomayor, B., and Zhang, X. 2006. Virtual clusters for grid communities. In Proceedings of the IEEE International Symposium on Cluster Computing and the Grid (CCGrid). 513--520. Google ScholarDigital Library
- Fox, A., Gribble, S. D., Chawathe, Y., Brewer, E. A., and Gauthier, P. 1997. Cluster-based scalable network services. In Proceedings of the 16th Symposium on Operating Systems Principles (SOSP’97). 78--91. Google ScholarDigital Library
- Gabriel, E., Fagg, G. E., Bosilca, G., Angskun, T., Dongarra, J. J., Squyres, J. M., Sahay, V., Kambadur, P., Barrett, B., Lumsdaine, A., Castain, R. H., Daniel, D. J., Graham, R. L., and Woodall, T. S. 2004. Open MPI: Goals, concept, and design of a next generation MPI implementation. In Proceedings of the 11th European PVM/MPI Users’ Group Meeting. 97--104.Google Scholar
- Geist, A., Beguelin, A., Dongarra, J., Jiang, W., Manchek, R., and Sunderam, V. 1994. PVM: Parallel Virtual Machine -- A Users’ Guide and Tutorial for Networked Parallel Computing. MIT Press. Google ScholarDigital Library
- Gentzsch, W. 2001. Sun grid engine: Towards creating a compute power grid. In Proceedings of the IEEE International Symposium on Cluster Computing and the Grid (CCGrid’01). 35--36. Google ScholarDigital Library
- GoGrid. GoGrid Cloud Hosting. http://www.gogrid.com/.Google Scholar
- Gupta, D., Lee, S., Vrable, M., Savage, S., Snoeren, A. C., Varghese, G., Voelker, G. M., and Vahdat, A. 2008. Difference engine: Harnessing memory redundancy in virtual machine. In Proceedings of the 8th Symposium on Operating Systems Design and Implementation (OSDI’08). 309--322. Google ScholarDigital Library
- Hibler, M., Stoller, L., Lepreau, J., Ricci, R., and Barb, C. 2003. Fast, scalable disk imaging with Frisbee. In Proceedings of the USENIX Annual Technical Conference. 283--296.Google Scholar
- Hibler, M., Ricci, R., Stoller, L., Duerig, J., Guruprasad, S., Stack, T., Webb, K., and Lepreau, J. 2008. Large-scale virtualization in the emulab network testbed. In Proceedings of the USENIX Annual Technical Conference. 113--128. Google ScholarDigital Library
- Higgins, D., Thompson, J., and Gibson, T. 1994. CLUSTAL W: Improving the sensitivity of progressive multiple sequence alignment through sequence weighting, position-specific gap penalties and weight matrix choice. Nucleic Acids Res. 22, 22, 4673--4680.Google ScholarCross Ref
- Hines, M. and Gopalan, K. 2009. Post-copy based live virtual machine migration using adaptive pre-paging and dynamic self-ballooning. In Proceedings of the Conference on Virtual Execution Environments (VEE’09). 51--60. Google ScholarDigital Library
- Huang, W., Gao, Q., Liu, J., and Panda, D. K. 2007. High performance virtual machine migration with RDMA over modern interconnects. In Proceedings of the IEEE International Conference on Cluster Computing (Cluster). 11--20. Google ScholarDigital Library
- Huelsenbeck, J. P. and Ronquist, F. 2001. MRBAYES: Bayesian inference of phylogenetic trees. Bioinformatics 17, 8, 754--755. http://mrbayes.csit.fsu.edu/.Google ScholarCross Ref
- Kawachiya, K., Ogata, K., Silva, D., Onodera, T., Komatsu, H., and Nakatani, T. 2007. Cloneable JVM: A new approach to start isolated Java applications faster. In Proceedings of the Conference on Virtual Execution Environments (VEE’07). 1--11. Google ScholarDigital Library
- Kozuch, M. and Satyanarayanan, M. 2002. Internet suspend/resume. In Proceedings of the 4th Workshop on Mobile Computing Systems and Applications (WMCSA’02). 40--46. Google ScholarDigital Library
- Lagar-Cavilla, H. A., Tolia, N., de Lara, E., Satyanarayanan, M., and O’Hallaron, D. 2007. Interactive resource-intensive applications made easy. In Proceedings of the ACM/IFIP/USENIX 8th International Middleware Conference. 143--163. Google ScholarDigital Library
- Li, K.-B. 2003. ClustalW-MPI: Clustalw analysis using distributed and parallel computing. Bioinformatics 19, 12, 1585--1586. http://www.bii.a-star.edu.sg/achievements/applications/clustalw/index.php.Google ScholarCross Ref
- Lin, B. and Dinda, P. 2005. VSched: Mixing batch and interactive virtual machines using periodic real-time scheduling. In Proceedings of the ACM/IEEE Conference on Supercomputing. 8--20. Google ScholarDigital Library
- Litty, L., Lagar-Cavilla, H. A., and Lie, D. 2008. Hypervisor support for identifying covertly executing binaries. In Proceedings of the 17th USENIX Security Symposium. 243--258. Google ScholarDigital Library
- McLoughlin, M. 2008. The QCOW2 Image Format. http://people.gnome.org/~markmc/qcow-image-format.html.Google Scholar
- McNett, M., Gupta, D., Vahdat, A., and Voelker, G. 2007. Usher: An extensible framework for managing clusters of virtual machines. In Proceedings of the 21st Large Installation System Adminstration Conference (LISA’07). 167--181. Google ScholarDigital Library
- Meyer, D., Aggarwal, G., Cully, B., Lefebvre, G., Hutchinson, N., Feeley, M., and Warfield, A. 2008. Parallax: Virtual disks for virtual machines. In Proceedings of the Eurosys. 41--54. Google ScholarDigital Library
- Microsoft. 2009. Virtual hard disk image format specification. http://technet.microsoft.com/en-us/virtualserver/bb676673.aspx.Google Scholar
- Mosso. Mosso Cloud Service. http://www.mosso.com/.Google Scholar
- Murphy, M. A., Kagey, B., Fenn, M., and Goasguen, S. 2009. Dynamic provisioning of virtual organization clusters. In Proceedings of the IEEE International Symposium on Cluster Computing and the Grid (CCGrid’09). 364--371. Google ScholarDigital Library
- Nathuji, R. and Schwan, K. 2007. VirtualPower: Coordinated power management in virtualized enterprise systems. In Proceedings of the 21st Symposium on Operating Systems Principles (SOSP’07). 265--278. Google ScholarDigital Library
- Nelson, M., Lim, B., and Hutchins, G. 2005. Fast transparent migration for virtual machines. In Proceedings of the USENIX Annual Technical Conference. Google ScholarDigital Library
- Nightingale, E. B., Veeraraghavan, K., Chen, P., and Flinn, J. 2006. Rethink the sync. In Proceedings of the 7th Symposium on Operating Systems Design and Implementation (OSDI’06). 1--14. Google ScholarDigital Library
- Nishimura, H., Maruyama, N., and Matsuoka, S. 2007. Virtual clusters on the fly -- fast, scalable and flexible installation. In Proceedings of the IEEE International Symposium on Cluster Computing and the Grid (CCGrid’07). 549--556. Google ScholarDigital Library
- Norris, J., Coleman, K., Fox, A., and Candea, G. 2004. OnCall: Defeating spikes with a free-market application cluster. In Proceedings of the 1st International Conference on Autonomic Computing (ICAC). 198--205. Google ScholarDigital Library
- Open vSwitch. An Open Virtual Switch. http://openvswitch.org.Google Scholar
- Pixar. RenderMan. https://renderman.pixar.com/.Google Scholar
- Platform Computing. 2006. Platform Enterprise Grid Orchestrator (Platform EGO). http://www.platform.com/Products/platform-enterprise-grid-orchestrator.pNFS.com.Google Scholar
- pNFS.com. Parallel NFS. http://www.pnfs.com/.Google Scholar
- QuantLib.org. QuantLib: A free/open-source library for quantitative finance. http://quantlib.org/index.shtml.Google Scholar
- Ranjan, S., Rolia, J., Fu, H., and Knightly, E. 2003. Qos-driven server migration for internet data centers. In Proceedings of the 10th International Workshop on Quality of Service (IWQoS’03). 3--12.Google Scholar
- Red Hat. LVM2 Resource Page. http://sources.redhat.com/lvm2/.Google Scholar
- Ritchie, D. and Thompson, K. 1974. The UNIX time-sharing system. Comm. ACM 17, 7, 365--375. Google ScholarDigital Library
- Rumble, S. M., Lacroute, P., Dalca, A. V., Fiume, M., Sidow, A., and Brudno, M. 2009. SHRiMP: Accurate mapping of short color-space reads. PLoS Comput. Biology 5, 5.Google ScholarCross Ref
- Ruth, P., McGachey, P., Jiang, J., and Xu, D. 2005. VioCluster: Virtualization for dynamic computational domains. In Proceedings of the IEEE International Conference on Cluster Computing (Cluster). 1--10.Google Scholar
- Samba.org. DISTCC: A Fast, Free Distributed C/C++ Compiler. http://distcc.samba.org/.Google Scholar
- Sapuntzakis, C. P., Chandra, R., Pfaff, B., Chow, J., Lam, M. S., and Rosenblum, M. 2002. Optimizing the migration of virtual computers. In Proceedings of the 5th Symposium on Operating Systems Design and Implementation (OSD’02I). 377--390. Google ScholarDigital Library
- SPEC. 2005. Standard Performance Evaluation Corporation SPECweb2005. http://www.spec.org/web2005/.Google Scholar
- Steinder, M., Whalley, I., Carrera, D., Gaweda, I., and Chess, D. 2007. Server virtualization in autonomic management of heterogeneous workloads. In Proceedings of the 10th Integrated Network Management (IM) Conference. 139--148.Google Scholar
- Stone, J. Tachyon Parallel/Multiprocessor Ray Tracing System. http://jedi.ks.uiuc.edu/~johns/raytracer/.Google Scholar
- Sun Microsystems. Solaris ZFS - Redefining File Systems as Virtualised Storage. http://nz.sun.com/learnabout/solaris/10/ds/zfs.html.Google Scholar
- Surie, A., Lagar-Cavilla, H. A., de Lara, E., and Satyanarayanan, M. 2008. Low-bandwidth vm migration via opportunistic replay. In Proceedings of the 9th Workshop on Mobile Computing Systems and Applications (HotMobile). 74--79. Google ScholarDigital Library
- Tolia, N., Kozuch, M., Satyanarayanan, M., Karp, B., Perrig, A., and Bressoud, T. 2003. Opportunistic use of content addressable storage for distributed file systems. In Proceedings of the USENIX Annual Technical Conference. 127--140.Google Scholar
- Tolia, N., Kaminsky, M., Andersen, D. G., and Patil, S. 2006. An architecture for internet data transfer. In Proceedings of the 3rd Symposium on Networked Systems Design and Implementation (NSDI’06). 253--266. Google ScholarDigital Library
- Uhlig, V., LeVasseur, J., Skoglund, E., and Dannowski, U. 2004. Towards scalable multiprocessor virtual machines. In Proceedings of the 3rd Virtual Machine Research and Technology Symposium. 43--56. Google ScholarDigital Library
- UofT. University of Toronto -- SnowFlock: Swift VM cloning for cloud computing. http: //sysweb.cs.toronto.edu/snowflock.Google Scholar
- Urgaonkar, B. and Shenoy, P. 2008. Cataclysm: Scalable overload policing for internet applications. J. Netw. Comput. Appl. 31, 4, 891--920. Google ScholarDigital Library
- Urgaonkar, B., Shenoy, P., Chandra, A., Goyal, P., and Wood, T. 2008. Agile dynamic provisioning of multi-tier internet applications. ACM Trans. Auton. Adapt. Syst. 3, 1, 1--39. Google ScholarDigital Library
- VASP-Group. VASP -- Vienna Ab initio Simulation Package, Theoretical Physics Department, Vienna. http://cms.mpi.univie.ac.at/vasp/.Google Scholar
- VMware. vSphere Resource Management Guide. www.vmware.com/pdf/vsphere4/r41/vsp_41_resource_mgmt.pdf.Google Scholar
- Vrable, M., Ma, J., Chen, J., Moore, D., Vandekieft, E., Snoeren, A., Voelker, G., and Savage, S. 2005. Scalability, fidelity and containment in the potemkin virtual honeyfarm. In Proceedings of the 20th Symposium on Operating Systems Principles (SOSP’05). 148--162. Google ScholarDigital Library
- Waldspurger, C. A. 2002. Memory resource management in VMware ESX server. In Proceedings of the 5th Symposium on Operating Systems Design and Implementation (OSDI’02). 181--194. Google ScholarDigital Library
- Warfield, A., Hand, S., Fraser, K., and Deegan, T. 2005. Facilitating the development of soft devices. In Proceedings of the USENIX Annual Technical Conference. Google ScholarDigital Library
- Whitaker, A., Shaw, M., and Gribble, S. D. 2002. Scale and performance in the denali isolation kernel. In Proceedings of the 5th Symposium on Operating Systems Design and Implementation (OSDI’02). 195--209. Google ScholarDigital Library
- White, B., Lepreau, J., Stoller, L., Ricci, R., Guruprasad, S., Newbold, M., Hibler, M., Barb, C., and Joglekar, A. 2002. An integrated experimental environment for distributed systems and networks. In Proceedings of the 5th Symposium on Operating Systems Design and Implementation (OSDI’02). 255--270. Google ScholarDigital Library
- Wood, T., Shenoy, P., Venkataramani, A., and Yousif, M. 2007. Black-box and gray-box strategies for virtual machine migration. In Proceedings of the 4th Symposium on Networked Systems Design and Implementation (NSDI’07). 229--242. Google ScholarDigital Library
- Youseff, L., Wolski, R., Gorda, B., and Krintz, C. 2006. Evaluating the performance impact of xen on MPI and process execution for HPC systems. In Proceedings of the 2nd International Workshop on Virtualization Technology in Distributed Computing (VTDC’06). 1--8. Google ScholarDigital Library
- Zayas, E. 1987. Attacking the process migration bottleneck. In Proceedings of the 11th ACM Symposium on Operating System Principles (SOSP’87). 13--24. Google ScholarDigital Library
- Zhang, H., Jiang, G., Yoshihira, K., Chen, H., and Saxena, A. 2009. Resilient workload manager: Taming bursty workload of scaling internet applications. In Proceedings of the 6th International Conference on Autonomic Computing and Communications, Industry Session (ICAC-INDST). 19--28. Google ScholarDigital Library
- Zhang, X., Keahey, K., Foster, I., and Freeman, T. 2005. Virtual cluster workspaces for grid applications. Tech. rep. TR-ANL/MCS-P1246-0405, University of Chicago.Google Scholar
Index Terms
- SnowFlock: Virtual Machine Cloning as a First-Class Cloud Primitive
Recommendations
SnowFlock: rapid virtual machine cloning for cloud computing
EuroSys '09: Proceedings of the 4th ACM European conference on Computer systemsVirtual Machine (VM) fork is a new cloud computing abstraction that instantaneously clones a VM into multiple replicas running on different hosts. All replicas share the same initial state, matching the intuitive semantics of stateful worker creation. ...
Transparently bridging semantic gap in CPU management for virtualized environments
Consolidated environments are progressively accommodating diverse and unpredictable workloads in conjunction with virtual desktop infrastructure and cloud computing. Unpredictable workloads, however, aggravate the semantic gap between the virtual ...
Enabling Instantaneous Relocation of Virtual Machines with a Lightweight VMM Extension
CCGRID '10: Proceedings of the 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid ComputingWe are developing an efficient resource management system with aggressive virtual machine (VM) relocation among physical nodes in a data center. Existing live migration technology, however, requires a long time to change the execution host of a VM, it ...
Reviews
Elliot Jaffe
A somewhat unusual mechanism for achieving parallelism in cloud-based systems is explored in this paper. Traditionally, a cloud server would be instantiated on a multiprocessor, and the developer would run multiple processes or threads to take advantage of the underlying hardware. SnowFlock takes this one step further, allowing a developer to easily and efficiently run copies of the whole virtual machine (VM) on other hardware instances. The challenge is to make this efficient, and SnowFlock does so by using "three key observations. First, it is possible to drastically reduce the time it takes to clone a VM by copying only the critical state and [by] fetching the VM's memory image efficiently on demand." Second, it is possible to significantly reduce the VM memory that must be transferred if the operating system knows about new allocations. Third, the similarity of the memory access patterns among "cloned VMs makes it beneficial to distribute [a] VM state using multicast. This allows for the instantiation of a large number of VMs at a (low) cost similar to that of forking a single copy." The paper is interesting as an alternative approach to cloud design. It is detailed enough to understand and grasp the concepts without getting lost in the specifics of implementation. It is definitely an important paper for cloud designers. Online Computing Reviews Service
Access critical reviews of Computing literature here
Become a reviewer for Computing Reviews.
Comments