Skip to main content
SpringerLink
Log in

A cost-benefit analysis of using cloud computing to extend the capacity of clusters

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

In this paper, we investigate the benefits that organisations can reap by using “Cloud Computing” providers to augment the computing capacity of their local infrastructure. We evaluate the cost of seven scheduling strategies used by an organisation that operates a cluster managed by virtual machine technology and seeks to utilise resources from a remote Infrastructure as a Service (IaaS) provider to reduce the response time of its user requests. Requests for virtual machines are submitted to the organisation’s cluster, but additional virtual machines are instantiated in the remote provider and added to the local cluster when there are insufficient resources to serve the users’ requests. Naïve scheduling strategies can have a great impact on the amount paid by the organisation for using the remote resources, potentially increasing the overall cost with the use of IaaS. Therefore, in this work we investigate seven scheduling strategies that consider the use of resources from the “Cloud”, to understand how these strategies achieve a balance between performance and usage cost, and how much they improve the requests’ response times.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Amazon Inc. Amazon Elastic Compute Cloud (Amazon EC2). http://aws.amazon.com/ec2

  2. Armbrust, M., Fox, A., Griffith, R., Joseph, A.D., Katz, R.H., Konwinski, A., Lee, G., Patterson, D.A., Rabkin, A., Stoica, I., Zaharia, M.: Above the clouds: A Berkeley view of Cloud computing. Technical report UCB/EECS-2009-28, Electrical Engineering and Computer Sciences, University of California at Berkeley, Berkeley, USA, February 2009

  3. Balazinska, M., Balakrishnan, H., Stonebraker, M.: Contract-based load management in federated distributed systems. In: 1st Symposium on Networked Systems Design and Implementation (NSDI), San Francisco, USA, March 2004, pp. 197–210. USENIX Association

  4. Barham, P., Dragovic, B., Fraser, K., Hand, S., Harris, T., Ho, A., Neugebauer, R., Pratt, I., Warfield, A.: Xen and the art of virtualization. In: 19th ACM Symposium on Operating Systems Principles (SOSP ’03), pp. 164–177. ACM, New York (2003)

    Chapter  Google Scholar 

  5. Buyya, R., Murshed, M.: GridSim: a toolkit for the modeling and simulation of distributed resource management and scheduling for Grid computing. Concurr. Comput. 14(13–15), 1175–1220 (2002)

    Article  MATH  Google Scholar 

  6. Chase, J.S., Irwin, D.E., Grit, L.E., Moore, J.D., Sprenkle, S.E.: Dynamic virtual clusters in a Grid site manager. In: 12th IEEE International Symposium on High Performance Distributed Computing (HPDC 2003), Washington, DC, USA, 2003, p. 90. IEEE Computer Society, Los Alamitos (2003)

    Chapter  Google Scholar 

  7. Cormen, T.H., Leiserson, C.E., Rivest, R.L., Stein, C.: Introduction to Algorithms, 2nd edn. MIT Press/McGraw-Hill, Cambridge/New York (2001)

    MATH  Google Scholar 

  8. de Assunção, M.D., Buyya, R., Venugopal, S.: InterGrid: a case for internetworking islands of Grids. Concurr. Comput. 20(8), 997–1024 (2008)

    Article  Google Scholar 

  9. Deelman, E., Singh, G., Livny, M., Berriman, B., Good, J.: The cost of doing science on the cloud: the montage example. In: 2008 ACM/IEEE Conference on Supercomputing (SC 2008), Piscataway, NJ, USA, 2008, pp. 1–12. IEEE Press, New York (2008)

    Google Scholar 

  10. di Costanzo, A., de Assunção, M.D., Buyya, R.: Harnessing cloud technologies for a virtualized distributed computing infrastructure. IEEE Int. Comput. 13(5), 24–33 (2009)

    Article  Google Scholar 

  11. Emeneker, W., Jackson, D., Butikofer, J., Stanzione, D.: Dynamic virtual clustering with Xen and Moab. In: Frontiers of High Performance Computing and Networking with ISPA 2006. LNCS, vol. 4331, pp. 440–451. Springer, Berlin/Heidelberg (2006)

    Chapter  Google Scholar 

  12. England, D., Weissman, J.B.: Costs and benefits of load sharing in the computational Grid. In: 10th International Workshop on Job Scheduling Strategies for Parallel Processing (JSSPP ’04), New York, USA, 2004. LNCS, vol. 3277, pp. 160–175. Springer, Berlin/Heidelberg (2004)

    Google Scholar 

  13. Feitelson, D.G., Rudolph, L., Schwiegelshohn, U., Sevcik, K.C., Wong, P.: Theory and practise in parallel job scheduling. In: Job Scheduling Strategies for Parallel Processing (IPPS ’97), London, UK, 1997, pp. 1–34. Springer, Berlin (1997)

    Google Scholar 

  14. Fontán, J., Vázquez, T., Gonzalez, L., Montero, R.S., Llorente, I.M.: OpenNEbula: the open source virtual machine manager for cluster computing. In: Open Source Grid and Cluster Software Conference—Book of Abstracts, San Francisco, USA, May 2008

  15. Foster, I., Freeman, T., Keahey, K., Scheftner, D., Sotomayor, B., Zhang, X.: Virtual clusters for Grid communities. In: 6th IEEE International Symposium on Cluster Computing and the Grid (CCGRID 2006), Washington, DC, USA, 2006, pp. 513–520. IEEE Comput. Soc., Los Alamitos (2006)

    Chapter  Google Scholar 

  16. Grimme, C., Lepping, J., Papaspyrou, A.: Prospects of collaboration between compute providers by means of job interchange. In: Job Scheduling Strategies for Parallel Processing. Lecture Notes in Computer Science, vol. 4942, pp. 132–151. Springer, Berlin/Heidelberg (2008)

    Chapter  Google Scholar 

  17. Grit, L., Inwin, D., Yumerefendi, A., Chase, J.: Virtual machine hosting for networked clusters: building the foundations for ‘autonomic’ orchestration. In: 1st International Workshop on Virtualization Technology in Distributed Computing (VTDC 2006), Tampa, Florida, November 2006

  18. Iosup, A., Epema, D.H.J., Tannenbaum, T., Farrellee, M., Livny, M.: Inter-operating Grids through delegated matchmaking. In: 2007 ACM/IEEE Conference on Supercomputing (SC 2007), New York, USA, 2007, pp. 1–12. ACM, New York (2007)

    Chapter  Google Scholar 

  19. Irwin, D., Chase, J., Grit, L., Yumerefendi, A., Becker, D., Yocum, K.G.: Sharing networked resources with brokered leases. In: USENIX Annual Technical Conference, pp. 199–212. Berkeley, USA, June 2006. USENIX Association

  20. Islam, M., Balaji, P., Sadayappan, P., Panda, D.K.: QoPS: A QoS based scheme for parallel job scheduling. In: 9th International Workshop on Job Scheduling Strategies for Parallel Processing (JSSPP ’03), Seattle, WA, USA, 2003. LNCS, vol. 2862, pp. 252–268. Springer, Berlin (2003)

    Chapter  Google Scholar 

  21. Keahey, K., Foster, I., Freeman, T., Zhang, X.: Virtual workspaces: Achieving quality of service and quality of life in the Grids. Sci. Program. 13(4), 265–275 (2006)

    Google Scholar 

  22. Lifka, D.A.: The ANL/IBM SP scheduling system. In: Workshop on Job Scheduling Strategies for Parallel Processing (IPPS’95), London, UK, 1995, pp. 295–303. Springer, Berlin (1995)

    Google Scholar 

  23. Lublin, U., Feitelson, D.G.: The workload on parallel supercomputers: Modeling the characteristics of rigid jobs. J. Parallel Distrib. Comput. 63(11), 1105–1122 (2003)

    Article  MATH  Google Scholar 

  24. Montero, R.S., Huedo, E., Llorente, I.M.: Dynamic deployment of custom execution environments in Grids. In: 2nd International Conference on Advanced Engineering Computing and Applications in Sciences (ADVCOMP ’08), Valencia, Spain, 2008, pp. 33–38. IEEE Comp. Soc., Los Alamitos (2008)

    Chapter  Google Scholar 

  25. Mu’alem, A.W., Feitelson, D.G.: Utilization, predictability, workloads, and user runtime estimates in scheduling the IBM SP2 with backfilling. IEEE Trans. Parallel Distrib. Syst. 12(6), 529–543 (2001)

    Article  Google Scholar 

  26. Nurmi, D., Wolski, R., Crzegorczyk, C., Obertelli, G., Soman, S., Youseff, L., Zagorodnov, D.: Eucalyptus: a technical report on an elastic utility computing architecture linking your programs to useful systems. Technical Report 2008-10, Department of Computer Science, University of California, Santa Barbara, CA, USA (2008)

  27. Palankar, M.R., Iamnitchi, A., Ripeanu, M., Garfinkel, S.: Amazon S3 for science Grids: a viable solution?. In: International Workshop on Data-Aware Distributed Computing (DADC’08) in Conjunction with HPDC 2008, New York, NY, USA, 2008, pp. 55–64. ACM, New York (2008)

    Chapter  Google Scholar 

  28. Ramakrishnan, L., Irwin, D., Grit, L., Yumerefendi, A., Iamnitchi, A., Chase, J.: Toward a doctrine of containment: grid hosting with adaptive resource control. In: 2006 ACM/IEEE Conference on Supercomputing (SC 2006), New York, NY, USA, 2006, p. 101. ACM, New York (2006)

    Chapter  Google Scholar 

  29. Rubio-Montero, A., Huedo, E., Montero, R., Llorente, I.: Management of virtual machines on globus grids using GridWay. In: IEEE International Parallel and Distributed Processing Symposium (IPDPS 2007), Long Beach, USA, March 2007, pp. 1–7. IEEE Comput. Soc., Los Alamitos (2007)

    Chapter  Google Scholar 

  30. Ruth, P., McGachey, P., Xu, D.: VioCluster: virtualization for dynamic computational domain. In: IEEE International on Cluster Computing (Cluster 2005), pp. 1–10, Burlington, USA, September 2005. IEEE

  31. Shoykhet, A., Lange, J., Dinda, P.: Virtuoso: a system for virtual machine marketplaces. Technical Report NWU-CS-04-39, Electrical Engineering and Computer Science Department, Northwestern University, Evanston/Chicago, IL, July 2004

  32. Singh, G., Kesselman, C., Deelman, E.: Adaptive pricing for resource reservations in shared environments. In: 8th IEEE/ACM International Conference on Grid Computing (Grid 2007) (Austin, USA, September 2007), pp. 74–80. ACM/IEEE, New York (2007)

  33. Sotomayor, B., Keahey, K., Foster, I.: Combining batch execution and leasing using virtual machines. In: 17th International Symposium on High performance Distributed Computing (HPDC 2008), New York, NY, USA, 2008, pp. 87–96. ACM, New York (2008)

    Chapter  Google Scholar 

  34. Srinivasan, S., Kettimuthu, R., Subramani, V., Sadayappan, P.: Selective reservation strategies for backfill job scheduling. In: 8th International Workshop on Job Scheduling Strategies for Parallel Processing (JSSPP ’02), London, UK, 2002. LNCS, vol. 2537, pp. 55–71. Springer, Berlin/Heidelberg (2002)

    Chapter  Google Scholar 

  35. Surana, S., Godfrey, B., Lakshminarayanan, K., Karp, R., Stoica, I.: Load balancing in dynamic structured peer-to-peer systems. Perform. Eval. 63(3), 217–240 (2006)

    Article  Google Scholar 

  36. Tatezono, M., Maruyama, N., Matsuoka, S.: Making wide-area, multi-site MPI feasible using Xen VM. In: Workshop on Frontiers of High Performance Computing and Networking (held with ISPA 2006). LNCS, vol. 4331, pp. 387–396. Springer, Berlin/Heidelberg (2006)

    Chapter  Google Scholar 

  37. Wang, Y.-T., Morris, R.J.T.: Load sharing in distributed systems. IEEE Trans. Comput. C-34(3), 204–217 (1985)

    Article  Google Scholar 

  38. Weiss, A.: Computing in the Clouds. netWorker 11(4), 16–25 (2007)

    Article  Google Scholar 

  39. Wolski, R., Plank, J.S., Brevik, J., Bryan, T.: Analyzing market-based resource allocation strategies for the computational Grid. Int. J. High Perform. Comput. Appl. 15(3), 258–281 (2001)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. INRIA RESO/LIP, École Normale Supérieure de Lyon, 46, allée d’Italie, 69364, Lyon Cedex 07, France

    Marcos Dias de Assunção

  2. The University of Melbourne, Melbourne, Victoria, Australia

    Alexandre di Costanzo & Rajkumar Buyya

Authors
  1. Marcos Dias de Assunção
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alexandre di Costanzo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rajkumar Buyya
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marcos Dias de Assunção.

Additional information

This work is partially supported by research grants from the Australian Research Council (ARC) and Australian Department of Innovation, Industry, Science and Research (DIISR). Marcos’ Ph.D. research was partially supported by National ICT Australia.

Rights and permissions

Reprints and permissions

About this article

Cite this article

de Assunção, M.D., di Costanzo, A. & Buyya, R. A cost-benefit analysis of using cloud computing to extend the capacity of clusters. Cluster Comput 13, 335–347 (2010). https://doi.org/10.1007/s10586-010-0131-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-010-0131-x

Keywords

Search

Navigation