Skip to main content
SpringerLink
Log in

Semantic-enabled CARE Resource Broker (SeCRB) for managing grid and cloud environment

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Grid computing is mainly helpful for executing high-performance computing applications. However, conventional grid resources sometimes fail to offer a dynamic application execution environment and this increases the rate at which the job requests of users are rejected. Integrating emerging virtualization technologies in grid and cloud computing facilitates the provision of dynamic virtual resources in the required execution environment. Resource brokers play a significant role in managing grid and cloud resources as well as identifying potential resources that satisfy users’ application requests. This research paper proposes a semantic-enabled CARE Resource Broker (SeCRB) that provides a common framework to describe grid and cloud resources, and to discover them in an intelligent manner by considering software, hardware and quality of service (QoS) requirements. The proposed semantic resource discovery mechanism classifies the resources into three categories viz., exact, high-similarity subsume and high-similarity plug-in regions. To achieve the necessary user QoS requirements, we have included a service level agreement (SLA) negotiation mechanism that pairs users’ QoS requirements with matching resources to guarantee the execution of applications, and to achieve the desired QoS of users. Finally, we have implemented the QoS-based resource scheduling mechanism that selects the resources from the SLA negotiation accepted list in an optimal manner. The proposed work is simulated and evaluated by submitting real-world bio-informatics and image processing application for various test cases. The result of the experiment shows that for jobs submitted to the resource broker, job rejection rate is reduced while job success and scheduling rates are increased, thus making the resource management system more efficient.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25

Similar content being viewed by others

Abbreviations

NS:

Number of Schedule

\(S\,\leftarrow \,\left\{ {S_1 ,S_2 , \ldots ,S_p.S_\mathrm{NS}} \right\} \) :

It represents set of schedules

\(S_j \) :

It represents pth schedule

M :

It represents the total number of resources

\(R\leftarrow \{{R_1 ,R_2,\ldots ,R_M} \}\) :

It represents the list of resources

\(J_i\) :

It represents ith job

\(R_j\) :

It represents jth resource

CB:

Cloud broker

\((\mathrm{ER})_{J_i}\) :

Exact region contain resources that exactly satisfies the demands of job \(J_i\)

\((\mathrm{SR})_{J_i}\) :

Subsume region contain resources that has more capability than demand of job \(J_i\)

\((\mathrm{PR})_{J_i}\) :

Plug-in region contain resources that has less capability than demand of Job \(J_i\)

\(\mathrm {Sim\_mat}\) :

Similarity matrix holds the cosine similar measure between jobs and resources in \((\mathrm{SR})_{J_i}\) and \((\mathrm{PR})_{J_i}\). Its dimension are \(({1 *\left| {(\mathrm{SR})_{J_i}} \right| })\) and \(({1*\left| {(\mathrm{PR})_{J_i}} \right| })\), respectively

\(({\mathrm{hs}\_\mathrm{SR}})_{J_i}\) :

High-similarity resources that satisfy demand specified by job \(J_i\) from subsume region

\(({\mathrm{hs}\_\mathrm{PR}})_{J_i}\) :

High-similarity resources that satisfy demand specified by job \(J_i \) from plug-in region

\(\mathrm{threshold}_\mathrm{SR}\) :

To filter the resources from subsume region whose similarity is greater or equal to specified value

\(\mathrm{threshold}_\mathrm{PR}\) :

To filter the resources from plug-in region whose similarity is greater or equal to specified value

\(\mathrm{Req}({J_i})\) :

\(\{J_i^\mathrm{ID} ,\,\mathrm{no}\_\mathrm{Nodes}_{J_i} ,\,\mathrm{Ram}\_\mathrm{needed}_{J_i} ,\mathrm{Hd}\_\mathrm{needed}_{J_i }, \mathrm{Deadline}_{J_i} ,\,\mathrm{OS}_{J_i} ,\,\mathrm{Sw}_{J_i} \}\)

\(\mathrm{Avail}({R_j})\) :

\(\left\{ \!\!{\begin{array}{l} R_j^\mathrm{ID} ,\mathrm{no}\_\mathrm{Nodes}_{R_j}, \mathrm{Ram}\_\mathrm{Capacity}_{R_j}, \mathrm{Hd}\_\mathrm{Capacity}_{R_j},\\ \mathrm{Availability}_{R_j},\mathrm{OS}_{R_j} ,\mathrm{Sw}_{R_j}\!, \mathrm{Type}_{R_j} \\ \end{array}} \!\!\right\} \)

\(\left| {(\mathrm{ER})_{J_i}} \right| \) :

Number of resources in exact region of job \(J_i \) that is represented as \(N\_\mathrm{ER}\)

\(\left| {\,({\mathrm{hs}\_\mathrm{SR}})_{J_i} \,} \right| \) :

Number of resources in subsume region of job \(J_i \) that is represented as \(N\_\mathrm{SR}\)

\(\left| {\,({\mathrm{hs}\_\mathrm{PR}})_{J_i} \,} \right| \) :

Number of resources in plug-in region that satisfies job \(J_i \) that is represented as \(N\_\mathrm{PR}\)

\(N_{(\mathrm{ER})_{J_i}}\) :

List of resources to which negotiation is agreed in exact region by broker

\(N_{(\mathrm{SR})_{J_i}}\) :

List of resources to which negotiation is agreed in subsume region by broker

\(N_{(\mathrm{PR})_{J_i}}\) :

List of resources to which negotiation is agreed in plug-in region by broker

\(P_\mathrm{Init}\) :

Initial proposal send by cloud broker

\(U_\mathrm{Init}\) :

Utility value of the cloud broker

\(({P_\mathrm{c}})_{({R_j})}\) :

Counter proposal given by resource \(R_j\) that receives \(P_\mathrm{Init}\)

\((\mathrm{No. proposal})_\mathrm{ER}\) :

Number of resources in exact region that send counter proposals which is received by broker

\((\mathrm{No. proposal})_\mathrm{SR}\) :

Number of resources in subsume region that send counter proposals which is received by broker

\((\mathrm{No. proposal})_\mathrm{PR}\) :

Number of resources in plug-in region that send counter proposals which is received by broker

\(({U_\mathrm{c}})_{({R_j})}\) :

Utility counter of the resource \(R_j\)

\(U_{(\mathrm{ER})_{J_i}}\) :

Set of utilities received by broker from the resources in exact region of the job \(J_i \)

\(U_{(\mathrm{SR})_{J_i}}\) :

Set of utilities received by broker from the resources in qualified subsume region of the job \(J_i \)

\(U_{(\mathrm{PR})_{J_i}}\) :

Set of utilities received by broker from the resources in qualified plug-in region of the job \(J_i \)

\(P_\mathrm{Avg}\) :

Average proposal received by the cloud broker

\(({R_{(\mathrm{PR})}})_{J_i}\) :

Resources in plug-in region that is scheduled to Job \(J_i \)

\(\mathrm{JS}_{S_p}\) :

Number of Jobs submitted for the Schedule \(S_p \)

\(\mathrm{JR}_{S_p}\) :

Number of Jobs rejected for the schedule\(S_p \)

\(\mathrm{JRR}_{S_p}\) :

Job rejection Rate for the schedule \(S_p \)

\(\mathrm{JSTS}_{S_p}\) :

Number of Jobs submitted to the scheduler in the schedule \(S_p\)

\(\mathrm{JSR}_{S_p}\) :

Job Success Rate for the schedule \(S_p \)

\(\mathrm{JSD}_{S_p}\) :

Number of Jobs successfully met deadline in the schedule \(S_p \)

\(\mathrm{JFD}_{S_p}\) :

Number of jobs failed to meet deadline in the schedule\(S_p \)

\(\mathrm{SSR}_{S_p}\) :

Scheduling success rate for the schedule \(S_p \)

\(\hbox {TPDefected}_{\mathrm{ith}\_\mathrm{fruit}}\) :

Total pixels defected in ith fruit

References

  1. Price Water House Coopers (2002) Powerful technology trends continue despite downturn. PWC Global Technology Center, Menlo Park

  2. National Institute of Standards and Technology (NIST). http://csrc.nist.gov/publications/drafts/800-145/Draft-SP-800-145_Cloud-definition.pdf. Accessed Jan 2013

  3. Virtualization. http://theCloudtutorial.com/what-is-virtualization.html. Accessed Jan 2013

  4. Multitenancy. http://theCloudtutorial.com/multitenancy.html. Accessed Jan 2013

  5. Czajkowski K, Foster I, Karonis N, Kesselman C, Martin S, Smith W, Tuecke S (1998) A resource management architecture for metacomputing systems. In: Proceedings of the workshop on job scheduling strategies for parallel processing. Springer, Berlin, Heidelberg, pp 62–82

  6. Missier P, Ziegler W, Wieder P (2007) Semantic support for meta-scheduling in grids. In: Talia D, Bilas A, Dikaiakos MD (eds) Knowledge and Data Management in GRIDs. Springer, USA, pp 169–183

  7. Berners-Lee T, Hendler J, Lassila O (2001) The semantic web. Sci Am 284(5):34–43

    Google Scholar 

  8. Ontology. http://www.ksl.stanford.edu/kst/what-is-an-ontology.html. Accessed Oct 2011

  9. Goble C, De Roure D (2004) The Semantic Grid: Myth Busting and Bridge Building. In: Proceedings of the 16th European Conference on Artificial Intelligence (ECAI 2004). Valencia, Spain

  10. Resource Description Framework (RDF) (2004) A W3C recommendation. http://www.w3.org/RDF

  11. Web Ontology Language (OWL). http://www.w3.org/2004/OWL. Accessed Jan 2013

  12. Protégé (2011) http://protege.stanford.edu/. Accessed Jan 2013

  13. Somasundaram TS, Balachander RA, Kumar R, Balakrishnan P, Rajendar K, Rajiv R, Kannan G, Rajesh Britto GR, Mahendran E, Madusudhanan B (2010) CARE resource broker: a framework for scheduling and supporting virtual resource management. Futur Gener Comput Syst 26(3): 337–347. doi:10.1016/j.future.2009.10.005

  14. Globus Toolkit. http://www.globus.org/toolkit/downloads/4.0.7/. Accessed Oct 2012

  15. Somasundaram TS, Balachander RA, Balakrishnan P, Kumar R, Rajendar K, Rajiv R, Rajesh Britto G, Eahendran E, Madusudhanan B (2009) Achieving co-allocation through virtualization in grid environment. In: Abdennadher N, Petcu D (eds) Advances in grid and pervasive computing, vol 5529. Springer, Berlin, Heidelberg, pp 235–243

  16. Somasundaram TS, Balachandar RA (2009) Extending conventional grid scheduler for leasing resources. In 10th IEEE/ACM International Conference in Grid Computing 2009 (Grid 2009), Banff, Canada, pp: 173–174

  17. Amarnath BR, Somasundaram TS, Mahendran E, Buyya R (2009) Ontology based grid resource management. Softw Pract Exper, vol 39 (17): pp 1419–1438

  18. Somasundaram TS, Rangasamy K, Govindarajan K (2011) Intelligent semantic discovery in virtualized grid environment. In: International Conference on Recent Trends in Information Technology (ICRTIT), 3–5 June 2011, Chennai, Tamil Nadu pp 644–649

  19. The Condor Project. http://research.cs.wisc.edu/condor/. Accessed Nov 2012

  20. Computing Resource Execution and Management (CREAM) (2012) http://grid.pd.infn.it/cream/

  21. gLite Workload Management System (WMS) (2012). http://iopscience.iop.org/1742-6596/219/6/062039/pdf/1742-6596_219_6_062039.pdf. Accessed Nov 2012

  22. Classified Advertisements (ClassAds). http://research.cs.wisc.edu/condor/classad/. Accessed Nov 2012

  23. Job Description Language (JDL). http://server11.infn.it/workload-grid/docs/DataGrid-01-TEN-0102-0_2-Document.pdf. Accessed Oct 2012

  24. Grid Interoperability Project. http://www.Grid-interoperability.eu/. Accessed Jan 2013

  25. InteliGrid (2004). http://inteliGrid.eu-project.info/. Accessed Jan 2013

  26. The myGrid Project (2008). http://www.myGrid.org.uk. Accessed Jan 2013

  27. Roure D, Jennings N, Shadbolt N (2003) The semantic grid: a future e-science infrastructure. In: Berman F, Fox G, Hey AJG (eds) Grid Computing - Making the Global Infrastructure a Reality. Wiley, New York

  28. Somasundaram TS, Balachander RA, Kandasamy V, Buyya R, Rajagopalan Raman M, Varun S (2006) Semantic-based grid resource discovery and its integration with the grid service broker. In: Proceedings of the 14th International Conference on Advanced Computing and Communications (ADCOM), pp 84–89

  29. Harth A, Decker S, He Y, Tangmunarunkit H, Kesselman C (2004) A semantic matchmaker service on the grid. In: Proceedings of the 13th international World Wide Web conference on Alternate track papers & posters, May 17–20, 2004, New York, pp 326–327

  30. Nurmi D, Wolski R, Grzegorczyk C, Obertelli G, Soman S, Youseff L, Zagorodnov D (2009) The Eucalyptus open-source cloud-computing system. In: IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGRID’09), Shanghai, pp 124–131

  31. OpenNebula. http://www.opennebula.org. Accessed Mar 2013

  32. Emeneker W, Jackson D, Butikofer J, Stanzione D (2006) Dynamic virtual clustering with Xen and Moab. In: Proceedings of the 2006 international conference on frontiers of high performance computing and networking. Springer, Berlin, Heidelberg, pp 440–451

  33. Agarwal A, Desmarais R, Gable I, Norton A, Sobie R, Vanderster D (2006) Evaluation of virtual machines for HEP grids. In: Proceedings of computing in high energy physics, Mumbai, India

  34. Xen. http://www.xen.org/. Accessed Mar 2013

  35. Vmware. http://www.vmware.com/. Accessed Mar 2013

  36. Kernel-based Virtual Machine (KVM). http://www.linux-kvm.org/. Accessed Mar 2013

  37. Amazon EC2 Cloud. http://www.amazon.org/ec2. Accessed Mar 2013

  38. Ma YB, Jang SH, Lee JS (2011) Ontology-based resource management for cloud computing. In: Intelligent information and database systems. Lecture Notes in computer science, vol 6592. Springer, Berlin, pp 343–352

  39. Xu B, Wang N, Li C (2010) Providing a cloud infrastructure on clusters. In: Proceedings of the third international symposium on electronic commerce and security workshops (ISECS ’10), Guangzhou, P.R. China, 29–31 July 2010, pp 317–320

  40. Buyya R, Pandey S, Vecchiola C (2009) Cloudbus toolkit for market-oriented cloud computing. In: Cloud Computing. Lecture Notes in Computer Science. Springer, Berlin

  41. Ejarque J, Sirvent R, Badia RM (2010) A multi-agent approach for semantic resource allocation. In: IEEE second international conference on cloud computing technology and science, Indianapolis, pp 335–342

  42. Somasundaram TS, Balakrishnan P (2011) SLA Enabled Care Resource Broker. Futur Gener Comput Syst. 27(3):265–279

  43. Ejarque J, Palol M, Goiri I, Julia F, Guitart J, Badia RM, Torres J (2009) SLA-driven semantically-enhanced dynamic resource allocator for virtualized service providers. In: IEEE Fourth international conference on eScience, Indianapolis

  44. Czajkowski K, Foster IT, Kesselman C, Sander V, Tuecke S (2002) SNAP: a protocol for negotiating service level agreements and coordinating resource management in distributed systems. In: 8th International workshop on job scheduling strategies for parallel processing, Edinburgh, Scotland, pp 153–183, July 2002

  45. Venugopal S, Chu X, Buyya R (2008) A negotiation mechanism for advance resource reservation using the alternate offers protocol. In: 16th international workshop on quality of service (IWQoS (2008) June 2–4, 2008, Enschede, pp 40–49

  46. Yan J, Kowalczyk R, Lin J, Chhetri MB, Goh SK, Zhang J (2007) Autonomous service level agreement negotiation for service composition provision. Futur Gener Comput Syst 23(6):748–759

    Google Scholar 

  47. Son S, Sim KM (2011) A negotiation mechanism that facilitates the price-timeslot-QoS negotiation for establishing SLAs of cloud service reservation. In: Fong S (ed) Networked digital technologies. Communications in computer and information science, vol 136. Springer, Berlin, pp 432–446

  48. StratusLab. http://stratuslab.eu/doku.php/start. Accessed Feb 2013

  49. WNoDeS (2012). http://web.infn.it/wnodes/index.php/wnodes. Accessed Feb 2013

  50. GLUE Schema V1.3. http://glueschema.forge.cnaf.infn.it/spec/v13/. Accessed Oct 2012

  51. Open Cloud Computing Interface (OCCI). http://occi-wg.org/. Accessed Jan 2013

  52. de Assunção MD, di Costanzo A, Buyya R (2010) A cost-benefit analysis of using cloud computing to extend the capacity of clusters. Cluster Comput 13(3): 335–347

    Google Scholar 

  53. Ostermann S, Prodan R, Fahringer T (2009) Extending grids with cloud resource management for scientific computing. In: 10th IEEE/ACM International Conference on Grid Computing, Banff, pp 42–49

  54. Iosup A, Ostermann S, Yigitbasi N, Prodan R, Fahringer T, Epema DHJ (2011) Performance analysis of cloud computing services for many-tasks scientific computing. IEEE Trans Parallel Distrib Syst 22(6):931–945

    Google Scholar 

  55. Mao M, Humphrey M (2012) A performance study on the VM startup time in the cloud, In: 2012 IEEE 5th International Conference on Cloud Computing(CLOUD), Honolulu, pp 423–430

  56. Algernon. http://algernon-j.sourceforge.net/doc/overview.html. Accessed Jan 2013

  57. Fast Fourier Transfer (FFT) (2011). http://www.fftw.org/. Accessed Dec 2012

  58. Message Passing Interface (MPI). http://www.mcs.anl.gov/research/projects/mpi/. Accessed Dec 2012

  59. GROMACS. http://www.gromacs.org/. Accessed Feb 2013

  60. Globus Toolkit Auto Install (GTAI) Utility. http://dev.globus.org/wiki/Incubator/GT_Auto_Install. Accessed Dec 2012

  61. Protein Data Bank (PDB). http://www.pdb.org/pdb/. Accessed Feb 2013

  62. Feitelson D (1997) Parallel workloads archive. http://www.cs.huji.ac.il/labs/parallel/workload. Accessed Jan 2013

  63. Irwin DE, Grit LE, Chase JS (2004) Balancing risk and reward in a market-based task service. In: Proceedings of the 13th IEEE International Symposium on High Performance Distributed Computing. Honolulu, USA, pp 160–169

Download references

Acknowledgments

The authors sincerely thank the Ministry of communication and Information Technology and the Government of India for financially supporting the Centre for Advanced Computing Research and Education of Anna University Chennai, India. Furthermore, the authors acknowledge the Central Electronics Engineering Research Institute (CEERI), India and Department of Biotechnology at the Anna University for their help in guiding the execution of applications.

Author information

Authors and Affiliations

  1. Anna University, Chennai, India

    Thamarai Selvi Somasundaram, Kannan Govindarajan & Usha Kiruthika

  2. Cloud Computing and Distributed Systems Lab, The University of Melbourne, Parkville, Australia

    Rajkumar Buyya

Authors
  1. Thamarai Selvi Somasundaram
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kannan Govindarajan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Usha Kiruthika
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rajkumar Buyya
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kannan Govindarajan.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Somasundaram, T.S., Govindarajan, K., Kiruthika, U. et al. Semantic-enabled CARE Resource Broker (SeCRB) for managing grid and cloud environment. J Supercomput 68, 509–556 (2014). https://doi.org/10.1007/s11227-013-1047-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-013-1047-z

Keywords

Search

Navigation