A computational economy for grid computing and its implementation in the Nimrod-G resource broker
Introduction
The accelerated development of grid computing systems has positioned them as promising next generation computing platforms. They enable the sharing, selection, and aggregation of geographically distributed resources for solving large-scale problems in science, engineering, and commerce [1], [7], [30]. However, application composition, resource management and scheduling in these environments is a complex undertaking. This is due to the geographic distribution of resources that are often owned by different organizations having different usage policies and cost models, and varying loads and availability patterns. To address these resource management challenges, we have proposed and developed a computational economy framework for resource allocation and regulation of supply and demand for resources. The new framework offers an incentive to resource owners for being part of the grid and motivates resource users to strike a balance between time for results delivery and heir economic cost, i.e., deadline and budget [5].
We are exploring the use of an economic paradigm for grid computing. We have developed an economy-driven grid resource broker within the Nimrod-G system that supports soft-deadline and budget-based scheduling of applications on the computational grid [7]. Depending on users’ quality of service (QoS) requirements, our resource broker dynamically leases grid services at runtime depending on their cost, quality, and availability. The scheduler allows minimization of time or cost within specific deadline and budget constraints.
Resource management systems need to provide mechanisms and tools that realize the goals of both service providers and consumers. The resource consumers need a utility model, representing their resource demand and preferences, and brokers that automatically generate strategies for choosing providers based on this model. Further, the brokers need to manage all issues associated with the execution of the underlying application. The service providers need price generation schemes that increase system utilization, as well as economic protocols that help them to offer competitive services. For the market to be competitive and efficient, coordination mechanisms are required that help the market reach an equilibrium price, that is, the market price at which the supply of a service equals the quantity demanded [13]. Numerous economic theories have been proposed in the literature and many commonly used economic models for selling goods and services can be employed as negotiation protocols in grid computing. Some of these market- or social-driven economic models are shown in Table 1 along with the identity of the distributed system that adopted the approach [8].
These economic models regulate the supply and demand for resources in grid-based virtual enterprises. We demonstrate the power of these models in scheduling computations using the Nimrod-G resource broker on a grid testbed, called the World Wide Grid (WWG) spanning across five continents. Whilst it is not the goal of the system to earn revenue for the resource providers, this approach does provide an economic incentive for resource owners to share their facilities on the grid. Further, it encourages the emergence of a new service-oriented computing industry. Importantly, it provides mechanisms to trade-off QoS parameters such as deadline and computational cost, and offers incentive for users to relax their requirements. For example, a user may be prepared to accept a later deadline if the computation can be achieved more cheaply. Current grid computing toolkits and applications do not provide this functionality.
Section snippets
Objectives and goals
Nimrod-G is a tool for automated modeling and execution of parameter sweep applications (parameter studies) over global computational grids [1], [2], [3]. It provides a simple declarative parametric modeling language for expressing parametric experiments. A domain expert can easily create a plan for a parametric experiment and use the Nimrod system to submit jobs for execution. It uses novel resource management and scheduling algorithms based on economic principles. Specifically, it supports
Scheduling experiments on the WWG testbed
We have performed deadline and budget constrained scheduling experiments at two different times (Australian peak and off-peak hours) on resources distributed in two major time zones [7] using a “cost-optimization scheduling algorithm” [6] on the WWG [14] testbed shown in Fig. 5. Currently, the testbed has heterogeneous computational resources owned by different organizations distributed across five continents: Asia, Australia, Europe, North America, and South America. This testbed contains
Conclusions and future work
The emerging grid computing technologies are enabling the creation of virtual organizations and enterprises for sharing distributed resources for solving large-scale problems in science, engineering, and commerce. The resource management and scheduling systems in grid environments need to be adaptive to handle dynamic changes in availability of resources and user requirements. At the same time, they need to provide scalable, controllable, measurable, and easily enforceable policies for
David Abramson is a Professor and the Head of the School of Computer Science and Software Engineering (CSSE) at Monash University, Melbourne, Australia. He has been involved in computer architecture and high performance computing research since 1979. Previous to joining Monash University in 1997, he has held appointments at Griffith University, CSIRO, and RMIT. He is currently project leader in the Co-operative Research Centre for Distributed Systems Nimrod Project. He is also Chief
References (30)
- D. Abramson, J. Giddy, L. Kotler, High performance parametric modeling with Nimrod-G: killer application for the Global...
- D. Abramson, R. Sosic, J. Giddy, B. Hall, Nimrod: a tool for performing parameterised simulations using distributed...
- D. Abramson, I. Foster, J. Giddy, A. Lewis, R. Sosic, R. Sutherst, N. White, The Nimrod Computational Workbench: a case...
- R. Buyya, D. Abramson, J. Giddy, Nimrod-G: an architecture for a resource management and scheduling system in a global...
- R. Buyya, D. Abramson, J. Giddy, An economy driven resource management architecture for global computational power...
- R. Buyya, J. Giddy, D. Abramson, An evaluation of economy-based resource trading and scheduling on computational power...
- R. Buyya, D. Abramson, J. Giddy, A case for economy grid architecture for service-oriented grid computing, in:...
- R. Buyya, H. Stockinger, J. Giddy, D. Abramson, Economic models for management of resources in peer-to-peer and grid...
- K. Czajkowski, I. Foster, N. Karonis, C. Kesselman, S. Martin, W. Smith, S. Tuecke, A resource management architecture...
- S. Chapin, J. Karpovich, A. Grimshaw, The Legion resource management system, in: Proceedings of the Fifth Workshop on...
Cited by (319)
Integrating selective undo of feature-based modeling operations for real-time collaborative CAD systems
2019, Future Generation Computer SystemsSupporting selective undo of string-wise operations for collaborative editing systems
2018, Future Generation Computer SystemsResource allocation decision model for dependable and cost-effective grid applications based on Grid Bank
2017, Future Generation Computer SystemsMarket-based autonomous resource and application management in private clouds
2017, Journal of Parallel and Distributed ComputingA Concept of Semantic Description for e-Production Systems in Manufacturing
2017, Procedia CIRP
David Abramson is a Professor and the Head of the School of Computer Science and Software Engineering (CSSE) at Monash University, Melbourne, Australia. He has been involved in computer architecture and high performance computing research since 1979. Previous to joining Monash University in 1997, he has held appointments at Griffith University, CSIRO, and RMIT. He is currently project leader in the Co-operative Research Centre for Distributed Systems Nimrod Project. He is also Chief Investigator in the following ARC funded research project: Guard—a relative debugger and a software environment for building high performance optimizing decision support systems from computational models.
Rajkumar Buyya is a lecturer in the School of Computer Science and Software Engineering at the University of Melbourne, Australia. He directs research activities of the Grid Computing and Distributed Systems (GRIDS) Laboratory. He was awarded Dharma Ratnakara Memorial Trust Gold Medal for his academic excellence during 1992 by Mysore and Kuvempu Universities. He has authored three books Microprocessor x86 Programming, Mastering C++, and Design of PARAS Microkernel. He has edited a popular two volumes book on High Performance Cluster Computing published by Prentice-Hall, USA. He also edited proceedings of six international conferences and served as guest editor for major research journals. He has contributed to the development of system software for PARAM supercomputers produced by the Centre for Development of Advanced Computing (C-DAC), India. At Monash University, he is conducting R&D on next generation Internet/grid computing technologies and its applications.
Jonathan Giddy recently joined as a coordinator for grid technologies in the Welsh e-Science Centre at Cardiff University, UK. Prior to that, he worked as Research Scientist at the Distributed Systems Technology Centre (DSTC), Monash University, Melbourne, Australia. He holds BSc honours in Computer Science from the University of Wollongong. He worked on a number of DSTC funded projects—Security Unit at the Queensland University of Technology and Tools Unit at the Griffith University. In the context of Nimrod project, he is involved in design and development of number of software tools for high performance distributed computing. He enjoys programming in Python!