Skip to main content
SpringerLink
Log in

Coordinated load management in Peer-to-Peer coupled federated grid systems

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

This paper proposes a coordinated load management protocol for Peer-to-Peer (P2P) coupled federated Grid systems. The participants in the system, such as the resource providers and the consumers who belong to multiple control domains, work together to enable a coordinated federation. The coordinated load management protocol embeds a logical spatial index over a Distributed Hash Table (DHT) space for efficient management of the coordination objects; the DHT-based space serves as a kind of decentralized blackboard system. We show that our coordination protocol has a message complexity that is logarithmic to the number of nodes in the system, which is significantly better than existing broadcast based coordination protocols.

The proposed load management protocol can be applied for efficiently coordinating resource brokering services of distributed computing systems such as grids and PlanetLab. Resource brokering services are the main components that control the way applications are scheduled, managed and allocated in a distributed, heterogeneous, and dynamic Grid computing environments. Existing Grid resource brokers, e-Science application work-flow schedulers, operate in tandem but still lack a coordination mechanism that can lead to efficient application schedules across distributed resources. Further, lack of coordination exacerbates the utilization of various resources (such as computing cycles and network bandwidth). The feasibility of the proposed coordinated load management protocol is studied through extensive simulations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. Abramson D, Buyya R, Giddy J (2002) A computational economy for grid computing and its implementation in the Nimrod-G resource broker. Future Gener Comput Syst 18(8):1061–1074

    Article  MATH  Google Scholar 

  2. Allen AO (1978) Probability, statistics and queuing theory with computer science applications. Academic Press, San Diego

    Google Scholar 

  3. Andrade N, Cirne W, Brasileiro F, Roisenberg P (2003) OurGrid: an approach to easily assemble grids with equitable resource sharing. In: JSSPP’03: proceedings of the 9th workshop on job scheduling strategies for parallel processing, Lecture Notes in Computer Science. Springer, Berlin

    Google Scholar 

  4. Auyoung A, Chun B, Snoeren A, Vahdat A (2004) Resource allocation in federated distributed computing infrastructures. In: OASIS ’04: 1st workshop on operating system and architectural support for the on-demand IT infrastructure, Boston, MA, October, 2004

  5. Raza Butt A, Zhang R, Hu YC (2003) A self-organizing flock of condors. In: SC ’03: proceedings of the 2003 ACM/IEEE conference on supercomputing. IEEE Computer Society, Los Alamitos, CA, USA

  6. Buyya R, Murshed M (2002) Gridsim: a toolkit for the modeling and simulation of distributed resource management and scheduling for grid computing. Concurr Comput Pract Exp 14(13–15):1175–1220

    Article  MATH  Google Scholar 

  7. Frey J, Tannenbaum T, Livny M, Foster I, Tuecke S (2001) Condor-G: a computation management agent for multi-institutional grids. In: 10th IEEE international symposium on high performance distributed computing (HPDC-10 ’01), 2001. IEEE Computer Society, Los Alamitos, pp 237–246

    Google Scholar 

  8. Fu Y, Chase J, Chun B, Schwab S, Vahdat A (2003) SHARP: an architecture for secure resource peering. In: SOSP ’03: proceedings of the nineteenth ACM symposium on operating systems principles, Bolton Landing, NY, USA. ACM Press, New York, pp 133–148

    Chapter  Google Scholar 

  9. Garca P, Pairot C, Mondjar R, Pujol J, Tejedor H, Rallo R (2005) Planetsim: a new overlay network simulation framework. In: Software engineering and middleware, SEM 2004, Linz, Austria, Lecture Notes in Computer Science. Springer, Berlin, pp 123–137

    Chapter  Google Scholar 

  10. Gong L (2001) JXTA: a network programming environment. IEEE Internet Comput 5(3):88–95

    Article  Google Scholar 

  11. Gupta A, Sahin OD, Agrawal D, El Abbadi A (2004) Meghdoot: content-based publish/subscribe over p2p networks. In: Proceedings of the 5th ACM/IFIP/USENIX international conference on middleware, Toronto, Canada. Springer, Berlin, pp 254–273

    Google Scholar 

  12. Irwin D, Chase J, Grit L, Yumerefendi A, Becker D, Yocum KG (2006) Sharing networked resources with brokered leases. In: 2006 USENIX annual technical conference, Boston, MA, USA, pp 199–212

  13. Lai K, Huberman BA, Fine L (2004) Tycoon: a distributed market-based resource allocation system. Technical report, HP Labs

  14. Lublin U, Feitelson DG (2003) The workload on parallel supercomputers: modeling the characteristics of rigid jobs. J Parallel Distrib Comput 63(11):1105–1122

    Article  MATH  Google Scholar 

  15. Ranjan R, Harwood A, Buyya R (2006) A study on peer-to-peer based discovery of grid resource information. Technical report, GRIDS-TR-2006-17, Grid Computing and Distributed Systems Laboratory, University of Melbourne, Australia

  16. Ranjan R, Harwood A, Buyya R (2008) Coordinated load management in peer-to-peer coupled federated grid systems. Technical report GRIDS-TR-2008-2, Grids Laboratory, CSSE Department, The University of Melbourne, Australia

  17. Ranjan R, Harwood A, Buyya R (2007) A case for cooperative and incentive based coupling of distributed clusters. Future Generation Computer Systems. Elsevier, Amsterdam (in press). Available online 15 June 2007

  18. Rowstron A, Druschel P (2001) Pastry: scalable, decentralized object location, and routing for large-scale peer-to-peer systems. In: Middleware’01: proceedings of IFIP/ACM international conference on distributed systems platforms. Springer, Heidelberg, pp 329–359

    Google Scholar 

  19. Samet H (1989) The design and analysis of spatial data structures. Addison-Wesley, Reading

    Google Scholar 

  20. Shan H, Oliker L, Biswas R (2003) Job superscheduler architecture and performance in computational grid environments. In: SC ’03: proceedings of the 2003 ACM/IEEE conference on supercomputing. IEEE Computer Society, Los Alamitos, pp 44–51

    Chapter  Google Scholar 

  21. Stoica I, Morris R, Karger D, Kaashoek MF, Balakrishnan H (2001) Chord: a scalable peer-to-peer lookup service for internet applications. In: SIGCOMM ’01: proceedings of the 2001 conference on applications, technologies, architectures, and protocols for computer communications, San Diego, California, USA. ACM Press, New York, pp 149–160

    Chapter  Google Scholar 

  22. Tanin E, Harwood A, Samet H (2005) A distributed quad-tree index for peer-to-peer settings. In: ICDE’05: proceedings of the international conference on data engineering. IEEE Computer Society, Los Alamitos, pp 254–255

    Google Scholar 

  23. Tanin E, Harwood A, Samet H (2007) Using a distributed quadtree index in peer-to-peer networks. VLDB J 16(2):165–178

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Service Oriented Computing (SOC) Research Group, School of Computer Science and Engineering, The University of New South Wales, Sydney, Australia

    Rajiv Ranjan

  2. Clouds Lab and P2P Group, Department of Computer Science and Software Engineering, The University of Melbourne, Melbourne, Australia

    Aaron Harwood & Rajkumar Buyya

Authors
  1. Rajiv Ranjan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Aaron Harwood
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rajkumar Buyya
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rajiv Ranjan.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ranjan, R., Harwood, A. & Buyya, R. Coordinated load management in Peer-to-Peer coupled federated grid systems. J Supercomput 61, 292–316 (2012). https://doi.org/10.1007/s11227-010-0426-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-010-0426-y

Keywords

Search

Navigation