Skip to main content
SpringerLink
Log in

Master–Worker: An Enabling Framework for Applications on the Computational Grid

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

The goal of this work is to create a tool that allows users to easily distribute large scientific computations on computational grids. Our tool MW relies on the simple master–worker paradigm. MW provides both a top Level interface to application software and a bottom Level interface to existing Grid computing toolkits. Both interfaces are briefly described. We conclude with a case study, where the necessary Grid services are provided by the Condor high-throughput computing system, and the MW-enabled application code is used to solve a combinatorial optimization problem of unprecedented complexity.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. D. Abramson, R. Sosic, J. Giddy and B. Hall, Nimrod: A tool for performing parameterised simulations using distributed workstations, in: Symposium on High Performance Distributed Computing, Virginia (August 1995). Available from http://www.dgs.monash. edu.au/davida/papers/nimrod.ps.Z.

  2. K. Anstreicher and N. Brixius, A new bound for the quadratic assignment problem based on convex quadratic programming, Technical report, Department of Management Sciences, University of Iowa (1999). Available from http://www.biz.uiowa.edu/ faculty/anstreicher/qapqp.ps.

  3. K. Anstreicher, N. Brixius, J.-P. Goux, G. Hudek-Davis and J. Linderoth, Location theory gives rise to QAP problem, data link (March 2000). The Alliance Online Technical News Letter, available from http://www.ncsa.uiuc.edu/SCD/Alliance/ datalink/0003/QA.Condor.html.

  4. A. Baratloo, M. Karaul, Z. Kedem and P. Wyckoff, Charlotte: Metacomputing on the Web, International Journal on Future Generation Computer Systems 15 (1999) 559–570.

    Google Scholar 

  5. J. Basney, R. Raman and M. Livny, High throughput Monte Carlo, in: Proceedings of the 9th SIAM Conference on Parallel Processing for Scientific Computing, San Antonio, Texas (1999).

  6. E. Cantu-Paz, Designing efficient master-slave parallel genetic algorithms, in: Genetic Programming: Proceedings of the Third Annual Conference, eds. J. Koza, W. Banzhaf, K. Chellapilla, K. Deb, M. Dorigo, D. Fogel, M. Garzon, D.E. Goldberg, H. Iba and R. Riolo (Morgan Kaufmann, San Francisco, 1998).

    Google Scholar 

  7. H. Casanova and J. Dongarra, NetSolve: Network enabled solvers, IEEE Computational Science and Engineering 5(3) (1998) 57–67.

    Google Scholar 

  8. E. Cela, The Quadratic Assignment Problem - Theory and Algorithms (Kluwer, 1998).

  9. Q. Chen, M. Ferris and J. Linderoth, Fatcop 2.0: Advanced features in an opportunistic mixed integer programming solver, Annals of Operations Research (2000). To appear, available from http://www. mcs.anl.gov/metaneos/fatcop2.ps.

  10. The Condor Team, Extending your Condor pool by gliding into Globus-controlled machines (2000), http://www.cs.wisc. edu/condor/manual/v6.1/2_12Extending_your. html.

  11. D.H.J. Epema, M. Livny, R. van Dantzig, X. Evers and J. Pruyne, A worldwide flock of Condors: Load sharing among workstation clusters, International Journal on Future Generation of Computer Systems 12 (1996) 67–85.

    Google Scholar 

  12. G. Fagg, K. Moore and J. Dongarra, Scalable networked information processing environment (SNIPE), International Journal on Future Generation of Computer Systems 15 (1999) 595–605.

    Google Scholar 

  13. I. Foster and C. Kesselman, Globus: A metacomputing infrastructure toolkit, International Journal on Supercomputer Applications (1997). Available as ftp://ftp.globus.org/pub/globus/ papers/globus.ps.gz.

  14. I. Foster and C. Kesselman, The Grid: Blueprint for a New Computing Infrastructure (Morgan-Kaufmann, 1999).

  15. D. Gelernter and D. Kaminsky, Supercomputing out of recycled garbage: preliminary experience with Piranha, in: ACM International Conference on Supercomputing, July 1992.

  16. J.-P. Goux and S. Leyffer, Mixed-integer nonlinear programming on metacomputing platform, Working Paper (1999).

  17. A. Grimshaw, A. Ferrari, F. Knabe and M. Humphrey, Legion: An operating system for wide-area computing (1999). Available as http://legion.virginia.edu/papers/CS–99–12.ps. Z.

  18. V. Kumar, K. Ramesh and V.N. Rao, Parallel best-first search of statespace graphs: A summary of results, in: Proceedings of the 1988 National Conference on Artificial Intelligence (August 1988) pp. 122–127.

  19. J. Linderoth and S. Wright, Solving large stochastic programs in a metacomputing environment, Invited Presentation at APMOD - Applied Modelling for Optimization (April 2000).

  20. M. Litzkow, Remote Unix - Turning idle workstations into cycle servers, in: Proceedings of Usenix Summer Conference (1987).

  21. M. Livny, J. Basney, R. Raman and T. Tannenbaum, Mechanisms for high throughput computing, SPEEDUP 11 (1997). Available from http://www.cs.wisc.edu/condor/doc/htc_mech.ps.

  22. M. Livny and R. Raman, High-throughput resource management, in: The Grid: Blueprint for a New Computing Infrastructure, eds. I. Foster and C. Kesselman (Morgan-Kauffmann, 1999).

  23. The MetaNEOS Project, Metacomputing Environments for Optimization (2000), http://www.mcs.anl.gov/metaneos.

  24. J. Pruyne and M. Livny, Providing resource management services to parallel applications, in: Proceedings of the Second Workshop on Environments and Tools for Parallel Scientific Computing (1994). Available as http://www.cs.wisc.edu/condor/ doc/condor_pvm_framework.ps.Z.

  25. J. Pruyne and M. Livny, Interfacing Condor and PVM to harness the cycles of workstation clusters, International Journal on Future Generation of Computer Systems 12 (1996) 53–65.

    Google Scholar 

  26. L. Sarmenta and S. Hirano, Bayanihan: Building and studying volunteer computing systems using Java, International Journal on Future Generation of Computer Systems 15(5/6) (1999) 675–686.

    Google Scholar 

  27. M. van Steen, P. Homburg and A. Tanenbaum, Globe: A wide-area distributed system, IEEE Concurrency 7 (1999) 70–78.

    Google Scholar 

  28. R. Wolski, J. Brevik, C. Krintz, G. Obertelli, N. Spring and A. Su, Running everyware on the computational Grid, in: SC99 Conference on High-performance Computing (1999). Available from http://www.cs.utk.edu/ rich/papers/ev-sc99.ps. gz.

  29. R. Wolski, N. Spring and J. Hayes, The Network Weather Service: A distributed resource performance forecasting service for metacomputing, International Journal on Future Generation of Computer Systems 15 (1999) 757–768.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, Northwestern University, USA

    Jean-Pierre Goux

  2. Mathematics and Computer Science Division, Argonne National Laboratory, 9700 South Cass Avenue, Argonne, IL, 60439, USA

    Jean-Pierre Goux & Jeff Linderoth

  3. Computer Sciences Department, University of Wisconsin –, Madison, 1210 West Dayton Street, Madison, WI, 53706, USA

    Sanjeev Kulkarni & Michael Yoder

Authors
  1. Jean-Pierre Goux
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sanjeev Kulkarni
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michael Yoder
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jeff Linderoth
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Goux, JP., Kulkarni, S., Yoder, M. et al. Master–Worker: An Enabling Framework for Applications on the Computational Grid. Cluster Computing 4, 63–70 (2001). https://doi.org/10.1023/A:1011416310759

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1011416310759

Search

Navigation