Skip to main content
SpringerLink
Log in

Memetic Algorithms for Parallel Code Optimization

  • Published:
International Journal of Parallel Programming Aims and scope Submit manuscript

Discovering the optimum number of processors and the distribution of data on distributed memory parallel computers for a given algorithm is a demanding task. A memetic algorithm (MA) is proposed here to find the best number of processors and the best data distribution method to be used for each stage of a parallel program. Steady state memetic algorithm is compared with transgenerational memetic algorithm using different crossover operators and hill-climbing methods. A self-adaptive MA is also implemented, based on a multimeme strategy. All the experiments are carried out on computationally intensive, communication intensive, and mixed problem instances. The MA performs successfully for the illustrative problem instances.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Anderson J., Lam M., (1993) Global Optimizations for Parallelism and Locality on Scalable Parallel Machines, in Proc. of the ACM SIGPLAN Conf. on Programming Language Design and Implementation, 112–125 (1993).

  2. P. Banerjee, J. Chandy, M. Gupta, E. Hodge, J. Holm, A. Lain, D. Palermo, S. Ramaswamy, and E. Su, The Paradigm Compiler for Distributed-Memory Multicomputers, IEEE Comput. October: 37–47 (1995).

  3. Beaumont O., Boudet V., Petitet A., Rastello F., Robert Y. (2001) A Proposal for Heterogeneous Cluster Scalapack. IEEE Trans. Comput. 50(10):1052–1071

    Article  MathSciNet  Google Scholar 

  4. Beaumont O., Boudet V., Rastello F., Robert Y., (2001) Matrix Multiplication on Heterogeneous Platforms, IEEE Trans. Parall. Distri. Syst. 12(10):1033–1051

    Article  MathSciNet  Google Scholar 

  5. R. Bixby, K. Kennedy, and U. Kremer, Automatic Data Layout using 0-1 Integer Programming, in Proc. Int. Conf. on Parallel Architectures and Compilation Techniques (PACT’94) pp. 111–122 (1994).

  6. H. Bremermann, Optimization Through Evolution and Recombination, Self-Organizing Syst. 93–106 (1962).

  7. Chen T., Sheu J. (1994) Communication-Free Data Allocation Techniques for Parallelizing Compilers on Multicomputers, IEEE Trans. Parall. Distri. Syst. 5:924–938

    Article  Google Scholar 

  8. J. Choi, J. Dongarra, and D. W. Walker, The design of a parallel, Dense Linear Algebra Software Library: Reduction to Hessenberg, Tridiagonal and Bidiagonal Form, in Proc. of the 2nd Workshop on Environments and Tools for Parallel Scientific Computing, pp. 98–111 (1994).

  9. H.G. Cobb, An investigation into the use of hypermutation as an adaptive operator in Genetic Algorithms Having Continuous, Time-dependent Nonstationary Environment, NRL Memorandum Report 6760 (1990).

  10. K.A. De Jong, An Analysis of the Behavior of a Class of Genetic Adaptive Systems, PhD Thesis, University of Michigan (1975).

  11. Fraser A. (1957) Simulation of Genetic Systems by Automatic Digital Computers, 2 Vols, Aust. J Biol. Sci. 10:484–491

    Google Scholar 

  12. Friedman G.J. (1959) Digital Simulation of An Evolutionary Process. Gen. Syst. Yearbook 4:171–184

    Google Scholar 

  13. Goldberg D.E. (1989) Genetic Algorithms in Search, Optimization, and Machine Learning. Addison-Wesley, Reading, MA

    MATH  Google Scholar 

  14. Holland J. H. Adaptation in Natural and Artificial Systems. University of Michican Press, (1975).

  15. Huang C.H., Sadayappan P. (1993) Communication-Free Hyperplane Partitioning of Nested Loops. J. Parall. and Distr. Comput. 19:90–102

    Article  MATH  Google Scholar 

  16. K. Ikudome, G. Fox, A. Kolawa, and J. Flower, An Automatic and Symbolic Parallelization System for Distributed Memory Parallel Computers, in Proc. of 5th Distributed Memory Computing Conference, pp.1105–1114 (1990).

  17. Karp A.H. (1987) Programming for Parallelism. IEEE Comput. 20(5):43–57

    MathSciNet  Google Scholar 

  18. Knobe K., Lucas J., Steele G. (1990). Data Optimizations: Allocation of Arrays to Reduce Communication on SIMD machines. J. arall. Distr. Compu. 8(2):102–118

    Article  Google Scholar 

  19. N. Krasnogor, Studies on the Theory and Design Space of Memetic Algorithms, PhD Thesis, University of the West of England (2002).

  20. N. Krasnogor and J. Smith, Emergence of Profitable Search Strategies Based on a Simple Inheritance Mechanism, in Proc. of the 2001 International Genetic and Evolutionary Computation Conference (GECCO2001) (2001).

  21. Krasnogor N., Gustafson S. (2004) A Study on the use of “Self-Generation” in Memetic Algorithms. Nat. Comput. 3(1):53–76

    Article  MATH  MathSciNet  Google Scholar 

  22. Lee P. (1997) Efficient algorithms for Data Distribution on Distributed Memory Parallel Computers. IEEE Trans. Parall. Distr. Syst. 8(8):825–839

    Article  Google Scholar 

  23. J. Li, and M. Chen, Index Domain Alignment: Minimizing Costs of Cross-referencing between Distributed Arrays, in Proc. of the Third Symp. on the Frontiers of Massively Parallel Computation, pp. 424–433 (1990).

  24. Mace M. (1987) Memory Storage Patterns in Parallel Processing. Kluwer Academic, Dordrecht

    Google Scholar 

  25. Mitchell M., Forrest S. (1994) Genetic Algorithms and Artificial Life. Artificial Life 1(3):267–289

    Article  Google Scholar 

  26. J. Mohan, Performance of Parallel Programs, PhD dissertation, Dept. of Computer Science, Carnegie-Mellon University, (1984).

  27. P. Moscato and M. G. Norman, A Memetic Approach for the Traveling Salesman Problem Implementation of a Computational Ecology for Combinatorial Optimization on Message-Passing Systems, Parall. Comput. Transputer Appl. 177–186 (1992).

  28. Onbasioglu E., Ozdamar L. (2003) Optimization of Data Distribution and Processor Allocation Problem using Simulated Annealing. J. Supercomp. 25:237–253

    Article  MATH  Google Scholar 

  29. Onbasioglu E., Paker Y. (1997). A Comparative Workload-based Methodology for Performance Evaluation of Parallel Computers. Future Gen. Comput. Syst. 12:512–545

    Article  Google Scholar 

  30. E. Ozcan An Empirical Investigation on Memes, Self-generation and Nurse Rostering, in Proc. of the 6th International Conference on the Practice and Theory of Automated Timetabling, 246–263 (2006).

  31. E. Ozcan, Memetic Algorithms for Nurse Rostering, in P. Yolum, T. Güngör, F. Gürgen, and C. Özturan, (Eds.) Proc. of the 20th International Symposium on Computer and Information Sciences, Lecture Notes in Computer Science, Springer-Verlag. pp.482–492 (2005).

  32. E. Ozcan and C. K. Mohan, Steady State Memetic Algorithm for Partial Shape Matching, V. William Porto, N. Saravanan, Donald E. Waagen, and A. E. Eiben (Eds.) in Evolutionary Programming VII. 7th International Conference, EP98, Lecture Notes in Computer Science. Springer. vol. 1447:527–536. 1998.

  33. E. Ozcan and E. Onbasioglu, Genetic Algorithms for Parallel Code Optimization. in Proc. of 2004 IEEE Congress on Evolutionary Computation, 2:pp.1775–1781 (2004).

  34. D. Palermo, Compiler Techniques for Optimizing Communication and Data Distribution for Distributed Memory Multicomputers, PhD thesis, University of Illinois at Urbana-Champaign (1996).

  35. Prügel-Bennett A., Shapiro J.L. (1997) An Analysis of Genetic Algorithms Using Statistical Mechanics. Physica D 104:75–114

    Article  MathSciNet  Google Scholar 

  36. N. J. Radcliffe and P. D. Surry, Formal Memetic Algorithms, Evol. Comput. AISB Workshop. 1–16 (1994).

  37. Ramanujan J., Sadayappan P. (1991) Compile-time Techniques for Data Distribution in Distributed Memory Machines. IEEE Trans. Parall. Distr. Syst. 2(4):472–482

    Article  Google Scholar 

  38. Rauber T., Runger G. (2000) Deriving Array Distributions by Optimization Techniques. J. Supercomput. 15:271–293

    Article  MATH  Google Scholar 

  39. R. Sierra, Margarita, M. and C. A. Coello Coello, On the Study of Some Theoretical Aspects of Genetic Algorithms, Technical Report EVOCINV-02–2003, Evolutionary Computation Group at CINVESTAV, Sección de Computación, Departamento de Ingeniería Eléctrica, CINVESTAV-IPN, México (2003).

  40. Smith J.E., Fogarty T.C. (1997) Operator and Parameter Adaptation in Genetic Algorithms. Soft Comput. 12:81–87

    Article  Google Scholar 

  41. L. D. Whitley and M. D. Vose, Foundations of Genetic Algorithms 3, Morgan Kaufmann Los Altos, (1995).

  42. S. Wholey, Automatic Data Mapping for Distributed Memory Parallel Computers, Proc. Int. Conf. Supercomput. pp. 25–34 (1992).

  43. M.E. Wolf and M. S. Lam, A Data Locality Optimizing Algorithm. in. Proc. ACM SIGPLAN Conf. on Programming Language Design and Implementation, pp. 30–44 (1991).

Download references

Author information

Authors and Affiliations

  1. Bilgisayar Muhendisligi Bolumu, Inonu Mah. KAYISDAGI Cad., Yeditepe Universitesi, 34755, Kadikoy/Istanbul, Turkey

    Ender Özcan & Esin Onbaşioğlu

Authors
  1. Ender Özcan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Esin Onbaşioğlu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Esin Onbaşioğlu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Özcan, E., Onbaşioğlu, E. Memetic Algorithms for Parallel Code Optimization. Int J Parallel Prog 35, 33–61 (2007). https://doi.org/10.1007/s10766-006-0026-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10766-006-0026-x

Keywords

Search

Navigation