Skip to main content
SpringerLink
Log in

Numerical Comparison of Augmented Lagrangian Algorithms for Nonconvex Problems

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

Augmented Lagrangian algorithms are very popular tools for solving nonlinear programming problems. At each outer iteration of these methods a simpler optimization problem is solved, for which efficient algorithms can be used, especially when the problems are large. The most famous Augmented Lagrangian algorithm for minimization with inequality constraints is known as Powell-Hestenes-Rockafellar (PHR) method. The main drawback of PHR is that the objective function of the subproblems is not twice continuously differentiable. This is the main motivation for the introduction of many alternative Augmented Lagrangian methods. Most of them have interesting interpretations as proximal point methods for solving the dual problem, when the original nonlinear programming problem is convex. In this paper a numerical comparison between many of these methods is performed using all the suitable problems of the CUTE collection.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. R.R. Allran and S.E.J. Johnsen, “An algorithm for solving nonlinear programming problems subject to nonlinear inequality constraints,” The Computer Journal, vol. 13, pp. 171–177, 1970.

    Article  Google Scholar 

  2. A. Auslender, M. Teboulle, and S. Ben-Tiba, “Interior proximal and multiplier methods based on second order homogeneous kernels,” Mathematics of Operations Research, vol. 24, pp. 645–668, 1999.

    Google Scholar 

  3. A. Ben Tal, I. Yuzefovich, and M. Zibulevsky, “Penalty/barrier multiplier methods for minimax and constrained smooth convex programs,” Research Report 9/92, Optimization Laboratory, Faculty of Industrial Engineering Management, Technion, Haifa, Israel, 1992.

  4. A. Ben-Tal and M. Zibulevsky, “Penalty/barrier multiplier methods for convex programming problems,” SIAM Journal on Optimization, vol. 7, pp. 347–366, 1997.

    Article  Google Scholar 

  5. D.P. Bertsekas, Constrained Optimization and Lagrange Multiplier Methods, Academic Press: New York, 1982.

    Google Scholar 

  6. D.P. Bertsekas, Nonlinear Programming, 2nd edition, Athena Scientific: Belmont, MA, 1999.

    Google Scholar 

  7. E.G. Birgin and J.M. Martínez, “Large-scale active-set box-constrained optimization method with spectral projected gradients,” Computational Optimization and Applications, vol. 23, pp. 101–125, 2002.

    Article  Google Scholar 

  8. E.G. Birgin, J.M. Martínez, and M. Raydan, “Nonmonotone spectral projected gradient methods on convex sets,” SIAM Journal on Optimization, vol. 10, pp. 1196–1211, 2000.

    Article  Google Scholar 

  9. E.G. Birgin, J.M. Martínez, and M. Raydan, “Algorithm 813: SPG—Software for convex-constrained optimization,” ACM Transactions on Mathematical Software, vol. 27, pp. 340–349, 2001.

    Article  Google Scholar 

  10. R.A. Castillo, “Métodos de Lagrangiano Aumentado usando penalidades general-izadas para programação não linear,” Tese de Doutorado, COPPE, Universidade Federal do Rio de Janeiro, Rio de Janeiro, Brasil, 1998.

    Google Scholar 

  11. A.R. Conn, N.I.M. Gould, and Ph.L. Toint, “A globally convergent Augmented Lagrangian algorithm for optimization with general constraints and simple bounds,” SIAM Journal on Numerical Analysis, vol. 28, pp. 545–572, 1991.

    Article  Google Scholar 

  12. E.D. Dolan and J.J. Moré, “Benchmarking optimization software with performance profiles,” Mathematical Programming, vol. 91, pp. 201–213, 2002.

    Article  Google Scholar 

  13. Z. Dostál, A. Friedlander, and S.A. Santos, “Augmented Lagrangians with adaptive precision control for quadratic programming with equality constraints,” Computational Optimization and Applications, vol. 14, pp. 37–53, 1999.

    Article  Google Scholar 

  14. Z. Dostál, A. Friedlander, and S.A. Santos, “Augmented Lagrangians with adaptive precision control for quadratic programming with simple bounds and equality constraints,” SIAM Journal on Optimization, vol. 13, pp. 1120–1140, 2003.

    Article  Google Scholar 

  15. Z. Dostál, F.A.M. Gomes, and S.A. Santos, “Solution of contact problems by FETI domain decomposition with natural coarse space projection,” Computational Method in Applied Mechanics and Engineering, vol. 190, pp. 1611–1627, 2002.

    Article  Google Scholar 

  16. R. Fletcher, Practical Methods for Optimization, John Wiley and Sons: Chichester, 1987.

    Google Scholar 

  17. C.C. Gonzaga and R.A. Castillo, “A nonlinear programming algorithm based on non-coercive penalty functions,” Mathematical Programming, vol. 96, pp. 87–101, 2003.

    Article  Google Scholar 

  18. W.W. Hager, “Dual techniques for constrained optimization,” Journal of Optimization Theory and Applications, vol. 55, pp. 33–72, 1987.

    Article  Google Scholar 

  19. W.W. Hager, “Analysis and implementation of a dual algorithm for constrained optimization,” Journal of Optimization Theory and Applications, vol. 79, pp. 427–462, 1993.

    Article  Google Scholar 

  20. M.R. Hestenes, “Multiplier and gradient methods,” Journal of Optimization Theory and Applications, vol 4, pp. 303–320, 1969.

    Article  Google Scholar 

  21. C. Humes and P.S. Silva, “Strict convex regularizations, proximal points and Augmented Lagrangians,” RAIRO Operations Research, vol. 34, pp. 283–303, 2000.

    Article  Google Scholar 

  22. A.N. Iusem, “Augmented Lagrangian methods and proximal point methods for convex optimization,” Investigacion Operativa, vol. 8, pp. 11–50, 1999.

    Google Scholar 

  23. K.C. Kiwiel, “Proximal minimization methods with generalized Bregman functions,” SIAM Journal on Control and Optimization, vol. 35, pp. 1142–1168, 1997.

    Article  Google Scholar 

  24. B.W. Kort and D.P. Bertsekas, “Multiplier methods for convex programming,” in Proceeding of the IEEE Decision and Control Conference, San Diego, CA, 1973, pp. 260–264.

  25. B.W. Kort and D.P. Bertsekas, “Combined primal-dual and penalty methods for convex programming,” SIAM Journal on Control and Optimization, vol. 14, pp. 268–294, 1976.

    Article  Google Scholar 

  26. O.L. Mangasarian and S. Fromovitz, “The Fritz-John necessary optimality conditons in presence of equality and inequality constraints,” Journal of Mathematical Analysis and Applications, vol. 17, pp. 37–47, 1967.

    Article  Google Scholar 

  27. L.C. Matioli, “Uma nova metodologia para construção de funções de penalização para algoritoms de Lagrangiano Aumentado,” Tese de Doutorado, Universidade Federal de Santa Catarina, Florianópolis, Brasil, 2001.

    Google Scholar 

  28. F.H. Murphy, “A class of exponential penalty functions,” SIAM Journal on Control, vol. 12, pp. 679–687, 1974.

    Article  Google Scholar 

  29. H. Nakayama, H. Samaya, and Y. Sawaragi, “A generalized Lagrangian function and multiplier method,” Journal of Optimization Theory and Applications, vol. 17, pp. 211–227, 1975.

    Article  Google Scholar 

  30. J. Nocedal and S.J. Wright, Numerical Optimization, Springer Verlag: New York, 1999.

    Google Scholar 

  31. R.A. Polyak, “Log-sigmoid multiplier method in constrained optimization,” Annals of Operation Research, vol. 101, pp. 427–460, 2001.

    Article  Google Scholar 

  32. R. Polyak and M. Teboulle, “Nonliner rescaling and proximal-like methods in convex optimization,” Mathematical Programming, vol. 76, pp. 265–284, 1997.

    Article  Google Scholar 

  33. M.J.D. Powell, “A method for nonlinear constraints in minimization problems,” in Optimization, R. Fletcher (Ed.), Academic Press, New York, NY, 1969, pp. 283–298.

    Google Scholar 

  34. R.T. Rockafellar, “The multiplier method of Hestenes and Powell applied to convex programming,” Journal of Optimization Theory and Applications, vol. 12, pp. 555–562, 1973.

    Article  Google Scholar 

  35. P. Tseng and D. Bertseaks, “On the convergence of the exponential multiplier method for convex programming,” Mathematical Programming, vol. 17, pp. 670–690, 1993.

    Google Scholar 

  36. A.E. Xavier, “Penalização hiperbolica,” Tese de Doutorado, COPPE, Universidade Federal do Rio de Janeiro, Rio de Janeiro, Brasil, 1992.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science IME-USP, University of São Paulo, Rua do Matão 1010, Cidade Universitária, 05508-090, São Paulo, SP, Brazil

    E. G. Birgin

  2. Department of Mathematics, Universidad Centroccidental Lisandro Alvarado, Barquisimeto, Venezuela

    R. A. Castillo

  3. Department of Applied Mathematics, IMECC-UNICAMP, University of Campinas, CP 6065, 13081-970, Campinas, SP, Brazil

    J. M. MartÍnez

Authors
  1. E. G. Birgin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. R. A. Castillo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. J. M. MartÍnez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to E. G. Birgin.

Additional information

This author was supported by ProNEx MCT/CNPq/FAPERJ 171.164/2003, FAPESP (Grants 2001/04597-4 and 2002/00094-0 and 2003/09169-6) and CNPq (Grant 302266/2002-0).

This author was partially supported by CNPq-Brasil and CDCHT-Venezuela.

This author was supported by ProNEx MCT/CNPq/FAPERJ 171.164/2003, FAPESP (Grant 2001/04597-4) and CNPq.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Birgin, E.G., Castillo, R.A. & MartÍnez, J.M. Numerical Comparison of Augmented Lagrangian Algorithms for Nonconvex Problems. Comput Optim Applic 31, 31–55 (2005). https://doi.org/10.1007/s10589-005-1066-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-005-1066-7

Keywords

Search

Navigation