Skip to main content
SpringerLink
Log in

A quasi-second-order proximal bundle algorithm

  • Published:
Mathematical Programming Submit manuscript

Abstract

This paper introduces an algorithm for convex minimization which includes quasi-Newton updates within a proximal point algorithm that depends on a preconditioned bundle subalgorithm. The method uses the Hessian of a certain outer function which depends on the Jacobian of a proximal point mapping which, in turn, depends on the preconditioner matrix and on a Lagrangian Hessian relative to a certain tangent space. Convergence is proved under boundedness assumptions on the preconditioner sequence.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. M. Al-Baali, “Variational quasi-Newton methods for unconstrained optimization,”JOTA 77 (1993) 127–143.

    Article  MATH  MathSciNet  Google Scholar 

  2. A. Auslender, “Numerical methods for nondifferentiable convex optimization,”Mathematical Programming Study 30 (1987) 102–126.

    MATH  MathSciNet  Google Scholar 

  3. J. Bonnans, J.Ch. Gilbert, C. Lemaréchal and C. Sagastizabal, “A family of variable metric proximal methods,”Mathematical Programming 68 (1995) 15–47.

    MathSciNet  Google Scholar 

  4. E.W. Cheney and A.A. Goldstein, “Newton's method for convex programming and Tchebycheff approximation,”Numerische Mathematik 1 (1959) 253–268.

    Article  MATH  MathSciNet  Google Scholar 

  5. A.R. Conn, N.I.M. Gould and Ph.L. Toint, “Convergence of quasi-Newton matrices generated by the symmetric rank one update,”Mathematical Programming 50 (1991) 177–195.

    Article  MATH  MathSciNet  Google Scholar 

  6. R. Correa and L. Lemaréchal, “Convergence of some algorithms for convex minimization,”Mathematical Programming 62 (1993) 261–275.

    Article  MathSciNet  Google Scholar 

  7. W.C. Davidon, “Variable metric methods for mimimization,” Research and Development Report ANL-5990 (Rev.), Argonne National Laboratory (Argonne, IL, 1959); (reprinted inSIAM Journal on Optimization 1 (1991) 1–17).

    Google Scholar 

  8. J.E. Dennis and J.J. Moré, “Quasi-Newton methods, motivation and theory,”SIAM Review 19 (1977) 46–84.

    Article  MATH  MathSciNet  Google Scholar 

  9. R. Fletcher, “Resolving degeneracy in quadratic programming,”Annals of Operations Research 47 (1993) 307–334.

    Article  MathSciNet  Google Scholar 

  10. M. Fukushima, “A descent algorithm for nonsmooth convex programming,”Mathematical Programming 30 (1984) 163–175.

    MATH  MathSciNet  Google Scholar 

  11. P.E. Gill, W. Murray and M.H. Wright,Practical Optimization (Academic Press, New York, 1981).

    MATH  Google Scholar 

  12. J. Guo, “A class of variable-metric-secant algorithms for unconstrained optimization,” Ph.D. Dissertation, Washington State University (Pullman, WA, 1993).

    Google Scholar 

  13. J. Hiriart-Urruty and C. Lemaréchal,Convex Analysis and Minimization Algorithms, I and II (Springer, Berlin, 1993).

    Google Scholar 

  14. J.E. Kelley, “The cutting plane method for solving convex programs,”Journal of the SIAM 8 (1960) 703–712.

    MathSciNet  Google Scholar 

  15. H. Khalfan, R.H. Byrd and R.B. Schnable, “A theoretical and experimental study of the symmetric rank one update,”SIAM Journal on Optimization 3 (1993) 1–24.

    Article  MATH  MathSciNet  Google Scholar 

  16. K.C. Kiwiel,Methods of Descent for Nondifferentiable Optimization (Springer, New York, 1985).

    MATH  Google Scholar 

  17. K.C. Kiwiel, “Proximity control in bundle methods for convex nondifferentiable minimization,”Mathematical Programming 46 (1990) 105–122.

    Article  MATH  MathSciNet  Google Scholar 

  18. C. Lemaréchal, “An extension of Davidon methods to nondifferentiable problems,”Mathematical Programming Study 3 (1975) 95–109.

    Google Scholar 

  19. C. Lemaréchal, “Numerical experiments in nonsmooth optimization,” in: E.A. Nurminski, ed.,Progress in Nondifferentiable Optimization (IIASA, Laxenburg, 1982) 61–84.

    Google Scholar 

  20. C. Lemaréchal, “Nondifferentiable Optimization,” in: G.L. Nemhauser, A.H.G. Rinnooy Kan and M.J. Todd, eds.,Optimization, Handbooks in Operations, Research and Management Science, Vol. 1 (North-Holland, Amsterdam, 1989) 529–572.

    Google Scholar 

  21. C. Lemaréchal, “Lagrangian decomposition and nonsmooth optimization: Bundle algorithm, prox iteration, augmented Lagrangian,” in: F. Giannessi, ed.,Nonsmooth Optimization Methods and Applications (Gordon and Breath, Amsterdam, 1992) 201–216.

    Google Scholar 

  22. C. Lemaréchal and R. Mifflin, “A globally and superlinearly convergent algorithm for one-dimensional minimization of convex functions,”Mathematical Programming 24 (1982) 241–256.

    Article  MATH  MathSciNet  Google Scholar 

  23. C. Lemaréchal and C. Sagastizabal, “An approach to variable metric bundle methods,” in:Proceedings of the 16th IFIP Conference on System Modelling and Optimization (Compiègne, 1993) 144–162.

  24. C. Lemaréchal and C. Sagastizabal, “Practical aspects of the Moreau-Yosida regularization I: Theoretical properties,” INRIA report (Le Chesnay, 1994).

  25. B. Martinet, “Régularisation d'inéquations variationnelles par aproximations successives,”Revue Française d'informatique et Recherche Opérationnelle R 3 (1970) 154–158.

    MathSciNet  Google Scholar 

  26. R. Mifflin, “Stationarity and superlinear convergence of an algorithm for univariate locally Lipschitz constrained minimization,”Mathematical Programming 28 (1984) 50–71.

    MATH  MathSciNet  Google Scholar 

  27. R. Mifflin, “Better than linear convergence and safeguarding in nonsmooth minimization,” in: P. Thoft-Christensen ed.,System Modelling and Optimization (Springer, New York, 1984) 321–330.

    Chapter  Google Scholar 

  28. R. Mifflin, “On superlinear convergence in univariate nonsmooth minimization,”Mathematical Programming 49 (1991) 273–279.

    Article  MathSciNet  Google Scholar 

  29. R. Mifflin and J.L. Nazareth, “The least prior deviation quasi-Newton update,”Mathematical Programming 65 (1994) 247–261.

    Article  MathSciNet  Google Scholar 

  30. J.J. Moreau, “Proximité et dualité dans un espace hilbertien,”Bulletin de la Société Mathématique de France 93 (1965) 273–299.

    MATH  MathSciNet  Google Scholar 

  31. L. Qi and J. Sun, “A nonsmooth version of Newton's method,”Mathematical Programming 58 (1993) 353–367.

    Article  MathSciNet  Google Scholar 

  32. M. Qian, “The variable metric proximal point algorithm: global and superlinear convergence,” Manuscript, Department of Mathematics, University of Washington (Seattle, WA, 1992).

    Google Scholar 

  33. R.T. Rockafellar,Convex Analysis (Princeton University Press, Princeton, NJ, 1970).

    MATH  Google Scholar 

  34. R.T. Rockafellar, “Augmented Lagrangians and applications of the proximal point algorithm in convex programming,”Mathematics of Operations Research 1 (1976) 97–116.

    Article  MATH  MathSciNet  Google Scholar 

  35. R.T. Rockafellar, “Monotone operators and the proximal point algorithm,”SIAM Journal on Control and Optimization 14 (1976) 877–898.

    Article  MATH  MathSciNet  Google Scholar 

  36. H. Schramm and J. Zowe, “A version of the bundle idea for minimizing a nonsmooth function: conceptual idea, convergence analysis, numerical results,”SIAM Journal on Optimization 2 (1992) 121–152.

    Article  MATH  MathSciNet  Google Scholar 

  37. J.E. Spingarn, “Applications of the method of partial inverse to convex programming: Decomposition,”Mathematical Programming 32 (1985) 199–223.

    Article  MATH  MathSciNet  Google Scholar 

  38. P. Wolfe, “A method of conjugate subgradients for minimizing nondifferentiable functions,”Mathematical Programming Study 3 (1975) 145–173.

    MathSciNet  Google Scholar 

  39. K. Yosida,Functional Analysis (Springer, Berlin, 1964).

    Google Scholar 

  40. J. Zowe, “Nondifferentiable optimization,” in: K. Schittkowski, ed.,Computational Mathematical Programming (Springer, New York, 1985) 323–356.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Pure and Applied Mathematics, Washington State University, 99164-3113, Pullman, WA, USA

    Robert Mifflin

Authors
  1. Robert Mifflin
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

Research supported by NSF Grant No. DMS-9402018 and by Institut National de Recherche en Informatique et en Automatique, France.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Mifflin, R. A quasi-second-order proximal bundle algorithm. Mathematical Programming 73, 51–72 (1996). https://doi.org/10.1007/BF02592098

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02592098

Keywords

Search

Navigation