Abstract
The penalty-function approach is an attractive method for solving constrained nonlinear programming problems, since it brings into play all of the well-developed unconstrained optimization techniques, If, however, the classical steepest-descent method is applied to the standard penalty-function objective, the rate of convergence approaches zero as the penalty coefficient is increased to yield a close approximation to the true solution.
In this paper, it is shown that, ifm+1 steps of the conjugate-gradient method are successively repeated (wherem is the number of constraints), the convergence rate when applied to the penalty-function objective conveges at a rate predicted by the second derivative of the Lagrangian. This rate is independent of the penalty coefficient and, hence, the scheme yields reasonable convergence for a first-order method.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Courant, T.,Calculus of Variations and Supplementary Notes and Exercises, Supplementary Notes by M. Kruskal and H. Rubin, Revised and Amended by J. Moser, New York University, New York, 1962.
Butler, T., andMartin, A. V.,On a Method of Courant for Minimizing Functionals, Journal of Mathematics and Physics, Vol. 41, No. 1, 1962.
Luenberger, D. G.,Optimization by Vector Space Methods, John Wiley and Sons, New York, 1969.
Fiacco, A. V., andMcCormick, G. P.,Nonlinear Programming: Sequential Unconstrained Minimization Techniques, John Wiley and Sons, New York, 1968.
Zangwill, W. I.,Nonlinear Programming Via Penalty Functions, Management Science, Vol. 13, No. 5, 1967.
Bryson, A. E., andHo, Y. C.,Applied Optimal Control, Blaisdell Publishing Company, Waltham, Massachusetts, 1969.
Goldstine, H. H.,Minimum Problems in the Functional Calculus, Bulletin of the American Mathematical Society, Series 2, Vol. 46, No. 1, 1940.
Antosiewicz, H. A., andRheinboldt, W. C.,Numerical Analysis and Functional Analysis, Survey of Numerical Analysis, Edited by J. Todd, McGraw-Hill Book Company, New York, 1962.
Akaike, H.,On a Successive Transformation of Probability Distributions and its Application to the Analysis of the Optimum Gradient Method, Annals of Mathematical Statistics, Vol. 11, No. 3, 1959–1960.
Fletcher, R., andReeves, C. M.,Functional Minimization by Conjugate Gradients, Computer Journal, Vol. 7, No. 2, 1964.
Shah, B. V., Buehler, R. J., andKempthorne, O.,Some Algorithms for Minimizing a Function of Several Variables, SIAM Journal on Applied Mathematics, Vol. 12, No. 1, 1964.
Daniel, J. W.,The Conjugate Gradient Method for Linear and Nonlinear Operator Equations, Stanford University, Department of Mathematics, Ph.D. Thesis, 1965.
Hestenes, M. R., andStiefel, E.,Method of Conjugate Gradients for Solving Linear Systems, Journal of Research for the National Bureau of Standards, Vol. 49, No. 6, 1952.
Author information
Authors and Affiliations
Additional information
Communicated by M. R. Hestenes
This research was supported by National Science Foundation, Grant No. NSF-GK-1683.
Rights and permissions
About this article
Cite this article
Luenberger, D.G. Convergence rate of a penalty-function scheme. J Optim Theory Appl 7, 39–51 (1971). https://doi.org/10.1007/BF00933591
Received:
Issue Date:
DOI: https://doi.org/10.1007/BF00933591