Hostname: page-component-76fb5796d-5g6vh Total loading time: 0 Render date: 2024-04-25T21:11:50.271Z Has data issue: false hasContentIssue false
  • English
  • Français

Sequential Quadratic Programming *

Published online by Cambridge University Press:  07 November 2008

Paul T. Boggs  and
Jon W. Tolle
Paul T. Boggs
Affiliation:
Applied and Computational Mathematics DivisionNational Institute of Standards and TechnologyGaithersburg, Maryland 20899USA E-mail: boggs@cam.nist.gov
Jon W. Tolle
Affiliation:
Departments of Mathematics and Operations ResearchUniversity of North CarolinaChapel Hill, North Carolina 27599USA E-mail: tolle@math.unc.edu
Get access Rights & Permissions [Opens in a new window]

Extract

Since its popularization in the late 1970s, Sequential Quadratic Programming (SQP) has arguably become the most successful method for solving nonlinearly constrained optimization problems. As with most optimization methods, SQP is not a single algorithm, but rather a conceptual method from which numerous specific algorithms have evolved. Backed by a solid theoretical and computational foundation, both commercial and public-domain SQP algorithms have been developed and used to solve a remarkably large set of important practical problems. Recently large-scale versions have been devised and tested with promising results.

Type
Research Article
Information
Acta Numerica , Volume 4 , January 1995 , pp. 1 - 51
Copyright
Copyright © Cambridge University Press 1995

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

REFERENCES

Bertsekas, D. P. (1980), ‘Variable metric methods for constrained optimization based on differentiable exact penalty functions’, in Proceedings of the Eighteenth Allerton Conference on Communications, Control and Computing, Allerton Park, Illinois, pp. 584–593.Google Scholar
Bertsekas, D. P. (1982), Constrained Optimization and Lagrange Multipliers Methods, Academic Press, London.Google Scholar
Boggs, P. T., Domich, P. D., Rogers, J. E. and Witzgall, C. (1991), ‘An interior point method for linear and quadratic programming problems’, Mathematical Programming Society Committee on Algorithms Newsletter 19, 32–40.Google Scholar
Boggs, P. T., Domich, P. D., Rogers, J. E. and Witzgall, C. (1994), ‘An interior-point method for general large scale quadratic programming problems’, Internal report (in progress), National Institute of Standards and Technology.Google Scholar
Boggs, P. T. and Tolle, J. W. (1980), ‘Augmented Lagrangians which are quadratic in the multiplier’, Journal of Optimization Theory and Applications 31, 17–26.CrossRefGoogle Scholar
Boggs, P. T. and Tolle, J. W. (1984), ‘A family of descent functions for constrained optimization’, SIAM Journal on Numerical Analysis 21, 1146–1161.CrossRefGoogle Scholar
Boggs, P. T. and Tolle, J. W. (1989), ‘A strategy for global convergence in a sequential quadratic programming algorithm’, SIAM Journal on Numerical Analysis 21, 600–623.CrossRefGoogle Scholar
Boggs, P. T. and Tolle, J. W. (1994), ‘Convergence properties of a class of rank-two updates’, SIAM Journal on Optimization 4, 262–287.CrossRefGoogle Scholar
Boggs, P. T., Tolle, J. W. and Kearsley, A. J. (1991), ‘A merit function for inequality constrained nonlinear programming problems’, Internal Report 4702, National Institute of Standards and Technology.CrossRefGoogle Scholar
Boggs, P. T., Tolle, J. W. and Kearsley, A. J. (1994), ‘A practical algorithm for general large scale nonlinear optimization problems’, Internal report, National Institute of Standards and Technology.Google Scholar
Boggs, P. T., Tolle, J. W. and Wang, P. (1982), ‘On the local convergence of quasi-Newton methods for constrained optimization’, SIAM Journal on Control and Optimization 20, 161–171.CrossRefGoogle Scholar
Bonnans, J. F., Panier, E. R., Tits, A. L. and Zhou, J. L. (1992), ‘Avoiding the Maratos effect by means of a nonmonotone line search II. Inequality constrained problems—feasible iterates’, SIAM Journal on Numerical Analysis 29, 1187–1202.CrossRefGoogle Scholar
Broyden, C. G., Dennis, J. E. Jr and Moré, J. J. (1973), ‘On the local and superlinear convergence of quasi-Newton methods’, Journal of the Institute of Mathematical Applications 12, 223–246.CrossRefGoogle Scholar
Byrd, R. H. and Nocedal, J. (1991), ‘An analysis of reduced Hessian methods for constrained optimization’, Mathematical Programming 49, 285–323.CrossRefGoogle Scholar
Byrd, R. H. and Schnabel, R. B. (1986), ‘Continuity of the null space basis and constrained optimization’, Mathematical Programming 35, 32–41.CrossRefGoogle Scholar
Byrd, R. H., Schnabel, R. B. and Schultz, G. A. (1985), ‘A trust-region strategy for nonlinearly constrained optimization’, SIAM Journal on Numerical Analysis 24, 1152–1170.CrossRefGoogle Scholar
Byrd, R. H., Tapia, R. A. and Zhang, Y. (1992), ‘An SQP augmented Lagrangian BFGS algorithm for constrained optimization’, SIAM Journal on Optimization 2, 210–241.CrossRefGoogle Scholar
Celis, M. R., Dennis, J. E. Jr and Tapia, R. A. (1985), ‘A trust-region strategy for nonlinear equality constrained optimization’, in Numerical Optimization 1984 (Boggs, P., Byrd, R. and Schnabel, R., eds.), SIAM, Philadelphia, pp. 71–82.Google Scholar
Chamberlain, R., Lemarechal, C., Pedersen, H. C. and Powell, M. J. D. (1982), ‘The watchdog technique for forcing convergence in algorithms for constrained optimization’, Mathematical Programming Study 16, 1–17.CrossRefGoogle Scholar
Coleman, T. F. (1990), ‘On characterizations of superlinear convergence for constrained optimization’, in Lectures in Applied Mathematics, American Mathematical Society, Providence, Rhode Island, pp. 113–133.Google Scholar
Coleman, T. F. and Conn, A. R. (1984), ‘On the local convergence of a quasi-Newton method for the nonlinear programming problem’, SIAM Journal on Numerical Analysis 21, 755–769.CrossRefGoogle Scholar
Coleman, T. F. and Feynes, P. A. (1992), ‘Partitioned quasi-Newton methods for nonlinear equality constrained optimization’, Mathematical Programming 53, 17–44.CrossRefGoogle Scholar
Coleman, T. F. and Sorensen, D. C. (1984), ‘A note on the computation of an orthonormal basis for the null space of a matrix’, Mathematical Programming 29, 234–242.CrossRefGoogle Scholar
Dennis, J. E. Jr and Moré, J. J. (1977), ‘Quasi-Newton methods, motivation and theory’, SIAM Review 19, 46–89.CrossRefGoogle Scholar
Dennis, J. E. Jr and Schnabel, R. B. (1983), Numerical Methods for Unconstrained Optimization and Nonlinear Equations, Prentice-Hall, Englewood Cliffs, New Jersey.Google Scholar
El-Alem, M. M. (1991), ‘A global convergence theory for the Celis-Dennis-Tapia trust region algorithm for constrained optimization’, SIAM Journal on Numerical Analysis 28, 266–290.CrossRefGoogle Scholar
El-Alem, M. M. (1992), ‘A robust trust region algorithm with nonmonotonic penalty parameter scheme for constrained optimization’, Department of Mathematical Sciences 92–30, Rice University.Google Scholar
Fletcher, R. (1972), ‘A class of methods for nonlinear programming, iii: Rates of convergence’, in Numerical Methods for Nonlinear Optimization (Lootsma, F. A., ed.), Academic Press, New York, pp. 371–382.Google Scholar
Fletcher, R. (1981), Practical Methods of Optimization, volume 2, Wiley, New York.Google Scholar
Fontecilla, R. (1988), ‘Local convergence of secant methods for nonlinear constrained optimization’, SIAM Journal on Numerical Analysis 25, 692–712.CrossRefGoogle Scholar
Fontecilla, R., Steihaug, T. and Tapia, R. A. (1987), ‘A convergence theory for a class of quasi-Newton methods for constrained optimization’, SIAM Journal on Numerical Analysis 24, 1133–1151.CrossRefGoogle Scholar
Fukushima, M. (1986), ‘A successive quadratic programming algorithm with global and superlinear convergence properties’, Mathematical Programming 35, 253–264.CrossRefGoogle Scholar
Gabay, D. (1982), ‘Reduced quasi-Newton methods with feasibility improvement for nonlinearly constrained optimization’, Mathematical Programming Study 16, 18–44.CrossRefGoogle Scholar
Garcia-Palomares, U. M. and Mangasarian, O. L. (1976), ‘Superlinearly convergent quasi-Newton methods for nonlinearly constrained optimization problems’, Mathematical Programming 11, 1–13.CrossRefGoogle Scholar
Gay, D. M. (1981), ‘Computing optimal locally constrained steps’, SIAM Journal on Scientific and Statistical Computing 2, 186–197.CrossRefGoogle Scholar
Ge, R. and Powell, M. J. D. (1983), ‘The convergence of variable metric matrices in unconstrained optimization’, Mathematical Programming 27, 123–143.Google Scholar
Gilbert, J. C. (1993), ‘Superlinear convergence of a reduced BFGS method with piecewise line-search and update criterion’, Rapport de Recherche 2140, Institut National de Recherche en Informatique et en Automatique.Google Scholar
Gill, P. E., Murray, W., Saunders, M. A., Stewart, G. W. and Wright, M. H. (1985), ‘Properties of a representation of a basis for the null space’, Mathematical Programming 33, 172–186.CrossRefGoogle Scholar
Gill, P. E., Murray, W., Saunders, M. A. and Wright, M. H. (1986), ‘User's guide for NPSOL (version 4.0): A Fortran package for nonlinear programming’, Technical Report SOL 2, Department of Operations Research, Stanford University.CrossRefGoogle Scholar
Gill, P. E., Murray, W. and Wright, M. H. (1981), Practical Optimization, Academic Press, New York.Google Scholar
Glad, S. T. (1979), ‘Properties of updating methods for the multipliers in augmented Lagrangians’, Journal of Optimization Theory and Applications 28, 135–156.CrossRefGoogle Scholar
Goodman, J. (1985), ‘Newton's method for constrained optimization’, Mathematical Programming 33, 162–171.CrossRefGoogle Scholar
Grippo, L., Lampariello, F. and Lucidi, S. (1986), ‘A nonmonotone line search technique for Newton's method’, SIAM Journal on Numerical Analysis 23, 707–716.CrossRefGoogle Scholar
Han, S.-P. (1976), ‘Superlinearly convergent variable metric algorithms for general nonlinear programming problems’, Mathematical Programming 11, 263–282.CrossRefGoogle Scholar
Han, S.-P. (1977), ‘A globally convergent method for nonlinear programming’, Journal of Optimization Theory and Applications 22, 297–309.CrossRefGoogle Scholar
Lalee, M., Nocedal, J. and Plantega, T. (1993), On the implementation of an algorithm for large-scale equality optimization, Department of Electrical Engineering and Computer Science NAM 09–93, Northwestern University.Google Scholar
Luenberger, D. G. (1984), Linear and Nonlinear Programming, 2d edition, Addison-Wesley, Reading, Massachusetts.Google Scholar
Moré, J. and Sorensen, D. C. (1983), ‘Computing a trust region step’, SIAM Journal of Scientific and Statistical Computing 4, 553–572.CrossRefGoogle Scholar
Murray, W. (1994), ‘Algorithms for large nonlinear problems’, in Mathematical Programming: State of the Art 1994 (Birge, J. R. and Murty, K. G., eds.), University of Michigan, Ann Arbor, MI, pp. 172–185.Google Scholar
Murray, W. and Prieto, F. J. (1995), ‘A sequential quadratic programming algorithm using an incomplete solution of the subproblem’, SIAM Journal on Optimization, to appear.CrossRefGoogle Scholar
Nash, S. G. and Sofer, A. (1995), Linear and Nonlinear Programming, McGraw-Hill, New York.Google Scholar
Nocedal, J. (1992), ‘Theory of algorithms for unconstrained optimization’, Acta Numerica 1991, 199–242.CrossRefGoogle Scholar
Nocedal, J. and Overton, M. L. (1985), ‘Projected Hessian updating algorithms for nonlinearly constrained optimization problems’, SIAM Journal on Numerical Analysis 22, 821–850.CrossRefGoogle Scholar
Omojokun, E. M. (1989), ‘Trust region algorithms for optimization with nonlinear equality and inequality constraints’, PhD thesis, University of Colorado.Google Scholar
Ortega, J. M. and Rheinboldt, W. C. (1970), Iterative Solution of Nonlinear Equations In Several Variables, Academic Press, New York.Google Scholar
Panier, E. R. and Tits, A. L. (1993), ‘On combining feasibility, descent and superlinear convergence in inequality constrained optimization’, Mathematical Programming 59, 261–276.CrossRefGoogle Scholar
Polak, E. (1989), ‘Basics of minimax algorithms’, in Nonsmooth Optimization and Related Topics (Clark, F. H., Dem'yanov, V. F. and Giannessi, F., eds.), Plenum, New York, pp. 343–369.CrossRefGoogle Scholar
Powell, M. J. D. (1977), ‘A fast algorithm for nonlinearly constrained optimization calculations’, in Numerical Analysis Dundee, 1977 (Watson, G. A., ed.), Springer-Verlag, Berlin, pp. 144–157.Google Scholar
Powell, M. J. D. (1978a), ‘Algorithms for nonlinear constraints that use Lagrangian functions’, Mathematical Programming 14, 224–248.CrossRefGoogle Scholar
Powell, M. J. D. (1978b), ‘The convergence of variable metric methods for nonlinearly constrained optimization calculations’, in Nonlinear Programming 3 (Mangasarian, O., Meyer, R. and Robinson, S., eds.), Academic Press, New York, pp. 27–64.CrossRefGoogle Scholar
Powell, M. J. D. and Yuan, Y. (1986), ‘A recursive quadratic programming algorithm that uses differentiable exact penalty functions’, Mathematical Programming 35, 265–278.CrossRefGoogle Scholar
Robinson, S. M. (1974), ‘Perturbed Kuhn-Tucker points and rates of convergence for a class of nonlinear-programming algorithms’, Mathematical Programming 7, 1–16.CrossRefGoogle Scholar
Schittkowski, K. (1981), ‘The nonlinear programming method of Wilson, Han, and Powell with an augmented Lagrangian type line search function, part 1: Convergence analysis’, Numerische Mathematik 38, 83–114.CrossRefGoogle Scholar
Schittkowski, K. (1983), ‘On the convergence of a sequential quadratic programming method with an augmented Lagrangian line search function’, Math. Operationsforsch. U. Statist., Ser. Optimization 14, 197–216.CrossRefGoogle Scholar
Stoer, J. (1984), ‘The convergence of matrices generated by rank-2 methods from the restricted β-class of Broyden’, Numerische Mathematik 44, 37–52.CrossRefGoogle Scholar
Tapia, R. A. (1974), ‘A stable approach to Newton's method for optimization problems with equality constraints’, Journal of Optimization Theory and Applications 14, 453–476.CrossRefGoogle Scholar
Tapia, R. A. (1977), ‘Diagonalized multiplier methods and quasi-Newton methods for constrained optimization’, Journal of Optimization Theory and Applications 22, 135–194.CrossRefGoogle Scholar
Tapia, R. A. (1978), ‘Quasi-Newton methods for equality constrained optimization: Equivalence of existing methods and a new implementation’, in Nonlinear Programming 3 (Mangasarian, O., Meyer, R. and Robinson, S., eds.), Academic Press, New York, pp. 125–164.CrossRefGoogle Scholar
Tapia, R. A. (1988), ‘On secant updates for use in general constrained optimization’, Mathematics of Computation 51, 181–202.CrossRefGoogle Scholar
Vanderbei, R. J. and Carpenter, T. J. (1993), ‘Symmetric indefinite systems for interior point methods’, Mathematical Programming 58, 1–32.CrossRefGoogle Scholar
Vardi, A. (1985), ‘A trust-region algorithm for equality constrained minimization and implementation’, SIAM Journal on Numerical Analysis 22, 575–591.CrossRefGoogle Scholar
Wilson, R. B. (1963), A simplicial algorithm for concave programming, PhD thesis, Harvard University.Google Scholar
Wolfe, P. (1975), ‘A method of conjugate subgradients for minimizing nondifferentiable functions’, in Mathematical Programming Study 3 (Balinski, M. and Wolfe, P., eds.), North-Holland, Amsterdam, pp. 145–173.Google Scholar
Yuan, Y. (1985), ‘An only 2-step q-superlinear convergence example for some algorithms that use reduced Hessian approximations’, Mathematical Programming 32, 224–231.CrossRefGoogle Scholar
Yuan, Y. (1990), ‘On a subproblem of trust region algorithms for constrained optimization’, Mathematical Programming 47, 53–63.CrossRefGoogle Scholar