Abstract
This paper mainly concerns with the primal superlinear convergence of the quasi-Newton sequential quadratic programming (SQP) method for piecewise linear-quadratic composite optimization problems. We show that the latter primal superlinear convergence can be justified under the noncriticality of Lagrange multipliers and a version of the Dennis-Moré condition. Furthermore, we show that if we replace the noncriticality condition with the second-order sufficient condition, this primal superlinear convergence is equivalent with an appropriate version of the Dennis-Moré condition. We also recover Bonnans’ result in (Appl. Math. Optim. 29, 161–186, 1994) for the primal-dual superlinear of the basic SQP method for this class of composite problems under the second-order sufficient condition and the uniqueness of Lagrange multipliers. To achieve these goals, we first obtain an extension of the reduction lemma for convex Piecewise linear-quadratic functions and then provide a comprehensive analysis of the noncriticality of Lagrange multipliers for composite problems. We also establish certain primal estimates for KKT systems of composite problems, which play a significant role in our local convergence analysis of the quasi-Newton SQP method.
Similar content being viewed by others
References
Bonnans, J.F.: Local analysis of Newton-type methods for variational inequalities and nonlinear programming. Appl. Math. Optim. 29, 161–186 (1994)
Bonnans, J.F., Gilbert, J.C.h., Lemaréchal, C., Sagastizabal, C.: Numerical Optimization: Theoretical and Practical Aspects, 2nd edn. Springer, Berlin (2006)
Boggs, P.T., Tolle, J.W., Wang, P.: On the local convergence of quasi-Newton methods for constrained optimization. SIAM J. Control Optim. 20, 161–171 (1982)
Bolte, J., Chen, Z., Pauwels, E.: The multiproximal linearization method for convex composite problems. Math. Program. 182, 1–36 (2020)
Burke, J.V., metric, A. Engle.: Strong (sub)regularity of KKT mappings for piecewise linear-quadratic convex composite optimization. To appear in Math. Oper. Res., arXiv:1805.01073 (2019)
Bonnans, J.F., Shapiro, A.: Perturbation Analysis of Optimization Problems. Springer, New York (2000)
Do, H., Mordukhovich, B.S., Sarabi, M.E.: Criticality of Lagrange multipliers in extended nonlinear optimization. Optimization, https://doi.org/10.1080/02331934.2020.1723585 (2020)
Dontchev, A.L., Rockafellar, R.T.: Implicit Functions and Solution Mappings: A View from Variational Analysis, 2nd edn. Springer, Dordrecht (2014)
Dontchev, A.L., Rockafellar, R.T.: Characterizations of Lipschitzian stability in nonlinear programming. In: Fiacco, A.V. (ed.) Mathematical Programming with Data Perturbations, pp 65–82. Marcel Dekker, New York (1997)
Drusvyatskiy, D., Lewis, A.S.: Error bounds, quadratic growth, and linear convergence of proximal methods. Math. Oper. Res. 43, 919–948 (2018)
Fernández, D., Izmailov, A.F., Solodov, M.V.: Sharp primal superlinear convergence results for some newtonian methods for constrained optimization. SIAM J. Optim. 20, 3312–3334 (2010)
Ioffe, A.D.: Variational Analysis of Regular Mappings: Theory and Applications. Springer, Cham (2017)
Izmailov, A.F.: On the analytical and numerical stability of critical Lagrange multipliers. Comput. Math. Math. Phys. 45, 930–946 (2005)
Izmailov, A.F., Solodov, M.V.: Stabilized SQP revisited. Math. Program. 133, 93–120 (2012)
Izmailov, A.F., Solodov, M.V.: Newton-Type Methods for Optimization and Variational Problems. Springer, New York (2014)
Izmailov, A.F., Solodov, M.V.: Newton-type methods: A broader view. J. Optim. Theory Appl. 164, 577–620 (2015)
Levy, A.B.: Second-order epi-derivatives of composite functionals. Ann. Oper. Res. 101, 267–281 (2001)
Lewis, A.S., Wright, S.J.: A proximal method for composite minimization. Math. Program. 158, 501–546 (2016)
Mordukhovich, B.S.: Variational Analysis and Applications. Springer Monographs in Mathematics Variational. Springer, Cham (2018)
Mordukhovich, B., Sarabi, M.E.: Critical multipliers in variational systems via second-order generalized differentiation. Math Program. 169, 605–648 (2018)
Mordukhovich, B., Sarabi, M.E.: Criticality of Lagrange multipliers in variational systems. SIAM J. Optim. 29, 1524–1557 (2019)
Mohammadi, A., Mordukhovich, B., Sarabi, M.E.: Variational analysis of composite models with applications to continuous optimization. To appear in Math. Oper. Res., arXiv:1905.08837 (2020)
Mohammadi, A., Mordukhovich, B., Sarabi, M.E.: Parabolic regularity via geometric variational analysis. To appear in Trans. Amer. Math Soc. arXiv:1909.00241 (2019)
Mohammadi, A., Mordukhovich, B., Sarabi, M.E.: Stability of KKT systems and superlinear convergence of the SQP method under parabolic regularity. J. Optim Theory Appl. 186, 731–758 (2020)
Mohammadi, A., Sarabi, M.E.: Twice epi-differentiability of extended-real-valued functions with applications in composite optimization. SIAM J. Optim. 30, 2379–2409 (2020)
Poliquin, R.A., Rockafellar, R.T.: Amenable functions in optimization. In: Giannessi, F. (ed.) Non- smooth Optimization Methods and Applications, pp 338–353 (1992)
Robinson, S.M.: Perturbed Kuhn-Tucker points and rates of convergence for a class of nonlinear programming algorithms. Math. Program. 7, 1–16 (1974)
Robinson, S.M.: Strongly regular generalized equations. Math. Oper. Res. 5, 3–62 (1980)
Robinson, S.M.: Some continuity properties of polyhedral multifunctions. Math. Program. Stud. 14, 206–214 (1981)
Robinson, S.M.: An implicit-function theorem for a class of nonsmooth functions. Math. Oper. Res. 16, 292–309 (1991)
Rockafellar, R.T.: Proto-differentiability of set-valued mappings and its applications in optimization Attouch, H., et al. (eds.) . Gathier-Villars, Paris (1989)
Rockafellar, R.T.: First- and second-order epi-differentiability in nonlinear programming. Trans. Amer. Math Soc. 307, 75–108 (1988)
Rockafellar, R.T.: Second-order optimality conditions in nonlinear programming obtained by way of epi-derivatives. Math. Oper. Res. 14, 462–484 (1989)
Rockafellar, R.T.: Extended nonlinear programming. In: Di Pillo, G., Giannessi, F. (eds.) Nonlinear Optimization and Related Topics, vol. 36, pp 381–399,. Kluwer Academic Publishers, Dordrecht (2000)
Rockafellar, R.T., Wets, R.J.-B.: Variational Analysis Grundlehren Series (Fundamental Principles of Mathematical Sciences), vol. 317. Springer, Berlin (2006)
Rockafellar, R.T., Wets, R.J-B: A Lagrangian finite-generation technique for solving linear-quadratic problems in stochastic programming. Math Program. Stud. 28, 63–93 (1986)
Acknowledgements
We thank the two anonymous reviewers whose comments and suggestions helped improve the original presentation of the paper.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Sarabi, M.E. Primal Superlinear Convergence of Sqp Methods in Piecewise Linear-Quadratic Composite Optimization. Set-Valued Var. Anal 30, 1–37 (2022). https://doi.org/10.1007/s11228-021-00580-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11228-021-00580-6
Keywords
- SQP methods
- Primal superlinear convergence
- Noncriticality
- Second-order sufficient conditions
- Piecewise linear-quadratic composite problems