Abstract
In this paper, we present new methods for black-box convex minimization. They do not need to know in advance the actual level of smoothness of the objective function. Their only essential input parameter is the required accuracy of the solution. At the same time, for each particular problem class they automatically ensure the best possible rate of convergence. We confirm our theoretical results by encouraging numerical experiments, which demonstrate that the fast rate of convergence, typical for the smooth optimization problems, sometimes can be achieved even on nonsmooth problem instances.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
English translation of this paper was included in Sect. 2.3 in [4].
Since in this problem the optimal value is known, we use it in the stopping criterion.
References
Babonneau, F., Nesterov, Yu., Vial, J.-P.: Design and operating of gas transmission networks. In: Operations Research, pp. 1–14, Feb. 2012. doi:10.1287/opre.1110.1001
Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2(1), 183–202 (2009)
Devolder, O., Glineur, F., Nesterov, Yu.: First-order methods of smooth convex optimization with inexact oracle. Math. Program. (2013). doi:10.1007/s10107-013-0677-5
Elster, K.-H. (ed.): Modern Mathematical Methods in Optimization. Academie Verlag, Berlin (1993)
Lan, G.: Bundle-level methods uniformly optimal for smooth and nonsmooth convex optimization. Math. Program. (2013). doi:10.1007/s10107-013-0737-x
Lemarechal, C., Nemirovskii, A., Nesterov, Yu.: New variants of bundle methods. Math. Program. 69, 111–147 (1995)
Nemirovskii, A., Nesterov, Yu.: Optimal methods for smooth convex optimization. Zh. Vychisl. Mat. i Mat. Fiz. 25(3), 356–369 (1985). in Russian
Nemirovsky, A., Yudin, D.: Problem Complexity and Method Efficiency in Optimization. Wiley, New York (1983)
Nesterov, Yu.: Introductory Lectures on Convex Optimization. Kluwer, Boston (2004)
Nesterov, Yu.: Smooth minimization of non-smooth functions. Math. Program. A 103(1), 127–152 (2005)
Nesterov, Yu.: Gradient methods for minimizing composite functions. Math. Program. 140(1), 125–161 (2013)
Nesterov, Yu.: Primal-dual subgradient methods for convex problems. Math. Program. 120(1), 261–283 (2009)
Acknowledgments
The author is very thankful to three anonymous referees for careful reading and many suggestions, which significantly improved the initial variant of the paper.
Author information
Authors and Affiliations
Corresponding author
Additional information
The research results presented in this paper have been supported by a grant “Action de recherche concertè ARC 04/09-315” from the “Direction de la recherche scientifique - Communautè française de Belgique”. The author also acknowledges the support from Laboratory of Structural Methods of Data Analysis in Predictive Modelling, through RF government grant 11.G34.31.0073, and RFBR research projects 13-01-12007 ofi_m, 14-01-00722-a.
Rights and permissions
About this article
Cite this article
Nesterov, Y. Universal gradient methods for convex optimization problems. Math. Program. 152, 381–404 (2015). https://doi.org/10.1007/s10107-014-0790-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10107-014-0790-0