Ciyou Zhu
Abstract
L-BFGS-B is a limited-memory algorithm for solving large nonlinear optimization problems subject to simple bounds on the variables. It is intended for problems in which information on the Hessian matrix is difficult to obtain, or for large dense problems. L-BFGS-B can also be used for unconstrained problems and in this case performs similarly to its predessor, algorithm L-BFGS (Harwell routine VA15). The algorithm is implemented in Fortran 77.
Supplemental Material
Available for Download
Software for "L-BFGS-B: Fortran Subroutines for Large-Scale Bound-Constrained Optimization"
- BERTSEKAS, D.P. 1982. Projected Newton methods for optimization problems with simple constraints. SIAM J. Contr. Optim. 20, 221-246.]]Google Scholar
Cross Ref
- BONGARTZ, I., CONN, A. R., GOULD, N., AND TOINT, PH. L. 1995. CUTE: Constrained and unconstrained testing environment. ACM Trans. Math. Softw. 21, 1 (Mar.), 123-160.]] Google ScholarDigital Library
- BUCKLEY, A. 1989. Remark on Algorithm 630. ACM Trans. Math. Softw. 15, 3 (Sept.), 262-274.]] Google ScholarDigital Library
- BUCKLEY, A. AND LENIR, A. 1985. ALGORITHM 630: BBVSCG--a variable-storage algorithm for function minimization. ACM Trans. Math. Softw. 11, 2 (June), 103-119.]] Google ScholarDigital Library
- BYRD, R. H., Lu, P., NOCEDAL, J., AND ZHU, C. 1995. A limited memory algorithm for bound constrained optimization. SIAM J. Sci. Comput. 16, 5 (Sept.), 1190-1208.]] Google ScholarDigital Library
- BYRD, R. g., NOCEDAL, J., AND SCHNABEL, R. B. 1994. Representations of quasi-Newton matrices and their use in limited memory methods. Math. Program. 63, 2 (Jan. 31), 129-156.]]Google ScholarCross Ref
- CONN, A. R., GOULD, N. I. M., AND TOINT, PH. L. 1988. Testing a class of methods for solving minimization problems with simple bounds on the variables. Math. Comput. 50, 182, 399-430.]]Google ScholarCross Ref
- CONN, A. R., GOULD, N. I. M., AND TOINT, PH. L. 1992. LANCELOT: A FORTRAN Package for Large-Scale Nonlinear Optimization (Release A). Springer Series in Computational Mathematics, vol. 17. Springer-Verlag, New York, NY.]] Google ScholarCross Ref
- DENNIS, J. E. AND SCHNABEL, R.B. 1983. Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Prentice-Hall, Inc., Upper Saddle River, NJ.]] Google ScholarDigital Library
- GILBERT, J. C. AND LEMARI~CHAL, C. 1989. Some numerical experiments with variablestorage quasi-Newton algorithms. Math. Program. 45, 3 (Dec.), 407-435.]]Google ScholarCross Ref
- GILL, P. E., MURRAY, W., AND WRIGHT, M.H. 1981. Practical Optimization. Academic Press Ltd., London, UK.]]Google Scholar
- LEVITIN, E. S. AND POLYAK, B.T. 1966. Constrained minimization problems. USSR Comput. Math. Math. Phys. 6, 1-50.]]Google ScholarCross Ref
- LIU, D. C. AND NOCEDAL, J. 1989. On the limited memory BFGS method for large scale optimization. Math. Program. 45, 3 (Dec.), 503-528.]]Google ScholarCross Ref
- MOR}~, J. J. AND THUENTE, D.J. 1994. Line search algorithms with guaranteed sufficient decrease. ACM Trans. Math. Softw. 20, 3 (Sept.), 286-307.]] Google ScholarDigital Library
- MOR}~, J. J. AND TORALDO, G. 1989. Algorithms for bound constrained quadratic programming problems. Numer. Math. 55, 377-400.]]Google ScholarDigital Library
- SIEGEL, D. 1992. Implementing and modifying Broyden class updates for large scale optimization. Rep. DAMPT 1992/NA12. Cambridge University, Cambridge, MA.]]Google Scholar
- ZHU, C., BYRD, R. H., Lu, P., AND NOCEDAL, J. 1995. L-BFGS-B: Fortran subroutines for large-scale bound constrained optimization. EECS Tech. Rep. NAM12. Northwestern University, Evanston, IL.]]Google Scholar
Index Terms
- Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound-constrained optimization
Recommendations
An interior-point trust-funnel algorithm for nonlinear optimization
We present an interior-point trust-funnel algorithm for solving large-scale nonlinear optimization problems. The method is based on an approach proposed by Gould and Toint (Math Prog 122(1):155---196, 2010) that focused on solving equality constrained ...
Limited-memory BFGS with displacement aggregation
AbstractA displacement aggregation strategy is proposed for the curvature pairs stored in a limited-memory BFGS (a.k.a. L-BFGS) method such that the resulting (inverse) Hessian approximations are equal to those that would be derived from a full-memory ...
Analysis of the BFGS Method with Errors
The classical convergence analysis of quasi-Newton methods assumes that function and gradient evaluations are exact. In this paper, we consider the case when there are (bounded) errors in both computations and establish conditions under which a slight ...
Reviews
Michael Minkoff
The authors provide an excellent algorithmic description of the software known as L-BFGS-B, an extension of a well-known limited-memory BFGS algorithm and software (due to Liu and Nocedal), L-BFGS. Bound constraints are often not treated thoroughly, yet the effective handling of simple bounds requires addressing most of the issues that arise in general linear constraints. The treatment of constrained optimization, even in the bound constrained case, provides an important practical tool for the application of optimization techniques to a wide range of application areas, such as simulation and optimization. Thus, this software provides a much-needed quality tool. Additionally, the authors provide theoretical, implementation, and benchmarking treatments of the software development process. After the problem statement, a summary of related algorithms clarifies the relative position of the limited-memory BFGS within the literature. The paper also relates the unconstrained algorithm to the simple bound version of the algorithm. The paper presents three drivers for the software, which appear to be well designed. A thorough presentation of termination criteria follows. Implementation aspects include details regarding the bound algorithm (its predecessor unconstrained algorithm). The use of a line search is then presented. In order to obtain high accuracy in nonlinear optimization, attention in adapting algorithms to treat optimization techniques in the presence of rounding errors is necessary. The authors address this issue and summarize implementation aspects. The implementation section also includes a treatment of machine constants and scale dependency, including limitations on scale dependency. The paper concludes with numerical results. The testing presented is quite thorough (something I do not always see). A comparison with the LANCELOT package is presented, and the results are analyzed. The paper concludes with issues for future study.
Access critical reviews of Computing literature here
Become a reviewer for Computing Reviews.
Comments