An interior-point method for efficient solution of block-structured NLP problems using an implicit Schur-complement decomposition
Introduction
Large-scale, rigorous, nonlinear models are used in many fields of science and engineering to provide accurate simulation of complex systems. As these models are increasingly used in decision making applications, there is new demand for efficient solution approaches to large-scale optimization problems. Coinciding with this need to solve larger nonlinear problems, there has been a change in typical desktop computing hardware. CPU clock rates are no longer increasing significantly, and hardware manufacturers are instead focusing on increasing the number of cores and improving parallel architectures. As the capabilities of a single CPU or workstation are often not enough to tackle emerging problems, it is imperative that we develop effective parallel algorithms to solve nonlinear optimization problems. Interior-point algorithms have been highly effective for solving large-scale nonlinear programming (NLP) problems and are currently considered among the most powerful methods available to tackle these problems (Nocedal and Wright, 2006). The dominant computational expense in an interior-point based algorithm like that described in Wächter and Biegler (2006) is the solution of a linear set of equations, known as the augmented system, arising from the application of a modified Newton's method to the optimality conditions. One promising approach for parallel solution of large-scale NLPs is to parallelize the solution of the augmented system along with other linear algebra operations in the algorithm. Parallel solution of linear systems is typically accomplished in one of the two ways: parallel iterative solution or structural decomposition.
Interior-point algorithms are possible utilizing iterative linear solvers with the so-called normal equations, the augmented system, and the doubly augmented system ([Forsgren et al., 2007], [Dollar, 2007]). Efficient solution requires suitable preconditioners, and significant research has been conducted in this field ([Oliveira and Sorensen, 2005], [Dollar, 2007], [Bergamaschi et al., 2004]). While a truly effective general preconditioner for use in interior-point methods has been elusive, promising results exist for specific problem classes. For example, Biros and Ghattas (2005) propose the Lagrange–Newton–Krylov–Schur (LNKS) method for steady-state PDE-constrained optimization. This method uses Krylov iterations to solve the full space linearized augmented system utilizing an approximate reduced space quasi-Newton preconditioner.
Effective parallelization of iterative linear solvers is possible, even for unstructured systems. However, real, large-scale, nonlinear optimization problems often possess inherent block structure. Example problem classes that exhibit block structure include optimization under uncertainty, parameter estimation, and spatially decomposable systems. Structure in the optimization problem induces inherent structure in the augmented system solved at each iteration of the interior-point algorithm. Several specialized decomposition-based interior-point algorithms have been developed that exploit block-angular or more general block-bordered structure in the solution of the augmented system. Examples include OOPS (Gondzio and Grothey, 2009), PIPS ([Petra and Anitescu, 2012], [Lubin et al., 2011]), Schur-Ipopt (Zavala et al., 2008), and PRBLOCK_IP (Castro, 2007). The work of Zavala et al. (2008) utilizes an explicit Schur-complement decomposition approach to provide efficient solutions to large-scale optimization problems. Built on the Ipopt algorithm, this approach is appropriate for general NLP problems. This algorithm is appropriate for block structured problems with complicating variables like those arising in parameter estimation and multi-scenario optimization under uncertainty. In this approach, the Schur-complement is formed explicitly, requiring one backsolve of the system for each complicating variables. As the number of complicating variable increases, forming and solving the Schur-complement explicitly is computationally prohibitive. To overcome the factorization cost, the preconditioned Schur-complement method in PIPS solves the Schur-complement system using a Krylov subspace iterative method with a stochastic preconditioner. The algorithm requires the formation of the Schur-complement system and factorization of the preconditioner (Petra and Anitescu, 2012). Castro (2007) solves the primal block-angular problem by performing Cholesky factorizations on the diagonal blocks of the system and using a preconditioned conjugate gradient method with a truncating power series preconditioner for the linking constraints. This algorithm solves convex QP problems with constraint coupling by exploiting structure in the normal equations.
This paper expands on the work of Zavala et al. (2008) to solve large-scale nonlinear block-angular problems using a parallel interior-point method with a Schur-complement decomposition approach. To avoid the computational expense of explicitly forming and solving the Schur-complement, we solve the Schur-complement system using an iterative method that only requires matrix vector products across the equation that describes the Schur-complement. This algorithm solves the Schur-complement system using a preconditioned conjugate gradient method with a limited memory quasi-Newton preconditioner generated using information from the previous CG iterations (Morales and Nocedal, 2000). For example problems with a large number of coupling variables, this algorithm gives an order of magnitude decrease in solution time over methods that explicitly form and solve the Schur-complement.
The paper is organized as follows. A description of the NLP problem formulation addressed by our algorithm is given in Section 2. The derivation of the interior-point method for the NLP problem is shown in Section 3. The explicit Schur-complement decomposition and the new PCG Schur-complement approach are introduced in Section 4. Section 5 demonstrates the performance of our approach on some large-scale problems. In Section 6 we discuss the significance of these results and propose some future work.
Section snippets
Problem formulation
In this paper, we consider a problem of the form, where the vector xl contains all the variables corresponding to one particular block l in the set of blocks . The vectors and are the lower and upper variable bounds on xl. The vector d contains the common variables that induce coupling between the blocks. The common variables impacting block l are replicated within xl, and then mapped to d via Eq. (1d). Pl and
Interior-point algorithm
Consider the general form of an NLP problem with n variables and m nonlinear equality constraints,
where and are assumed to have continuous first and second derivatives, and . The vectors and are the set of lower and upper variable bounds for x. We solve this problem using an interior-point method with a filter-based line-search based on that described in Wächter and Biegler (2006), and a detailed description of the algorithm and its derivation
Decomposition approach for structured systems
The algorithm description given in the previous section was derived based on the general problem from Eq. (2). While the problem described in Eq. (1) fits this form, it provides additional structure that can be exploited to allow for parallel solution techniques. Given the problem shown in Eq. (1), the augmented system from Eq. (13), can be rearranged into the block-bordered structure (Zavala et al., 2008),
where
Numerical results
To test the computational performance of our approach, we implemented the interior-point algorithm described in Section 3. Solution of the augmented system is the dominant computational expense in every interior-point iteration, and as described in the previous section, the major algorithm variations in our implementation differ in their solution strategy for this linear system. These variations are described as follows:
- 1
Full-space serial (FS-S): The full-space serial option solves the KKT
Conclusions and future work
Given the demand for efficient optimization of large nonlinear systems, coupled with the change in focus of hardware manufacturers toward multi-core and parallel architectures, there is a distinct need for development of effective parallel algorithms for solution of NLP problems. Fortunately, large-scale problems frequently possess inherent structure arising from distinct problem classes, and this structure can often be exploited in developing parallel solution strategies.
In this paper, we
Acknowledgements
The authors thank Yu Zhu and Victor M. Zavala for insightful comments on the algorithm, and Johan Akesson for helpful discussion regarding the distillation column example problem. The authors gratefully acknowledge Ali El-Halwagi and Tylan Higley for their efforts testing the linear algebra routines in our algorithms. Thanks is also extended for the financial support provided to Jia Kang by the National Science Foundation Cyber-Enabled Discovery and Innovation (CDI)-Type II (Award # 0941313),
References (25)
- et al.
An improved method for nonlinear model reduction using balancing of empirical Gramians
Comput Chem Eng
(2002) - et al.
A new class of preconditioners for large-scale linear systems from interior point methods for linear programming
Linear Algebr Appl
(2005) - et al.
Interior-point decomposition approaches for parallel solution of large-scale nonlinear parameter estimation problems
Chem Eng Sci
(2008) - et al.
Optimal operation of cryogenic air separation systems with demand uncertainty and contractual obligations
Chem Eng Sci
(2011) - et al.
Preconditioning indefinite systems in interior point methods for optimization
Comput Optim Appl
(2004) - (2010)
- et al.
Parallel Lagrange–Newton–Krylov–Schur methods for PDE-constrained optimization. Part I: The Krylov–Schur solver
SIAM J Sci Comput
(2005) An interior-point approach for primal block-angular problems
Comput Optim Appl
(2007)Constraint-style preconditioners for regularized saddle point problems
SIAM J Matrix Anal Appl
(2007)- et al.
Interior methods for nonlinear optimization
SIAM Rev
(2002)
Iterative solution of augmented systems arising in interior methods
SIAM J Optim
AMPL: a modeling language for mathematical programming
Cited by (46)
Utilizing modern computer architectures to solve mathematical optimization problems: A survey
2024, Computers and Chemical EngineeringA Nested Schur decomposition approach for multiperiod optimization of chemical processes
2021, Computers and Chemical EngineeringSensitivity-Assisted multistage nonlinear model predictive control: Robustness, stability and computational efficiency
2021, Computers and Chemical EngineeringScalable preconditioning of block-structured linear algebra systems using ADMM
2020, Computers and Chemical EngineeringDeep neural network approximation of nonlinear model predictive control
2020, IFAC-PapersOnLine