Elsevier

Computers & Chemical Engineering

Volume 71, 4 December 2014, Pages 563-573
Computers & Chemical Engineering

An interior-point method for efficient solution of block-structured NLP problems using an implicit Schur-complement decomposition

https://doi.org/10.1016/j.compchemeng.2014.09.013Get rights and content

Abstract

In this work, we address optimization of large-scale, nonlinear, block-structured problems with a significant number of coupling variables. Solving these problems using interior-point methods requires the solution of a linear system that has a block-angular structure at each iteration. Parallel solution is possible using a Schur-complement decomposition. In an explicit Schur-complement decomposition, the computational cost of forming and factorizing the Schur-complement is prohibitive for problems with many coupling variables. In this paper, we show that this bottleneck can be overcome by solving the Schur-complement equations implicitly, using a quasi-Newton preconditioned conjugate gradient method. This new algorithm avoids explicit formation and factorization of the Schur-complement. The computational efficiency of this algorithm is compared with the serial full-space approach, and the serial and parallel explicit Schur-complement approach. These results show that the PCG Schur-complement approach dramatically reduces the computational cost for problems with many coupling variables.

Introduction

Large-scale, rigorous, nonlinear models are used in many fields of science and engineering to provide accurate simulation of complex systems. As these models are increasingly used in decision making applications, there is new demand for efficient solution approaches to large-scale optimization problems. Coinciding with this need to solve larger nonlinear problems, there has been a change in typical desktop computing hardware. CPU clock rates are no longer increasing significantly, and hardware manufacturers are instead focusing on increasing the number of cores and improving parallel architectures. As the capabilities of a single CPU or workstation are often not enough to tackle emerging problems, it is imperative that we develop effective parallel algorithms to solve nonlinear optimization problems. Interior-point algorithms have been highly effective for solving large-scale nonlinear programming (NLP) problems and are currently considered among the most powerful methods available to tackle these problems (Nocedal and Wright, 2006). The dominant computational expense in an interior-point based algorithm like that described in Wächter and Biegler (2006) is the solution of a linear set of equations, known as the augmented system, arising from the application of a modified Newton's method to the optimality conditions. One promising approach for parallel solution of large-scale NLPs is to parallelize the solution of the augmented system along with other linear algebra operations in the algorithm. Parallel solution of linear systems is typically accomplished in one of the two ways: parallel iterative solution or structural decomposition.

Interior-point algorithms are possible utilizing iterative linear solvers with the so-called normal equations, the augmented system, and the doubly augmented system ([Forsgren et al., 2007], [Dollar, 2007]). Efficient solution requires suitable preconditioners, and significant research has been conducted in this field ([Oliveira and Sorensen, 2005], [Dollar, 2007], [Bergamaschi et al., 2004]). While a truly effective general preconditioner for use in interior-point methods has been elusive, promising results exist for specific problem classes. For example, Biros and Ghattas (2005) propose the Lagrange–Newton–Krylov–Schur (LNKS) method for steady-state PDE-constrained optimization. This method uses Krylov iterations to solve the full space linearized augmented system utilizing an approximate reduced space quasi-Newton preconditioner.

Effective parallelization of iterative linear solvers is possible, even for unstructured systems. However, real, large-scale, nonlinear optimization problems often possess inherent block structure. Example problem classes that exhibit block structure include optimization under uncertainty, parameter estimation, and spatially decomposable systems. Structure in the optimization problem induces inherent structure in the augmented system solved at each iteration of the interior-point algorithm. Several specialized decomposition-based interior-point algorithms have been developed that exploit block-angular or more general block-bordered structure in the solution of the augmented system. Examples include OOPS (Gondzio and Grothey, 2009), PIPS ([Petra and Anitescu, 2012], [Lubin et al., 2011]), Schur-Ipopt (Zavala et al., 2008), and PRBLOCK_IP (Castro, 2007). The work of Zavala et al. (2008) utilizes an explicit Schur-complement decomposition approach to provide efficient solutions to large-scale optimization problems. Built on the Ipopt algorithm, this approach is appropriate for general NLP problems. This algorithm is appropriate for block structured problems with complicating variables like those arising in parameter estimation and multi-scenario optimization under uncertainty. In this approach, the Schur-complement is formed explicitly, requiring one backsolve of the system for each complicating variables. As the number of complicating variable increases, forming and solving the Schur-complement explicitly is computationally prohibitive. To overcome the factorization cost, the preconditioned Schur-complement method in PIPS solves the Schur-complement system using a Krylov subspace iterative method with a stochastic preconditioner. The algorithm requires the formation of the Schur-complement system and factorization of the preconditioner (Petra and Anitescu, 2012). Castro (2007) solves the primal block-angular problem by performing Cholesky factorizations on the diagonal blocks of the system and using a preconditioned conjugate gradient method with a truncating power series preconditioner for the linking constraints. This algorithm solves convex QP problems with constraint coupling by exploiting structure in the normal equations.

This paper expands on the work of Zavala et al. (2008) to solve large-scale nonlinear block-angular problems using a parallel interior-point method with a Schur-complement decomposition approach. To avoid the computational expense of explicitly forming and solving the Schur-complement, we solve the Schur-complement system using an iterative method that only requires matrix vector products across the equation that describes the Schur-complement. This algorithm solves the Schur-complement system using a preconditioned conjugate gradient method with a limited memory quasi-Newton preconditioner generated using information from the previous CG iterations (Morales and Nocedal, 2000). For example problems with a large number of coupling variables, this algorithm gives an order of magnitude decrease in solution time over methods that explicitly form and solve the Schur-complement.

The paper is organized as follows. A description of the NLP problem formulation addressed by our algorithm is given in Section 2. The derivation of the interior-point method for the NLP problem is shown in Section 3. The explicit Schur-complement decomposition and the new PCG Schur-complement approach are introduced in Section 4. Section 5 demonstrates the performance of our approach on some large-scale problems. In Section 6 we discuss the significance of these results and propose some future work.

Section snippets

Problem formulation

In this paper, we consider a problem of the form, minxl,dlNfl(xl)s.t.cl(xl)=0lNx_lxlx¯llNPlxlPldd=0lN, where the vector xl contains all the variables corresponding to one particular block l in the set of blocks N={1,,N}. The vectors x_l and x¯l are the lower and upper variable bounds on xl. The vector d contains the common variables that induce coupling between the blocks. The common variables impacting block l are replicated within xl, and then mapped to d via Eq. (1d). Pl and Pld

Interior-point algorithm

Consider the general form of an NLP problem with n variables and m nonlinear equality constraints,

minf(x)s.t.c(x)=0x_xx¯,where f:n and c:nm are assumed to have continuous first and second derivatives, and xn. The vectors x_ and x¯ are the set of lower and upper variable bounds for x. We solve this problem using an interior-point method with a filter-based line-search based on that described in Wächter and Biegler (2006), and a detailed description of the algorithm and its derivation

Decomposition approach for structured systems

The algorithm description given in the previous section was derived based on the general problem from Eq. (2). While the problem described in Eq. (1) fits this form, it provides additional structure that can be exploited to allow for parallel solution techniques. Given the problem shown in Eq. (1), the augmented system from Eq. (13), can be rearranged into the block-bordered structure (Zavala et al., 2008),

W1kA1W2kA2W3kA3WNkANA1TA2TA3TANTδHkI·Δv1kΔv2kΔv3kΔvNkΔdk=r1kr2kr3krNkrdk,where

(rlk)T=

Numerical results

To test the computational performance of our approach, we implemented the interior-point algorithm described in Section 3. Solution of the augmented system is the dominant computational expense in every interior-point iteration, and as described in the previous section, the major algorithm variations in our implementation differ in their solution strategy for this linear system. These variations are described as follows:

  • 1

    Full-space serial (FS-S): The full-space serial option solves the KKT

Conclusions and future work

Given the demand for efficient optimization of large nonlinear systems, coupled with the change in focus of hardware manufacturers toward multi-core and parallel architectures, there is a distinct need for development of effective parallel algorithms for solution of NLP problems. Fortunately, large-scale problems frequently possess inherent structure arising from distinct problem classes, and this structure can often be exploited in developing parallel solution strategies.

In this paper, we

Acknowledgements

The authors thank Yu Zhu and Victor M. Zavala for insightful comments on the algorithm, and Johan Akesson for helpful discussion regarding the distillation column example problem. The authors gratefully acknowledge Ali El-Halwagi and Tylan Higley for their efforts testing the linear algebra routines in our algorithms. Thanks is also extended for the financial support provided to Jia Kang by the National Science Foundation Cyber-Enabled Discovery and Innovation (CDI)-Type II (Award # 0941313),

References (25)

  • A. Forsgren et al.

    Iterative solution of augmented systems arising in interior methods

    SIAM J Optim

    (2007)
  • R. Fourer et al.

    AMPL: a modeling language for mathematical programming

    (1993)
  • Cited by (46)

    View all citing articles on Scopus
    View full text