Invited Review
The Benders decomposition algorithm: A literature review

https://doi.org/10.1016/j.ejor.2016.12.005Get rights and content

Highlights

  • A state-of-the-art survey of Benders decomposition algorithm.

  • Synthesizing the state-of-the-art in acceleration methods.

  • Focusing on combinatorial optimization.

Abstract

The Benders decomposition algorithm has been successfully applied to a wide range of difficult optimization problems. This paper presents a state-of-the-art survey of this algorithm, emphasizing its use in combinatorial optimization. We discuss the classical algorithm, the impact of the problem formulation on its convergence, and the relationship to other decomposition methods. We introduce a taxonomy of algorithmic enhancements and acceleration strategies based on the main components of the algorithm. The taxonomy provides the framework to synthesize the literature, and to identify shortcomings, trends and potential research directions. We also discuss the use of the Benders Decomposition to develop efficient (meta-)heuristics, describe the limitations of the classical algorithm, and present extensions enabling its application to a broader range of problems.

Introduction

It has been more than five decades since the Benders Decomposition (BD) algorithm was proposed by Benders (1962), with the main objective of tackling problems with complicating variables, which, when temporarily fixed, yield a problem significantly easier to handle. The BD method (also referred to as variable partitioning, Zaourar and Malick (2014), and outer linearization, Trukhanov, Ntaimo, and Schaefer (2010)) has become one of the most widely used exact algorithms, because it exploits the structure of the problem and decentralizes the overall computational burden. Successful applications are found in many divers fields, including planning and scheduling (Canto, 2008, Hooker, 2007), health care (Luong, 2015), transportation and telecommunications (Costa, 2005), energy and resource management (Cai, McKinney, Lasdon, Watkins, 2001, Zhang, Ponnambalam, 2006), and chemical process design (Zhu & Kuno, 2003), as illustrated in Table 1.

The BD method is based on a sequence of projection, outer linearization, and relaxation (Geoffrion, 1970a, Geoffrion, 1970b). The model is first projected onto the subspace defined by the set of complicating variables. The resulting formulation is then dualized, and the associated extreme rays and points respectively define the feasibility requirements (feasibility cuts) and the projected costs (optimality cuts) of the complicating variables. Thus, an equivalent formulation can be built by enumerating all the extreme points and rays. However, performing this enumeration and, then, solving the resulting formulation is generally computationally exhausting, if not impossible. Hence, one solves the equivalent model by applying a relaxation strategy to the feasibility and optimality cuts, yielding a Master Problem (MP) and a subproblem, which are iteratively solved to respectively guide the search process and generate the violated cuts.

The BD algorithm was initially proposed for a class of mixed-integer linear programming (MILP) problems. When the integer variables are fixed, the resulting problem is a continuous linear program (LP) for which we can use standard duality theory to develop cuts. Many extensions have since been developed to apply the algorithm to a broader range of problems (e.g., Geoffrion, 1972, Hooker, Ottosson, 2003). Other developments were proposed to increase the algorithm’s efficiency on certain optimization classes (e.g., Costa, Cordeau, Gendron, Laporte, 2012, Crainic, Hewitt, Rei, 2014). In addition, BD often provides a basis for the design of effective heuristics for problems that would otherwise be intractable (Côté, Laughton, 1984, Raidl, 2015). The BD approach has thus become widely used for linear, nonlinear, integer, stochastic, multi-stage, bilevel, and other optimization problems, as illustrated in Table 2.

Fig. 1 depicts the increasing interest in the BD algorithm over the years. Despite this level of interest, there has been no comprehensive survey of the method in terms of its numerical and theoretical challenges and opportunities. The now out-of-date survey by Costa (2005) reviews only applications to fixed-charge network design problems. The main goal of this paper therefore is to contribute to filling this gap by reviewing the current state-of-the-art, focusing on the main ideas for accelerating the method, discussing the main variants and extensions aimed to handle more general problems involving, e.g., nonlinear/integer/constraint programming subproblems, and identifying trends and promising research directions. Many different enhancement strategies were proposed to address the shortcomings of the BD method and accelerate it. This effort contributed significantly to the success of the method. We propose a taxonomy of the enhancement and acceleration strategies based on the main components of the algorithm: the decomposition strategy, the strategies to handle the MP and subproblem, and the strategies to generate solutions and cuts. The taxonomy provides the framework to classify and synthesize the literature and to identify relations among strategies and between these and the BD method.

The remainder of this paper is organized as follows. Section 2 presents the classical BD algorithm, the associated model selection criteria, and its relationship to other decomposition methods. Section 3 presents the proposed taxonomy, used to survey the acceleration strategies in Sections 4–7. Section 8 presents Benders-type heuristics, and Section 9 describes extensions of the classical algorithm. Finally, Section 10 provides concluding remarks and describes promising research directions.

Section snippets

The Benders decomposition method

We present in this section the classical version of the Benders algorithm (Benders, 1962). We review its extensions to a broader range of optimization problems in Section 9.

Taxonomy of the enhancement strategies

A straightforward application of the classical BD algorithm may require excessive computing time and memory (Magnanti, Wong, 1981, Naoum-Sawaya, Elhedhli, 2013). Its main drawbacks include: time-consuming iterations; poor feasibility and optimality cuts; ineffective initial iterations; zigzagging behavior of the primal solutions and slow convergence at the end of the algorithm (i.e., a tailing-off effect); and upper bounds that remain unchanged in successive iterations because equivalent

Decomposition strategies

Recent studies have presented various modified decomposition strategies. Crainic, Hewitt, Rei, 2014, Crainic, Hewitt, Rei, 2016 emphasized that the BD method causes the MP to lose all the information associated with the noncomplicating variables. This results in instability, erratic progression of the bounds, and a large number of iterations. Moreover, the problem structure associated with the linking constraints (3) is lost, and thus many classical VIs are not applicable. The authors proposed

Solution procedure

The iterative solution of the MP and subproblem is a major computational bottleneck. In particular, the MP, an MILP formulation, is often lacking special structure, and is continually growing in size becoming more and more difficult to solve. Classically, the MP is solved to optimality via branch-and-bound, while the subproblem is handled with the simplex method. In this section, we survey the various alternatives that have been proposed. These strategies exploit the structure of the MP and

Solution generation

The quality of the solutions for the set of complicating variables directly determines the number of iterations, as they are used to generate cuts and bounds. These solutions are traditionally found by exactly or approximately solving the regular MP. Three approaches have been proposed to improve the quality of the solutions or generate them more quickly: (1) using alternative formulations, (2) improving the MP formulation, and (3) using heuristics to independently generate solutions or to

Cut generation

The number of iterations is closely related to the strength of the cuts, i.e., the values selected for the dual variables. Researchers have explored ways to select or strengthen the traditional feasibility and optimality cuts or to generate additional valid cuts.

Magnanti and Simpson (1978) and Magnanti and Wong (1981) were the first to consider the degeneracy of the subproblems, when the dual subproblem has multiple optimal solutions that do not yield cuts of equal strength. Hence, to find the

Benders-type heuristics

Because of time and memory limitations, the execution of the BD method might be stopped before its convergence is established. Moreover, in many practical applications, decision-makers do not need a provably optimal solution, a good feasible solution being deemed sufficient. Such a solution is often obtained somewhat early in the solution process.

From a heuristic point of view, the BD method is an attractive methodology because it can take advantage of special structures and provides a rich

Extensions of the classical Benders decomposition method

The classical BD algorithm was proposed for certain classes of MILPs for which the integer variables were considered to be complicating, and standard duality theory could be applied to the subproblem to develop cuts. Extensions of the method have allowed it to address a broader range of optimization problems, including integer subproblems (e.g., Carøe & Tind, 1998), nonlinear functions (e.g., Cai, McKinney, Lasdon, Watkins, 2001, Geoffrion, 1972), logical expressions (e.g., Eremin & Wallace,

Conclusions and future research

We have presented a state-of-the-art survey of the BD method. We have discussed the classical algorithm, the impact of the problem formulation on its convergence, and the relationship to other decomposition methods. We have developed a taxonomy to classify the literature on acceleration strategies, based on the main components of the algorithm, which provided rich guidelines to analyze various enhancements, identifying shortcomings, trends and potential research directions. We have also

Acknowledgments

Partial funding for this project has been provided by the Natural Sciences and Engineering Council of Canada (NSERC), through its Discovery Grant program and by the Fonds de recherche du Québec through its Team Grant program. We also gratefully acknowledge the support of Fonds de recherche du Québec through their strategic infrastructure grants.

References (157)

  • K. Fukuda et al.

    Analysis of backtrack algorithms for listing all vertices and all faces of a convex polyhedron

    Computational Geometry

    (1997)
  • V. Gabrel et al.

    Exact solution of multicommodity network optimization problems with general step cost functions

    Operations Research Letters

    (1999)
  • S. Gelareh et al.

    Multi-period hub location problems in transportation

    Transportation Research Part E: Logistics and Transportation Review

    (2015)
  • I. Harjunkoski et al.

    A decomposition approach for the scheduling of a steel plant production

    Computers & Chemical Engineering

    (2001)
  • I. Harjunkoski et al.

    Decomposition techniques for multistage scheduling problems using mixed-integer and constraint programming methods

    Computers & Chemical Engineering

    (2002)
  • K. Holmberg

    On using approximations of the Benders master problem

    European Journal of Operational Research

    (1994)
  • M. Jeihoonian et al.

    Accelerating Benders decomposition for closed-loop supply chain network design: Case of used durable products with different quality levels

    European Journal of Operational Research

    (2016)
  • LaiM.-C. et al.

    A hybrid algorithm for capacitated plant location problem

    Expert Systems with Applications

    (2010)
  • G. Laporte et al.

    The integer L-shaped method for stochastic integer programs with complete recourse

    Operations Research Letters

    (1993)
  • LiX.

    Parallel nonconvex generalized Benders decomposition for natural gas production network planning under uncertainty

    Computers & Chemical Engineering

    (2013)
  • U. Lorenz et al.

    Solving multistage quantified linear optimization problems with the alpha–beta nested Benders decomposition

    EURO Journal on Computational Optimization

    (2015)
  • A. Mercier et al.

    A computational study of Benders decomposition for the integrated aircraft routing and crew scheduling problem

    Computers & Operations Research

    (2005)
  • A. Mercier et al.

    An integrated aircraft routing, crew scheduling and flight retiming model

    Computers & Operations Research

    (2007)
  • Y. Adulyasak et al.

    Benders decomposition for production routing under demand uncertainty

    Operations Research

    (2015)
  • Angulo, G., Ahmed, S., & Dey, S. S. (2014). Improving the integer L-shaped method. Optimization Online, Available at...
  • K.A. Ariyawansa et al.

    Performance of a benchmark parallel implementation of the Van Slyke and Wets algorithm for two-stage stochastic programs on the sequent/balance

    Concurrency: Practice and Experience

    (1991)
  • M.S. Bazaraa et al.

    Nonlinear Programming: Theory and Algorithms

    (2013)
  • J.C. Beck

    Checking-up on branch-and-check

  • J.F. Benders

    Partitioning procedures for solving mixed-variables programming problems

    Numerische Mathematik

    (1962)
  • T. Benoist et al.

    Constraint programming contribution to Benders decomposition: a case study

    Proceedings of the 8th international conference on principles and practice of constraint programming. CP ’02.

    (2002)
  • J.R. Birge

    Decomposition and partitioning methods for multistage stochastic linear programs

    Operations Research

    (1985)
  • J.R. Birge et al.

    Introduction to stochastic programming

    (1997)
  • J.A. Bloom

    Solving an electricity generating capacity expansion planning problem by generalized Benders’ decomposition

    Operations Research

    (1983)
  • Bodur, M., Dash, S., & Luedtke, J. (2014). Strengthened Benders cuts for stochastic integer programs with continuous...
  • N. Boland et al.

    Proximity Benders: a decomposition heuristic for stochastic programs

    Journal of Heuristics

    (2016)
  • M. Boschetti et al.

    Benders decomposition, lagrangean relaxation and metaheuristic design

    Journal of Heuristics

    (2009)
  • Q. Botton et al.

    Benders decomposition for the hop-constrained survivable network design problem

    INFORMS Journal on Computing

    (2013)
  • K.T. Brandes

    Implementierung und analyse verschiedener strategien zur aggregation und disaggregation von multi-cuts im benders dekompositionsverfahren

    (2011)
  • CaiX. et al.

    Solving large nonconvex water resources management models using generalized Benders decomposition

    Operations Research

    (2001)
  • C.C. Carøe et al.

    L-Shaped decomposition of two-stage stochastic programs with integer recourse

    Mathematical Programming

    (1998)
  • Chermakani, D. P. (2015). Optimal aggregation of blocks into subproblems in linear programs with...
  • G. Codato et al.

    Combinatorial Benders’ cuts for mixed-integer linear programming

    Operations Research

    (2006)
  • I. Contreras et al.

    Benders decomposition for large-scale uncapacitated hub location

    Operations Research

    (2011)
  • J.-F. Cordeau et al.

    An integrated model for logistics network design

    Annals of Operations Research

    (2006)
  • J.-F. Cordeau et al.

    Simultaneous assignment of locomotives and cars to passenger trains

    Operations Research

    (2001)
  • J.-F. Cordeau et al.

    Benders decomposition for simultaneous aircraft routing and crew scheduling

    Transportation Science

    (2001)
  • A.M. Costa et al.

    Benders, metric and cutset inequalities for multicommodity capacitated network design

    Computational Optimization and Applications

    (2009)
  • A.M. Costa et al.

    Accelerating Benders decomposition with heuristic master problem solutions

    Pesquisa Operacional

    (2012)
  • J.-F. Côté et al.

    Combinatorial Benders’ cuts for the strip packing problem

    Operations Research

    (2014)
  • T.G. Crainic

    Parallel meta-heuristic search

    Publication CIRRELT-2015-42

    (2015)
  • Cited by (528)

    View all citing articles on Scopus
    View full text