Model Predictive Control of Internal Combustion Engines: A Review and Future Directions

Abstract

An internal combustion engine (ICE) is a highly nonlinear dynamic and complex engineering system whose operation is constrained by operational limits, including emissions, noise, peak in-cylinder pressure, combustion stability, and actuator constraints. To optimize today’s ICEs, seven to ten control actuators and 10–20 feedback sensors are often used, depending on the engine applications and target emission regulations. This requires extensive engine experimentation to calibrate the engine control module (ECM), which is both cumbersome and costly. Despite these efforts, optimal operation, particularly during engine transients and to meet real driving emission (RDE) targets for broad engine speed and load conditions, has still not been obtained. Methods of model predictive control (MPC) have shown promising results for real-time multi-objective optimal control of constrained multi-variable nonlinear systems, including ICEs. This paper reviews the application of MPC for ICEs and analyzes the recent developments in MPC that can be utilized in ECMs. ICE control and calibration can be enhanced by taking advantage of the recent developments in the field of Artificial Intelligence (AI) in applying Machine Learning (ML) to large-scale engine data. Recent developments in the field of ML-MPC are investigated, and promising methods for ICE control applications are identified in this paper.

1. Introduction

1.1. Progress in Engine Control

Internal Combustion Engines (ICEs) are widely used for small power applications, such as lawnmower and string trimmers, to large applications, such as power generation and commercial transportation, such as ships, locomotives, and heavy-duty trucks [1,2,3,4,5,6,7,8,9,10]. Due to widespread and broad application, ICEs contribute more than 20% of total GHG (greenhouse gas) emissions in the world [11].

Reducing GHG emissions and improving the fuel economy of ICEs under real driving conditions are increasing challenges in the area of engine research [12,13,14]. The complexity of combustion phenomena combined with more stricter emission regulations and higher fuel economy demands requires more advanced engine controller. The improvement of micro controllers and the availability of online optimization methods allows the automotive industry to utilize even more advanced control methods.

In automotive applications—especially in ICE control—feedback control often couples with feedforward control to deal with the influence of varying operating points. One of the most common techniques to design a feedforward controller is two-dimensional look-up tables—so-called calibration maps. The feedforward controller enables fast changes in operating points, while the feedback controller performs the error compensation. The conventional controller type for feedback controller is a Proportional Integral Derivative (PID) controller.

The gains of PID controllers are tuned using the procedure of parameter optimization and fine tuning using the trial-and-error method. The optimization process results in finding look-up tables, and controller gains are usually referred to as engine calibration [15,16]. Due to the demands mentioned above on low-fuel consumption and emissions, the number of control inputs have increased substantially, making manual test-bench calibration extremely difficult and time-consuming.

However, systematic optimization methods based on a model developed and identified using experimental results are ideal ways to tackle this problem. Several model-based controllers have been used in engine feedback control to address this, such as the Linear Quadratic Regulator (LQR) controller [17], Linear Quadratic Gaussian (LQG) controller [18], Sliding Model Controller (SMC) [5,19,20], Adaptive [20], and Model Predictive Control (MPC) controller [21,22].

Among these mode-based controllers, MPC is one of the most promising controllers that can deal with the highly constrained nonlinear system of ICEs. MPC can provide an optimal real-time solution for meeting multi-objective goals while addressing system and operational constraints. New variants of MPC utilize optimization solvers and packages that are suitable for the real-time operation of time-critical systems [21,22].

Model-based engine control techniques have been applied to ICEs for over five decades [23]; however, conventional MPC techniques have been applied for ICE applications over the past 23 years. Two examples of early MPC on ICEs include: (i) air-fuel ratio (AFR) control of an SI gasoline engine using a linear AFR model by linear approximation of a neural network model [24] in 1998, and (ii) idle speed control of an SI gasoline engine using a linear model by applying system identification techniques on GT-Power engine model simulations [25] in 1999.

These early works were done in simulation environments, while recent work [26] includes experimental implementation of nonlinear multi-objective MPC on a real engine. MPC has been successfully implemented for ICEs control with an increasing trend [21,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54]. The integration of Machine Learning (ML) and MPC is a new emerging area that provides additional opportunities for the control and optimization of ICEs.

1.2. Rationale for Using Model Predictive Control (MPC) in ICE

Advanced ICEs exhibit highly nonlinear and stochastic dynamic behavior due to complex thermo-kinetic reactions coupled with nonlinear turbulent in-cylinder flow dynamics causing engine cyclic variability. These factors make the design and calibration of controllers for ICEs a challenging and time consuming task that lead to trade-offs that limit the performance and robustness of these engines. For instance, the current ECM for a compression ignition engine often has over 12,000 calibration parameters. The controller calibration and validation process for this ECM is time- and labor-intensive and can cost several million dollars.

Despite extensive controller calibration, the optimum and robust engine performance for a broad operational range cannot be guaranteed. In addition, the engine control needs to be coordinated with other control modules in a vehicle, e.g., the transmission control unit (TCU), anti-jerk control, vehicle stability control, etc. This makes the engine control a constrained, multi-objective, multi-variable, optimal control problem that needs to be solved with a millisecond timescale to allow cycle-by-cycle or within-cycle engine combustion control.

MPC is a control technique that has been increasingly used in industry during the past four decades due to the following five main advantages: (1) implicitly considers constraints on state, input, and output variables, (2) provides closed loop control performance and stability for the optimal problem with constraints, (3) exploits the use of a future horizon while optimizing the current control law, (4) offers the possibility of both offline and real-time implementations, and (5) provides the capability to handle uncertainty in the system’s parameters, delays, and non-linearity in the model [55].

MPC can be employed for different purposes, which are generally categorized into four main groups: setpoint stabilization, trajectory tracking, path following, and economic operation [56]. A survey of MPC shows that it is one of the most common control approaches. The superiority of the MPC controller over classical PID controller is well documented [57].

1.3. MPC Background

1.3.1. A Short History of MPC

The idea of using MPC begun around 1960s [58]; however, the first reported application of MPC in industry was in 1978 [59]. Then, after the initial applications of MPC in the late 80s, MPC usage grew rapidly in several industries. In particular, the process industry was an early adopter of MPC as it was able to handle both input constraints and states constraints. Some of these processes were slow enough to allow MPC implementations with the processors of that time.

A survey in 1997 estimated 2233 applications of MPC from five different vendors [60]. A graphical depiction of MPC development and implementation is shown in Figure 1. Increased interest in MPC stability and robustness started in the early 1990s [60]. At the same time, multiple algorithms were developed to control the nonlinear systems using MPC [61]. Starting at about the year 2000, new approaches began to be developed. For example, hybrid MPC, which considers both continuous and discrete variables [62], and also explicit MPC [63].

In explicit MPC, the goal is to perform all computations offline for all the feasible inputs and then implement the optimal control law using a lookup table stored in computer memory. The control law trades off computation with computer memory. Explicit MPC becomes untenable when the number of states or variables increases. To overcome this problem, Fast MPC was proposed to solve the MPC problem implicitly but much faster than the earlier algorithms [61].

These advances in solving the MPC problem with low computational cost and improving MPC performance have expanded the applications of MPC from the process industry to the other industries, such as manufacturing, automotive, power and energy system, aerospace, healthcare, and finance [61]. Increasing processor speeds has also allowed the application of MPC to control nonlinear and nonconvex systems in real time [64].

1.3.2. Terminology

MPC uses a receding horizon to minimize a cost function to calculate the optimal control inputs for a finite control horizon. Over the finite prediction horizon the cost is minimized with respect to dynamics of the system, current states of the system, and constraints applied to the system. From these calculated control inputs, only the first step is applied to control the system output, and then, for the next time interval, MPC repeats the same process.

A schematic about MPC operation in an ICE application is depicted graphically in Figure 2 where the Indicated Mean Effective Pressure (IMEP) is controlled using the injection fuel quantity as control variable. In this figure, $H_u$ is the control horizon, and $H_p$ is the prediction horizon. The prediction horizon is longer than the control horizon with more computational costs for longer horizons [65].

In ICEs, the control horizon could be one cycle for an ICE in highly transient operation in a conventional vehicle and could be three or more engine cycles for operation in a hybrid electric vehicle for the operating mode that the ICE is decoupled from the road load conditions. MPC formulation can be defined as:

$$\begin{array}{cc}\hfill \underset{u_0,\dots ,u_{N-1}}{min}& J_f\left(x_N\right)+\sum _{k=0}^{N-1}J(x_k,y_k,r_k,u_k,s_k)\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& x_{k+1}=f(x_k,u_k,d_k),y_k=g(x_k,u_k,d_k)k\in {\mathbb{N}}_0^{N-1}\hfill \\ \hfill & x_k\in \mathcal{X},u_k\in \mathcal{U}k\in {\mathbb{N}}_0^{N-1}\hfill \\ \hfill & X_N\in {\mathcal{X}}_f{x}_0=x\left(t\right)\hfill \end{array}$$

(1)

where x, y, u, r, s, d, and N represent states, outputs, inputs, references, slack variables, disturbances, and the prediction horizon, respectively. In this equation, J, f, g, $\mathcal{X}$, $\mathcal{U}$, and ${\mathcal{X}}_f$ represent the state function, output function, cost function, state constraint set, input constraint set, and terminal state constraint set, respectively.

1.3.3. Methods

The broad range of industry applications of MPC control has resulted in different MPC methods being developed. The selection of MPC methods depends on the solution method, uncertainties in the system, dynamics of the system, and scale of the system. MPC types are categorized based on these criteria and are shown in Figure 3.

Economic MPC uses economic objectives directly as the cost function of the control process, and thus instead of tracking a set point based on a given economic solution, the economic performance is optimized in real-time [90]. This combines two layers of the control system. The upper layer, Real-Time Optimization (RTO), computes the economically optimal solution using a steady state model of the process, while the lower control layer considers the economics of the process [91]. Using economic MPC over MPC can enhance the efficiency and performance of the controller [91].

Explicit MPC is used to speed up the optimization process by pre-computing the control law offline. This requires solving a multi-parametric Quadratic Programming (mpQP) problem and then replacing the online optimization with a simple linear function to find the corresponding region for the given states, reference, and disturbance while maintaining MPC characteristics [63,92]. Explicit MPC can reduce the online computational cost of optimization, which can speed up the control law calculation for a problem with smaller number of dimensions. For problems with large dimensions, the computational cost and memory footprint increase rapidly and can be prohibitive [93].

To allow MPC to be implemented on systems with a fast dynamics controller, fast MPC was developed. In fast MPC, customized algorithms are used to solve the optimization problem online much faster than other MPC methods [89]. The computational cost and complexity of fast MPC is highly dependent on the application—the linearity and convexity of the system. If the application is nonlinear and nonconvex, the solution is more difficult and computationally more expensive.

To ensure closed loop performance and stability, despite system uncertainties, robust MPC was developed [71]. One common approach in robust MPC is using a Min-Max algorithm. Here, the worst-case scenario among all admissible solutions is considered. In another approach, the constraints can be rewritten in the form of linear matrix inequalities (LMIs), and this can be used to limit the worst-case performance of the system under bounded uncertainty [76].

Robust MPC can increase the robustness of MPC for unknown and bounded disturbances and uncertainties in the model. However, when some knowledge about the disturbances or uncertainty of the model is available, this approach does not take this into account. Stochastic MPC was developed to considering the possibility of some optimality or economic benefits, which can be gained by moving towards or violating those disturbances’ boundaries [77,94]. Stochastic MPC can handle states and model parametric uncertainties and independent disturbances by employing the information about the mean and variance of the prediction states, parameters, and disturbances to make sure that possible violation of the constraints remain admissible relative to a predefined threshold [78].

A hybrid system consists of discrete and continuous states, and hybrid MPC has been developed for this application [73]. Mixed Logical Dynamical systems (MLD), piecewise affine systems, and Discrete Hybrid Automata (DHA) are frameworks that can be used to model hybrid MPC. A common approach in hybrid MPC is to reformulate the problem as a Mixed Integer Nonlinear Programming (MINLP) (or, in the case of a linear system, Mixed Integer Quadratic Programming (MIQP)) and then solve the optimization problem using numerical methods [74].

MPC was developed to gather all the information about system dynamics and its variables at a single location and then perform all the computation and optimization. This approach is not feasible on large-scale systems, such as water networks, urban traffic design, and power grids, due to the computing and data gathering requirements [95]. To overcome this, techniques, such as decentralized MPC [71], distributed MPC, or hierarchical MPC [72], can be employed.

1.4. Scope of the Paper

This paper builds upon our prior experience for the design and implementation of model-based ICE controllers [2,4,5,7,21,22,46,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130]. In this review paper, we searched for relevant papers in the field of MPC and ICE using the Scopus, Web of Science, IEEE Xplore, ScienceDirect, Springer, SAE, ASME, Wiley, Taylor & Francis, arXiv, and SAGE databases. Then, papers were carefully reviewed, and key publications were identified.

The previous review papers were not focused on internal combustion control specifically, and most of them focused on the general application of MPC [131] or MPC application in automotive applications, including vehicle dynamics control [132] and thermal management systems [132,133,134]. In this paper, we perform a comprehensive review of research articles in the fields of MPC for ICE controls. First, the MPC applications and design methods are analyzed. Real-time implementation is investigated based on categorizing by optimization and solver types. Finally, recent developments of MPC using Machine Learning (ML) for ICE applications are discussed.

This paper is organized into six sections. In the “Introduction”, the definition of MPC and a brief history is provided. In “MPC applications in ICEs”, the structure of MPC along with methods and design of MPC for ICEs are presented. The “Real-time Implementation of MPC” section focuses on the implementation of MPC in ICEs. The “AI and MPC integration” section provides examples of AI and MPC integration for ICE applications. Finally, the main conclusions and recommendations are detailed in the “Recommendation and Future Directions” and “Summary and Conclusions” sections.

2. MPC Applications in ICEs

MPC has been used in ICE control for a wide variety of ICE control problems. Among these control problems, MPC has been used widely for the control of fuel consumption [28,30,33,34,35,36,52,54,135,136,137], combustion phasing [21,22,27,28,37,38,39,52,54,108,129,130,138], cyclic variability [54,130], torque and load (IMEP) [22,27,28,33,37,38,39,40,41,42,52,54,108,129,130,136,137,138], idle speed [25,36,43,139,140], airpath (P${}_{man}$, EGR) [29,30,31,32,34,40,44,45,46,47,48,49,50,52,54,137,141,142,143,144], knock and Maximum Pressure Rise Rate (MPRR) [22,54], engine-out emissions [26,28,30,35,41,47,51,137,145], exhaust after treatment [29,145,146], multi-mode operation [53], and waste heat recovery [147,148,149].

MPC applications are not limited to a specific engine type, and, due to the capability of MPC, they have been implemented in various types of engines. MPC has been implemented for different ICE combustion modes, including Compression Ignition Combustion (CI) [26,28,29,30,31,34,35,40,45,47,48,49,50,53,135,137,139,141,142,143,145,147,148,150], Spark Ignition (SI) [25,33,36,41,42,43,51,52,54,136,138,140,144], and Low Temperature Combustion (LTC) [21,22,27,37,38,39,108,129,130].

MPC has also been implemented in wide varieties of engine size ranging including a single cylinder [37,108,138], four cylinder [21,22,25,27,35,42,45,51,52,53,129,130,137,139,140,143], six cylinder [28,29,30,54,135,147], and eight cylinder [36,43,149] with different displacement volume ($V_d$) including $V_d$≤ 2.0 [21,22,27,28,37,53,108,129,130,137,138,143], 2.0 $<V_d\le$ 7.0 [25,30,31,35,36,38,39,41,42,45,51,52,54,139,142], and 7.0 $<V_d\le$ 15.0 L [29,40,135]. In addition, MPC is widely used in different application of ICEs including light duty engines [21,22,25,26,27,35,37,38,39,41,42,45,51,52,53,54,108,136,137,138,139,140,142,143,144], medium duty engines [34,36,43,47,48,147,149], and heavy duty engines [28,29,30,31,40,135,147].

3. MPC Designs for ICEs

In this part, first, different topologies of applying MPC into engine control are discussed, and then MPC methods and ICE models for the design of MPC in different topologies of ICEs are discussed.

The main benefit of MPC is providing real-time optimal solution to a constrained, multi-objective, multi-variable control problem. Thus, MPC is a natural choice for use as a supervisory ICE controller to set optimal trajectories, e.g., optimal CA50, EGR, or AFR trajectories to constrain engine-out emissions while minimizing fuel consumption. However, MPC has been also widely used as a tracking controller, e.g., tracking reference IMEP, CA50, or engine idle speed trajectory. MPC has been also used in ICEs to provide a combination of supervisory and tracking control functions.

3.1. Topology/Structure

Depending on how MPC is used in the engine control hierarchy, three different topologies can be identified in the existing literature. Schematics of these three topologies are shown in Figure 4.

The first and second typologies (Figure 4a,b) include three main control levels:

• Level 1 (L1A): Optimizer/supervisory controller.
• Level 2 (L2A): Feedback optimal tracking controllers.
• Level 3 (L3A): Actuator controllers.

The third typology (Figure 4c) includes two main control levels:

• Level 1 (L1B): Combined supervisory and feedback controller.
• Level 2 (L2B): Actuator controllers.

Level 1A acts as an optimizer to design the reference for tracking controllers in level 2A. Examples of objective functions in Level 1A include minimizing fuel consumption and/or maximizing use of natural gas or hydrogen fuel in a dual fuel engine. Examples of optimization constraints include the maximum allowable engine-out NOx, soot, uHC emissions, MPRR, peak in-cylinder pressure, combustion stability limit (COV${}_{\mathrm{IMEP}}$), and drivability (e.g., the maximum allowable deviation from the driver requested torque). The output of the MPC supervisory controller in level 1A will depend on the engine type and application and could include optimal reference trajectories for AFR, CA50, idling speed, desired heat release shape, or desired combustion mode in a multi-mode engine.

Level 2A includes feedback tracking control to follow the generated optimal reference from level 1A. In general, these tracking controllers can be MPC or non-MPC controllers. For cases using MPC, minimizing the tracking error is typically part of the cost function and the optimization is subject to engine operational constraints and actuator constraints. The MPC will determine the optimal control action for low-level actuator controllers (level 3A/2B).

Depending on engine control applications, MPC may command optimal values for EGR percentage, variable geometry turbine (VGT) level or waste-gate position, valve timing, throttle position, fuel injection quantity, fuel injection pressure, number of injections, injection timing, and the duration of each injection. Level 3A/2B often includes PI or PID controllers to implement the requested control values and adjust engine actuators, such as VGT, EGR, exhaust back pressure flap, variable valve actuation (VVA), throttle body, DI injectors, and the fuel pump.

Depending on the availability of computational resources, different topologies have been used. The most common structure is topology I with three divided control levels (Figure 4a). Since the supervisory and feedback controllers are in a separate loop, low computational resources are required. In topology II (Figure 4b), injection control variables are determined by the supervisory controller in level 1A, while airpath control is done in level 2A. This has been used in engines where in-cylinder pressure measurement for real-time combustion feedback was not available for engine load and combustion phasing controllers. Thus the level 2A controller focuses on airpath feedback control for which air flow measurement, compressor air pressure, and intake manifold pressure measurements are available for real-time feedback controllers.

The last and most complex structure is topology III (Figure 4c), where both airpath and fuel path controls are conducted by a combined supervisory and feedback controller (Level L1B) and actuator control (level L2B). This topology will require the highest amount of computational resources, but it can provide the most optimal control actions since all dynamics, including the system dynamics, feedback control dynamics, and actuator dynamics, are taken into account in a unified control layer while optimizing the engine performance.

All three topologies include the actuator control level, which can be implemented using MPC or some other control methods. It is critical that the limitations of the actuators response are considered in level 1B of topology III or levels 1A and 2A of topologies 2 and 3; otherwise, the engine will operate in an suboptimal manner and may even become unstable.

All three of these topologies have been implemented experimentally or in simulation in the literature. Most studies focused on the topology I—level 2A, i.e., optimal tracking controllers, and these studies assumed that the optimal reference values were given. Implementation of L2A of topology I has been done both in simulation [22,25,108,129,130,136,137,138,139,140,141,142,143,144,145,147,148,149,150,151] and in real-time experimental platforms [21,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51].

For instance, both L1A and L2A have been done for IMEP control of SI engines with external EGR, while minimizing fuel consumption; constraining COV${}_{imep}$, and knock intensity in L1A and tracking of CA50, Manifold Absolute Pressure (MAP), and Mass Air Flow (MAF) in L2A [54]. Another example includes a three-level hierarchy control with focus on Level 2A [30]. Schematic of supervisory (L1A), fuel path and air path feedback controllers (L2A) are shown in Figure 5. In this study, explicit MPC is used to regulate ${\mathrm{NO}}_{\mathrm{x}}$ and PM emissions, while the controller ensures the engine follows the requested torque.

The L1A controller sets the optimal air-fuel equivalence ratio and the optimal dilution level to regulate emissions. Then, the L2A controllers will determine EGR level, variable geometry turbine (VGT) position, injection pressure, start of injection timing, and pilot fuel/total injected fuel ratio.

Few articles have focused on topology II [26], and III [135] to generate comprehensive optimal control actions (to achieve level 1B). This is partly due to the complexity and required computational resources in topology II and III on an actual engine. The example of topology II was published in a recent study [26] and is shown in Figure 6. In this study, a supervisory MPC controller (L1A) sets both the optimum EGR rate and fueling rate while an MPC feedback controller (L2A) tracks those set points. The airpath L2A controller adjusts the engine EGR level by commanding the required throttle angle, EGR valve position, and VGT position.

As discussed, topology II was implemented in a real-time engine application [26]; however, to date, no experimental implementation of topology III was found in the literature. The only example of topology III included the implementation into processor-in-loop (PIL) platform [135]. The schematic of this example is shown in Figure 7 where the supervisory MPC controller generates optimal control action to control output torque and emission level (NOx and Soot). In this schematic, both the fuel path, including the fuel rail pressure ($P_{inj}$) and SOI, and the air path, including VGT, EGR, and EF (Exhaust flap), are optimized by the supervisory controller.

3.1.1. Methods of MPC in ICEs

Different MPC methods, previously discussed in the “MPC Background” section, have been applied to ICEs. The three main MPC methods that have been implemented on ICEs are Linear, Economic, and Nonlinear MPC as shown in

Table 1. Types of MPC controllers used in the literature that are demonstrated for ICE controls.

MPC Types		Experimental	Simulation	Processor-In-Loop (PIL)
Linear	Explicit	[30,38,39,40,43,45,47,50]	–	–
	Single	[24,28,36,37,40,49]	[108,142,145,151]	–
	Switching	[27,29,33,46,53]	[147,149]	–
	LPV	[21,29,42,48]	[22,129,143,150]	–
Economic	Linear	[35]	–	–
	Nonlinear	[52,54,136]	[137]	–
Nonlinear	–	[26,32,34,41,51]	[130,138,139,140,141,144,148,150]	[135]

. Due to the limited computational capability of engine ECU, linear methods, such as Explicit MPC [30,38,39,40,43,45,47,50], single MPC [24,28,36,37,40,49,108,142,145,151], switching MPC [27,29,33,46,53,147,149], and linear parameter varying (LPV) MPC [21,22,29,42,48,129,143,150], have been widely implemented in ICEs.

Most linear methods have been implemented experimentally, and their control performance has been verified on real ICEs [21,24,27,28,29,30,33,36,37,38,39,40,42,43,45,46,47,48,49,50,53]. These include implementation on mass production ECU [33] and prototype ECU [21,24,27,28,29,30,36,37,38,39,40,42,43,45,46,47,48,49,50,53] for broad engine applications. Recent developments include the implementation of real-time nonlinear MPC for emission control of a light-duty diesel engine [26,32,34,41,51]. Using newly developed solvers along with real-time iteration scheme and symbolic code generation tools, estimated computational time under 8 ms for real-time prototyping was achieved [26].

3.1.2. Dynamic Models for MPC

A core part of an MPC structure includes the dynamic model that is used to predict the system performance over a prediction horizon. The system dynamics are often included as equality constraints in an MPC formulation. Typically, MPC has high sensitivity to model uncertainty, and classical MPC performance will significantly deteriorate in the presence of a large model uncertainty. The three most important factors for selecting models for MPC are: (i) accuracy in predicting the required states, (ii) linear vs. nonlinear and the type of nonlinearities in the model, and (iii) convex vs. non-convex and the possibility of convexification.

These three factors will directly affect the MPC performance, computational cost, and solver type for solving the optimization problem. Often, these three factors must be trade-off against each other. For instance, one can choose a detailed non-convex model to predict engine-out soot emissions accurately; however, this will lead to high computation cost and will likely require a more complex solver.

On the other hand, one can create a linearized model or develop multi linear models to predict soot emission. This will result in a less accurate model that can lead to suboptimal MPC control actions, but it will be computationally efficient and can use a common Quadratic Programming (QP) or Sequential Quadratic Programming (SQP) solver. Developing appropriate dynamic predictive models for MPC is often a large part of the development effort when designing MPCs for ICEs.

Classification of the ICE models that have been used for model predictive control of different ICE control problems is shown in

Table 2. Classification of ICE control problems based on the engine model and optimization problem.

Problems	Control Applications
Multi linear	Waste heat recovery [147,149]
	Burn rate/combustion phasing [21,22,27,28,37,38,39,108,129]
	Torque and IMEP [22,27,28,33,35,37,38,39,40,42,108]
	Idle speed [25,36,43,140]
Convex or convexifiable	NOx emission [26,28,30,137]
	Exhaust aftertreatment system [145]
	Combustion mode transition [53]
	Airpath of turbocharged engines [32,34,52,141,144]
Non-Convex	Cyclic Variability [54,152]
	Soot and uHC emissions [153,154]
	Knock and MPRR [155,156,157]

. Often, ICE dynamic models are nonlinear and nonconvex. However, provided that acceptable performance by the MPC feedback controller is obtained, some of the ICE nonlinear models can be linearized or represented by multiple linear models at different engine speeds and loads. In addition, some of the nonconvex ICE models can be convexified, and this will lead to more flexibility for selecting solvers and also obtaining globally optimal solutions.

Emission models, such as soot, uHC, and CO, have a complex structure, and model-based emission regulation usually results in a non-convex problem. Knock and maximum pressure rise rate, and cyclic variability are also problems that result in the nonconvex optimization problems. Fortunately, airpath turbocharged engine, NOx emission, combustion mode transition, and exhaust after-treatment systems are often convex or convexifiable, thus, making it possible to use a convex solver.

Multilinear problems, developed using either LPV or piecewise modeling, result in linear MPC for which there are a wide variety of effective solvers to choose from. Waste heat recovery, burn rate, combustion phasing, airpath of turbocharged engines, torque, IMEP, and idle speed controls have been successfully represented as multilinear problems in the literature (see Table 2).

Engine dynamics have different time scales as shown in Figure 8. The response time of the controlled dynamics is a critical factor in design of the appropriate MPC framework. Depending on the control target in an ICE, different phenomena with different time scales need to be considered. This affects the structure of MPC model, and also the selection of the control horizon and prediction horizon, which could be either fixed or time-varying based on the engine speed and load.

Figure 8 shows fuel-air mixing, combustion knock, in-cylinder residual gas fraction and temperature. In-cylinder emission formation is considered to have fast dynamics and typically results in a non-convex optimization problem. Most of the subsystems that involve thermal dynamics have a relatively slow dynamic (more than a second) as shown by red color in Figure 8. Most thermal dynamics result in either convex or convexifiable problems as well as multi-linear problems.

The blue color in Figure 8 represents flow dynamics that are also a convex problem or convertible to a convex problem. Depending on the ICE control problem and the time scale of associated dynamics, a coupled or decoupled MPC model structure should be chosen. If, in a control problem, the subsystems have substantially different time scales, the problem can be solved decoupled for simplicity; however, for systems with overlapping response times, a coupled system needs to be considered for the MPC model.

For example, as shown in Figure 8, NOx formation and in-cylinder residual gas fraction and temperature have similar response times; therefore, cycle-by-cycle control of the NOx control problem must be coupled with engine dynamics. However, the in-cylinder residual gas fraction and temperature can be decoupled from the cylinder wall temperature as they have a substantially different response time. In this case, a decoupled MPC model can be used for simplicity for cycle-by-cycle engine combustion control.

An important factor for selecting control horizon is knowing the engine actuator dynamics. Typical engine actuator response times are shown in Figure 9 where the ignition coil in SI engines and direct fuel injectors in both GDI and CI engines have the fastest actuator dynamics. Slower actuators, sorted from the fastest to the slowest, include the cam phaser, fuel pump, throttle valve, wastegate and EGR valve. Consideration of these response times is critical for cycle-by-cycle engine control particularly at high engine speeds. For instance, an engine cycle in a single-cylinder four-stroke engine running at 6000 RPM takes only 20 ms, while most control actions need to be implemented within 1–2 ms.

Equally important to the response time is the structure of the model, i.e., the number of states and control inputs, since the structure directly affects the MPC computational cost and provides feasible optimal solutions in real-time. The classification of engine-related modeling based on number of states is shown in Figure 10 for a large number of prior ICE control studies. The problems are divided into three main classes based on number of states: two to three states, four to six states, and seven to nine states, and then sorted further based on the number of control input (u).

Experimentally demonstrated MPCs in the ICE literature are mostly done for the problems with the maximum of six states. The studies with seven to nine states are usually only demonstrated in the simulation and include offline optimization selection of number of system states and control inputs depend on (i) ICE control application, (ii) MPC topology and centralized versus decentralized approach, and (iii) available computation power and memory in ECU.

4. Real-Time Implementation of MPC

This section focus on the experimental implementation of real-time MPC controllers on real engines.

4.1. MPC Optimization Methods

MPC is realized by numerically solving an Optimal Control Problem (OCP) at each time step. The three common approaches to solve an OCP problem are: Dynamic Programming (DP), the Indirect Method, and the Direct Method [158]. The Hamilton–Jacobi–Bellman (HJB) equation can be formed by minimizing a cost function, and it provides the rule for defining the optimals of a continuous time system using DP. Due to the curse of dimensionality, DP is not applicable for the problems with a large number of states and control variables.

The indirect method uses the Pontryagin Minimum Principle and considers boundary conditions and equality constraints in the form of the Two-Point Boundary Value Problem (TPBVP). Indirect methods cannot handle inequality constraints smoothly [158,159]. Finally, the direct method forms a Nonlinear Programming (NLP) by first discretizing the OCP using single shooting, multiple shooting, or collocation methods. For the MPC context, the direct method is the most common approach for transforming OCP to an optimization problem that can handle inequalities and provides a robust solution [159].

Constrained or unconstrained optimization is one way to categorize the optimization problem. Convexity is another metric that can be used to categorize optimization problems. Convex problems are normally easier to solve with well-established algorithms to find the global optima. For nonconvex problems, it is difficult to find a general algorithm for solving the problem, and normally finding the global solution cannot be guaranteed.

For nonconvex problems, the feasibility and optimality of the solution highly depend strongly on choosing an appropriate initial point. Further, when solving a nonconvex problem, local minima, saddle point, flat region, and widely varying curvature of the function cause difficulties [160]. An optimization taxonomy is presented in Figure 11.

Numerical optimization can also be categorized based on input or state variables. In some cases, the variables can only take integer values, while, in other cases, problems can take continuous variables, and finally there is a possibility of having both types of variables in a problem. The first case forms discrete optimization, the second is continuous optimization, and the third is called the mixed integer programming problem [160]. Another category could be based on uncertainty in the system. Although most of the optimization problems use a deterministic approach and assume that all the variables of the system are well known, some optimization methods, such as robust optimization and stochastic programming, build uncertainty into the optimization formulation [160].

4.2. Non Linear Programming (NLP)

NLP is the most general form of constrained optimization. A NLP can be written as:

$$\begin{array}{cc}\hfill min& f\left(x\right)\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& g_i\left(x\right)\le 0\mathrm{for}i=1,\dots ,m\hfill \\ \hfill & h_i\left(x\right)=0\mathrm{for}i=1,\dots ,l\hfill \\ \hfill & x\in X\hfill \end{array}$$

(2)

If the cost function and constraints are linear, the NLP transforms into a Linear Programming (LP). A quadratic cost function with linear constraints is a Quadratic Programming (QP) problem, and a quadratic cost function and quadratic constraints define Quadratically Constrained Quadratic Programming (QCQP) [161]. In Semi-Infinite Programming (SIP) problems, many finite variables are subject to infinite inequality constraints. This type of NLP can arise in robust optimization [162].

Second Order Cone Programming (SOCP) problems are part of convex optimization problems that minimize a linear cost function with respect to intersection of affine linear constraints with second-order cones [163]. SOCP can be considered as a subset of Semi-Definite Programming (SDP), which minimizes a cost function over the intersection of affine constraints and the cone of positive semidefinite matrices. Even though LP and QP can be reformulated as SOCP problem, the computation cost of SOCP problems is higher than LP and QP problems [163].

Mixed Integer Nonlinear Programming (MINLP) is another class of NLP that has both integer and real variables in its cost function or constraints. Due to nonconvexity, which can occur in these types of problems, they are computationally expensive to solve [161].

To solve NLP problems, a variety of algorithms can be used. Some of the most prevalent methods are: First Order Methods (FOM), Active Set (AS) methods, Interior Point (IP) methods, and Sequential Quadratic Programming (SQP) methods. For FOM, alternating direction method of multipliers (ADMM), gradient methods, and forward backward newton (FBN) use the gradient of the function as opposed to using the Hessian to solve the optimization problem [164]. These methods are often simple to implement, but they can suffer from slow convergence and are sensitive to problem scaling.

Active Set (AS) methods solve a sequence of equality-constrained quadratic sub-problems. They attempt to find set of equality constraints that are satisfied at the solution of the problem. The classical approach for AS methods considers two phases. The first phase is focused on feasibility, and the second one emphasizes optimality [165]. These methods are simple to implement, fast for the small to medium size QPs, and can benefit from a warm start when they are used in SQPs. Due to the existence of exact complexity certification for these methods [166,167], they are an excellent candidate for fast embedded MPC applications [168].

Interior Points (IP) methods consider iterations that fall inside one of the feasible regions based on the constraints instead of considering the boundary of these regions. IP methods follow this path until they find the optimal solution [165] and can find the solution in less iterations compared to active set methods. The computational cost of IP is higher per iteration due to more complex linear algebra operations needed [165]. A warm start is also difficult to implement for IP methods, as the warm start point could be far from the path they follow [165].

SQP methods linearize the cost function of NLP and solve a quadratic programming subproblem in each iteration. This is performed so that the sequence finds the local minimum of the original NLP [160]. To reduce the computational cost of SQP, some algorithms limit the number of iterations and return a suboptimal solution [169]. SQP methods employ either AS or IP methods to solve the QP at each iteration. Generally SQP methods are faster than IP methods; however, they can end up with a suboptimal solution [169].

Commercial or open source solvers are available to solve NLP problems. A list of common solvers, which can be used for MPC is given in

Table 3. Common solvers for MPC along with the application type, optimization method, and availability of open source).

Optimization Solver	MPC Problem Type	Optimization	Commercial/Free
IPOPT * [171]	LMPC, Convex NMPC, Nonconvex NMPC	IP	F
BARON [172]	LMPC, Convex NMPC, Nonconvex NMPC	IP, SQP	C
Gurobi [173]	LMPC, Convex NMPC, Nonconvex NMPC	IP	C
ForcesPro [169,174]	LMPC, Convex NMPC, Nonconvex NMPC	IP, SQP, FOM	C
SNOPT * [175]	LMPC, Convex NMPC, Nonconvex NMPC	IP, SQP	C
OOQP [176]	LMPC	IP	F
duQuad [177]	LMPC, Convex NMPC	FOM	F
HPIPM [178]	LMPC, Convex NMPC	IP	F
qpOASES * [179]	LMPC, Convex NMPC, Nonconvex NMPC	SQP	F
OpEn [179]	LMPC, Convex NMPC, Nonconvex NMPC	AGD	F
WORHP * [180]	LMPC, NMPC	SQP, IP	C
CPLEX [181]	LMPC, Convex NMPC		C
CVXOPT [182]	LMPC, Convex NMPC	IP	F
KNITRO * [183]	LMPC, Convex NMPC, Nonconvex NMPC	IP, SQP	C
Mathworks MPC Toolbox * [184]	LMPC, NMPC		C
FiOrdOs [185]	LMPC	FOM	F
APOPT [186]	LMPC, Convex NMPC	SQP	F
LOQO [187]	LMPC, Convex NMPC, Nonconvex NMPC	IP, SQP	C
OnRAMP (Honeywell) *	LMPC, Convex NMPC, Nonconvex NMPC		C
YALMIP * [188]	LMPC, Convex NMPC, Nonconvex NMPC	IP, SQP	F
ACADO * [189]	LMPC, Convex NMPC, Nonconvex NMPC	IP, SQP	F

* Implemented in ICE application.

. To help select a solver for a specific type of problem, , ref. [170] provided a decision tree for optimization software and benchmarks for different solvers.

To mathematically formulate the optimization problem and solve an optimal control problem, many software tools are available. One of the free and open-source software programs that easily integrates with MATLAB^® and Python for nonlinear optimization and algorithmic differentiation (AD) is CasADi, which has been used for solving NMPC in ICEs [137,138]. CasADi is an open-source software that uses most of mentioned solvers in Table 3, and it has been used in NMPC [190]. Other well-known software includes CVXOPT [191], YALMIP [188], ACADO [189], and ForcesPro [174].

4.3. MPC Implementation

MPC control on ICE engines has been extensively studied using simulation. However, for implementation of MPC controllers on a real engine, only a limited amount of published work can be found. MPC has been implemented in a real-time system in various universities, industry, and service providers. A summary of studies that have experimentally implemented MPC controller on ICE engines is shown in Figure 12.

Among these studies, only General Motors deployed mass production of engine controller using MPC. Due to the recent developments in microprocessors and availability of efficient solvers, it is anticipated that MPC will be implemented in other ICE industries in the near future. Other than GM, other companies, such as Cummins, Toyota, Ford, Fiat Chrysler Automobiles (now Stellantis), and Hyundai Motor have implemented MPC for ICE operations under real-time transient conditions.

Due to the fast dynamics of engines, explicit MPC and fast MPC are the two MPC methods that have been often used for real-time implementation of engine studies on prototype ECU. Explicit MPC has the advantage that it has lower computational requirements but has the disadvantage of a high memory footprint. For a typical ECM, systems with more than five states are impractical to implement with explicit MPC due to the computational expense and parameter tuning difficulty. On a real engine, different solvers (optimizers) have been employed to apply MPC.

Figure 13 show the solvers that have been used in the literature for implementing MPC experimentally on spark ignition (SI) engines and compression ignition (CI) engines. For SI engines, SQP is used more often for different control problems, including IMEP, fuel, airpath controller [52,54,136]. In CI engine, mpQP is used for fuel path [21,28,30,40], and airpath [21,30]. OnRAMP solver is used for after-treatment controller [29,147] and for engine combustion optimization broad type of IP, and SQP-based solvers, including IPOPT, WORHP, KNITRO, and SNOPT, are implemented in the literature [135].

For most studies, a single linear MPC or a set of linear MPCs have been used to model the dynamics of the engine [21,27,28,29,30,33,147]. This simplification in defining the model enables an MPC controller to be implemented in mass production embedded processors [33]. Conversely, nonlinear MPC is used to capture the non-linearity of the system more accurately but is more complex [26,31,32,135,141].

In a recent study, a nonlinear MPC was implemented on a prototype CI engine controller that could be implemented in a production ECM [26]. This study is an example of topology II that includes two levels of control: Level 1A as supervisory NMPC (SNMPC) and Level 2A as optimal feedback controller. The main objective of SNMPC is safety enforcement by guaranteeing fuel input and EGR rate upper band to prevent engine damage and satisfying fuel-air ratio constraints for limiting smoke in engine transients.

The level 2A controller includes both feedforward and feedback parts to address both high performance and robustness to disturbance and uncertainties to track the supervisory provided EGR rate and intake manifold pressure targets as shown schematically in Figure 6. These three controllers are implemented on a dSPACE DS1006 rapid prototyping unit.

All necessary derivatives of three optimal control problems, including cost function, constraints, and dynamics, were calculated symbolically using the Maple symbolic language. Then, a symbolic control design environment (SCDE) was used to translate these symbolic calculations into highly optimized C code to embed in MATLAB^®/Simulink S-function. The QP solver for supervisory NMPC was then embedded in MATLAB^®. Feedback/feedforward control was Cholesky factorization routines, which were directly implemented in SCDE.

They scaled the execution time of MPC to a standard ECM, as shown in

Table 4. Problem size and execution time for MPC controllers reported in [26]—ECU execution times are estimated for a 256 MHz ECU clocking scaling.

	Supervisory SMPC	NMPC Feedforward	NMPC Feedback
Average execution time [$\mathsf{\mu }$s]	530	31	127
Maximum execution time [$\mathsf{\mu }$s]	550	32	133
Maximum estimated ECU execution time [ms]	5.6	0.33	1.5
Number of decision variables	17	18	18
Number of hard constraints	41	0	0

where the average execution time for SMPC, NMPC feedforward, and NMPC feedback is 530, 31, and 127 $\mathsf{\mu }$s that sum up to total average execution time of lower than 700 $\mathsf{\mu }$s. The maximum estimated ECU execution time shows that, for all three controllers, the summation of the Maximum estimated ECU execution time is 7.43 ms, which is below the given sampling time of 8 ms in their engine setup [26].

To achieve this low execution time, a sequence of tasks were performed. These included using the symbolic calculation for the derivative, linearizing the nonlinear model for supervisory control, using Gauss–Newton Hessian approximation instead of Lagrangian in the cost function derivative, solving a single-linear system per timestep for each NMPC in feedforward/feedback control, selecting an optimal solver and well tuned MPC controllers [26].

Although their results show that their nonlinear MPC could be implemented in real-time on a dedicated ECM; they note this is still not feasible for mass production as the ECM still needs to handle many other functions in addition to the emission control strategies tested. Further improvements on CPU usage or using a more capable ECM are still needed before the proposed method is ready for mass production [26].

For [26] the results in Figure 14 show that the designed nonlinear MPC outperforms a well-calibrated production ECM control strategy for reducing engine-out NOx, soot, and total hydrocarbon (THC) emissions. In Figure 14, the MPC L2A controller tracks with high accuracy supervisory controller (L1A) generated fueling rate, intake pressure, and EGR rate, which results in the optimal reduction of THC, NOx, and smoke opacity.

As shown, the designed hierarchical MPC satisfies fuel-air ratio and smoke opacity limits while NOx and THC levels are lower than Benchmark (BM) controller. Their results show no tailpipe visible smoke and also a 10% to 15% reduction in the NOx and total HC emissions compared to a state-of-the-art industry controller when testing a diesel engine under transient drive cycle as part of worldwide harmonized light-duty vehicles test cycle (WLTC) [26].

Another example of MPC experimental implementation for ICE control applications is found in [27]. Switching linear MPC to control combustion phasing (CA50) and engine load (IMEP) of an RCCI engine under transient conditions are used in [27]. Their results, depicted in Figure 15, show CA50 and IMEP tracking errors less than 1.5 CAD and 20 kPa, while the engine combustion stability was controlled by keeping COV${}_{IMEP}$ less than 5%.

Table 5. Summary of the main accomplishments of articles reviewed about MPC in ICEs in comparison with conventional control methods.

Achievements	References
Fuel Saving (0.3 to 12%)	[26,28,29,34,35,36,47,52,54,135,136,137]
NOx reduction (up to 15%)	[26,28,30,41,45,47,135,137,145]
PM reduction (up to 11%)	[29,47,137]
HC reduction (10 to 15%)	[26]
Calibration effort reduction	[36,46,52,53]
Tracking improvement (up to 40%)	[21,22,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,108,136,137,138,139,140,141,142,143,144,147,148,149,150,151]

lists the main results for successful applications of MPC for ICEs. These results show that MPC can provide substantial fuel saving, engine-out emission (NOx, PM, and HC) reduction, and improving reference tracking performance, while significantly reducing the engine calibration efforts particularly for transient speed and load conditions.

5. AI and MPC Integration

This section discusses the integration of Artificial Intelligence (AI) and Model Predictive Control (MPC) for ICE control. AI, in general, refers to the broad discipline of creating intelligent machines, while Machine Learning (ML) predominantly refers to algorithmic subsystems of AI that can learn from experience. The integration of MPC with AI mainly refers to ML-based MPC or data-driven MPC in the literature. The integration of ML and MPC started in the early 2000s but has accelerated in the last few years in broad disciplines, especially in engineering applications [192,193,194,195,196].

Due to the complexity of combustion phenomena and the high number of subsystems in ICE, physical-based model development is time-consuming and may become nonlinear and non-convex. In addition, the accuracy of the physics-based method is typically compromised, mainly due to linearization or model-order reduction techniques [120]. The requirement of accurate modeling to guarantee MPC robust performance while simultaneously having a simpler model has opened an opportunity to utilize the ML method in developing required models in MPC platforms.

In ML-based MPC, a machine learning-based model is used to develop a model. This model is used directly to design MPC or implement optimization. The training process of this model can be either adaptive or based on offline learning. Adaptive learning improves accuracy by deploying ML in real-time, and MPC model coefficients change online in real-time [197,198,199] while, in offline modeling, the coefficient and structure of the model are developed ahead of time using offline statistical data [192,200,201,202].

Several ML-based data-driven modeling techniques have been used, including Artificial Neural Network (ANN) [197,203], Extreme Learning Machine (EL)M [198,204], Bayesian Neural Network (BNN) [205], and Least-Square Support Vector Machine (LS-SVM) [21,22] to provide a predictive model of sufficient accuracy for control of ICEs. MPC and ML integration in ICEs applications is depicted in Figure 16. In this figure, the structure of the ML-based data-driven model, control problems, modeling methods, and engine combustion types are summarized.

Among all data-driven modeling using ML, ANN is the most common in ICEs. A classical time-series system identification technique to identify a dynamic system in the literature is Nonlinear AutoRegressive eXogenous (NARX) [209] where the discrete-time structure of the model can be defined as

$$\begin{array}{cc}\hfill {\mathbf{x}}_{k+1}& =f({\mathbf{x}}_k,{\mathbf{x}}_{k-1},...,{\mathbf{x}}_{k-d},\hfill \\ \hfill & {\mathbf{u}}_k,{\mathbf{u}}_{k-1},{\mathbf{u}}_{k-2},...,{\mathbf{u}}_{k-n})\hfill \end{array}$$

(3)

where $x_k$ is predicted state of system based on d past values of $x_k$ and n past values of control input, $u_k$. In most of these cases, the output of the system, $y_k$, can be $y_k=C{x}_k$. Here, C is the system matrix to map states to outputs. The function f used to approximate the input–output response can be any nonlinear function, such as polynomials and Neural Networks (NNs).

An ANN can be used, and adding shallow networks (low number of hidden layers) could enable an accurate function approximation f [24,203]. This model, Equation (3), in general, is nonlinear and usually requires NLP for MPC implementation (Equation (1)) [130,198,203,208]. The model of Equation (3) provides the required model, $f\left(x\right)$, to perform NLP optimization in Equation (1). Alternatively, by linearizing Equation (3), linear MPC can be used [24,204]. By performing linearization around one specific operating point of engine, the state-space equation of the model is derived as

$$\begin{array}{cc}\hfill {\mathbf{x}}_{k+1}& =A{\mathbf{x}}_k+B{\mathbf{u}}_k\hfill \\ \hfill {\mathbf{x}}_k& =C{\mathbf{x}}_k+D{\mathbf{u}}_k\hfill \end{array}$$

(4)

where A, B, C, and D are state-space model matrices that are found based on linearizing f in Equation (3)

$$\begin{array}{cc}\hfill A& =\frac{\partial f}{\partial \mathbf{x}}{|}_{(x_{op},u_{op})}\hfill \\ \hfill B& =\frac{\partial f}{\partial \mathbf{u}}{|}_{(x_{op},u_{op})}\hfill \end{array}$$

(5)

where $x_{op}$ and $u_{op}$ are states and control inputs of the operating point when the system is linearized. The output matrix, C can be defined based on the required output, and, usually, D is zero. This equation provides the predicted state, output, and control input for MPC optimization in Equation (1). A linearized ANN-based model is used to simplify MPC and change the nonlinear optimization problem of NMPC to the linear MPC that could be interpreted as a quadratic programming problem [210]. However, through this linearization, the main benefits of using neural networks to capture nonlinear system optimization for MPC are lost, and thus using NLP is typically preferred [203].

Extreme Learning Machine (ELM) is a well-known method of ANN for adaptive learning where a gradient-based backpropagation is not required to update the network weights. In ELM, hidden nodes are randomly chosen, and the weights of the network are determined analytically. It has a very efficient training time; however, the accuracy may vary in different cases [211].

In ICEs, ELM was combined with MPC to provide a model for both offline and online learning. An HCCI engine was modeled using ELM, and then nonlinear MPC was used to design IMEP and stability controller [204].

In this study, seven variables of the engine model, including IMEP, CA50, maximum in-cylinder pressure $P_{max}$, maximum pressure rise rate $R_{max}$, output torque, and equivalent air-fuel ratio (EAFR), were predicted for the given input variables including injected fuel mass $m_f$, EVC, and SOI. Then, each output was modeled using one network where inputs are $m_f$, EVC, and SOI. For example, to model CA50 in this study, a single hidden layer was used, which is shown schematically in Figure 17.

The output of this network can be calculated as

$$\widehat{\mathbf{x}}=\sum _{i=1}^L{\beta }_{i}g(w_i{\mathbf{u}}_j+b_i)j=1,\dots ,N$$

(6)

where $\widehat{x}$ is the state of system, L is the number of hidden units (neuron in hidden layer), N is the number of training samples, $\beta$ is the weight vector between the hidden layer and output, w is the weight vector between the input and hidden layer, b is the bias vector, g is the activation function, and u is the control input vector. In the case ib [204], $\widehat{x}=\mathrm{CA}50$ and $u={\left[mfSOIEVC\right]}^T$. The ELM algorithm first randomly assigns weights for $w_i$ and $b_i$, then, based on the following equation, $\beta$ is analytically calculated as

$$\beta ={\mathbf{H}}^{-1}{u}_{tr}$$

(7)

where $x_{tr}$ is training output (actual value of CA50 from experimental data) and $\mathbf{H}$ is $N\times L$ matrix with elements of $g(w_i{u}_j+b_i)$ where i is neuron index while j is training data index. Then, the model can be evaluated for new data by evaluating $\widehat{x}=\mathbf{H}\beta$ [204]. Figure 18 shows the comparison of the predicted outputs and actual experimental data.

The ELM model of this study was trained offline, and the coefficients of linearized ELM were updated in real-time application. This model then used to design MPC to control IMEP and CA50 while constraining the maximum pressure rise rate $R_{max}$. In this study, a fast QP algorithm is used for optimizing MPC in simulation. The optimum control inputs generated by MPC is EVC, SOI, and Fuel Mass. The performance of ELM-MPC control of [204] is shown in Figure 19.

Another standard method of machine learning for MPC modeling is Support Vector Machine (SVM) [212]. In SVM, a convex quadratic programming is solved to find a correlation between input–output or classification. A least-squares version of SVM to solve a set of linear equations to lower the computation cost for the constrained optimization programming is described in [114,115,117,118].

Both regression SVM and least-squares SVM, so-called LS-SVM, have been used to provide ICE models for MPC. Linear Parameter Varying (LPV) formulation of a Reactivity Controlled Compression Ignition (RCCI) model for CA50 and engine load control is driven based on LS-SVM in [21]. The LPV version of the state-space structure discrete-time model (LPV-SS) can be described as

$$\begin{array}{cc}\hfill {\mathbf{x}}_{k+1}& =A\left(p_k\right){\mathbf{x}}_k+B\left(p_k\right){\mathbf{u}}_k\hfill \\ \hfill {\mathbf{y}}_k& =C\left(p_k\right)\mathbf{x}\left(p_k\right)+D\left(p_k\right){\mathbf{u}}_k\hfill \end{array}$$

(8)

where $u_k$ and $x_k$ are inputs and states at k, respectively; $p_k$ is a scheduling variable; and A, B, C, and D are state-space model matrix functions of the scheduling variable. This model can be directly used in Equation (1). Using this LPV model can improve linear MPC performance. To find model matrices, both ANN and Least-Squared versions of SVM are used [21,22,129,130]. One example of implementing the LS-SVM-based model for MPC was presented in [21]. State-space model matrices in the LS-SVM framework can be calculated as

$$\begin{array}{cc}\hfill A\left(p_k\right)& =\sum _{j=1}^N{\alpha }_j{\mathbf{x}}_j^T{K}_{RBF}(p_j,p_k)\hfill \\ \hfill B\left(p_k\right)& =\sum _{j=1}^N{\alpha }_j{\mathbf{u}}_j^T{K}_{RBF}(p_j,p_k)\hfill \\ \hfill C\left(p_k\right)& =\sum _{j=1}^N{\beta }_j{\mathbf{x}}_j^T{K}_{RBF}(p_j,p_k)\hfill \end{array}$$

(9)

where the j index shows data in the training set where the model developed using the training set of ${\mathbf{x}}_{j}j\in (1,2,\dots ,N)$ and ${\mathbf{u}}_{j}j\in (1,2,\dots ,N)$. Additionally, the scheduling parameter, p, is also given in the training set where $p_{j}j\in (1,2,\dots ,N)$. In these equations, $K_{RBF}$ is a Radial Basis Function (RBF) kernel, which is defined as

$$\begin{array}{c}\hfill K_{RBF}(p_j,p_k)=exp(-\frac{\left|\right|p_j-p_k{\left|\right|}^2}{2\sigma })\end{array}$$

(10)

where $\left|\right|p_j-p_k{\left|\right|}^2$ is the L2 norm between the two feature vectors and $\sigma$ is a free parameter. This method of updating model matrices was used in [21,22,129,130].

For instance, fuel-injected per engine cycle, $FQ(mg/cycle)$, is used as a scheduling parameter, and the goal is to control CA50 by varying Start of Injection (SOI) in the framework of RCCI engine in [21]. The true output of the system along with the predicted value are shown in Figure 20a. Experimental implementation of LPS-SS based design MPC performance is shown in Figure 20b. In this implementation, a dSPACE MicroAutoBox (MABX) unit along with dSPACE RapidPro are used for experimental implementation.

LPV-SS modeling is not limited to using LS-SVM, and ANN was also used in LPV-SS modeling to improve the accuracy of modeling inside the MPC. Then, MPC was used to control IMEP and CA50 of the RCCI engine. In an LPV-SS structure, the system matrix function (A, B, C, and D) was represented by a fully-connected ANN. Then, these matrices were updated based on defined scheduling parameters [112,197,205]. An online learning technique was used to refine the model to improve model accuracy for MPC control applications in [197].

Bayesian neural networks (BNNs) were also augmented to LPV-SS in [205]. BNN is a stochastic artificial neural network, which is trained by using Bayesian inference and is another neural network-based method that can be used to mimic MPC behavior. The main advantage of BNN is that the neural network is trained and contains a probability distribution attached to its weights [213]. The ANN-based LPV-SS modeling results in high accuracy prediction of dynamic of CA50 and IMEP where the Best Fit Ratio (BFR) is more than 95% for both cases [112].

In addition to using ML in the model of MPC, ML can be also used to tune the MPC optimization gains is ICE applications. An ANN-based method is used to optimize MPC weights for diesel engine boost pressure and EGR rate [207]. In this study, how humans tune the matrix of MPC (P, Q, and R in MPC optimization) is learned by an ANN. The output of ANN is an approximated human learned cost function based on given performance of manual tuned MPC time response to characterstics, such as overshoot, settling time, and undershoot. Therefore, based on NN’s results, the cost function can be created where can help to tune MPC parameters optimally [207]. This method has the potential to decrease calibration effort but has not been explicitly discussed in the literature.

6. Recommendation and Future Directions

6.1. MPC in ICEs

A summary of MPC benefits and its current status based on the latest developments for ICE control are shown in Figure 21. Implementing MPC uses a systemic approach and provides real-time optimal control solutions. This can result in fuel savings up to 12% while providing a 10–15% reduction in soot, UHC, and NOx emissions. These make MPC a promising method for engine control to assist in meeting future RDE regulations. In addition, MPC can improve reference tracking performance up to 40% for ICE control applications, while substantially reducing controller calibration efforts under transient operating conditions (Table 5).

This can provide benefits for the control of modern ICEs that have several actuators and include coupled dynamics that make calibration of ICE controllers time consuming and often sub-optimal for transient operating conditions.

Two main well-recognized challenges of MPC include the computational cost and sensitivity to model accuracy. This directly affects the required modeling efforts. Recent developments in the areas of fast MPC and applications of ML for modeling and integration with ICE control offer promising solutions. Successful experimental implementation of NMPC for ICE control applications show implementing MPC using techniques of NMPC, EMPC, and LMPC. Implementing NMPC and EMPC in production ECMs is being realized, while there is already evidence of implementing LMPC on production ECMs. To implement NMPC and EMPC on production ECMs, an integrated approach is necessary.

This approach includes reducing computational costs by (i) using techniques of model order reduction and incorporating adaptive model complexity depending on engine speed and load conditions and considering the relative importance of the engine dynamics involved, (ii) optimal and adaptive selection of the control horizon and prediction horizon for transient engine control, (iii) optimal selection of a solver/optimizer for specific engine control problems, and (iv) optimizing the solver structure and use of relaxation techniques based on the required accuracy for certain engine control functions and operating conditions.

Based on the literature, three main topologies are introduced for ICE control (see Figure 4) and depending on the availability of computational resources the appropriate method can be selected. Selection of the best topology for an engine control application should be decided based on the available ECM computational resources, availability of feedback sensors, type of feedback controllers, and required model fidelity for the coupled engine dynamics in each specified ICE control application.

In topology I, the controller has three levels: supervisory control, feedback control, and actuator controllers. This is the most common topology that has been implemented for many ICE applications, though most studies focused on one of these three levels. The most common model structure for implementing MPC in ICEs has four to six states and two to four control inputs. This is linked with the status of computational resources available in an ECM.

Topology III can provide the best performance. This topology is a promising method where supervisory control is combined with the feedback controller but requires more computational resources than the other two topologies. An example of topology II is the fuel path control is done in the supervisory optimal controller, while the air path control is implemented in the feedback control level. The computational load in this topology is between that in topology I and topology III.

6.2. Optimization of MPC in ICEs

Several different methods for implementation on embedded real-time MPC were investigated in this paper. Active Set methods and Gradient Projection methods are the two most promising methods for implementation on embedded real-time MPC. This is due to their simplicity to code, speed, and feasibility/optimality of their solution. Despite these advantages of Active Set and Gradient Projection methods, their speed advantage degrades quickly for larger problems.

Thus for large-scale problems, Interior Point methods are usually a more effective solution in comparison with the other methods. This is summarized schematically in Figure 22. In addition to selecting an appropriate method, using a fast and reliable solver is crucial for the real-time MPC embedded on the ECM. While Gurobi and BARON are state of the arts solvers, they are mostly suitable to desktop computers solving the nonlinear nonconvex problem, which can arise from NMPC. Solvers, such as IPOPT, ForcesPro, and qpOASES provide decent performance for a wide range of MPC problems including real-time MPC—the solvers are summarized in Table 3.

The compromise between reduced order modeling and solver computational cost when formulating the optimal control problem as a nonlinear programming affects the solver selection for NMPC. This trade off is shown in Figure 23. Many ICE control problems are nonlinear and nonconvex, and thus there is a trade-off between high effort to find a set of linearized models that capture the features of the system or higher computational cost to solve a more complex optimization problem, such as MINLP.

To implement MPC for a specific ICE problem, both the method and solver must be determined. To find the best formulation for NMPC, different solvers and methods should be assessed using a suitable benchmark ICE problems.

6.3. Integration of AI and MPC

The integration of AI and MPC were reviewed by focusing on ICE applications. In ICE, the use of ML methods as a subset of AI by focusing on supervised learning in developing high-fidelity plant model inside MPCs were used widely in the literature. In general, a data-driven model is used to predict future steps of MPC by employing a wide variety of learning algorithms and training methods. Training methods can be either offline or online (adaptive). A data-driven model is developed via offline learning, and the model’s coefficients stay constant during the implementation of MPC.

On the other hand, the offline learning process is used in adaptive learning, except that coefficients are updated in real-time. It is essential to update the model based on changing the engine conditions. Although retraining networks and then using MPC optimization based on a current ECM hardware seems infeasible due to the computational load, connected vehicles using cloud/edge computing could make this a future possibility for specific engine applications.

The ECM of the engine can be connected to cloud/edge servers, and data can be transferred to the server using high-quality internet, and updated coefficients of the networks can be transferred back to ECM without adding additional computational load to ECM. The first beta version of this idea can be realistically implemented in stationary engines in the near term. For mobile applications and real-time engine control, network latency needs to be considered unless a real-time ECM update is not required.

Despite the fact that machine learning methods have been used widely in modeling ICEs for MPC application in the literature, it still needs further improvement to make it more compatible with the existing solvers used inside MPC. Additionally, adaptive learning has only been used in limited publications in the literature that seems promising. The integration of ML and MPC was only reported for developing a model; however, other disciplines have used other methods to enhance MPC.

For example, robotics, process control, and heating, ventilation, and air conditioning used (i) learning dynamic modeling for MPC by adjusting the model structure of MPC [192,200,201,202,214,215,216], (ii) the controller design of MPC [193,217,218,219,220], (iii) optimization of MPC solvers [196], (iv) imitation of MPC [195,221], and (v) MPC-based safe-learning of ML [194,222,223]. These methods seem promising for future implementation in ICE applications but must be comprehensively assessed.

7. Summary and Conclusions

MPC provides an effective and systematic framework to optimize and control internal combustion engines (ICEs). The utilization of MPC for multi-objective transient engine control and reducing real driving emission (RDE) is promising. This can substantially reduce ICE calibration efforts and provide more optimal performance, compared to conventional industrial PID-based engine control methods. Recent developments of fast MPC algorithms and solvers have made it possible to implement nonlinear MPC on engine control modules (ECMs).

For the successful implementation of MPC on today’s ECMs, a comprehensive and integrated design approach is required to minimize the MPC computational footprint. This can be done using these four integrated steps: (i) applying model order reduction techniques to reduce nonlinearities, simplifying coupled engine dynamics, and convexifying ICE models wherever possible, (ii) the design of a mechanism to implement variable prediction and control horizons based on ICE system and actuator dynamics, (iii) the selection of an optimal solver based on the complexity and convexity of ICE target dynamics and constraints, optimization of the solver for each specific ICE control problem (e.g., by time-distributed SQP solver), and (iv) selection of appropriate ECM hardware and processor to compute ICE control action within 1–2 ms for cycle-by-cycle engine combustion control.

Integration of AI and machine learning to enhance MPC for ICE control is a promising area to help complex modern ICE comply with future stringent emission targets. The MPC enhancement can be achieved via use of accurate machine learning (ML) or grey-box models for predicting ICE complex dynamics, such as transient soot emissions and/or use of AI within MPC structure, or improving MPC solvers. There is a gap in the literature for techniques to modify ML models for direct use in existing MPC structure. Finally, further investigation is needed to fully assess the potential of AI-MPC methods for a variety of challenging optimal engine control problems.