Perspectives on Characteristics Based Curse-of-Dimensionality-Free Numerical Approaches for Solving Hamilton–Jacobi Equations

Abstract

This paper extends the considerations of the works (Darbon and Osher, Res Math Sci 3:19, 2016; Chow et al., 2017, arxiv.org/abs/1704.02524) regarding curse-of-dimensionality-free numerical approaches to solve certain types of Hamilton–Jacobi equations arising in optimal control problems, differential games and elsewhere. A rigorous formulation and justification for the extended Hopf–Lax formula of(Chow et al., 2017, arxiv.org/abs/1704.02524) is provided together with novel theoretical and practical discussions including useful recommendations. By using the method of characteristics, the solutions of some problem classes under convexity/concavity conditions on Hamiltonians (in particular, the solutions of Hamilton–Jacobi–Bellman equations in optimal control problems) are evaluated separately at different initial positions. This allows for the avoidance of the curse of dimensionality, as well as for choosing arbitrary computational regions. The corresponding feedback control strategies are obtained at selected positions without approximating the partial derivatives of the solutions. The results of numerical simulations demonstrate the high potential of the proposed techniques. It is also pointed out that, despite the indicated advantages, the related approaches still have a limited range of applicability, and their extensions to Hamilton–Jacobi–Isaacs equations in zero-sum two-player differential games are currently developed only for sufficiently narrow classes of control systems. These extensions require further investigation.

Appendix

Let us introduce one more class of differential games for which a single programmed iteration is enough to reach the closed-loop game value. The corresponding result was derived in [54, §V.2].

Consider the linear differential game (92), (90) under Assumptions 4.2, 4.6. In addition to the notations presented in the introduction and Sect. 4, adopt that

$$\begin{aligned}&\Vert y \Vert _j^0 \, {\mathop {=}\limits ^{\mathrm {def}}} \, \max \limits _{i \in \{1, \,2, \ldots , j\}} |y_i|, \quad \Vert y \Vert _j^1 \, {\mathop {=}\limits ^{\mathrm {def}}} \, \sum \limits _{i = 1}^j |y_i|, \nonumber \\&\mathcal {B}_j^0(y,\, \varepsilon ) \, {\mathop {=}\limits ^{\mathrm {def}}} \, \left\{ v \in \mathbb {R}^j \, :\, \Vert v - y \Vert _j^0 \leqslant \varepsilon \right\} \end{aligned}$$

(133)

for all $j \in \mathbb {N} ,$ $y = (y_1,\, y_2, \ldots , y_j) \in \mathbb {R}^j$ and $\varepsilon \geqslant 0 .$ Several more conditions are imposed.

Assumption A.1

There exist a vector $y_0 \in \mathbb {R}^k ,$ numbers $r \in [0, \,+\infty ) ,$ $\bar{r} \in [0,\, +\infty )$ and continuous functions

$$\begin{aligned}&\hat{y} \, :\, [0,\, T] \rightarrow \mathbb {R}^k, \quad \hat{z} \, :\, [0,\, T] \rightarrow \mathbb {R}^k, \\&h_1 \, :\, [0,\, T] \rightarrow [0,\, +\infty ), \quad h_{21} \, :\, [0,\, T] \rightarrow [0,\, +\infty ), \\&h_{22} \, :\, [0,\, T] \rightarrow [0,\, +\infty ), \end{aligned}$$

such that the set $\mathcal {M}$ from Assumption 4.6 is determined by

$$\begin{aligned} \mathcal {M} = y_0 + \mathcal {B}_k^0(O_k,\, r) + \mathcal {B}_k(O_k,\, \bar{r}) \end{aligned}$$

(134)

and, for all $t \in [0,\, T] ,$ $l \in L_k,$ one has

$$\begin{aligned}&\{ \varPhi (T,\, t) \, B_1(t) \, U_1(t) \}_k = \mathcal {B}^0_k \left( \hat{y}(t), \, h_1(t) \right) , \nonumber \\&\int \limits _t^T \max \limits _{u_2 \in U_2} \left\langle l, \, \{ \varPhi (T, \,\xi ) \, B_2(\xi ) \, u_2 \}_k \right\rangle \, d \xi = h_{21}(t) + h_{22}(t) \, \Vert l \Vert ^1_k + \langle l, \, \hat{z}(t) \rangle , \nonumber \\&h_{22}(t) \leqslant \int \limits _t^T h_1(\xi ) \, d \xi + r. \end{aligned}$$

(135)

Denote also

$$\begin{aligned} h_3(t) \, {\mathop {=}\limits ^{\mathrm {def}}} \, \int \limits _t^T h_1(\xi ) \, d \xi + r - h_{22}(t) \quad \forall t \in [0, \,T]. \end{aligned}$$

(136)

The sought-after result can now be formulated.

Theorem A.2

[54, §V.2] Under Assumptions 4.2, 4.6, A.1, the closed-loop game value for (92), (90) at any position $(t_0,\, x_0) \in [0,\, T] \times \mathbb {R}^n$ is represented as

$$\begin{aligned} V(t_0,\, x_0) = \max \, \left\{ V^*(t_0,\, x_0), \, \max \limits _{t \in [t_0,\, T]} \, \{ h_{21}(t) - h_3(t) \} - \bar{r} \right\} , \end{aligned}$$

(137)

where $V^*$ is the programmed maximin function specified in Proposition 4.7.

Theorem A.2 can be applied in the two subsequent examples.

Example A.3

[54, §V.2] Consider the problem that appears from the game of Example 4.10 just by replacing $U_1 = \mathcal {B}_2(O_2,\, a_1)$ with $U_1 = \mathcal {B}^0_2(O_2,\, a_1) .$ Then the closed-loop game value function is also represented as in (113), but the programmed maximin function $V^*$ is now different. $\square$

Example A.4

[54, §V.2] For the game

$$\begin{aligned} \left\{ \begin{array}{l} \dot{x}_1(t) = x_3(t) + v_1(t), \\ \dot{x}_2(t) = x_4(t) + v_2(t), \\ \dot{x}_3(t) = -\alpha \, x_3(t) + u_1(t) + v_3(t), \\ \dot{x}_4(t) = -\alpha \, x_4(t) + u_2(t) + v_4(t), \\ x(t) = (x_1(t), \, x_2(t), \, x_3(t), \, x_4(t))^{\top } \in \mathbb {R}^4, \\ u(t) = (u_1(t), \, u_2(t))^{\top } \in U_1 = \mathcal {B}^0_2 ( \omega ^0, \,a ), \\ v(t) = (v_1(t), \, v_2(t), \, v_3(t), \, v_4(t))^{\top } \\ \quad \in U_2 = (-\mathcal {B}_2(\omega _*, \,b_*)) \times \mathcal {B}^0_2(\omega ^*, \,b^*), \\ \alpha> 0, \, b_* \geqslant 0, \, b^* \geqslant 0, \, a \geqslant b^* \, \text{ are } \text{ scalar } \text{ constants }, \\ \omega ^0, \, \omega _*, \, \omega ^* \, \text{ are } \text{ constant } \text{ vectors } \text{ in } \mathbb {R}^2 , \\ t \in [0,\, T], \, T > 0 \text{ is } \text{ fixed }, \\ k = 2, \, \mathcal {M} = \{ O_2 \}, \\ \sigma (x(T)) = \Vert \{ x(T) \}_2 \Vert _2 = \sqrt{x_1^2(T) + x_2^2(T)} \\ \qquad \qquad \qquad \qquad \quad \longrightarrow \inf _{u(\cdot )} \, \sup _{v(\cdot )} \, \text{ or } \, \sup _{v(\cdot )} \, \inf _{u(\cdot )} , \end{array} \right. \end{aligned}$$

(138)

Theorem A.2 leads to the representation

$$\begin{aligned} \begin{aligned}&V \left( t_0, \, x^0 \right) = \max \, \left\{ V^* \left( t_0, \, x^0 \right) , \, \max \limits _{t \in [t_0,\, T]} \{ (T - t) \, b_* - R_{\alpha }(T,\, t) \, (a - b^*) \} \right\} \\&\forall \, \left( t_0, \, x^0 \right) \in [0,\, T] \times \mathbb {R}^4, \end{aligned} \end{aligned}$$

(139)

where $R_{\alpha }$ is defined as in (116) and $V^*$ is determined according to Proposition 4.7. $\square$