A max-plus primal space fundamental solution for a class of differential Riccati equations

Abstract

A class of differential Riccati equations (DREs) is considered for which the evolution of any solution can be identified with the propagation of a value function of a corresponding optimal control problem arising in ${\mathscr {L}_2}$-gain analysis. By exploiting the semigroup properties inherited from the attendant dynamic programming principle, a max-plus primal space fundamental solution semigroup of max-plus linear max-plus integral operators is developed that encapsulates all such value function propagations. Using this semigroup, a one-parameter fundamental solution semigroup of matrices is constructed for the aforementioned class of DREs. It is demonstrated that this semigroup can be used to compute particular solutions of these DREs, and to characterize finite escape times (should they exist) in a simple way.

Appendix A: Proofs

1.1 A.1 Proof of Theorem 1

Since $Q_t\in {\mathbb {R}}^{2n\times 2n}$ satisfies DRE (22), (23) for all $t\in {\mathbb {R}}_{\ge 0}$, see (28), it may be represented by a corresponding symplectic fundamental solution of the form (5), denoted here by $\widehat{\varSigma }_t\in {\mathbb {R}}^{4n\times 4n}$. In order to apply (5), define ${\widehat{{\mathcal {H}}}},{\varDelta }\in {\mathbb {R}}^{4n\times 4n}$ by

$$\begin{aligned} {\widehat{{\mathcal {H}}}}&\doteq \left[ \begin{array}{c|c} -{\hat{A}} &{} -{\hat{B}} {\hat{B}}' \\ &{} \\ \hline &{} \\ \ {\hat{C}}' {\hat{C}} &{} {\hat{A}}' \end{array} \right] , \quad {\varDelta }\doteq \left[ \begin{array}{cc|cc} I &{} 0 &{} 0 &{} 0 \\ 0 &{} 0 &{} I &{} 0 \\ \hline 0 &{} I &{} 0 &{} 0 \\ 0 &{} 0 &{} 0 &{} I \end{array} \right] , \end{aligned}$$

where ${\hat{A}}\in {\mathbb {R}}^{2n\times 2n}$, ${\hat{B}}\in {\mathbb {R}}^{2n\times m}$, $\hat{C}\in {\mathbb {R}}^{p\times 2n}$ are as per (24). Note by inspection that ${\varDelta }= {\varDelta }' = {\varDelta }^{-1}$. By substitution, a straightforward calculation yields that

$$\begin{aligned} {\widehat{{\mathcal {H}}}} = {\varDelta }\left[ \begin{array}{c|c} {\mathcal {H}} &{} 0 \\ \hline 0 &{} 0 \end{array} \right] {\varDelta }\,, \end{aligned}$$

where ${\mathcal {H}}\in {\mathbb {R}}^{2n\times 2n}$ is as per (5). Hence, the symplectic fundamental solution $\widehat{\varSigma }_t$ for DRE (22), (23) is, again by (5),

$$\begin{aligned}&\widehat{\varSigma }_t = \exp ({\widehat{{\mathcal {H}}}} t) = {\varDelta }\left[ \begin{array}{c|c} \exp ({\mathcal {H}} t) &{} 0 \\ \hline 0 &{} I \end{array} \right] {\varDelta }= {\varDelta }\left[ \begin{array}{c|c} {\varSigma }_t &{} 0 \\ \hline 0 &{} I \end{array} \right] {\varDelta }= \left[ \begin{array}{cc|cc} {\varSigma }_t^{11} &{} 0 &{} {\varSigma }_t^{12} &{} 0 \\ 0 &{} I &{} 0 &{} 0 \\ \hline {\varSigma }_t^{21} &{} 0 &{} {\varSigma }_t^{22} &{} 0 \\ 0 &{} 0 &{} 0 &{} I \end{array} \right] , \end{aligned}$$

(61)

for all $t\in {\mathbb {R}}_{\ge 0}$, where the notation of (3) has been applied. Hence, the particular solution $Q_t$ of DRE (22), (23) is given in terms of the symplectic fundamental solution (5), with respect to $\widehat{\varSigma }_t$, by

$$\begin{aligned}&Q_t = \widehat{Y}_t \widehat{X}_t^{-1} \end{aligned}$$

(62)

for all $t\in [0,t^*(Q_0))\equiv {\mathbb {R}}_{\ge 0}$, see (28), in which

(63)

and $\mu (M)$ is defined by (10). For any fixed $t\in {\mathbb {R}}_{\ge 0}$, note in particular that

$$\begin{aligned} \widehat{X}_t^{-1}&= \left[ \begin{array}{ll} \left( {\varSigma }_t^{11} + {\varSigma }_t^{12} M\right) ^{-1} &{} \ \left( {\varSigma }_t^{11} + {\varSigma }_t^{12} M\right) ^{-1} {\varSigma }_t^{12} M \\ 0 &{} \ I \end{array} \right] , \end{aligned}$$

in which $({\varSigma }_t^{11} + {\varSigma }_t^{12} M)^{-1}$ is well defined as $t^*(M)=+\infty$ by Assumption 1 and (6), (28). That is, $\widehat{X}_t^{-1}$ is well defined. Its substitution in (62), along with $\widehat{Y}_t$ from (63), yields $Q_t = \widehat{Y}_t \widehat{X}_t^{-1} \doteq {\varXi }({\varSigma }_t)$, where ${\varXi }:{\mathbb {R}}^{2n\times 2n}\rightarrow {\mathbb {R}}^{2n\times 2n}$ is defined by

$$\begin{aligned} {\varXi }({\varSigma })&\doteq \left[ \begin{array}{ll} {\varXi }^{11}({\varSigma }) &{} {\varXi }^{12}({\varSigma }) \\ {\varXi }^{21}({\varSigma }) &{} {\varXi }^{22}({\varSigma }) \end{array}\right] , \nonumber \\ {\mathsf {dom}\, }({\varXi })&\doteq \left\{ {\varSigma }\in {\mathbb {R}}^{2n\times 2n}\, \left| \begin{array}{l} {\varSigma }^{11}+{\varSigma }^{12}M\in {\mathbb {R}}^{n\times n}\\ \text {invertible} \end{array} \right. \right\} , \end{aligned}$$

(64)

using the notation of (3), with

$$\begin{aligned} {\varXi }^{11}({\varSigma })&\doteq ({\varSigma }^{21}+{\varSigma }^{22}M)({\varSigma }^{11}+{\varSigma }^{12}M)^{-1}\,, \\ {\varXi }^{12}({\varSigma })&\doteq {\varXi }^{11}({\varSigma }) \, {\varSigma }^{12} M - {\varSigma }^{22} M\,, \\ {\varXi }^{21}({\varSigma })&\doteq -M({\varSigma }^{11}+{\varSigma }^{12}M)^{-1}\,, \\ {\varXi }^{22}({\varSigma })&\doteq {\varXi }^{21}({\varSigma })\, {\varSigma }^{12}M+M. \end{aligned}$$

As M is invertible by Assumption 1, it may be verified directly that ${\varXi }$ of (64) is invertible, with ${\varXi }^{-1}:{\mathbb {R}}^{2n\times 2n}\rightarrow {\mathbb {R}}^{2n\times 2n}$ given by

$$\begin{aligned} {\varXi }^{-1}(Q)&\doteq \left[ \begin{array}{ll} ({\varXi }^{-1})^{11}(Q) &{} ({\varXi }^{-1})^{12}(Q) \\ ({\varXi }^{-1})^{21}(Q) &{} ({\varXi }^{-1})^{22}(Q) \end{array}\right] , \nonumber \\ {\mathsf {dom}\, }({\varXi }^{-1})&\doteq \left\{ Q\in {\mathbb {R}}^{2n\times 2n} \, \biggl | \, Q^{21}\in {\mathbb {R}}^{n\times n} \text { invertible} \right\} . \end{aligned}$$

(65)

where

$$\begin{aligned} ({\varXi }^{-1})^{11}(Q)&\doteq -(Q^{21})^{-1}Q^{22}\,, \\ ({\varXi }^{-1})^{12}(Q)&\doteq -(Q^{21})^{-1}(M-Q^{22})M^{-1}\,, \\ ({\varXi }^{-1})^{21}(Q)&\doteq {Q^{11}}\, ({\varXi }^{-1})^{11}(Q) +Q^{12}\,, \\ ({\varXi }^{-1})^{22}(Q)&\doteq Q^{11}\, ({\varXi }^{-1})^{12}(Q) -Q^{12}M^{-1}\,. \end{aligned}$$

That is, (29) holds. $\square$

1.2 A.2 Proof of Lemma 1

Fix $M_0\in {\mathbb {S}}_{\ge 0}^{n\times n}$ as the stabilizing solution of ARE (30) indicated in the lemma statement. Let $t^*(M_0)\in {\mathbb {R}}_{>0}^+$ denote the maximal horizon of existence (6) of the DRE

$$\begin{aligned} {\dot{R}}_t&= A' R_t + R_t A + R_t B B' R_t + C'C, \quad R_0 = M_0. \end{aligned}$$

(66)

As $M_0$ is the stabilizing solution of ARE (30), note that $R_t \doteq M_0$ is the unique solution of this DRE for all $t\in {\mathbb {R}}_{\ge 0}$. Consequently, $t^*(M_0) = +\infty$. Choose any invertible $M\in {\mathbb {S}}^{n\times n}$ such that (31) holds, and note that such a choice is always possible. Recalling (25), let $Q_t^{11}\in {\mathbb {S}}^{n\times n}$, $t\in [0,t^*(M))$ denote the unique solution of DRE (25) initialized with $Q_0^{11} = M$. As DREs (25) and (66) are identical, Proposition 1 (see below) and (31) imply that solutions $Q_t^{11}$ and $R_t$ satisfy the monotonicity property

$$\begin{aligned}&{Q_t^{11} - M_0 = Q_t^{11} - R_t} \in {{\mathbb {S}}_{\le 0}^{n\times n}} \end{aligned}$$

(67)

for all $t\in [0,t^*(M))$. By inspection, this provides an upper bound for $Q_t^{11}$. In order to determine a lower bound, choose $w_s = 0$ for all $s\in [0,t]$ suboptimal in the definition (20) of $S_t(x,0)$. Recalling (16), (17), (21),

$$\begin{aligned}&{{\textstyle {\frac{1}{2}}}}x' Q_t^{11} x = S_t(x,0) \ge {{\textstyle {\frac{1}{2}}}}x' O_t \, x \end{aligned}$$

(68)

for all $x\in {\mathbb {R}}^n$, in which $O_t\in {\mathbb {S}}^{n\times n}$ is well defined by

$$\begin{aligned} O_t \doteq {\exp (A' t) \, M \exp (A \, t) + \int _0^t \exp (A' s) \, C' C \exp (A\, s) \, ds} \end{aligned}$$

for all $t\in {\mathbb {R}}_{\ge 0}$. Note that $O_t\in {\mathbb {S}}^{n\times n}$ is finite for all $t\in {\mathbb {R}}_{\ge 0}$, and provides a lower bound for $Q_t^{11}\in {\mathbb {S}}^{n\times n}$. Hence, combining (67) and (68),

$$\begin{aligned} Q_t^{11}\in {\mathbb {S}}_{\ge O_t}^{n\times n} \cap {{\mathbb {S}}_{\le M_0}^{n\times n}} \end{aligned}$$

for all $t\in [0,t^*(M))$. A simple contradiction argument subsequently implies that $Q_t^{11}\in {\mathbb {S}}^{n\times n}$ is finite for all $t\in {\mathbb {R}}_{\ge 0}$, so that $t^*(M) = +\infty$. As $M^{-1}$ exists by definition, it follows that Assumption 1 holds, as required. $\square$

Proposition 1

Given initializations $P_0, {\widetilde{P}}_0\in {\mathbb {S}}^{n\times n}$ satisfying $P_0 - {\widetilde{P}}_0\in {\mathbb {S}}_{\le 0}^{n\times n}$, and $t^*\doteq \min (t^*(P_0), t^*({\widetilde{P}}_0))$, the respective unique solutions $P_t, {\widetilde{P}}_t\in {\mathbb {S}}^{2n\times 2n}$ of DRE (1) defined for all $t\in [0,t^*)$, satisfy

$$\begin{aligned}&{P_t - {\widetilde{P}}_t} \in {\mathbb {S}}_{\le 0}^{n \times n} \end{aligned}$$

(69)

for all $t\in [0,t^*)$.

Proof

Fix $t\in [0,t^*)$. Let ${\mathcal {T}}:{\varDelta }_{0,t}\rightarrow {\mathbb {R}}^{n\times n}$ denote the evolution operator associated with the time-dependent ordinary differential equation (ODE)

$$\begin{aligned} {\dot{Y}}_s&= \left( {\hat{A}} + {{\textstyle {\frac{1}{2}}}}{\hat{B}} {\hat{B}}' (P_s + {\widetilde{P}}_s)\right) ' \, Y_s, \end{aligned}$$

defined for $s\in [0,t]$, with ${\varDelta }_{0,t} \doteq \{ (r,s)\in {\mathbb {R}}_{\ge 0}^2 \, \bigl | \, 0\le r \le s \le t \}$. By definition, see, for example, [23, Proposition 3.6, p.138],

$$\begin{aligned} \begin{aligned} {\mathcal {T}}(\sigma ,\sigma )&= I\,, \\ {\textstyle {{\frac{\partial {}}{\partial {s}}}}} {\mathcal {T}}(s,\sigma )&= \left( {\hat{A}} + {{\textstyle {\frac{1}{2}}}}{\hat{B}} \hat{B}' \left( P_s + {\widetilde{P}}_s\right) \right) ' \, {\mathcal {T}}(s,\sigma )\,, \\ {\textstyle {{\frac{\partial {}}{\partial {\sigma }}}}} {\mathcal {T}}(s,\sigma )&= -{\mathcal {T}}(s,\sigma ) \, \left( {\hat{A}} + {{\textstyle {\frac{1}{2}}}}{\hat{B}} {\hat{B}}' \left( P_\sigma + \widetilde{P}_\sigma \right) \right) '\,, \end{aligned} \end{aligned}$$

(70)

for all $(s,\sigma )\in {\varDelta }_{0,t}$. Define $\pi :[0,t]\rightarrow {\mathbb {S}}^{n\times n}$ by

$$\begin{aligned} \pi _\sigma&\doteq {\mathcal {T}}(t,\sigma )\, ( P_\sigma - \widetilde{P}_\sigma ) \, {\mathcal {T}}(t,\sigma )' \end{aligned}$$

(71)

for all $\sigma \in [0,t]$. Differentiating with respect to $\sigma$,

$$\begin{aligned} \dot{\pi }_\sigma =&{\textstyle {{\frac{\partial {}}{\partial {\sigma }}}}} {\mathcal {T}}(t,\sigma ) \, \left( P_\sigma - {\widetilde{P}}_\sigma \right) \, {\mathcal {T}}(t,\sigma )' + {\mathcal {T}}(t,\sigma ) \, \left( {\dot{P}}_\sigma - \dot{{\widetilde{P}}}_\sigma \right) \, {\mathcal {T}}(t,\sigma )' \nonumber \\&+ {\mathcal {T}}(t,\sigma ) \left( P_\sigma - {\widetilde{P}}_\sigma \right) {\textstyle {{\frac{\partial {}}{\partial {\sigma }}}}} {\mathcal {T}}(t,\sigma )' \nonumber \\ =&{\mathcal {T}}(t,\sigma ) \, {\varGamma }_\sigma \, {\mathcal {T}}(t,\sigma )' \end{aligned}$$

(72)

for all $\sigma \in [0,t]$, where

$$\begin{aligned} {\varGamma }_\sigma \doteq&\left( {\dot{P}}_\sigma - \dot{{\widetilde{P}}}_\sigma \right) - \left( {\hat{A}} + {{\textstyle {\frac{1}{2}}}}{\hat{B}} {\hat{B}}' \left( P_\sigma + {\widetilde{P}}_\sigma \right) \right) ' (P_\sigma - {\widetilde{P}}_\sigma ) \\&- \left( P_\sigma - {\widetilde{P}}_\sigma \right) \, \left( {\hat{A}} + {{\textstyle {\frac{1}{2}}}}{\hat{B}} {\hat{B}}' \left( P_\sigma + {\widetilde{P}}_\sigma \right) \right) \\&= 0\,, \end{aligned}$$

in which the equality with zero follows by virtue of the fact that $P_\sigma$, ${\widetilde{P}}_\sigma$ both satisfy the DRE (1). Consequently, (72) implies that $\dot{\pi }_\sigma = 0$ for all $\sigma \in [0,t]$, so that integration with respect to $\sigma \in [0,t]$ yields $\pi _t = \pi _0$. Recalling (71), it follows immediately that

$$\begin{aligned} P_t - {\widetilde{P}}_t&= \pi _t = \pi _0 = {\mathcal {T}}(t,0)\, (P_0 - {\widetilde{P}}_0)' \, {\mathcal {T}}(t,0)' \end{aligned}$$

Recalling that $P_0 - {\widetilde{P}}_0\in {\mathbb {S}}_{\le 0}^{n\times n}$, and noting that $t\in [0,t^*)$ is arbitrary, yields the required assertion (69). $\square$

1.3 A.3 Proof of Lemma 2

Suppose that Assumption 1 holds. Fix $x,y\in {\mathbb {R}}^n$, $t\in {\mathbb {R}}_{>0}$. Note that $G_t(x,y)\in {\mathbb {R}}^-$ by Theorem 2.

(Necessity) Suppose that $G_t(x,y)\in {\mathbb {R}}$. Recalling the value function interpretation of $G_t(x,y)$, if the dynamics (18) are not controllable from x to y in time t, it immediately follows by definitions (16), (36) that $G_t(x,y) = -\infty$. Hence, the dynamics (18) must be controllable from x to y in time t. Necessity follows as $x,y\in {\mathbb {R}}^n$ and $t\in {\mathbb {R}}_{>0}$ are arbitrary.

(Sufficiency) Suppose that dynamics (18) are controllable. Proposition 2 (see below) implies that $Q_t^{22} \in {\mathbb {S}}_{>M}^{n\times n}$, where $Q_t^{22}$ is as per (27). Hence, $S_t(x,\cdot )\in {\mathscr {S}_+^{{-M}}} = {\mathsf {dom}\, }({\mathcal {D}}_\varphi )$, so that ${\mathcal {D}}_\varphi S_t(x,\cdot )\in {\mathscr {S}_-^{{-M}}}$ is well defined. So, applying the semiconvex transform (11) to $S_t(x,\cdot )$ yields

$$\begin{aligned} ({\mathcal {D}}_\varphi S_t(x,\cdot ))(y) =&- \int _{{\mathbb {R}}^n}^\oplus \varphi (\xi ,y)\otimes (-S_t(x,\xi )) \, d\xi \nonumber \\ =&- \int _{{\mathbb {R}}^n}^\oplus {{\textstyle {\frac{1}{2}}}}\left[ \begin{array}{l} \xi \\ y \end{array} \right] ' \mu (M) \left[ \begin{array}{l} \xi \\ y \end{array} \right] - {{\textstyle {\frac{1}{2}}}}\left[ \begin{array}{l} x \\ \xi \end{array} \right] ' Q_t \left[ \begin{array}{l} x \\ \xi \end{array} \right] d\xi \nonumber \\ =&- \int _{{\mathbb {R}}^n}^\oplus {{\textstyle {\frac{1}{2}}}} \left[ \begin{array}{l} x \\ y \\ \hline \xi \end{array} \right] ' \left[ \begin{array}{cc|c} -Q_t^{11} &{} \ 0 &{} -Q_t^{12} \\ 0 &{} \ +M &{} - M \\ \hline &{}&{} \\ -(Q_t^{12})' &{} \ -M &{} M - Q_t^{22} \end{array} \right] \left[ \begin{array}{l} x \\ y \\ \hline \xi \end{array} \right] d\xi \nonumber \\ =&- {{\textstyle {\frac{1}{2}}}}\left[ \begin{array}{l} x \\ y \end{array} \right] ' \left[ \begin{array}{l@{\quad }l} -Q_t^{11} &{} 0 \\ 0 &{} +M \end{array} \right] \left[ \begin{array}{l} x \\ y \end{array} \right] \nonumber \\&+ {{\textstyle {\frac{1}{2}}}}\left[ \begin{array}{l} x \\ y \end{array} \right] ' \left[ \begin{array}{l} -Q_t^{12} \\ -M \end{array} \right] \left( M - Q_t^{22}\right) ^{-1} \left[ \begin{array}{l} -Q_t^{12} \\ -M \end{array} \right] ' \left[ \begin{array}{l} x \\ y \end{array} \right] \nonumber \\ \doteq&{{\textstyle {\frac{1}{2}}}}\left[ \begin{array}{l} x \\ y \end{array} \right] ' {\varLambda }_t \left[ \begin{array}{l} x \\ y \end{array} \right] , \end{aligned}$$

(73)

in which $(M-Q_t^{22})^{-1}$ is guaranteed to exist, so that ${\varLambda }_t\in {\mathbb {R}}^{2n\times 2n}$ exists and is finite by definition. Hence, applying the right-hand equality of (36) in Theorem 2,

$$\begin{aligned} G_t(x,y)&= {{\textstyle {\frac{1}{2}}}}\left[ \begin{array}{l} x \\ y \end{array} \right] ' {\varLambda }_t \left[ \begin{array}{l} x \\ y \end{array} \right] \in {\mathbb {R}}, \end{aligned}$$

(74)

which is finite, thereby demonstrating sufficiency. $\square$

Proposition 2

Under Assumption 1, controllability of the dynamics (18) implies that $Q_t^{22} \in {\mathbb {S}}_{>M}^{n\times n}$ for all $t\in {\mathbb {R}}_{>0}$.

Proof

With $M\in {\mathbb {S}}^{n\times n}$ satisfying Assumption 1, recall that $t^*(M) = +\infty$ as per (28). Fix any $t\in {\mathbb {R}}_{>0}$. Consequently, the optimal dynamics associated with $S_t(x,y)$ of (20), (21) are well defined by the time-dependent ODE

$$\begin{aligned} \dot{x}_s^*&= \left( A + B B' Q_{t-s}^{11} \right) \, x_s^*\,, \qquad x_0 = x\,, \end{aligned}$$

(75)

for all $s\in [0,t]$. Let ${\mathcal {V}}_t:{\varDelta }_{0,t}\rightarrow {\mathbb {R}}^{n\times n}$ denote the evolution operator associated with (75), with ${\varDelta }_{0,t} \doteq \{ (r,s)\in {\mathbb {R}}_{\ge 0}^2 \, \bigl | \, 0\le r \le s \le t \}$. By definition, see, for example, [23, Proposition 3.6, p.138],

$$\begin{aligned} \begin{aligned} {\mathcal {V}}_t(\sigma ,\sigma )&= I, \\ {\textstyle {{\frac{\partial {}}{\partial {s}}}}} {\mathcal {V}}_t(s,\sigma )&= \left( A + B B' Q_{t-s}^{11} \right) \, {\mathcal {V}}_t(s,\sigma ), \\ {\textstyle {{\frac{\partial {}}{\partial {\sigma }}}}} {\mathcal {V}}_t(s,\sigma )&= -{\mathcal {V}}_t(s,\sigma ) \, \left( A + B B' Q_{t-\sigma }^{11} \right) \,, \end{aligned} \end{aligned}$$

(76)

for all $(s,\sigma )\in {\varDelta }_{0,t}$. Define ${\mathcal {U}}_t:{\varDelta }_{0,t}\rightarrow {\mathbb {R}}^{n\times n}$ via (76) by

$$\begin{aligned} {\mathcal {U}}_t(r,\tau )&\doteq {\mathcal {V}}_t(t-\tau ,t-r)' \end{aligned}$$

(77)

for all $(r,\tau )\in {\varDelta }_{0,t}$. By inspection of (76), (77),

$$\begin{aligned} \begin{aligned} {\mathcal {U}}_t(\tau ,\tau )&= I\,, \\ {\textstyle {{\frac{\partial {}}{\partial {r}}}}} {\mathcal {U}}_t(r,\tau )&= \left[ {\textstyle {{\frac{\partial {}}{\partial {\sigma }}}}} {\mathcal {V}}_t(s,\sigma ) \bigl |_{(s,\sigma )=(t-\tau , t-r)}\right] ' \, (-1) \\&= \left( A + B B' Q_{r}^{11} \right) ' \, {\mathcal {V}}_t(t-\tau ,t-r)' = \left( A + B B' Q_{r}^{11} \right) ' \, {\mathcal {U}}_t(r,\tau )\,, \\ {\textstyle {{\frac{\partial {}}{\partial {\tau }}}}} {\mathcal {U}}_t(r,\tau )&= \left[ {\textstyle {{\frac{\partial {}}{\partial {s}}}}} {\mathcal {V}}_t(s,\sigma ) \bigl |_{(s,\sigma )=(t-\tau , t-r)}\right] ' \, (-1) \\&= - {\mathcal {V}}_t(t-\tau ,t-r)' \left( A + B B' Q_{\tau }^{11}\right) ' = - {\mathcal {U}}_t(r,\tau ) \, \left( A + B B' Q_{\tau }^{11}\right) ' \end{aligned} \end{aligned}$$

(78)

That is, ${\mathcal {U}}_t:{\varDelta }_{0,t}\rightarrow {\mathbb {R}}^{n\times n}$ is the evolution operator for the dynamics associated with $(A + B B' Q_{s}^{11})'$, $s\in [0,t]$. Comparing with (26), it immediately follows that $Q_s^{12} = - {\mathcal {U}}_t(s,0) M$ for all $s\in [0,t]$. Hence, (27) implies that

$$\begin{aligned} Q_t^{22} - M&= \int _0^t \left( Q_s^{12}\right) ' B B' Q_s^{12} \, ds \nonumber \\&= \int _0^t M \, {\mathcal {U}}_t(s,0)' B B' \, {\mathcal {U}}_t(s,0) M \, ds = M \, {\mathcal {C}}_t \, M \end{aligned}$$

(79)

where ${\mathcal {C}}_t \doteq \int _0^t {\mathcal {V}}_t(t,t-s) B B' \, {{\mathcal {V}}_t(t,t-s)'}\, ds\in {\mathbb {S}}_{\ge 0}^{n\times n}$ is the controllability gramian for the pair $(A+B B' Q_{t-\cdot }^{11}, B)$ on [0, t], by definition of ${\mathcal {V}}_t$. However, recall that controllability is preserved under state feedback, see, for example, [1, p.48]. Hence, (A, B) completely controllable implies that $(A+B B' Q_{t-\cdot }^{11}, B)$ is completely controllable, which in turn implies that ${\mathcal {C}}_t$ is invertible for $t\in {\mathbb {R}}_{>0}$. That is, ${\mathcal {C}}_t\in {\mathbb {S}}_{>0}^{n\times n}$ for all $t\in {\mathbb {R}}_{>0}$. As M is invertible by Assumption 1, the assertion immediately follows by (79). $\square$

1.4 A.4 Proof of Theorem 3

Fix any $t\in {\mathbb {R}}_{>0}$, $x\in {\mathbb {R}}^n$. Applying Lemma 2, and in particular (73), (74), it follows that $Q_t\in {\mathbb {R}}^{2n\times 2n}$, ${\varLambda }_t\in {\mathbb {S}}^{2n\times 2n}$ of (22), (73) are related via

$$\begin{aligned} Q_t&= {\varPi }({\varLambda }_t)\,, \qquad {\varLambda }_t = {\varPi }^{-1}(Q_t)\,, \end{aligned}$$

with matrix operators ${\varPi }, {\varPi }^{-1}:{\mathbb {S}}^{2n\times 2n}\rightarrow {\mathbb {S}}^{2n\times 2n}$ defined using the notation of (3) by

$$\begin{aligned} {\varPi }({\varLambda })&\doteq \left[ \begin{array}{l@{\qquad }l} {\varLambda }^{11} - {\varLambda }^{12}(M+ {\varLambda }^{22})^{-1} ({\varLambda }^{12})' &{} {\varLambda }^{12} (M+ {\varLambda }^{22})^{-1} M \\ M(M+ {\varLambda }^{22})^{-1} ({\varLambda }^{12})' &{} M - M ( M + {\varLambda }^{22})^{-1} M \end{array} \right] \nonumber \\ {\mathsf {dom}\, }({\varPi })&\doteq \left\{ {\varLambda }\in {\mathbb {S}}^{2n\times 2n}\, \biggl | \, {\varLambda }^{22}\in {\mathbb {S}}_{<-M}^{n\times n}\right\} , \end{aligned}$$

(80)

$$\begin{aligned} {\varPi }^{-1}(Q)&\doteq \left[ \begin{array}{l@{\qquad }l} Q^{11} + Q^{12}(M - Q^{22})^{-1} (Q^{12})' &{} Q^{12} ( M - Q^{22})^{-1}M \\ M (M - Q^{22})^{-1} (Q^{12})' &{} M ( M - Q^{22})^{-1} M - M \end{array} \right] \nonumber \\ {\mathsf {dom}\, }({\varPi }^{-1})&\doteq \left\{ Q\in {\mathbb {S}}^{2n\times 2n}\, \biggl | \, Q^{22}\in {\mathbb {S}}_{>M}^{n\times n}\right\} . \end{aligned}$$

(81)

In particular, (81) follows by inspection of (73), while (80) may be verified by demonstrating that ${\varPi }\circ {\varPi }^{-1}$ is the identity. $\square$

1.5 A.5 Proof of Theorem 4

Throughout, it is assumed that Assumptions 1 and 2 hold, with $M\in {\mathbb {S}}^{n\times n}$ specified by the former, as per the theorem statement. Note in particular that $t^*(M) = +\infty$, so that $({\varSigma }_t^{11} + {\varSigma }_t^{12}\, M)^{-1}$ exists for all $t\in {\mathbb {R}}_{\ge 0}$, where ${\varSigma }_t$ is the symplectic fundamental solution identified in (5). Consequently, $Q_t\in {\mathbb {S}}^{2n\times 2n}$ is well defined as the unique solution of DRE (22), (23), for all $t\in {\mathbb {R}}_{\ge 0}$ by Assumption 1, see Theorem 1 and its proof. Note that $P_0\in {\mathbb {S}}_{>M}^{n\times n} = {\mathsf {dom}\, }({\varUpsilon })$ by hypothesis and (13).

The proof proceeds by demonstrating a sequence of implications concerning the following claims, posed with respect to arbitrary fixed $t\in {\mathbb {R}}_{>0}$ and $P_0\in {\mathbb {S}}_{>M}^{n\times n}$:

1):

$t\in (0,t^*(P_0))$;

2):

${\varUpsilon }(P_0) + Q_s^{22}\in {\mathbb {S}}_{<0}^{n\times n}$ for all $s\in (0,t]$;

3):

${\varUpsilon }(P_0) + Q_t^{22}\in {\mathbb {S}}_{<0}^{n\times n}$;

4):

$P_0 + {\varLambda }_t^{22}\in {\mathbb {S}}_{<0}^{n\times n}$; and

5):

(44) and (45) hold.

In particular, it is shown that (1) $\Leftrightarrow$ (2) $\Leftrightarrow$ (3) $\Leftrightarrow$ (4) $\Rightarrow$ (5).

2) $\Rightarrow$ 1): Suppose that ${\varUpsilon }(P_0) + Q_s^{22} \in {\mathbb {S}}_{<0}^{n\times n}$ for all $s\in (0,t]$. Applying (13) and Theorem 1,

$$\begin{aligned}&M^{-1} \left( {\varUpsilon }(P_0) + Q_s^{22} \right) \, M^{-1} = (M - P_0)^{-1} - \left( {\varSigma }_s^{11} + {\varSigma }_s^{12} M\right) ^{-1} {\varSigma }_s^{12} \end{aligned}$$

(82)

where it may be noted that the inverses on the right-hand side are guaranteed to exist. By hypothesis, the left-hand side is invertible, so that a matrix $K_s\in {\mathbb {R}}^{n\times n}$ is well defined for an arbitrary $s\in (0,t]$ by

$$\begin{aligned} K_s \doteq&\left( {\varSigma }_s^{11} + {{\varSigma }_s^{12}} M\right) ^{-1} + \left( {\varSigma }_s^{11} + {{\varSigma }_s^{12}} M\right) ^{-1} {\varSigma }_t^{12} \\&\times \left[ (M - P_0)^{-1} - \left( {\varSigma }_s^{11} + {\varSigma }_s^{12} M\right) ^{-1} {\varSigma }_s^{12} \right] ^{-1} \left( {\varSigma }_s^{11} + {{\varSigma }_s^{12}} M\right) ^{-1}\,. \end{aligned}$$

However, the Woodbury lemma implies that

$$\begin{aligned} K_s&= \left[ \left( {\varSigma }_s^{11} + {{\varSigma }_s^{12}} M\right) - {\varSigma }_s^{12} (M - P_0) \right] ^{-1} = \left( {{\varSigma }_s^{11}} + {\varSigma }_s^{12} P_0\right) ^{-1}\,. \end{aligned}$$

That is, ${{\varSigma }_s^{11}} + {\varSigma }_s^{12} P_0\in {\mathbb {S}}^{n\times n}$ is invertible. Recalling (6), and that $s\in (0,t]$ is arbitrary, immediately implies that 1) holds.

1) $\Rightarrow$ 2): Fix an arbitrary $t\in (0,t^*(P_0))$. Analogously to the proof of Theorem 1, let ${\widetilde{Q}}_s\in {\mathbb {S}}^{2n\times 2n}$ denote the unique solution of DRE (22) subject to the initialization

$$\begin{aligned}&{\widetilde{Q}}_0 = \mu (P_0) \end{aligned}$$

(83)

defined, via (10), for all $s\in [0, t^*({\widetilde{Q}}_0))$, where $t^*({\widetilde{Q}}_0)\in {\mathbb {R}}_{>0}$ is the corresponding maximal horizon of existence (6). Analogously to the argument yielding (28), observe that $t^*(\widetilde{Q}_0) = t^*(P_0)$, so that $t\in (0,t^*({\widetilde{Q}}_0))$. An application of the symplectic fundamental solution (4), (5), (61) yields

$$\begin{aligned}&{\widetilde{Q}}_s = {\widetilde{Y}}_s \widetilde{X}_s^{-1} \end{aligned}$$

(84)

for all $s\in [0,t]$, in which

for all $s\in [0,t]$. In particular,

$$\begin{aligned} {\widetilde{X}}_s^{-1}&= \left[ \begin{array}{ll} \left( {\varSigma }_s^{11} + {\varSigma }_s^{12} P_0\right) ^{-1} &{} \ \left( {\varSigma }_s^{11} + {\varSigma }_s^{12} P_0\right) ^{-1} {\varSigma }_s^{12} M \\ 0 &{} \ I \end{array} \right] , \end{aligned}$$

in which $({\varSigma }_s^{11} + {\varSigma }_s^{12} P_0)^{-1}$ is well defined for all $s\in [0,t]$, as $t\in (0,t^*(P_0))$, see (6). Consequently, recalling (84),

$$\begin{aligned} {\widetilde{Q}}_s^{22}&= M - M \left( {\varSigma }_s^{11} + {\varSigma }_s^{12} P_0\right) ^{-1} {\varSigma }_s^{12} M \end{aligned}$$

(85)

is well defined for all $s\in [0,t]$. Recalling (23), (83), as ${\widetilde{Q}}_0 = \mu (P_0) \ge \mu (M) = Q_0$, Proposition 1 applied to (22) implies that ${\widetilde{Q}}_s - Q_s\in {\mathbb {S}}_{\ge 0}^{2n\times 2n}$, so that in particular

$$\begin{aligned}&{\widetilde{Q}}_s^{22} - Q_s^{22}\in {\mathbb {S}}_{\ge 0}^{n\times n} \end{aligned}$$

(86)

for all $s\in [0,t]$. Fix any $s\in (0,t]$. Rearranging (85) and applying (86) and Proposition 2,

$$\begin{aligned} \left( {\varSigma }_s^{11} + {\varSigma }_s^{12} P_0\right) ^{-1} {\varSigma }_s^{12}&= M^{-1} \left( M - {\widetilde{Q}}_s^{22} \right) \, M^{-1} \nonumber \\&\le M^{-1} \left( M - Q_s^{22} \right) \, M^{-1}\in {\mathbb {S}}_{<0}^{n\times n}\,. \end{aligned}$$

(87)

Theorem 1 and (64) imply via the notation of (3) that

$$\begin{aligned} Q_s^{22}&= [{\varXi }({\varSigma }_s)]^{22} = M - M({\varSigma }_s^{11} + {\varSigma }_s^{12} M)^{-1} {\varSigma }_s^{12} M. \end{aligned}$$

(88)

Recall that ${\varSigma }_s\in {\mathsf {dom}\, }({\varXi })$ (i.e., the inverse involved is guaranteed to exist) by Assumption 1, as $s\in (0,t^*(M)) \equiv {\mathbb {R}}_{>0}$. Furthermore, as $s\in (0,t^*(P_0))$, definition (6) implies that ${\varSigma }_s^{11} + {\varSigma }_s^{12} P_0$ is invertible. Hence, a matrix $L_s\in {\mathbb {S}}^{n\times n}$ is well defined by

$$\begin{aligned} L_s&\doteq (M - P_0) + (M-P_0) \left( {\varSigma }_s^{11} + {\varSigma }_s^{12} P_0\right) ^{-1} {\varSigma }_s^{12} (M-P_0) \\&= (M - P_0) + (M-P_0) \left[ \left( {\varSigma }_s^{11} + {\varSigma }_s^{12} M\right) - {\varSigma }_s^{12} (M - P_0) \right] ^{-1} {\varSigma }_s^{12} \, (M-P_0). \end{aligned}$$

where the second equality follows by adding and subtracting ${\varSigma }_s^{12} M$ within the inverse. Applying (87), and the fact that $P_0\in {\mathbb {S}}_{>M}^{n\times n}$, note that $L_s\in {\mathbb {S}}_{<0}^{n\times n}$ by definition. The Woodbury lemma subsequently implies that

$$\begin{aligned} L_s&= \left[ (M-P_0)^{-1} - \left( {\varSigma }_s^{11} + {\varSigma }_s^{12} M\right) ^{-1} {\varSigma }_s^{12} \right] ^{-1} = M \left( {\varUpsilon }(P_0) + Q_s^{22} \right) ^{-1} M \end{aligned}$$

where the second equality follows as per (82). Consequently, as $M\in {\mathbb {S}}^{n\times n}$ is invertible and $L_s\in {\mathbb {S}}_{<0}^{n\times n}$,

$$\begin{aligned} {\varUpsilon }(P_0) + Q_s^{22} = M L_s^{-1} \, M \in {\mathbb {S}}_{<0}^{n\times n}. \end{aligned}$$

As $s\in (0,t]$ is arbitrary, claim 2) immediately follows.

2) $\Rightarrow$ 3): By hypothesis, ${\varUpsilon }(P_0) + Q_s^{22}\in {\mathbb {S}}_{<0}^{n\times n}$ for all $s\in (0,t]$. Selecting $s=t$ yields claim 3) as required.

3) $\Rightarrow$ 2): By hypothesis, ${\varUpsilon }(P_0) + Q_t^{22}\in {\mathbb {S}}_{<0}^{n\times n}$. Furthermore, ${\varUpsilon }(P_0)\in {\mathbb {S}}_{<-M}^{n\times n}$ by (13), (14). Hence, $Q_t^{22}\in {\mathbb {S}}^{n\times n}$, so that $(Q_\sigma ^{12})' B B' Q_\sigma ^{12}$ must be integrable with respect to $\sigma \in [0,t]$ by definition (27). In particular,

$$\begin{aligned} Q_t^{22} - M&= \int _0^t \left( Q_\sigma ^{12}\right) ' B B' Q_\sigma ^{12} \, d\sigma \ge \int _0^s \left( Q_\sigma ^{12}\right) ' B B' Q_\sigma ^{12} \, d\sigma = Q_s^{22} - M \end{aligned}$$

for any fixed $s\in (0,t]$. Hence, $Q_s^{22} - Q_t^{22}\in {\mathbb {S}}_{\le 0}^{n\times n}$, so that

$$\begin{aligned} {\varUpsilon }(P_0) + Q_s^{22}&= \left( {\varUpsilon }(P_0) + Q_t^{22}\right) + \left( Q_s^{22} - Q_t^{22}\right) \in {\mathbb {S}}_{<0}^{n\times n}. \end{aligned}$$

Recalling that $s\in (0,t]$ is arbitrary yields claim 2) as required.

3) $\Rightarrow$ 4): Recalling (13) and Theorem 3, see (42), (80),

$$\begin{aligned} {\varUpsilon }(P_0) + Q_t^{22}&= (-M - M(P_0-M)^{-1} M) + \left( M - M \left( M + {\varLambda }_t^{22} \right) ^{-1} M\right) \nonumber \\&= M \left[ (M-P_0)^{-1} - \left( M + {\varLambda }_t^{22} \right) ^{-1} \right] M. \end{aligned}$$

(89)

Recalling that ${\varUpsilon }(P_0) + Q_t^{22}\in {\mathbb {S}}_{<0}^{n\times n}$ by hypothesis,

$$\begin{aligned} {\varUpsilon }(P_0) + Q_t^{22}\in {\mathbb {S}}_{<0}^{n\times n}&\Leftrightarrow (M-P_0)^{-1} - \left( M + {\varLambda }_t^{22} \right) ^{-1}\in {\mathbb {S}}_{<0}^{n\times n} \nonumber \\&\Leftrightarrow \left( M + {\varLambda }_t^{22}\right) - (M-P_0) \in {\mathbb {S}}_{<0}^{n\times n} \nonumber \\&\Leftrightarrow P_0 + {\varLambda }_t^{22}\in {\mathbb {S}}_{<0}^{n\times n}\,. \end{aligned}$$

(90)

That is, claim 4) holds.

4) $\Rightarrow$ 3): Note that (89) holds as per the 3) $\Rightarrow$ 4) case above. By hypothesis, $P_0 + {\varLambda }_t^{22}\in {\mathbb {S}}_{<0}^{n\times n}$. Hence, the string of equivalences (90) implies that 3) holds.

4) $\Rightarrow$ 5): Recalling (15), (35) and (42), the value function $W_t$ of (15), (19) satisfies

$$\begin{aligned} W_t(x)&= {{\textstyle {\frac{1}{2}}}}x' P_t x = \int _{{\mathbb {R}}^n}^\oplus G_t(x,y)\otimes {\varPsi }(y)\, dy \\&= \int _{{\mathbb {R}}^n}^\oplus {{\textstyle {\frac{1}{2}}}}\left[ \begin{array}{l} x \\ y \end{array} \right] ' {\varLambda }_t \left[ \begin{array}{l} x \\ y \end{array} \right] \otimes {{\textstyle {\frac{1}{2}}}}y' P_0 \, y \, dy \\&= {{\textstyle {\frac{1}{2}}}}\int _{{\mathbb {R}}^n}^\oplus \left[ \begin{array}{l} x \\ y \end{array} \right] ' \left[ \begin{array}{ll} {\varLambda }_t^{11} &{} {\varLambda }_t^{12} \\ \left( {\varLambda }_t^{12}\right) ' &{} P_0 + {\varLambda }_t^{22} \end{array} \right] \left[ \begin{array}{l} x \\ y \end{array} \right] dy \end{aligned}$$

for all $x\in {\mathbb {R}}^n$. By hypothesis, $P_0 + {\varLambda }_t^{22}\in {\mathbb {S}}_{<0}^{n\times n}$, so that $(P_0 + {\varLambda }_t^{22})^{-1}$ exists. Hence, the above max-plus integration explicitly evaluates as

$$\begin{aligned} {{\textstyle {\frac{1}{2}}}}x' P_t\, x&= {{\textstyle {\frac{1}{2}}}}x' \left[ {\varLambda }_t^{11} - {\varLambda }_t^{12} \left( P_0 + {\varLambda }_t^{22} \right) ^{-1} \left( {\varLambda }_t^{12} \right) ' \right] x. \end{aligned}$$

As $x\in {\mathbb {R}}^n$ is arbitrary, (44) follows immediately. In addition, as 4) $\Leftrightarrow$ 1), it immediately follows that

$$\begin{aligned} \sup \left\{ t\in {\mathbb {R}}_{>0} \left| P_0 + {\varLambda }_t^{22}\in {\mathbb {S}}_{<0}^{n\times n} \right. \right\} = \sup \left\{ t\in {\mathbb {R}}_{>0} \bigl | t\in (0,t^*(P_0)) \right\} = t^*(P_0). \end{aligned}$$

That is, (45) holds. $\square$