Fast nonlinear risk assessment for autonomous vehicles using learned conditional probabilistic models of agent futures

Abstract

This paper presents fast non-sampling based methods to assess the risk for trajectories of autonomous vehicles when probabilistic predictions of other agents’ futures are generated by deep neural networks (DNNs). The presented methods address a wide range of representations for uncertain predictions including both Gaussian and non-Gaussian mixture models to predict both agent positions and control inputs conditioned on the scene contexts. We show that the problem of risk assessment when Gaussian mixture models of agent positions are learned can be solved rapidly to arbitrary levels of accuracy with existing numerical methods. To address the problem of risk assessment for non-Gaussian mixture models of agent position, we propose finding upper bounds on risk using nonlinear Chebyshev’s Inequality and sums-of-squares programming; they are both of interest as the former is much faster while the latter can be arbitrarily tight. These approaches only require higher order statistical moments of agent positions to determine upper bounds on risk. To perform risk assessment when models are learned for agent control inputs as opposed to positions, we propagate the moments of uncertain control inputs through the nonlinear motion dynamics to obtain the exact moments of uncertain position over the planning horizon. To this end, we construct deterministic linear dynamical systems that govern the exact time evolution of the moments of uncertain position in the presence of uncertain control inputs. The presented methods are demonstrated on realistic predictions from DNNs trained on the Argoverse and CARLA datasets and are shown to be effective for rapidly assessing the probability of low probability events.

Appendix

1.1 A. Moments and characteristic functions of mixture models

Let $f_X$ denote the pdf of a K-component mixture model X, with pdf components $f_{X_i}, \forall i\in [K]$ and let $f_Z$ denote the pdf of the K category Multinoulli. Then, by definition $f_X(x) = \sum _{i=1}^m f_{X_i}(x) f_Z(i)$. For any measurable function g, by interchanging the order of integration and summation, the following holds true

$$\begin{aligned} {\mathbb {E}}[g(X)]&= \int g(x) f_X(x) dx \end{aligned}$$

(25)

$$\begin{aligned}&= \sum _{i=1}^K f_Z(i)\int g(x) f_{X_i}(x) dx \end{aligned}$$

(26)

By letting $g(X) = X^n$ or $g(X) = e^{itX}$, the moments and characteristic function of X can both be computed as the weighted sum of those of their components.

1.2 B. Moments of trigonometric random variables

In this section, we show how trigonometric moments of the form ${\mathbb {E}}[\cos ^n(X)]$, ${\mathbb {E}}[\sin ^n(X)]$, and ${\mathbb {E}}[\cos ^m(X)\sin ^n(X)]$ can be computed in terms of the characteristic function of the random variable X, denoted by $\Phi _{X}$ (Jasour et al. 2021; Wang et al. 2020a). We begin by applying Euler’s Identity to the definition of the characteristic function as follows:

$$\begin{aligned} \begin{aligned} \Phi _{X}(t)&= {\mathbb {E}}[e^{itX}]\\&= {\mathbb {E}}[\cos (tX)] + i{\mathbb {E}}[\sin (tX)] \end{aligned} \end{aligned}$$

(27)

Thus, we have that ${\mathbb {E}}[\cos (tX)] = \text {Re}(\Phi _{X}(t))$ and ${\mathbb {E}}[\sin (tX)] = \text {Im}(\Phi _{X}(t))$. This immediately gives us the ability to compute the first moments of our trigonometric random variables. For higher moments, the trigonometric power formulas can be used to express quantities of the form $\cos ^n(X)$ as the sum of quantities of the form $\cos (mX)$ where $m\in {\mathbb {N}}$ and similarly for $\sin ^n(X)$ (Zwillinger 2002). Thus, higher moments of $\sin (X)$ and $\cos (X)$ can be computed using $\Phi _X(t)$. More precisely, given $n \in {\mathbb {N}},$ trigonometric moments of order n of the forms ${\mathbb {E}}[{cos}^{n}(X)]$ and ${\mathbb {E}}[{sin}^{n}(X)]$ reads as (Jasour et al. 2021):

$$\begin{aligned}&{\mathbb {E}}[{cos}^{n}(X)] =\frac{1}{2^{n}} \sum _{k=0}^{n} \left( {\begin{array}{c}n\\ k\end{array}}\right) \Phi _{X}(2k-n) \end{aligned}$$

(28)

$$\begin{aligned}&{\mathbb {E}}[{sin}^{n}(X)]=\frac{(-i)^{n}}{2^{n}} \sum _{k=0}^{n} \left( {\begin{array}{c}n\\ k\end{array}}\right) (-1)^{n-k}\Phi _{X}(2k-n) \end{aligned}$$

(29)

where $\left( {\begin{array}{c}n\\ k\end{array}}\right) =\frac{n !}{k!(n-k)!}$.

Trigonometric moments of the form:

$$\begin{aligned} {\mathbb {E}}[\cos ^m(X)\sin ^n(X)] \end{aligned}$$

(30)

can also ultimately be computed in terms of $\Phi _X(t)$. This can be seen if we make the substitutions $\cos (X) = \frac{1}{2}(e^{ix} + e^{-ix})$ and $\sin (X) = \frac{1}{2i}(e^{ix} - e^{-ix})$, then (30) can be expressed as:

$$\begin{aligned} {\mathbb {E}}\left[ \frac{1}{i^n2^{m + n}}(e^{iX} + e^{-iX})^m(e^{iX} - e^{-iX})^n\right] \end{aligned}$$

(31)

By applying the binomial theorem to both expressions in parentheses, and multipying the resulting expressions, we find the entire expression in the expectation operator can be expressed as a polynomial in $e^{iX}$ and $e^{-iX}$. Thus, the entire expression can be written as the sum of terms of the form ${\mathbb {E}}[e^{itX}]$ for $t\in {\mathbb {Z}}$ which is in the definition of $\Phi _X(t)$. More precisely, given $(n,m) \in {\mathbb {N}}^2$, trigonometric moment of the form ${\mathbb {E}}\left[ {cos}^{m}(X){sin}^{n}(X) \right]$ reads as (Jasour et al. 2021):

$$\begin{aligned} {\mathbb {E}}\left[ {cos}^{m}(X){sin}^{n}(X) \right]= & {} \frac{(-i)^{n}}{2^{m+n}} \sum \nolimits _{(k_1,k_2)=(0,0)}^{(m,n)} \left( {\begin{array}{c}m\\ k_1\end{array}}\right) \left( {\begin{array}{c}n\\ k_2\end{array}}\right) \nonumber \\&(-1)^{n-k_2}\Phi _{X}\left( 2(k_1+k_2)-m-n\right) \nonumber \\ \end{aligned}$$

(32)