Consistency of Bayesian inference with Gaussian process priors in an elliptic inverse problem

2.1.1. Main notation

Throughout the paper, $\mathcal{O}\subset {\mathbb{R}}^{d},\;d\in \mathbb{N}$, is a given nonempty open and bounded set with smooth boundary $\partial \mathcal{O}$ and closure $\overline{\mathcal{O}}$.

The spaces of continuous functions defined on $\mathcal{O}$ and $\overline{\mathcal{O}}$ are respectively denoted $C\left(\mathcal{O}\right)$ and $C\left(\overline{\mathcal{O}}\right)$, and endowed with the supremum norm ||⋅||_∞. For positive integers $\beta \in \mathbb{N}$, ${C}^{\beta }\left(\mathcal{O}\right)$ is the space of β-times differentiable functions with uniformly continuous derivatives; for non-integer β > 0, ${C}^{\beta }\left(\mathcal{O}\right)$ is defined as

$$\begin{equation*}{C}^{\beta }\left(\mathcal{O}\right)=\left\{f\in {C}^{\lfloor \beta \rfloor }\left(\mathcal{O}\right):\forall \vert i\vert =\lfloor \beta \rfloor ,\underset{x,y\in \mathcal{O},x\ne y}{\mathrm{sup}}\frac{\vert {D}^{i}f\left(x\right)-{D}^{i}f\left(y\right)\vert }{\vert x-y{\vert }^{\beta -\lfloor \beta \rfloor }}{< }\infty \right\},\end{equation*}$$

where ⌊β⌋ denotes the largest integer less than or equal to β, and for any multi-index i = (i₁, ..., i_d), Dⁱ is the ith partial differential operator. ${C}^{\beta }\left(\mathcal{O}\right)$ is normed by

$$\begin{equation*}{\Vert}f{{\Vert}}_{{C}^{\beta }\left(\mathcal{O}\right)}=\sum _{\vert i\vert {\leqslant}\lfloor \beta \rfloor }\underset{x\in \mathcal{O}}{\mathrm{sup}}\vert {D}^{i}f\left(x\right)\vert +\sum _{\vert i\vert =\lfloor \beta \rfloor }\underset{x,y\in \mathcal{O},\;x\ne y}{\mathrm{sup}}\frac{\vert {D}^{i}f\left(x\right)-{D}^{i}f\left(y\right)\vert }{\vert x-y{\vert }^{\beta -\lfloor \beta \rfloor }},\end{equation*}$$

where the second summand is removed for integer β. We denote by ${C}^{\infty }\left(\mathcal{O}\right)={\cap }_{\beta }{C}^{\beta }\left(\mathcal{O}\right)$ the set of smooth functions, and by ${C}_{c}^{\infty }\left(\mathcal{O}\right)$ the subspace of elements in ${C}^{\infty }\left(\mathcal{O}\right)$ with compact support contained in $\mathcal{O}$.

Denote by ${L}^{2}\left(\mathcal{O}\right)$ the Hilbert space of square integrable functions on $\mathcal{O}$, equipped with its usual inner product ${\langle \cdot ,\cdot \rangle }_{{L}^{2}\left(\mathcal{O}\right)}$. For integer α ⩾ 0, the order-α Sobolev space on $\mathcal{O}$ is the separable Hilbert space

$$\begin{align*}\hfill {H}^{\alpha }\left(\mathcal{O}\right)& =\left\{f\in {L}^{2}\left(\mathcal{O}\right):\forall \vert i\vert {\leqslant}\alpha ,\;\exists \;{D}^{i}f\in {L}^{2}\left(\mathcal{O}\right)\right\},\hfill \\ {\langle f,g\rangle }_{{H}^{\alpha }\left(\mathcal{O}\right)}\hfill & =\sum _{\vert i\vert {\leqslant}\alpha }{\langle {D}^{i}f,{D}^{i}g\rangle }_{{L}^{2}\left(\mathcal{O}\right)}.\hfill \end{align*}$$

For non-integer α ⩾ 0, ${H}^{\alpha }\left(\mathcal{O}\right)$ can be defined by interpolation, for example, Lions and Magenes (1972). For any α ⩾ 0, ${H}_{c}^{\alpha }\left(\mathcal{O}\right)$ will denote the completion of ${C}_{c}^{\infty }\left(\mathcal{O}\right)$ with respect to the norm ${\Vert}\cdot {{\Vert}}_{{H}^{\alpha }\left(\mathcal{O}\right)}$. Finally, if K is a nonempty compact subset of $\mathcal{O}$, we denote by ${H}_{K}^{\alpha }\left(\mathcal{O}\right)$ the closed subspace of functions in ${H}^{\alpha }\left(\mathcal{O}\right)$ with support contained in K. Whenever there is no risk of confusion, we will omit the reference to the underlying domain $\mathcal{O}$.

Throughout, we use the symbols ≲ and ≳ for inequalities holding up to a universal constant. Also, for two real sequences (a_N) and (b_N), we say that a_N ≃ b_N if both a_N ≲ b_N and b_N ≲ a_N for all N large enough. For a sequence of random variables Z_N we write Z_N = O_Pr(a_N) if for all ɛ > 0 there exists M_ɛ < ∞ such that for all N large enough, Pr(|Z_N| ⩾ M_ɛa_N) < ɛ. Finally, we will denote by $\mathcal{L}\left(Z\right)$ the law of a random variable Z.

2.1.2. Parameter spaces and link functions

Let $g\in {C}^{\infty }\left(\mathcal{O}\right)$ be an arbitrary source function, which will be regarded as fixed throughout. For $f\in {C}^{\beta }\left(\mathcal{O}\right),\;\beta { >}1$, consider the boundary value problem

$$\begin{equation}\begin{cases}\nabla \cdot \left(f\nabla u\right)=g\quad \text{on} \mathcal{O},\quad \hfill \\ u=0\quad \text{on} \partial \mathcal{O}.\quad \hfill \end{cases}\end{equation} \tag{ 3 }$$

If we assume that f ⩾ K_min > 0 on $\mathcal{O}$, then standard elliptic theory (e.g. Gilbarg and Trudinger (1998)) implies that (3) has a classical solution $G\left(f\right)\equiv {u}_{f}\in C\left(\overline{\mathcal{O}}\right)\cap {C}^{1+\beta }\left(\mathcal{O}\right)$.

We consider the following parameter space for f: for integer α > 1 + d/2, K_min ∈ (0, 1), and denoting by n = n(x) the outward pointing normal at $x\in \partial \mathcal{O}$, let

$$\begin{equation}{\mathcal{F}}_{\alpha ,{K}_{\mathrm{min}}}=\left\{f\in {H}^{\alpha }\left(\mathcal{O}\right):{\mathrm{inf}}_{x\in \mathcal{O}}f\left(x\right){ >}{K}_{\mathrm{min}},\;{f}_{\vert \partial \mathcal{O}}=1,\;{\frac{{\partial }^{j}f}{\partial {n}^{j}}}_{\vert \partial \mathcal{O}}=0\;\quad \text{for} 1{\leqslant}j{\leqslant}\alpha -1\right\}.\end{equation} \tag{ 4 }$$

Our approach will be to place a prior probability measure on the unknown conductivity f and base our inference on the posterior distribution of f given noisy observations of G(f), via Bayes' theorem. It is of interest to use Gaussian process priors. Such probability measures are naturally supported in linear spaces (in our case ${H}_{c}^{\alpha }\left(\mathcal{O}\right)$) and we now introduce a bijective re-parametrisation so that the prior for f is supported in the relevant parameter space ${\mathcal{F}}_{\alpha ,{K}_{\mathrm{min}}}$. We follow the approach of using regular link functions Φ as in Nickl et al (2020).

Condition 1. For given K_min > 0, let ${\Phi}:\mathbb{R}\to \left({K}_{\mathrm{min}},\infty \right)$ be a smooth, strictly increasing bijective function such that Φ(0) = 1, ${{\Phi}}^{\prime }\left(t\right){ >}0,t\in \mathbb{R}$, and assume that all derivatives of Φ are bounded on $\mathbb{R}$.

For some of the results to follow it will prove convenient to slightly strengthen the previous condition.

Condition 2. Let Φ be as in condition 1, and assume furthermore that Φ' is nondecreasing and that lim inf_t→−∞Φ'(t)t^a > 0 for some a > 0.

For a = 2, an example of such a link function is given in example 24 below. Note however that the choice of Φ = exp is not permitted in either condition.

Given any link function Φ satisfying condition 1, one can show (cf. Nickl et al (2020), section 3.1) that the set ${\mathcal{F}}_{\alpha ,{K}_{\mathrm{min}}}$ in (4) can be realised as the family of composition maps

$$\begin{equation*}{\mathcal{F}}_{\alpha ,{K}_{\mathrm{min}}}=\left\{{\Phi}{\circ}F:F\in {H}_{c}^{\alpha }\left(\mathcal{O}\right)\right\},\quad \alpha \in \mathbb{N}.\end{equation*}$$

We then regard the solution map associated to (3) as one defined on ${H}_{c}^{\alpha }$ via

$$\begin{equation}\mathcal{G}:{H}_{c}^{\alpha }\left(\mathcal{O}\right)\to {L}^{2}\left(\mathcal{O}\right),\quad F{\mapsto}\mathcal{G}\left(F\right){:=}G\left({\Phi}{\circ}F\right),\end{equation} \tag{ 5 }$$

where G(Φ◦F) is the solution to (3) now with $f={\Phi}{\circ}F\in {\mathcal{F}}_{\alpha ,{K}_{\mathrm{min}}}$. In the results to follow, we will implicitly assume a link function Φ to be given and fixed, and understand the re-parametrised solution map $\mathcal{G}$ as being defined as in (5) for such choice of Φ.

2.1.3. Measurement model

Define the uniform distribution on $\mathcal{O}$ by $\mu =\mathrm{d}x/\text{vol}\left(\mathcal{O}\right)$, where dx is the Lebesgue measure and $\text{vol}\left(\mathcal{O}\right)={\int }_{\mathcal{O}}\;\mathrm{d}x$, and consider random design variables

$$\begin{equation}{\left({X}_{i}\right)}_{i=1}^{N}\text{iid}{\sim }\mu ,\quad N\in \mathbb{N}.\end{equation} \tag{ 6 }$$

For unknown $f\in {\mathcal{F}}_{\alpha ,{K}_{\mathrm{min}}}$, we model the statistical errors under which we observe the corresponding measurements ${\left\{G\left(f\right)\left({X}_{i}\right)\right\}}_{i=1}^{N}$ by i.i.d. Gaussian random variables W_i ∼ N(0, 1), all independent of the X_i's. Using the re-parameterisation f = Φ◦F via a given link function from the previous subsection, the observation scheme is then

$$\begin{equation}{Y}_{i}=\mathcal{G}\left(F\right)\left({X}_{i}\right)+\sigma {W}_{i},\quad i=1,\dots ,\;N,\end{equation} \tag{ 7 }$$

where σ > 0 is the noise amplitude. We will often use the shorthand notation ${Y}^{\left(N\right)}={\left({Y}_{i}\right)}_{i=1}^{N}$, with analogous definitions for X^(N) and W^(N). The random vectors (Y_i, X_i) on $\mathbb{R}{\times}\mathcal{O}$ are then i.i.d with laws denoted as ${P}_{F}^{i}$. Writing dy for the Lebesgue measure on $\mathbb{R}$, it follows that ${P}_{F}^{i}$ has Radon–Nikodym density

$$\begin{equation}{p}_{F}\left(y,x\right){:=}\frac{\mathrm{d}{P}_{F}^{i}}{\mathrm{d}y{\times}\mathrm{d}\mu }\left(y,x\right)=\frac{1}{\sqrt{2\pi {\sigma }^{2}}}\;{\mathrm{e}}^{-{\left[y-\mathcal{G}\left(F\right)\left(x\right)\right]}^{2}/\left(2{\sigma }^{2}\right)},\quad y\in \mathbb{R},\;x\in \mathcal{O}.\end{equation} \tag{ 8 }$$

We will write ${P}_{F}^{N}={\otimes }_{i=1}^{N}{P}_{F}^{i}$ for the joint law of (Y^(N), X^(N)) on ${\mathbb{R}}^{N}{\times}{\mathcal{O}}^{N}$, with ${E}_{F}^{i}$, ${E}_{F}^{N}$ the expectation operators corresponding to the laws ${P}_{F}^{i}$, ${P}_{F}^{N}$ respectively. In the sequel we sometimes use the notation ${P}_{f}^{N}$ instead of ${P}_{F}^{N}$ when convenient.

2.1.4. The Bayesian approach

In the Bayesian approach one models the parameter $F\in {H}_{c}^{\alpha }\left(\mathcal{O}\right)$ by a Borel probability measure Π supported in the Banach space $C\left(\mathcal{O}\right)$. Since the map (F, (y, x)) ↦ p_F(y, x) can be shown to be jointly measurable, the posterior distribution Π(⋅|Y^(N), X^(N)) of F|(Y^(N), X^(N)) arising from data in model (7) equals, by Bayes' formula (p 7, Ghosal and van der Vaart 2017),

$$\begin{equation}{\Pi}\left(B\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right)=\frac{{\int }_{B}{\mathrm{e}}^{{\ell }^{\left(N\right)}\left(F\right)}\mathrm{d}{\Pi}\left(F\right)}{{\int }_{C\left(\mathcal{O}\right)}{\mathrm{e}}^{{\ell }^{\left(N\right)}\left(F\right)}\mathrm{d}{\Pi}\left(F\right)}\quad \text{any} \text{Borel} \text{set} B\subseteq C\left(\mathcal{O}\right),\end{equation} \tag{ 9 }$$

where

$$\begin{equation}{\ell }^{\left(N\right)}\left(F\right)=-\frac{1}{2{\sigma }^{2}}\sum _{i=1}^{N}{\left[{Y}_{i}-\mathcal{G}\left(F\right)\left({X}_{i}\right)\right]}^{2}\end{equation} \tag{ 10 }$$

is (up to an additive constant) the joint log-likelihood function.

In this section we will show that the posterior distribution arising from certain priors concentrates near any sufficiently regular ground truth F₀ (or, equivalently, f₀), and provide a bound on the rate of this contraction, assuming the observation (Y^(N), X^(N)) to be generated through model (7) of law ${P}_{{F}_{0}}^{N}$. We will regard σ > 0 as a fixed and known constant; in practice it may be replaced by the estimated sample variance of the Y_i's.

The priors we will consider are built around a Gaussian process base prior Π', but to deal with the non-linearity of the inverse problem, some additional regularisation will be required. We first show how this can be done by an N-dependent 'rescaling' step as suggested in Monard et al (2020). We then further show that a randomised truncation of a Karhunen–Loeve-type series expansion of the base prior also leads to a consistent, 'fully Bayesian' solution of this inverse problem.

2.2.1. Results with re-scaled Gaussian priors

We will freely use terminology from the basic theory of Gaussian processes and measures, see, for example, Giné and Nickl (2016), chapter 2 for details.

Condition 3. Let α > 1 + d/2, β ⩾ 1, and let $\mathcal{H}$ be a Hilbert space continuously imbedded into ${H}_{c}^{\alpha }\left(\mathcal{O}\right)$. Let Π' be a centred Gaussian Borel probability measure on the Banach space $C\left(\mathcal{O}\right)$ that is supported on a separable measurable linear subspace of ${C}^{\beta }\left(\mathcal{O}\right)$, and assume that the reproducing-kernel Hilbert space (RKHS) of Π' equals $\mathcal{H}$.

As a basic example of a Gaussian base prior Π' satisfying condition 3, consider a Whittle–Matérn process $M=\left\{M\left(x\right),\;x\in \mathcal{O}\right\}$ indexed by $\mathcal{O}$ and of regularity α (cf. example 25 below for full details). We will assume that it is known that ${F}_{0}\in {H}^{\alpha }\left(\mathcal{O}\right)$ is supported inside a given compact subset K of the domain $\mathcal{O}$, and fix any smooth cut-off function $\chi \in {C}_{c}^{\infty }\left(\mathcal{O}\right)$ such that χ = 1 on K. Then, ${{\Pi}}^{\prime }=\mathcal{L}\left(\chi M\right)$ is supported on the separable linear subspace ${C}^{{\beta }^{\prime }}\left(\mathcal{O}\right)$ of ${C}^{\beta }\left(\mathcal{O}\right)$ for any β < β' < α − d/2, and its RKHS $\mathcal{H}=\left\{\chi F,F\in {H}^{\alpha }\left(\mathcal{O}\right)\right\}$ is continuously imbedded into ${H}_{c}^{\alpha }\left(\mathcal{O}\right)$ (and contains ${H}_{K}^{\alpha }\left(\mathcal{O}\right)$). The condition ${F}_{0}\in \mathcal{H}$ that is employed in the following theorems then amounts to the standard assumption that ${F}_{0}\in {H}^{\alpha }\left(\mathcal{O}\right)$ be supported in a strict subset K of $\mathcal{O}$.

To proceed, if Π' is as above and F' ∼ Π', we consider the 're-scaled' prior

$$\begin{equation}{{\Pi}}_{N}=\mathcal{L}\left({F}_{N}\right),\quad {F}_{N}=\frac{1}{{N}^{d/\left(4\alpha +4+2d\right)}}{F}^{\prime },\end{equation} \tag{ 11 }$$

Then Π_N again defines a centred Gaussian prior on $C\left(\mathcal{O}\right)$, and a basic calculation (e.g., exercise 2.6.5 in Giné and Nickl (2016)) shows that its RKHS ${\mathcal{H}}_{N}$ is still given by $\mathcal{H}$ but now with norm

$$\begin{equation}{\Vert}F{{\Vert}}_{{\mathcal{H}}_{N}}={N}^{d/\left(4\alpha +4+2d\right)}{\Vert}F{{\Vert}}_{\mathcal{H}}\quad \forall F\in \mathcal{H}.\end{equation} \tag{ 12 }$$

Our first result shows that the posterior contracts towards F₀ in 'prediction'-risk at rate N^{−(α+1)/(2α+2+d)} and that, moreover, the posterior draws possess a bound on their C^β-norm with overwhelming frequentist probability.

Theorem 4. For fixed integer α > β + d/2, β ⩾ 1, consider the Gaussian prior Π_N in (11) with base prior F' ∼ Π' satisfying condition 3 for RKHS $\mathcal{H}$. Let Π_N(⋅|Y^(N), X^(N)) be the resulting posterior distribution arising from observations (Y^(N), X^(N)) in (7), set δ_N = N^{−(α+1)/(2α+2+d)}, and assume ${F}_{0}\in \mathcal{H}$.

Then for any D > 0 there exists L > 0 large enough (depending on σ, F₀, D, α, β, as well as on $\mathcal{O},d,g$) such that, as N → ∞,

$$\begin{equation}{{\Pi}}_{N}\left(F:{\Vert}\mathcal{G}\left(F\right)-\mathcal{G}\left({F}_{0}\right){{\Vert}}_{{L}^{2}}{ >}L{\delta }_{N}\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right)={O}_{{P}_{{F}_{0}}^{N}}\left({\mathrm{e}}^{-DN{\delta }_{N}^{2}}\right),\end{equation} \tag{ 13 }$$

and for sufficiently large M > 0 (depending on σ, D, α, β)

$$\begin{equation}{{\Pi}}_{N}\left(F:{\Vert}F{{\Vert}}_{{C}^{\beta }}{ >}M\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right)={O}_{{P}_{{F}_{0}}^{N}}\left({\mathrm{e}}^{-DN{\delta }_{N}^{2}}\right).\end{equation} \tag{ 14 }$$

Following ideas in Monard et al (2020), we can combine (13) with the regularisation property (14) and a suitable stability estimate for G⁻¹ to show that the posterior contracts about f₀ also in L²-risk. We shall employ the stability estimate proved in Nickl et al (2020), lemma 24, which requires the source function g in the base PDE (3) to be strictly positive, a natural condition ensuring injectivity of the map f ↦ G(f), see Richter (1981). Denote the push-forward posterior on the conductivities f by

$$\begin{equation}{\tilde {{\Pi}}}_{N}\left(\cdot \vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right){:=}\mathcal{L}\left(f\right),\quad f={\Phi}{\circ}F:F\sim {{\Pi}}_{N}\left(\cdot \vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right).\end{equation} \tag{ 15 }$$

Theorem 5. Let Π_N(⋅|Y^(N), X^(N)), δ_N and F₀ be as in theorem 4 for integer β > 1. Let f₀ = Φ◦F₀ and assume in addition that ${\mathrm{inf}}_{x\in \mathcal{O}}g\left(x\right){\geqslant}{g}_{\mathrm{min}}{ >}0$. Then for any D > 0 there exists L > 0 large enough (depending on $\sigma ,{f}_{0},D,\alpha ,\beta ,\mathcal{O}$, g_min, d) such that, as N → ∞,

$$\begin{equation*}{\tilde {{\Pi}}}_{N}\left(f:{\Vert}f-{f}_{0}{{\Vert}}_{{L}^{2}}{ >}L{N}^{-\lambda }\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right)={O}_{{P}_{{f}_{0}}^{N}}\left({\mathrm{e}}^{-DN{\delta }_{N}^{2}}\right),\lambda =\frac{\left(\alpha +1\right)\left(\beta -1\right)}{\left(2\alpha +2+d\right)\left(\beta +1\right)}.\end{equation*}$$

We note that as the smoothness α of f₀ increases, we can employ priors of higher regularity α, β. In particular, if F₀ ∈ C^∞ = ∩_α>0H^α, we can let the above rate N^−λ be as closed as desired to the 'parametric' rate N^−1/2.

We conclude this section showing that the posterior mean E^Π[F|Y^(N), X^(N)] of Π_N(⋅|Y^(N), X^(N)) converges to F₀ at the rate N^−λ from theorem 5. We formulate this result at the level of the vector space valued parameter F (instead of for conductivities f), as the most commonly used MCMC algorithms (such as pCN, see subsection 2.3.1) target the posterior distribution of F.

Theorem 6. Under the hypotheses of theorem 5, let ${\overline{F}}_{N}={E}^{{\Pi}}\left[F\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right]$ be the (Bochner-)mean of Π_N(⋅|Y^(N), X^(N)). Then, as N → ∞,

$$\begin{equation}{P}_{{F}_{0}}^{N}\left({\Vert}{\overline{F}}_{N}-{F}_{0}{{\Vert}}_{{L}^{2}}{ >}{N}^{-\lambda }\right)\to 0.\end{equation} \tag{ 16 }$$

The same result holds for the implied conductivities, that is, for ${\Vert}{\Phi}{\circ}{\overline{F}}_{N}-{f}_{0}{{\Vert}}_{{L}^{2}}$ replacing ${\Vert}{\overline{F}}_{N}-{F}_{0}{{\Vert}}_{{L}^{2}}$, since composition with Φ is Lipschitz.

2.2.2. Extension to high-dimensional Gaussian sieve priors

It is often convenient, for instance for computational reasons as discussed in subsection 2.3.1, to employ 'sieve'-priors that are concentrated on a finite-dimensional approximation of the parameter space supporting the prior. For example a truncated Karhunen–Loeve-type series expansion (or some other discretisation) of the Gaussian base prior Π' is frequently used (Dashti and Stuart 2011, Hairer et al 2014). The theorems of the previous subsection remain valid if the approximation spaces are appropriately chosen.

Let us illustrate this by considering a Gaussian series prior based on an orthonormal basis $\left\{{{\Psi}}_{\ell r},\;\ell {\geqslant}-1, r\in {\mathbb{Z}}^{d}\right\}$ of ${L}^{2}\left({\mathbb{R}}^{d}\right)$ composed of sufficiently regular, compactly supported Daubechies wavelets (see chapter 4 in Giné and Nickl (2016) for details). We assume that ${F}_{0}\in {H}_{K}^{\alpha }\left(\mathcal{O}\right)$ for some $K\subset \mathcal{O}$, and denote by ${\mathcal{R}}_{\ell }$ the set of indices r for which the support of Ψ_ℓr intersects K. Fix any compact ${K}^{\prime }\subset \mathcal{O}$ such that K ⊊ K', and a cut-off function $\chi \in {C}_{c}^{\infty }\left(\mathcal{O}\right)$ such that χ = 1 on K'. For any real α > 1 + d/2, consider the prior Π'_J arising as the law of the Gaussian random sum

$$\begin{equation}{{\Pi}}_{J}^{\prime }=\mathcal{L}\left(\chi F\right),\quad F=\sum _{\ell {\leqslant}J,r\in {\mathcal{R}}_{\ell }}{2}^{-\ell \alpha }{F}_{\ell r}{{\Psi}}_{\ell r},\quad \;{F}_{\ell r}\text{iid}{\sim }N\left(0,1\right),\end{equation} \tag{ 17 }$$

where J = J_N → ∞ is a (deterministic) truncation point to be chosen. Then Π'_J defines a centred Gaussian prior that is supported on the finite-dimensional space

$$\begin{equation}{\mathcal{H}}_{J}=\text{span}\left\{\chi {{\Psi}}_{\ell r},\;\ell {\leqslant}J,\;r\in {\mathcal{R}}_{\ell }\right\}\subset C\left(\mathcal{O}\right).\end{equation} \tag{ 18 }$$

Proposition 7. Consider a prior Π_N as in (11) where now F' ∼ Π'_J and $J={J}_{N}\in \mathbb{N}$ is such that 2^J ≃ N^1/(2α+2+d). Let Π_N(⋅|Y^(N), X^(N)) be the resulting posterior distribution arising from observations (Y^(N), X^(N)) in (7), and assume ${F}_{0}\in {H}_{K}^{\alpha }\left(\mathcal{O}\right)$. Then the conclusions of theorems 4–6 remain valid (under the respective hypotheses on α, β, g).

A similar result could be proved for more general Gaussian priors (not of wavelet type), but we refrain from giving these extensions here.

2.2.3. Randomly truncated Gaussian series priors

In this section we show that instead of rescaling Gaussian base priors Π', Π'_J in an N-dependent way to attain extra regularisation, one may also randomise the dimensionality parameter J in (17) by a hyper-prior with suitable tail behaviour. While this is computationally somewhat more expensive (by necessitating a hierarchical sampling method, see subsection 2.3.1), it gives a possibly more principled approach to ('fully') Bayesian regularisation in our inverse problem. The theorem below will show that such a procedure is consistent in the frequentist sense, at least for smooth enough F₀.

For the wavelet basis and cut-off function χ introduced before (17), we consider again a random (conditionally Gaussian) sum

$$\begin{equation}{\Pi}=\mathcal{L}\left(\chi F\right),\quad F=\sum _{\ell {\leqslant}J,r\in {\mathcal{R}}_{\ell }}{2}^{-\ell \alpha }{F}_{\ell r}{{\Psi}}_{\ell r},\;{F}_{\ell r}\text{iid}{\sim }N\left(0,1\right)\end{equation} \tag{ 19 }$$

where now J is a random truncation level, independent of the random coefficients F_ℓr, satisfying the following inequalities:

$$\begin{equation}\mathrm{Pr}\left(J{ >}j\right)={\mathrm{e}}^{-{2}^{jd}\mathrm{log}{2}^{jd}}\;\forall j{\geqslant}1;\quad \mathrm{Pr}\left(J=j\right)\gtrsim {\mathrm{e}}^{-{2}^{jd}\mathrm{log}{2}^{jd}},\quad j\to \infty .\end{equation} \tag{ 20 }$$

When d = 1, a (log-)Poisson random variable satisfies these tail conditions, and for d > 1 such a random variable J can be easily constructed too—see example 28 below.

Our first result in this section shows that the posterior arising from the truncated series prior in (19) achieves (up to a log-factor) the same contraction rate in L²-prediction risk as the one obtained in theorem 4. Moreover, as is expected in light of the results in van der Vaart and van Zanten (2009) and Ray (2013), the posterior adapts to the unknown regularity α₀ of F₀ when it exceeds the base smoothness level α.

Theorem 8. For any α > 1 + d/2, let Π be the random series prior in (19), and let Π(⋅|Y^(N), X^(N)) be the resulting posterior distribution arising from observations (Y^(N), X^(N)) in (7). Then, for each α₀ ⩾ α and any ${F}_{0}\in {H}_{K}^{{\alpha }_{0}}\left(\mathcal{O}\right)$, we have that for any D > 0 there exists L > 0 large enough (depending on $\sigma ,{F}_{0},D,\alpha ,\mathcal{O},d,g$) such that, as N → ∞,

$$\begin{equation*}{\Pi}\left(F:{\Vert}\mathcal{G}\left(F\right)-\mathcal{G}\left({F}_{0}\right){{\Vert}}_{{L}^{2}}{ >}L{\xi }_{N}\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right)={O}_{{P}_{{F}_{0}}^{N}}\left({\mathrm{e}}^{-DN{\xi }_{N}^{2}}\right),\end{equation*}$$

where ${\xi }_{N}={N}^{-\left({\alpha }_{0}+1\right)/\left(2{\alpha }_{0}+2+d\right)}\mathrm{log}N$. Moreover, for ${\mathcal{H}}_{J}$ the finite-dimensional subspaces in (18) and ${J}_{N}\in \mathbb{N}$ such that ${2}^{{J}_{N}}\simeq {N}^{1/\left(2{\alpha }_{0}+2+d\right)}$, we also have that for sufficiently large M > 0 (depending on D, α)

$$\begin{equation}{\Pi}\left(F:F\in {\mathcal{H}}_{{J}_{N}},\;{\Vert}F{{\Vert}}_{{H}^{\alpha }}{\leqslant}M{2}^{{J}_{N}\alpha }N{\xi }_{N}^{2}\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right)=1-{O}_{{P}_{{F}_{0}}^{N}}\left({\mathrm{e}}^{-DN{\xi }_{N}^{2}}\right).\end{equation} \tag{ 21 }$$

We can now exploit the previous result along with the finite-dimensional support of the posterior and again the stability estimate from Nickl et al (2020) to obtain the following consistency theorem for ${F}_{0}\in {H}^{{\alpha }_{0}}$ if α₀ is large enough (with a precise bound α₀ ⩾ α* given in the proof of lemma 12).

Theorem 9. Let the link function Φ in the definition (5) of $\mathcal{G}$ satisfy condition 2. Let Π(⋅|Y^(N), X^(N)), ξ_N be as in theorem 8, assume in addition g ⩾ g_min > 0 on $\mathcal{O}$, and let $\tilde {{\Pi}}\left(\cdot \vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right)$ be the posterior distribution of f as in (15). Then for f₀ = Φ◦F₀ with ${F}_{0}\in {H}_{K}^{{\alpha }_{0}}\left(\mathcal{O}\right)$ for α₀ large enough (depending on α, d, a) and for any D > 0 there exists L > 0 large enough (depending on $\sigma ,{f}_{0},D,\alpha ,\mathcal{O},{g}_{\mathrm{min}},d$) such that, as N → ∞,

$$\begin{equation*}\tilde {{\Pi}}\left(f:{\Vert}f-{f}_{0}{{\Vert}}_{{L}^{2}}{ >}L{N}^{-\rho }\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right)={O}_{{P}_{{f}_{0}}^{N}}\left({\mathrm{e}}^{-DN{\xi }_{N}^{2}}\right),\rho =\frac{\left({\alpha }_{0}+1\right)\left(\alpha -1\right)}{\left(2{\alpha }_{0}+2+d\right)\left(\alpha +1\right)}.\end{equation*}$$

Just as before, for f₀ ∈ C^∞ the above rate can be made as close as desired to N^−1/2 by choosing α large enough. Moreover, the last contraction theorem also translates into a convergence result for the posterior mean of F.

Theorem 10. Under the hypotheses of theorem 9, let ${\overline{F}}_{N}={E}^{{\Pi}}\left[F\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right]$ be the mean of Π(⋅|Y^(N), X^(N)). Then, as N → ∞,

$$\begin{equation}{P}_{{F}_{0}}^{N}\left({\Vert}{\overline{F}}_{N}-{F}_{0}{{\Vert}}_{{L}^{2}}{ >}{N}^{-\rho }\right)\to 0.\end{equation} \tag{ 22 }$$

We note that the proof of the last two theorems crucially takes advantage of the 'non-symmetric' and 'non-exponential' nature of the stability estimate from Nickl et al (2020), and may not hold in other non-linear inverse problems where such an estimate may not be available (e.g., as in Monard et al (2020), Abraham and Nickl (2020) or also in the Schrödinger equation setting studied in Nickl et al (2020), Nickl (2018)).

Let us conclude this section by noting that hierarchical priors such as the one studied here are usually devised to 'adapt to unknown' smoothness α₀ of F₀, see van der Vaart and van Zanten (2009) and Ray (2013). Note that while our posterior distribution is adaptive to α₀ in the 'prediction risk' setting of theorem 8, the rate N^−ρ obtained in theorems 9 and 10 for the inverse problem does depend on the minimal smoothness α, and is therefore not adaptive. Nevertheless, this hierarchical prior gives an example of a fully Bayesian, consistent solution of our inverse problem.

2.3.1. Posterior computation

2.3.2. Open problems: towards optimal convergence rates

Proof of theorem 5. The conclusions of theorem 4 can readily be translated for the push-forward posterior ${\tilde {{\Pi}}}_{N}\left(\cdot \vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right)$ from (15). In particular, (13) implies, for f₀ = Φ◦F₀, as N → ∞,

$$\begin{equation}{\tilde {{\Pi}}}_{N}\left(f:{\Vert}G\left(f\right)-G\left({f}_{0}\right){{\Vert}}_{{L}^{2}}{ >}L{\delta }_{N}\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right)={O}_{{P}_{{f}_{0}}^{N}}\left({\mathrm{e}}^{-DN{\delta }_{N}^{2}}\right);\end{equation} \tag{ 26 }$$

and using lemma 29 in Nickl et al (2020) and (14) we obtain for sufficiently large M' > 0

$$\begin{align}\hfill {\tilde {{\Pi}}}_{N}\left(f:{\Vert}f{{\Vert}}_{{C}^{\beta }}{ >}{M}^{\prime }\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right)& {\leqslant}{{\Pi}}_{N}\left(F:{\Vert}F{{\Vert}}_{{C}^{\beta }}{ >}M\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right)\hfill \\ \hfill & ={O}_{{P}_{{f}_{0}}^{N}}\left({\mathrm{e}}^{-DN{\delta }_{N}^{2}}\right).\hfill \end{align} \tag{ 27 }$$

From the previous bounds we now obtain the following result.□

Lemma 11. For Π_N(⋅|Y^(N), X^(N)), δ_N and F₀ as in theorem 4, let ${\tilde {{\Pi}}}_{N}\left(\cdot \vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right)$ be the push-forward posterior distribution from (15). Then, for f₀ = Φ◦F₀ and any D > 0 there exists L > 0 large enough such that, as N → ∞,

$$\begin{equation*}{\tilde {{\Pi}}}_{N}\left(f:{\Vert}G\left(f\right)-G\left({f}_{0}\right){{\Vert}}_{{H}^{2}}{ >}L{\delta }_{N}^{\left(\beta -1\right)/\left(\beta +1\right)}\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right)={O}_{{P}_{{F}_{0}}^{N}}\left({\mathrm{e}}^{-DN{\delta }_{N}^{2}}\right).\end{equation*}$$

Proof. Using the continuous imbedding of ${C}^{\beta }\subset {H}^{\beta },\beta \in \mathbb{N}$, and (27), for some M' > 0 as N → ∞,

$$\begin{equation*}{\tilde {{\Pi}}}_{N}\left(f:{\Vert}f{{\Vert}}_{{H}^{\beta }}{ >}{M}^{\prime }\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right)={O}_{{P}_{{F}_{0}}^{N}}\left({\mathrm{e}}^{-DN{\delta }_{N}^{2}}\right).\end{equation*}$$

Now if f ∈ H^β with ${\Vert}f{{\Vert}}_{{H}^{\beta }}{\leqslant}{M}^{\prime }$, lemma 23 in Nickl et al (2020) implies G(f), G(f₀) ∈ H^β+1, with

$$\begin{equation*}{\Vert}G\left({f}_{0}\right){{\Vert}}_{H\beta +1}< sim 1+{\Vert}{f}_{0}{{\Vert}}_{{H}^{\beta }}^{\beta \left(\beta +1\right)}{< }\infty , {\Vert}G\left(f\right){{\Vert}}_{{H}^{\beta +1}}< sim 1+{\Vert}f{{\Vert}}_{{H}^{\beta }}^{\beta \left(\beta +1\right)}{< }{M}^{{\prime\prime}}{< }\infty ;\end{equation*}$$

and by the usual interpolation inequality for Sobolev spaces (Lions and Magenes 1972),

$$\begin{align*}\hfill {\Vert}G\left(f\right)-G\left({f}_{0}\right){{\Vert}}_{{H}^{2}}& < sim {\Vert}G\left(f\right)-G{\left({f}_{0}\right){\Vert}}_{{L}^{2}}^{\left(\beta -1\right)/\left(\beta +1\right)}{\Vert}G\left(f\right)-G{\left({f}_{0}\right){\Vert}}_{{H}^{\beta +1}}^{2/\left(\beta +1\right)}\hfill \\ \hfill & < sim {\Vert}G\left(f\right)-G{\left({f}_{0}\right){\Vert}}_{{L}^{2}}^{\left(\beta -1\right)/\left(\beta +1\right)}.\hfill \end{align*}$$

Thus, by what precedes and (26), for sufficiently large L > 0

$$\begin{align*}\hfill & {\tilde {{\Pi}}}_{N}\left(f:{\Vert}G\left(f\right)-G\left({f}_{0}\right){{\Vert}}_{{H}^{2}}{ >}L{\delta }_{N}^{\left(\beta -1\right)/\left(\beta +1\right)}\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right)\hfill \\ \hfill & \quad \;\;\;\;{\leqslant}{\tilde {{\Pi}}}_{N}\left(f:{\Vert}G\left(f\right)-G\left({f}_{0}\right){{\Vert}}_{{L}^{2}}{ >}{L}^{\prime }{\delta }_{N}\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right)\hfill \\ \hfill & \quad \;\;\;\;\;\;\;\;+\;{\tilde {{\Pi}}}_{N}\left(f:{\Vert}f{{\Vert}}_{{H}^{\beta }}{ >}{M}^{{\prime\prime}}\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right)={O}_{{P}_{{F}_{0}}^{N}}\left({\mathrm{e}}^{-DN{\delta }_{N}^{2}}\right),\hfill \end{align*}$$

as N → ∞.□

To prove theorem 5 we use (25), (27) and lemma 11 to the effect that for any D > 0 we can find L, M > 0 large enough such that, as N → ∞,

$$\begin{align*}\hfill & {\tilde {{\Pi}}}_{N}\left(f:{\Vert}f-{f}_{0}{{\Vert}}_{{L}^{2}}{ >}L{\delta }_{N}^{\frac{\beta -1}{\beta +1}}\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right)\hfill \\ \hfill & \quad \;\;\;\;{\leqslant}{\tilde {{\Pi}}}_{N}\left(f:{\Vert}G\left(f\right)-G\left({f}_{0}\right){{\Vert}}_{{H}^{2}}{ >}{L}^{\prime }{\delta }_{N}^{\frac{\beta -1}{\beta +1}}\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right)\hfill \\ \hfill & \quad \;\;\;\;\;\;\;\;+{\tilde {{\Pi}}}_{N}\left(f:{\Vert}f{{\Vert}}_{{C}^{\beta }}{ >}M\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right)={O}_{{P}_{{F}_{0}}^{N}}\left({\mathrm{e}}^{-DN{\delta }_{N}^{2}}\right).\hfill \end{align*}$$

Proof of theorem 6. The proof largely follows ideas of Monard et al (2020) but requires a slightly more involved, iterative uniform integrability argument to also control the probability of events $\left\{F:{\Vert}F{{\Vert}}_{{C}^{\beta }}{ >}M\right\}$ on whose complements we can subsequently exploit regularity properties of the inverse link function Φ⁻¹.

Using Jensen's inequality, it is enough to show, as N → ∞,

$$\begin{equation*}{P}_{{F}_{0}}^{N}\left({E}^{{\Pi}}\left[{\Vert}F-{F}_{0}{{\Vert}}_{{L}^{2}}^{2}\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right]{ >}{N}^{-\lambda }\right)\to 0.\end{equation*}$$

For M > 0 sufficiently large to be chosen, we decompose

$$\begin{align}\hfill {E}^{{\Pi}}\left[{\Vert}F-{F}_{0}{{\Vert}}_{{L}^{2}}\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right]& ={E}^{{\Pi}}\left[{\Vert}F-{F}_{0}{{\Vert}}_{{L}^{2}}{1}_{{\Vert}F{{\Vert}}_{{C}^{\beta }}{\leqslant}M}\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right]\hfill \\ \hfill & \quad +\;{E}^{{\Pi}}\left[{\Vert}F-{F}_{0}{{\Vert}}_{{L}^{2}}{1}_{{\Vert}F{{\Vert}}_{{C}^{\beta }}{ >}M}\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right].\hfill \end{align} \tag{ 28 }$$

Using the Cauchy–Schwarz inequality we can upper bound the expectation in the second summand by

$$\begin{equation*}\sqrt{{E}^{{\Pi}}\left[{\Vert}F-{F}_{0}{{\Vert}}_{{L}^{2}}^{2}\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right]}\sqrt{{{\Pi}}_{N}\left(F:{\Vert}F{{\Vert}}_{{C}^{\beta }}{ >}M\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right)}.\end{equation*}$$

In view of (14), for all D > 0 we can choose M > 0 large enough to obtain

$$\begin{align*}\hfill & {P}_{{F}_{0}}^{N}\left({E}^{{\Pi}}\left[{\Vert}F-{F}_{0}{{\Vert}}_{{L}^{2}}^{2}\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right]{{\Pi}}_{N}\left(F:{\Vert}F{{\Vert}}_{{C}^{\beta }}{ >}M\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right){ >}{N}^{-2\lambda }\right)\hfill \\ \hfill & \quad \;\;\;\;{\leqslant}{P}_{{F}_{0}}^{N}\left({E}^{{\Pi}}\left[{\Vert}F-{F}_{0}{{\Vert}}_{{L}^{2}}^{2}\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right]{\mathrm{e}}^{-DN{\delta }_{N}^{2}}{ >}{N}^{-2\lambda }\right)+o\left(1\right).\hfill \end{align*}$$

To bound the probability in the last line, let ${\mathcal{B}}_{N}$ be the sets defined in (A4) below, note that lemmas 16 and 23 below jointly imply that ${{\Pi}}_{N}\left({\mathcal{B}}_{N}\right){\geqslant}\mathrm{a}{\mathrm{e}}^{-AN{\delta }_{N}^{2}}$ for some a,A > 0. Also, let $\nu \left(\cdot \right)={{\Pi}}_{N}\left(\cdot \cap {\mathcal{B}}_{N}\right)/{{\Pi}}_{N}\left({\mathcal{B}}_{N}\right)$, and let ${\mathcal{C}}_{N}$ be the event from (A10), for which lemma 7.3.2 in Giné and Nickl (2016) implies that ${P}_{{F}_{0}}^{N}\left({\mathcal{C}}_{N}\right)\to 1$ as N → ∞. Then

$$\begin{align*}\hfill & {P}_{{F}_{0}}^{N}\left({E}^{{\Pi}}\left[{\Vert}F-{F}_{0}{{\Vert}}_{{L}^{2}}^{2}\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right]{\mathrm{e}}^{-DN{\delta }_{N}^{2}}{ >}{N}^{-2\lambda }\right)\hfill \\ \hfill & \quad \;\;\;\;{\leqslant}{P}_{{F}_{0}}^{N}\left(\frac{{\int }_{C\left(\mathcal{O}\right)}{\Vert}F-{F}_{0}{{\Vert}}_{{L}^{2}}^{2}\prod _{i=1}^{N}{p}_{F}/{p}_{{F}_{0}}\left({Y}_{i},{X}_{i}\right)\mathrm{d}{{\Pi}}_{N}\left(F\right)}{{\Pi}\left({\mathcal{B}}_{N}\right){\int }_{{\mathcal{B}}_{N}}\prod _{i=1}^{N}{p}_{F}/{p}_{{F}_{0}}\left({Y}_{i},{X}_{i}\right)d\nu \left(F\right)}{\mathrm{e}}^{-DN{\delta }_{N}^{2}}{ >}{N}^{-2\lambda },{\mathcal{C}}_{N}\right)+o\left(1\right)\hfill \\ \hfill & \quad \;\;\;\;{\leqslant}{P}_{{F}_{0}}^{N}\left({\int }_{C\left(\mathcal{O}\right)}{\Vert}F-{F}_{0}{{\Vert}}_{{L}^{2}}^{2}\prod _{i=1}^{N}{p}_{F}/{p}_{{F}_{0}}\left({Y}_{i},{X}_{i}\right)\mathrm{d}{{\Pi}}_{N}\left(F\right){ >}{N}^{-2\lambda }\mathrm{a}{\mathrm{e}}^{\left(D-A-2\right)N{\delta }_{N}^{2}}\right)+o\left(1\right)\hfill \end{align*}$$

which is upper bounded, using Markov's inequality and Fubini's theorem, by

$$\begin{equation*}\frac{1}{\mathrm{a}}{\mathrm{e}}^{-\left(D-A-2\right)N{\delta }_{N}^{2}}{N}^{2\lambda }{\int }_{C\left(\mathcal{O}\right)}{\Vert}F-{F}_{0}{{\Vert}}_{{L}^{2}}^{2}{E}_{{F}_{0}}^{N}\left(\prod _{i=1}^{N}\frac{{p}_{F}}{{p}_{{F}_{0}}}\left({Y}_{i},{X}_{i}\right)\right)\mathrm{d}{{\Pi}}_{N}\left(F\right).\end{equation*}$$

Taking D > A + 2 (and M large enough in (28)), using the fact that ${E}_{{F}_{0}}^{N}\left({\prod }_{i=1}^{N}\;{p}_{F}/{p}_{{F}_{0}}\left({Y}_{i},{X}_{i}\right)\right)=1$, and that ${E}^{{{\Pi}}_{N}}{\Vert}F{{\Vert}}_{{L}^{2}}{< }\infty$ (by Fernique's theorem, e.g., Giné and Nickl (2016), exercise 2.1.5), we then conclude

$$\begin{equation}{P}_{{F}_{0}}^{N}\left({E}^{{\Pi}}\left[{\Vert}F-{F}_{0}{{\Vert}}_{{L}^{2}}^{2}{1}_{{\Vert}F{{\Vert}}_{{C}^{\beta }}{ >}M}\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right]{ >}{N}^{-\lambda }\right)\to 0,\quad N\to \infty .\end{equation} \tag{ 29 }$$

To handle the first term in (28), let f = Φ◦F and f₀ = Φ◦F₀. Then for all $x\in \mathcal{O}$, by the mean value and inverse function theorems,

$$\begin{equation*}\vert F\left(x\right)-{F}_{0}\left(x\right)\vert =\vert {{\Phi}}^{-1}{\circ}f\left(x\right)-{{\Phi}}^{-1}{\circ}{f}_{0}\left(x\right)\vert =\frac{1}{\vert {{\Phi}}^{\prime }\left({{\Phi}}^{-1}\left(\eta \right)\right)\vert }\vert f\left(x\right)-{f}_{0}\left(x\right)\vert \end{equation*}$$

for some η lying between f(x) and f₀(x). If ${\Vert}F{{\Vert}}_{{C}^{\beta }}{\leqslant}M$ then, as Φ is strictly increasing, necessarily f(x) = Φ(F(x)) ∈ [Φ(−M), Φ(M)] for all $x\in \mathcal{O}$. Similarly, the range of f₀ is contained in the compact interval [Φ(−M), Φ(M)] for M ⩾ ||F₀||_∞, so that

$$\begin{equation*}\vert {{\Phi}}^{-1}{\circ}f\left(x\right)-{{\Phi}}^{-1}{\circ}{f}_{0}\left(x\right)\vert {\leqslant}\frac{1}{{\mathrm{min}}_{z\in \left[-M,M\right]}{{\Phi}}^{\prime }\left(z\right)}\vert f\left(x\right)-{f}_{0}\left(x\right)\vert < sim \vert f\left(x\right)-{f}_{0}\left(x\right)\vert \end{equation*}$$

for a multiplicative constant not depending on $x\in \mathcal{O}$. It follows

$$\begin{equation*}{\Vert}F-{F}_{0}{{\Vert}}_{{L}^{2}}{1}_{{\Vert}F{{\Vert}}_{{C}^{\beta }}{\leqslant}M}< sim {\Vert}f-{f}_{0}{{\Vert}}_{{L}^{2}}{1}_{{\Vert}F{{\Vert}}_{{C}^{\beta }}{\leqslant}M},\end{equation*}$$

and

$$\begin{equation*}{E}^{{\Pi}}\left[{\Vert}F-{F}_{0}{{\Vert}}_{{L}^{2}}{1}_{{\Vert}F{{\Vert}}_{{C}^{\beta }}{\leqslant}M}\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right]< sim {E}^{\tilde {{\Pi}}}\left[{\Vert}f-{f}_{0}{{\Vert}}_{{L}^{2}}\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right].\end{equation*}$$

Noting that for each L > 0 the last expectation is upper bounded by

$$\begin{align*}\hfill & L{N}^{-\lambda }+{E}^{\tilde {{\Pi}}}\left[{\Vert}f-{f}_{0}{{\Vert}}_{{L}^{2}}{1}_{{\Vert}f-{f}_{0}{{\Vert}}_{{L}^{2}}{ >}L{N}^{-\lambda }}\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right]\hfill \\ \hfill & \quad \;\;\;\;{\leqslant}L{N}^{-\lambda }+\sqrt{{E}^{\tilde {{\Pi}}}\left[{\Vert}f-{f}_{0}{{\Vert}}_{{L}^{2}}^{2}\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right]}\sqrt{{\tilde {{\Pi}}}_{N}\left(f:{\Vert}f-{f}_{0}{{\Vert}}_{{L}^{2}}{ >}L{N}^{-\lambda }\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right)},\hfill \end{align*}$$

we can repeat the above argument, with the event $\left\{F:{\Vert}F{{\Vert}}_{{C}^{\beta }}{ >}M\right\}$ replaced by the event $\left\{f:{\Vert}f-{f}_{0}{{\Vert}}_{{L}^{2}}{ >}L{N}^{-\lambda }\right\}$, to deduce from theorem 5 that for D > A + 2 there exists L > 0 large enough such that

$$\begin{align*}\hfill & {P}_{{F}_{0}}^{N}\left({E}^{\tilde {{\Pi}}}\left[{\Vert}f-{f}_{0}{{\Vert}}_{{L}^{2}}^{2}\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right]{\tilde {{\Pi}}}_{N}\left(f:{\Vert}f-{f}_{0}{{\Vert}}_{{L}^{2}}^{2}{ >}L{N}^{-\lambda }\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right){ >}{N}^{-\lambda }\right)\hfill \\ \hfill & \quad \;\;\;\;< sim {\mathrm{e}}^{-\left(D-A-2\right)N{\delta }_{N}^{2}}{N}^{2\lambda }\hfill \end{align*}$$

which combined with (29) and the definition of δ_N concludes the proof.□

The proof only requires minor modification from the proofs of section 2.2.1. We only discuss here the main points: one first applies the L²-prediction risk theorem 14 with a sieve prior. In the proof of the small ball lemma 16 one uses the following observations: the projection ${P}_{{\mathcal{H}}_{J}}\left({F}_{0}\right)\in {\mathcal{H}}_{J}$ of ${F}_{0}\in {H}_{K}^{\alpha }$ defined in (B4) satisfies by (B6)

$$\begin{equation*}{\Vert}{F}_{0}-{P}_{{\mathcal{H}}_{J}}\left({F}_{0}\right){{\Vert}}_{{\left({H}^{1}\left(\mathcal{O}\right)\right)}^{{\ast}}}< sim {2}^{-J\left(\alpha +1\right)};\end{equation*}$$

hence choosing J such that 2^J ≃ N^1/(2α+2+d), and noting also that ${\Vert}{P}_{{\mathcal{H}}_{J}}\left({F}_{0}\right){{\Vert}}_{{C}^{1}}{\leqslant}{\Vert}{F}_{0}{{\Vert}}_{{C}^{1}}{< }\infty$ for all J by standard properties of wavelet bases, it follows from (23) that

$$\begin{equation*}{\Vert}\mathcal{G}\left({F}_{0}\right)-\mathcal{G}\left({P}_{{\mathcal{H}}_{J}}\left({F}_{0}\right)\right){{\Vert}}_{{L}^{2}}< sim {\Vert}{F}_{0}-{P}_{{\mathcal{H}}_{J}}\left({F}_{0}\right){{\Vert}}_{{\left({H}^{1}\right)}^{{\ast}}}< sim {N}^{-\left(\alpha +1\right)/\left(2\alpha +2+d\right)}={\delta }_{N}.\end{equation*}$$

Therefore, by the triangle inequality,

$$\begin{equation*}{{\Pi}}_{N}\left(F:{\Vert}\mathcal{G}\left(F\right)-\mathcal{G}\left({F}_{0}\right){{\Vert}}_{{L}^{2}}{\geqslant}q{\delta }_{N}\right){\geqslant}{{\Pi}}_{N}\left(F:{\Vert}\mathcal{G}\left(F\right)-\mathcal{G}\left({P}_{{\mathcal{H}}_{N}}\left({F}_{0}\right)\right){{\Vert}}_{{L}^{2}}{\geqslant}{q}^{\prime }{\delta }_{N}\right).\end{equation*}$$

The rest of the proof of lemma 16 then carries over (with ${P}_{{\mathcal{H}}_{J}}\left({F}_{0}\right)$ replacing F₀), upon noting that (B3) and a Sobolev imbedding imply

$$\begin{equation*}\underset{J\in \mathbb{N}}{\mathrm{sup}}{E}^{{{\Pi}}_{J}^{\prime }}{\Vert}{F{\Vert}}_{{C}^{1}}^{2}{< }\infty ,\;\;\text{as}\;\text{well}\;\text{as}\;\;{\Vert}F{{\Vert}}_{{H}^{\alpha }}{\leqslant}c{\Vert}F{{\Vert}}_{{\mathcal{H}}_{J}}\quad \text{for} \text{all} F\in {\mathcal{H}}_{J}\end{equation*}$$

for some constant c > 0 independent of J. Moreover, the last two properties are sufficient to prove an analogue of lemma 17 as well, so that theorem 14 indeed applies to the sieve prior. The proof from here onwards is identical to the ones of theorems 4–6 for the unsieved case, using also that what precedes implies that ${\mathrm{sup}}_{J} {E}^{{{\Pi}}_{J}^{\prime }}{\Vert}{F{\Vert}}_{{L}^{2}}^{2}{< }\infty$, relevant in the proof of convergence of the posterior mean.

Inspection of the proofs for rescaled priors implies that theorems 9 and 10 can be deduced from theorem 8 if we can show that posterior draws lie in a α-Sobolev ball of fixed radius with sufficiently high frequentist probability. This is the content of the next result.

Lemma 12. Under the hypotheses of theorem 9, there exists α* > 0 (depending on α, d and a) such that for each ${F}_{0}\in {H}_{K}^{{\alpha }_{0}}\left(\mathcal{O}\right),{\alpha }_{0}{ >}{\alpha }^{{\ast}}$, and any D > 0 we can find M > 0 large enough such that, as N → ∞,

$$\begin{equation*}{\Pi}\left(F:{\Vert}F{{\Vert}}_{{H}^{\alpha }}{\leqslant}M\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right)=1-{O}_{{P}_{{F}_{0}}^{N}}\left({\mathrm{e}}^{-DN{\xi }_{N}^{2}}\right).\end{equation*}$$

Proof. Theorem 8 implies that for all D > 0 and sufficiently large L, M > 0, if ${J}_{N}\in \mathbb{N}:{2}^{{J}_{N}}\simeq {N}^{1/\left(2{\alpha }_{0}+2+d\right)}$ and denoting by

$$\begin{equation*}{\mathcal{A}}_{N}=\left\{F\in {\mathcal{H}}_{{J}_{N}}:{\Vert}F{{\Vert}}_{{H}^{\alpha }}{\leqslant}M{2}^{{J}_{N}\alpha }\sqrt{N}{\xi }_{N},\;{\Vert}\mathcal{G}\left(F\right)-\mathcal{G}\left({F}_{0}\right){{\Vert}}_{{L}^{2}}{\leqslant}L{\xi }_{N}\right\},\end{equation*}$$

then as N → ∞

$$\begin{equation}{\Pi}\left({\mathcal{A}}_{N}\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right)=1-{O}_{{P}_{{F}_{0}}^{N}}\left({\mathrm{e}}^{-DN{\xi }_{N}^{2}}\right).\end{equation} \tag{ 30 }$$

Next, note that if $F\in {\mathcal{H}}_{{J}_{N}}$, then by standard properties of wavelet bases (cf. (63)), ${\Vert}F{{\Vert}}_{{H}^{\alpha }}< sim {2}^{{J}_{N}\alpha }{\Vert}F{{\Vert}}_{{L}^{2}}$ for all N large enough. Thus, for ${P}_{{\mathcal{H}}_{{J}_{N}}}\left({F}_{0}\right)$ the projection of F₀ onto ${\mathcal{H}}_{{J}_{N}}$ defined in (B4),

$$\begin{equation*}{\Vert}F{{\Vert}}_{{H}^{\alpha }}{\leqslant}{\Vert}F-{P}_{{\mathcal{H}}_{{J}_{N}}}\left({F}_{0}\right){{\Vert}}_{{H}^{\alpha }}+{\Vert}{P}_{{\mathcal{H}}_{{J}_{N}}}\left({F}_{0}\right){{\Vert}}_{{H}^{\alpha }}{\leqslant}{2}^{{J}_{N}\alpha }{\Vert}F-{F}_{0}{{\Vert}}_{{L}^{2}}+{\Vert}{F}_{0}{{\Vert}}_{{H}^{\alpha }},\end{equation*}$$

and a Sobolev imbedding further gives ${\Vert}F{{\Vert}}_{{L}^{\infty }}{\leqslant}{M}^{\prime }{2}^{{J}_{N}\alpha }\sqrt{N}{\xi }_{N}$, for some M' > 0. Now letting f = Φ◦F and f₀ = Φ◦F₀, by similar argument as in the proof of theorem 6 combined with monotonicity of Φ', we see that for all N large enough

$$\begin{equation*}{\Vert}F-{F}_{0}{{\Vert}}_{{L}^{2}}{\leqslant}\frac{1}{{{\Phi}}^{\prime }\left(-{M}^{\prime }{2}^{{J}_{N}\alpha }\sqrt{N}{\xi }_{N}\right)}{\Vert}f-{f}_{0}{{\Vert}}_{{L}^{2}}.\end{equation*}$$

Then, using the assumption on the left tail of Φ in condition 2, and the stability estimate (25),

$$\begin{equation*}{\Vert}F-{F}_{0}{{\Vert}}_{{L}^{2}}< sim {\left({2}^{{J}_{N}\alpha }\sqrt{N}{\xi }_{N}\right)}^{a}{\Vert}f{{\Vert}}_{{H}^{\alpha }}{\Vert}G\left(f\right)-G\left({f}_{0}\right){{\Vert}}_{{H}^{2}}.\end{equation*}$$

Finally, by the interpolation inequality for Sobolev spaces (Lions and Magenes1972) and lemma 23 in Nickl et al (2020),

$$\begin{align*}\hfill {\Vert}G\left(f\right)-G\left({f}_{0}\right){{\Vert}}_{{H}^{2}}& < sim {\Vert}G\left(f\right)-G\left({f}_{0}\right){{\Vert}}_{{L}^{2}}^{\left(\alpha -1\right)/\left(\alpha +1\right)}{\Vert}G\left(f\right)-G{\left({f}_{0}\right){\Vert}}_{{H}^{\alpha +1}}^{2/\left(\alpha +1\right)}\hfill \\ \hfill & < sim {\xi }_{N}^{\left(\alpha -1\right)/\left(\alpha +1\right)}{\left({\Vert}G\left(f\right){{\Vert}}_{{H}^{\alpha +1}}+{\Vert}G\left({f}_{0}\right){{\Vert}}_{{H}^{\alpha +1}}\right)}^{2/\left(\alpha +1\right)}\hfill \\ \hfill & < sim {\xi }_{N}^{\left(\alpha -1\right)/\left(\alpha +1\right)}{\left(1+{\Vert}{f{\Vert}}_{{H}^{\alpha }}^{{\alpha }^{2}+\alpha }\right)}^{2/\left(\alpha +1\right)},\hfill \end{align*}$$

so that, in conclusion, for each $F\in {\mathcal{A}}_{N}$ and sufficiently large N,

$$\begin{equation*}{\Vert}F{{\Vert}}_{{H}^{\alpha }}< sim 1+{2}^{{J}_{N}\alpha }{\left({2}^{{J}_{N}\alpha }\sqrt{N}{\xi }_{N}\right)}^{a}{\Vert}f{{\Vert}}_{{H}^{\alpha }}{\xi }_{N}^{\frac{\alpha -1}{\alpha +1}}{\left(1+{\Vert}f{{\Vert}}_{{H}^{\alpha }}^{{\alpha }^{2}+\alpha }\right)}^{\frac{2}{\alpha +1}}.\end{equation*}$$

The last term is bounded, using lemma 29 in Nickl et al (2020), by a multiple of

$$\begin{equation*}{\xi }_{N}^{\frac{\alpha -1}{\alpha +1}}{2}^{{J}_{N}\alpha }{\left({2}^{{J}_{N}\alpha }\sqrt{N}{\xi }_{N}\right)}^{2{\alpha }^{2}+2\alpha +a}={N}^{-\frac{\left({\alpha }_{0}+1\right)\left(\alpha -1\right)}{\left(2{\alpha }_{0}+2+d\right)\left(\alpha +1\right)}}{N}^{\frac{2{\alpha }^{3}+\left(2+d\right){\alpha }^{2}+\left(1+a+d\right)\alpha +ad/2}{2{\alpha }_{0}+2+d}}\end{equation*}$$

the last identity holding up to a log factor. Hence, if

$$\begin{equation*}{\alpha }_{0}{ >}{\alpha }^{{\ast}}{:=}\frac{\left[2{\alpha }^{3}+\left(2+d\right){\alpha }^{2}+\left(1+a+d\right)\alpha +ad/2\right]\left(\alpha +1\right)}{\left(\alpha -1\right)}\end{equation*}$$

then we conclude overall that ${\Vert}F{{\Vert}}_{{H}^{\alpha }}< sim 1+o\left(1\right)$ as N → ∞ for all $F\in {\mathcal{A}}_{N}$, proving the claim in view of (30).□

Replacing β by α in the conclusion of lemma 11, the proof of theorem 9 now proceeds as in the proof of theorem 5 without further modification. Likewise, theorem 10 can be shown following the same argument as in the proof of theorem 6, noting that for Π the random series prior in (19), it also holds that ${E}^{{\Pi}}{\Vert}{F{\Vert}}_{{L}^{2}}^{2}{< }\infty$.

Using the same notation as in section 2.1.2, and given a sequence of Borel prior probability measures Π_N on $\mathcal{F}$, we write Π_N(⋅|Y^(N), X^(N)) for the posterior distribution of F|(Y^(N), X^(N)) (arising as after (9) and (10)). We also continue to use the notation p_F for the densities from (8) now in the general observation model (A1) (and implicitly assume that the map (F, (y, x)) ↦ p_F(y, x) is jointly measurable to ensure existence of the posterior distribution). Below we formulate a general contraction theorem in the Hellinger distance that forms the basis of the proofs of the main results. It closely follows the general theory in Ghosal and van der Vaart (2017) and its adaptation to the inverse problem setting in Monard et al (2020)—we include a proof for conciseness and convenience of the reader.

Define the Hellinger distance h(⋅, ⋅) on the set of probabilities density functions on $\mathbb{R}{\times}\mathcal{D}$ (with respect to the product measure dy × dx) by

$$\begin{equation*}{h}^{2}\left({p}_{1},{p}_{2}\right){:=}{\int }_{\mathbb{R}{\times}\mathcal{D}}{\left[\sqrt{{p}_{1}\left(y,x\right)}-\sqrt{{p}_{2}\left(y,x\right)}\right]}^{2}\mathrm{d}y\mathrm{d}x.\end{equation*}$$

For any set $\mathcal{A}$ of such densities, let $N\left(\eta ;\mathcal{A},h\right),\;\eta { >}0$, be the minimal number of Hellinger balls of radius η needed to cover $\mathcal{A}$.

Theorem 13. Let Π_N be a sequence of prior Borel probability measures on $\mathcal{F}$, and let Π_N(⋅|Y^(N), X^(N)) be the resulting posterior distribution arising from observations (Y^(N), X^(N)) in model (A1). Assume that for some fixed ${F}_{0}\in \mathcal{F}$, and a sequence δ_N > 0 such that δ_N → 0 and $\sqrt{N}{\delta }_{N}\to \infty$ as N → ∞, the sets

$$\begin{equation}{\mathcal{B}}_{N}{:=}\left\{F:\;{E}_{{F}_{0}}^{1}\left[\mathrm{log}\frac{{p}_{{F}_{0}}\left({Y}_{1},{X}_{1}\right)}{{p}_{F}\left({Y}_{1},{X}_{1}\right)}\right]{\leqslant}{\delta }_{N}^{2},\;{E}_{{F}_{0}}^{1}{\left[\mathrm{log}\frac{{p}_{{F}_{0}}\left({Y}_{1},{X}_{1}\right)}{{p}_{F}\left({Y}_{1},{X}_{1}\right)}\right]}^{2}{\leqslant}{\delta }_{N}^{2}\right\},\end{equation} \tag{ A4 }$$

satisfy for all N large enough

$$\begin{equation}{{\Pi}}_{N}\left({\mathcal{B}}_{N}\right){\geqslant}a\;{\mathrm{e}}^{-AN{\delta }_{N}^{2}},\quad \mathrm{some}\;a,A{ >}0.\end{equation} \tag{ A5 }$$

Further assume that there exists a sequence of Borel sets ${\mathcal{A}}_{N}\subset \mathcal{F}$ for which

$$\begin{equation}{{\Pi}}_{N}\left({\mathcal{A}}_{N}^{c}\right)< sim {\mathrm{e}}^{-BN{\delta }_{N}^{2}},\quad \mathrm{some}\;B{ >}A+2\end{equation} \tag{ A6 }$$

for all N large enough, as well as

$$\begin{equation}\mathrm{log}N\left({\delta }_{N};{\mathcal{A}}_{N},h\right){\leqslant}CN{\delta }_{N}^{2},\quad \mathrm{some}\;C{ >}0.\end{equation} \tag{ A7 }$$

Then, for sufficiently large L = L(B, C) > 4 such that L² > 12(B ∨ C), and all 0 < D < B − A − 2, as N → ∞,

$$\begin{equation}{{\Pi}}_{N}\left(F\in {\mathcal{A}}_{N}:h\left({p}_{F},{p}_{{F}_{0}}\right){\leqslant}L{\delta }_{N}\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right)=1-{O}_{{P}_{{F}_{0}}^{N}}\left({\mathrm{e}}^{-DN{\delta }_{N}^{2}}\right).\end{equation} \tag{ A8 }$$

Proof. We start noting that by theorem 7.1.4 in Giné and Nickl (2016), for each L > 4 satisfying L² > 12(B ∨ C) we can find tests (random indicator functions) Ψ_N = Ψ_N(Y^(N), X^(N)) such that as N → ∞

$$\begin{equation}{E}_{{F}_{0}}^{N}{{\Psi}}_{N}\to 0,\quad \underset{F\in {\mathcal{A}}_{N}:h\left({p}_{F},{p}_{{F}_{0}}\right){\geqslant}L{\delta }_{N}}{\mathrm{sup}}{E}_{F}^{N}\left(1-{{\Psi}}_{N}\right){\leqslant}{\mathrm{e}}^{-BN{\delta }_{N}^{2}}.\end{equation} \tag{ A9 }$$

Next, denote the set whose posterior probability we want to lower bound by

$$\begin{equation*}{\tilde {\mathcal{A}}}_{N}=\left\{F\in {\mathcal{A}}_{N}:h\left({p}_{F},{p}_{{F}_{0}}\right){\leqslant}L{\delta }_{N}\right\};\end{equation*}$$

and, using the first display in (A9), decompose the probability of interest as

$$\begin{align*}\hfill {P}_{{F}_{0}}^{N}\left({{\Pi}}_{N}\left({\tilde {\mathcal{A}}}_{N}^{c}\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right){\geqslant}{\mathrm{e}}^{-DN{\delta }_{N}^{2}}\right)& ={P}_{{F}_{0}}^{N}\left({{\Pi}}_{N}\left({\tilde {\mathcal{A}}}_{N}^{c}\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right){\geqslant}{\mathrm{e}}^{-DN{\delta }_{N}^{2}},{{\Psi}}_{N}=0\right)\hfill \\ \hfill & \;\quad +{P}_{{F}_{0}}^{N}\left({{\Pi}}_{N}\left({\tilde {\mathcal{A}}}_{N}^{c}\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right){\geqslant}{\mathrm{e}}^{-DN{\delta }_{N}^{2}},{{\Psi}}_{N}=1\right)\hfill \\ \hfill & \;={P}_{{F}_{0}}^{N}\left({{\Pi}}_{N}\left({\tilde {\mathcal{A}}}_{N}^{c}\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right){\geqslant}{\mathrm{e}}^{-DN{\delta }_{N}^{2}},{{\Psi}}_{N}=0\right)+o\left(1\right).\hfill \end{align*}$$

Next, let $\nu \left(\cdot \right)={{\Pi}}_{N}\left(\cdot \cap {\mathcal{B}}_{N}\right)/{{\Pi}}_{N}\left({\mathcal{B}}_{N}\right)$ be the restricted normalised prior on ${\mathcal{B}}_{N}$, and define the event

$$\begin{equation}{\mathcal{C}}_{N}=\left\{{\int }_{{\mathcal{B}}_{N}}\prod _{i=1}^{N}\frac{{p}_{F}}{{p}_{{F}_{0}}}\left({Y}_{i},{X}_{i}\right)\mathrm{d}\nu \left(F\right){\geqslant}{\mathrm{e}}^{-2N{\delta }_{N}^{2}}\right\},\end{equation} \tag{ A10 }$$

for which lemma 7.3.2 in Giné and Nickl (2016) implies that ${P}_{{F}_{0}}^{N}\left({\mathcal{C}}_{N}\right)\to 1$ as N → ∞. We then further decompose

$$\begin{align*}\hfill & {P}_{{F}_{0}}^{N}\left({{\Pi}}_{N}\left({\tilde {\mathcal{A}}}_{N}^{c}\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right){\geqslant}{\mathrm{e}}^{-DN{\delta }_{N}^{2}},{{\Psi}}_{N}=0\right)\hfill \\ \hfill & \quad \;\;\;\;={P}_{{F}_{0}}^{N}\left({{\Pi}}_{N}\left({\tilde {\mathcal{A}}}_{N}^{c}\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right){\geqslant}{\mathrm{e}}^{-DN{\delta }_{N}^{2}},{{\Psi}}_{N}=0,{\mathcal{C}}_{N}\right)+o\left(1\right)\hfill \end{align*}$$

and in view of condition (A5) and the above definition of ${\mathcal{C}}_{N}$, we see that

$$\begin{align*}\hfill & {P}_{{F}_{0}}^{N}\left({{\Pi}}_{N}\left({\tilde {\mathcal{A}}}_{N}^{c}\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right){\geqslant}{\mathrm{e}}^{-DN{\delta }_{N}^{2}},{{\Psi}}_{N}=0,{\mathcal{C}}_{N}\right)\hfill \\ \hfill & \quad \;\;\;\;={P}_{{F}_{0}}^{N}\left(\frac{{\int }_{{\tilde {\mathcal{A}}}_{N}^{c}}\prod _{i=1}^{N}{p}_{F}/{p}_{{F}_{0}}\left({Y}_{i},{X}_{i}\right)\mathrm{d}{{\Pi}}_{N}\left(F\right)}{{\int }_{\mathcal{F}}\prod _{i=1}^{N}{p}_{F}/{p}_{{F}_{0}}\left({Y}_{i},{X}_{i}\right)\mathrm{d}{{\Pi}}_{N}\left(F\right)}{\geqslant}{\mathrm{e}}^{-DN{\delta }_{N}^{2}},{{\Psi}}_{N}=0,{\mathcal{C}}_{N}\right)\hfill \\ \hfill & \quad \;\;\;\;{\leqslant}{P}_{{F}_{0}}^{N}\left(\frac{{\int }_{{\tilde {\mathcal{A}}}_{N}^{c}}\left(1-{{\Psi}}_{N}\right)\prod _{i=1}^{N}{p}_{F}/{p}_{{F}_{0}}\left({Y}_{i},{X}_{i}\right)\mathrm{d}{{\Pi}}_{N}\left(F\right)}{{\int }_{{\mathcal{B}}_{N}}\prod _{i=1}^{N}{p}_{F}/{p}_{{F}_{0}}\left({Y}_{i},{X}_{i}\right)d\nu \left(F\right)}{\geqslant}{{\Pi}}_{N}\left({\mathcal{B}}_{N}\right){\mathrm{e}}^{-DN{\delta }_{N}^{2}},{\mathcal{C}}_{N}\right)\hfill \\ \hfill & \quad \;\;\;\;{\leqslant}{P}_{{F}_{0}}^{N}\left({\int }_{{\tilde {\mathcal{A}}}_{N}^{c}}\left(1-{{\Psi}}_{N}\right)\prod _{i=1}^{N}\frac{{p}_{F}}{{p}_{{F}_{0}}}\left({Y}_{i},{X}_{i}\right)\mathrm{d}{{\Pi}}_{N}\left(F\right){\geqslant}a{e}^{-\left(A+D+2\right)N{\delta }_{N}^{2}}\right).\hfill \end{align*}$$

We conclude applying Markov's inequality and Fubini's theorem, jointly with the fact that for all $F\in \mathcal{F}$

$$\begin{equation*}{E}_{{F}_{0}}^{N}\left[\left(1-{{\Psi}}_{N}\right)\prod _{i=1}^{N}\frac{{p}_{F}}{{p}_{{F}_{0}}}\left({Y}_{i},{X}_{i}\right)\right]={E}_{{F}_{0}}^{N}\left[\left(1-{{\Psi}}_{N}\right)\prod _{i=1}^{N}\frac{\mathrm{d}{P}_{F}^{1}}{\mathrm{d}{P}_{{F}_{0}}^{1}}\left({Y}_{i},{X}_{i}\right)\right]={E}_{F}^{N}\left[1-{{\Psi}}_{N}\right],\end{equation*}$$

to upper bound the last probability by

$$\begin{align*}\hfill & \frac{1}{a}{\mathrm{e}}^{\left(A+D+2\right)N{\delta }_{N}^{2}}\left({\int }_{{\mathcal{A}}_{N}^{c}}{E}_{F}^{N}\left[1-{{\Psi}}_{N}\right]\mathrm{d}{{\Pi}}_{N}\left(F\right)+{\int }_{\left\{F\in {\mathcal{A}}_{N}:h\left({p}_{{F}_{0}},{p}_{F}\right){ >}L{\delta }_{N}\right\}}{E}_{F}^{N}\left[1-{{\Psi}}_{N}\right]\mathrm{d}{{\Pi}}_{N}\left(F\right)\right.\hfill \\ \hfill & \quad \;\;\;\quad \left.+{\int }_{\left\{F\in {\mathcal{A}}_{N}^{c}:h\left({p}_{{F}_{0}},{p}_{F}\right){ >}L{\delta }_{N}\right\}}{E}_{F}^{N}\left[1-{{\Psi}}_{N}\right]\mathrm{d}{{\Pi}}_{N}\left(F\right)\right)\hfill \\ \hfill & \quad \;\;\;\;{\leqslant}\frac{1}{a}{\mathrm{e}}^{\left(A+D+2\right)N{\delta }_{N}^{2}}\left(2{{\Pi}}_{N}\left({\mathcal{A}}_{N}^{c}\right)+{\int }_{\left\{F\in {\mathcal{A}}_{N}:h\left({p}_{{F}_{0}},{p}_{F}\right){ >}L{\delta }_{N}\right\}}{E}_{F}^{N}\left[1-{{\Psi}}_{N}\right]\mathrm{d}{{\Pi}}_{N}\left(F\right)\right)\hfill \\ \hfill & \quad \;\;\;\;< sim {\mathrm{e}}^{-\left(B-A-D-2\right)N{\delta }_{N}^{2}}=o\left(1\right)\hfill \end{align*}$$

as N → ∞ since B > A + D + 2, having used the excess mass condition (A6) and the second display in (A9).□

While the previous result assumed a general sequence of priors, we now derive explicit contraction rates in L²-prediction risk for the specific choices of priors considered in section 2.2. We start with the 're-scaled' priors of section 2.2.1.

Theorem 14. Let the forward map $\mathcal{G}$ satisfy (A2) and (A3) for given β, γ, κ ⩾ 0 and S > 0. For integer α > β + d/2, consider a Gaussian prior Π_N constructed as in (11) with scaling N^{d/(4α+4κ+2d)} and with base prior F' ∼ Π' satisfying condition 3 with RKHS $\mathcal{H}$. Let Π_N(⋅|Y^(N), X^(N)) be the resulting posterior arising from observations (Y^(N), X^(N)) in (A1), assume ${F}_{0}\in \mathcal{H}$ and set δ_N = N^{−(α+κ)/(2α+2κ+d)}.

Then for any D > 0 there exists L > 0 large enough (depending on σ, F₀, D, α, and β, γ, κ, S, d) such that, as N → ∞,

$$\begin{equation}{{\Pi}}_{N}\left(F:{\Vert}\mathcal{G}\left(F\right)-\mathcal{G}\left({F}_{0}\right){{\Vert}}_{{L}^{2}\left(\mathcal{D}\right)}{ >}L{\delta }_{N}\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right)={O}_{{P}_{{F}_{0}}^{N}}\left({\mathrm{e}}^{-DN{\delta }_{N}^{2}}\right),\end{equation} \tag{ A11 }$$

and for sufficiently large M > 0 (depending on σ, D, α, β, γ, κ, d)

$$\begin{equation}{{\Pi}}_{N}\left(F:{\Vert}F{{\Vert}}_{{C}^{\beta }}{ >}M\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right)={O}_{{P}_{{F}_{0}}^{N}}\left({\mathrm{e}}^{-DN{\delta }_{N}^{2}}\right).\end{equation} \tag{ A12 }$$

Remark 15. Inspection of the proof shows that if κ=0 in (A12), then the RKHS $\mathcal{H}$ in condition 3 can be assumed to be continuously imbedded in H^α($\mathcal{O}$) only instead of H^α_c($\mathcal{O}$). The same remark in fact applies for κ < 1/2.

Proof. In view of the boundedness assumption (A3) on $\mathcal{G}$, we have by lemma 23 below that for some q > 0 (depending on σ, S)

$$\begin{equation*}{E}_{{F}_{0}}^{1}\left[\mathrm{log}\frac{{p}_{{F}_{0}}\left({Y}_{1},{X}_{1}\right)}{{p}_{F}\left({Y}_{1},{X}_{1}\right)}\right]\vee {E}_{{F}_{0}}^{1}{\left[\mathrm{log}\frac{{p}_{{F}_{0}}\left({Y}_{1},{X}_{1}\right)}{{p}_{F}\left({Y}_{1},{X}_{1}\right)}\right]}^{2}{\leqslant}q{\Vert}\mathcal{G}\left({F}_{0}\right)-\mathcal{G}{\left(F\right){\Vert}}_{{L}^{2}\left(\mathcal{D}\right)}^{2}.\end{equation*}$$

Hence, for ${\mathcal{B}}_{N}$ the sets from (A4) we have $\left\{F:{\Vert}\mathcal{G}\left({F}_{0}\right)-\mathcal{G}\left(F\right){{\Vert}}_{{L}^{2}\left(\mathcal{D}\right)}{\leqslant}{\delta }_{N}/q\right\}\subseteq {\mathcal{B}}_{N},$ which in turn implies the small ball condition (A5) since by lemma 16 below (premultiplying if needed δ_N by a sufficiently large but fixed constant):

$$\begin{equation*}{{\Pi}}_{N}\left(F:{\Vert}\mathcal{G}\left(F\right)-\mathcal{G}\left({F}_{0}\right){{\Vert}}_{{L}^{2}\left(\mathcal{D}\right)}{\leqslant}{\delta }_{N}/q\right)\gtrsim {\mathrm{e}}^{-AN{\delta }_{N}^{2}}\end{equation*}$$

for some A > 0 and all N large enough. Next, for all D > 0 and any B > A + D + 2, we can choose sets ${\mathcal{A}}_{N}$ as in lemmas 17 and 18 and verify the excess mass condition (A6) as well as the complexity bound (A7). Note that ${\Vert}F{{\Vert}}_{{C}^{\beta }}{\leqslant}M$ for all $F\in {\mathcal{A}}_{N}$. We then conclude by theorem 13 that for some L' > 0 large enough

$$\begin{equation*}{{\Pi}}_{N}\left(F\in {\mathcal{A}}_{N}:h\left({p}_{F},{p}_{{F}_{0}}\right){\leqslant}{L}^{\prime }{\delta }_{N}\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right)=1-{O}_{{P}_{{F}_{0}}^{N}}\left({\mathrm{e}}^{-DN{\delta }_{N}^{2}}\right)\end{equation*}$$

yielding the claim for some appropriate L > 0 using the first inequality in (A26).□

The following key lemma shows that the (non-Gaussian) prior induced on the regression functions $\mathcal{G}\left(F\right)$ assigns sufficient mass to a L²-neighbourhood of $\mathcal{G}\left({F}_{0}\right)$.

Lemma 16. Let Π_N, F₀ and δ_N be as in theorem 14. Then, for all sufficiently large q > 0 there exists A > 0 (depending on q, F₀, α, β, γ, κ, d) such that

$$\begin{equation}{{\Pi}}_{N}\left(F:{\Vert}\mathcal{G}\left(F\right)-\mathcal{G}\left({F}_{0}\right){{\Vert}}_{{L}^{2}\left(\mathcal{D}\right)}{\leqslant}q{\delta }_{N}\right)\gtrsim {\mathrm{e}}^{-AN{\delta }_{N}^{2}}\end{equation} \tag{ A13 }$$

for all N large enough.

Proof. Using (A2) and noting that ${\Vert}{F}_{0}{{\Vert}}_{{C}^{\beta }}{< }\infty$ for ${F}_{0}\in \mathcal{H}$ by a Sobolev imbedding, for any fixed constant M > 1∨ ||F₀||_C^β,

$$\begin{equation*}\begin{aligned}{{\Pi}}_{N}\left(F:{\Vert}\mathcal{G}\left(F\right)-\mathcal{G}\left({F}_{0}\right){{\Vert}}_{{L}^{2}\left(\mathcal{D}\right)}{\leqslant}q{\delta }_{N}\right)\hfill & {\geqslant}{{\Pi}}_{N}\left(F:{\Vert}F-{F}_{0}{{\Vert}}_{{\left({H}^{\kappa }\right)}^{{\ast}}}{\leqslant}{q}^{\prime }{\delta }_{N},{\Vert}F-{F}_{0}{{\Vert}}_{{C}^{\beta }}{\leqslant}M\right)\hfill \\ \hfill & ={{\Pi}}_{N}\left(F:F-{F}_{0}\in {C}_{1}\cap {C}_{2}\right),\hfill \end{aligned}\end{equation*}$$

having defined

$$\begin{equation*}{C}_{1}{:=}\left\{F:{\Vert}F{{\Vert}}_{{\left({H}^{\kappa }\right)}^{{\ast}}}{\leqslant}{q}^{\prime }{\delta }_{N}\right\},\quad {C}_{2}{:=}\left\{F:{\Vert}F{{\Vert}}_{{C}^{\beta }}{\leqslant}M\right\},\end{equation*}$$

whose intersection is a symmetric set in the ambient space $C\left(\mathcal{O}\right)$. Then, since ${F}_{0}\in \mathcal{H}$, recalling that the RKHS ${\mathcal{H}}_{N}$ of Π_N coincides with $\mathcal{H}$ with RKHS norm ${\Vert}\cdot {{\Vert}}_{{\mathcal{H}}_{N}}$ given in (12), now with scaling ${N}^{d/\left(4\alpha +4\kappa +2d\right)}=\sqrt{N}{\delta }_{N}$, we can use corollary 2.6.18 in Giné and Nickl (2016) to lower bound the last probability by

$$\begin{align*}\hfill {\mathrm{e}}^{-{\Vert}{F}_{0}{{\Vert}}_{{\mathcal{H}}_{N}}^{2}/2}{{\Pi}}_{N}\left({C}_{1}\cap {C}_{2}\right)& ={\mathrm{e}}^{-\frac{1}{2}N{\delta }_{N}^{2}{\Vert}{F}_{0}{{\Vert}}_{\mathcal{H}}^{2}}{{\Pi}}_{N}\left({C}_{1}\cap {C}_{2}\right)\hfill \\ \hfill & {\geqslant}{\mathrm{e}}^{-\frac{1}{2}N{\delta }_{N}^{2}{\Vert}{F}_{0}{{\Vert}}_{\mathcal{H}}^{2}}\left({{\Pi}}_{N}\left({C}_{1}\right)-{{\Pi}}_{N}\left({C}_{2}^{c}\right)\right).\hfill \end{align*}$$

We proceed finding a lower bound for the prior probability of C₁, which, by construction of Π_N, satisfies

$$\begin{equation*}{{\Pi}}_{N}\left(F\in {C}_{1}\right)={{\Pi}}^{\prime }\left({F}^{\prime }:{\Vert}{F}^{\prime }{{\Vert}}_{{\left({H}^{\kappa }\right)}^{{\ast}}}{\leqslant}{q}^{\prime }\sqrt{N}{\delta }_{N}^{2}\right).\end{equation*}$$

For any integer α > 0 and any κ ⩾ 0, letting ${B}_{c}^{\alpha }\left(r\right){:=}\left\{F\in {H}_{c}^{\alpha },\;{\Vert}F{{\Vert}}_{{H}^{\alpha }}{\leqslant}r\right\},\;r{ >}0$, we have the metric entropy estimate:

$$\begin{equation}\mathrm{log}N\left(\eta ;{B}_{c}^{\alpha }\left(r\right),{\Vert}\cdot {{\Vert}}_{{\left({H}^{\kappa }\right)}^{{\ast}}}\right)< sim {\left(r/\eta \right)}^{d/\left(\alpha +\kappa \right)}\quad \forall \eta { >}0;\end{equation} \tag{ A14 }$$

see the proof of lemma 19 in Nickl et al (2020) for the case κ ⩾ 1/2, and theorem 4.10.3 in Triebel (1978) for κ < 1/2 (in the latter case, we note in fact that the estimate holds true also for balls in the whole space H^α). Hence, since $\mathcal{H}$ is continuously imbedded into ${H}_{c}^{\alpha }$, letting ${B}_{\mathcal{H}}\left(1\right)$ be the unit ball of $\mathcal{H}$, we have ${B}_{\mathcal{H}}\left(1\right)\subseteq {B}_{c}^{\alpha }\left(r\right)$ for some r > 0, implying that for all η > 0

$$\begin{equation*}\mathrm{log}N\left(\eta ;{B}_{\mathcal{H}}\left(1\right),{\Vert}\cdot {{\Vert}}_{{\left({H}^{\kappa }\right)}^{{\ast}}}\right){\leqslant}\mathrm{log}N\left(\eta ;{B}_{c}^{\alpha }\left(r\right),{\Vert}\cdot {{\Vert}}_{{\left({H}^{\kappa }\right)}^{{\ast}}}\right)< sim {\eta }^{-d/\left(\alpha +\kappa \right)}.\end{equation*}$$

Then, for all N large enough, the small ball estimate in theorem 1.2 in Li and Linde (1999) yields

$$\begin{align}\hfill -\mathrm{log}{{\Pi}}^{\prime }\left({F}^{\prime }:{\Vert}{F}^{\prime }{{\Vert}}_{{\left({H}^{\kappa }\right)}^{{\ast}}}{\leqslant}{q}^{\prime }\sqrt{N}{\delta }_{N}^{2}\right)& < sim {\left(\sqrt{N}{\delta }_{N}^{2}\right)}^{-2\frac{d}{\alpha +\kappa }{\left(2-d/\left(\alpha +\kappa \right)\right)}^{-1}}\hfill \\ \hfill & ={\left[{N}^{-\left(\alpha +\kappa -d/2\right)/\left(2\alpha +2\kappa +d\right)}\right]}^{-\frac{2d}{2\alpha +2\kappa -d}}\hfill \\ \hfill & =N{\delta }_{N}^{2},\hfill \end{align} \tag{ A15 }$$

implying ${{\Pi}}_{N}\left({C}_{1}\right){\geqslant}{\mathrm{e}}^{-{q}^{{\prime\prime}}N{\delta }_{N}^{2}},$ for some q'' > 0. Note that q'' can be made as small as desired by taking q in (A13) large enough.

We conclude obtaining a suitable upper bound for Π_N(C₂^c). In particular, by construction of Π_N, recalling ${N}^{d/\left(2\alpha +2\kappa +d\right)}=N{\delta }_{N}^{2}$,

$$\begin{equation*}{{\Pi}}_{N}\left({C}_{2}\right)=\mathrm{Pr}\left({\Vert}{F}^{\prime }{{\Vert}}_{{C}^{\beta }}{\leqslant}MN{\delta }_{N}^{2}\right),\quad {F}^{\prime }\sim {{\Pi}}^{\prime }.\end{equation*}$$

By condition 3, F' defines a centred Gaussian Borel random element in a separable measurable subspace $\mathcal{C}$ of C^β, and by the Hahn–Banach theorem and the separability of $\mathcal{C}$, ${\Vert}{F}^{\prime }{{\Vert}}_{{C}^{\beta }}$ can then be represented as a countable supremum

$$\begin{equation*}{\Vert}{F}^{\prime }{{\Vert}}_{{C}^{\beta }}=\underset{T\in \mathcal{T}}{\mathrm{sup}}\vert T\left({F}^{\prime }\right)\vert \end{equation*}$$

of actions of bounded linear functionals $\mathcal{T}={\left({T}_{m}\right)}_{m\in \mathbb{N}}\subset {\left({C}^{\beta }\right)}^{{\ast}}$. It follows that the collection ${\left\{{T}_{m}\left({F}^{\prime }\right)\right\}}_{m\in \mathbb{N}}$ is a centred Gaussian process with almost surely finite supremum, so that by Fernique's theorem Giné and Nickl (2016), theorem 2.1.20:

$$\begin{equation*}E{\Vert}{F}^{\prime }{{\Vert}}_{{C}^{\beta }}=E\;\underset{m\in \mathbb{N}}{\mathrm{sup}}\vert {T}_{m}\left({F}^{\prime }\right)\vert {< }\infty ;\quad {\tau }^{2}{:=}\underset{m\in \mathbb{N}}{\mathrm{sup}}E\vert {T}_{m}\left({F}^{\prime }\right){\vert }^{2}{< }\infty .\end{equation*}$$

We then apply the Borell–Sudakov–Tirelson inequality Giné and Nickl (2016), theorem 2.5.8, to obtain for all N large enough,

$$\begin{equation*}\mathrm{Pr}\left({\Vert}{F}^{\prime }{{\Vert}}_{{C}^{\beta }}{\geqslant}M\sqrt{N}{\delta }_{N}\right){\leqslant}\mathrm{Pr}\left({\Vert}{F}^{\prime }{{\Vert}}_{{C}^{\beta }}{\geqslant}E{\Vert}{F}^{\prime }{{\Vert}}_{{C}^{\beta }}+M\sqrt{N}{\delta }_{N}/2\right){\leqslant}{\mathrm{e}}^{-\frac{1}{8}{\left(M/\tau \right)}^{2}N{\delta }_{N}^{2}}.\end{equation*}$$

Given our initial choice of M, the proof is then concluded taking q in lemma 16 sufficiently large so that q'' < (M/τ)²/8.□

We now construct suitable approximating sets for which we check the excess mass condition (A6).

Lemma 17. Let Π_N and δ_N be as in theorem 14. Define for any M, Q > 0

$$\begin{equation}{\mathcal{A}}_{N}=\left\{F:F={F}_{1}+{F}_{2}:{\Vert}{F}_{1}{{\Vert}}_{{\left({H}^{\kappa }\right)}^{{\ast}}}{\leqslant}Q{\delta }_{N},\;{\Vert}{F}_{2}{{\Vert}}_{\mathcal{H}}{\leqslant}M,\;{\Vert}F{{\Vert}}_{{C}^{\beta }}{\leqslant}M\right\}.\end{equation} \tag{ A16 }$$

Then for any B > 0 and for sufficiently large M, Q (both depending on B, α, β, γ, κ, d), for all N large enough,

$$\begin{equation}{{\Pi}}_{N}\left({\mathcal{A}}_{N}^{c}\right){\leqslant}2{\mathrm{e}}^{-BN{\delta }_{N}^{2}}.\end{equation} \tag{ A17 }$$

Proof. We note that the last inequality at the end of the proof of the previous lemma implies that for $M\gtrsim \sqrt{B}$ and all N large enough, ${{\Pi}}_{N}\left(F:{\Vert}F{{\Vert}}_{{C}^{\beta }}{\leqslant}M\right){\geqslant}1-{\mathrm{e}}^{-BN{\delta }_{N}^{2}}.$ Thus, the claim will follow if we can derive a similar lower bound for

$$\begin{align*}\hfill & {{\Pi}}_{N}\left(F:F={F}_{1}+{F}_{2}:{\Vert}{F}_{1}{{\Vert}}_{{\left({H}^{\kappa }\right)}^{{\ast}}}{\leqslant}Q{\delta }_{N},{\Vert}{F}_{2}{{\Vert}}_{\mathcal{H}}{\leqslant}M\right)\hfill \\ \hfill & \quad \;\;\;\;\quad \;\;\;\;\;{\geqslant}{{\Pi}}^{\prime }\left({F}^{\prime }:{F}^{\prime }={F}_{1}^{\prime }+{F}_{2}^{\prime },\;{\Vert}{F}_{1}^{\prime }{{\Vert}}_{{\left({H}^{\kappa }\right)}^{{\ast}}}{\leqslant}Q\sqrt{N}{\delta }_{N}^{2},{\Vert}{F}_{2}^{\prime }{{\Vert}}_{\mathcal{H}}{\leqslant}M\sqrt{N}{\delta }_{N}\right),\hfill \end{align*}$$

having used that ${N}^{d/\left(4\alpha +4\kappa +d\right)}=\sqrt{N}{\delta }_{N}$. Using theorem 1.2 in Li and Linde (1999) as before (A15), we deduce that for some q > 0

$$\begin{equation*}-\mathrm{log}{{\Pi}}^{\prime }\left({F}^{\prime }:{\Vert}{F}^{\prime }{{\Vert}}_{{\left({H}^{\kappa }\right)}^{{\ast}}}{\leqslant}Q\sqrt{N}{\delta }_{N}^{2}\right){\leqslant}q{\left(Q\sqrt{N}{\delta }_{N}^{2}\right)}^{-\frac{2d}{2\alpha +2\kappa -d}}\end{equation*}$$

so that taking any Q > (B/q)^{−(2α+2κ−d)/(2d)} implies

$$\begin{equation}-\mathrm{log}{{\Pi}}^{\prime }\left({F}^{\prime }:{\Vert}{F}^{\prime }{{\Vert}}_{{\left({H}^{\kappa }\right)}^{{\ast}}}{\leqslant}Q\sqrt{N}{\delta }_{N}^{2}\right){\leqslant}B{\left(\sqrt{N}{\delta }_{N}^{2}\right)}^{-\frac{2d}{2\alpha +2\kappa -d}}=BN{\delta }_{N}^{2}.\end{equation} \tag{ A18 }$$

Next, denote

$$\begin{equation*}{M}_{N}=-2{{\Phi}}^{-1}\left({\mathrm{e}}^{-BN{\delta }_{N}^{2}}\right)\end{equation*}$$

where Φ is the standard normal cumulative distribution function. Then by standard inequalities for Φ⁻¹ we have ${M}_{N}\simeq \sqrt{BN}{\delta }_{N}$ as N → ∞, so that taking $M\gtrsim \sqrt{B}$

$$\begin{align*}\hfill {{\Pi}}^{\prime }({F}^{\prime }:{F}^{\prime }& ={F}_{1}^{\prime }+{F}_{2}^{\prime },\;{\Vert}{F}_{1}^{\prime }{{\Vert}}_{{\left({H}^{\kappa }\right)}^{{\ast}}}{\leqslant}Q\sqrt{N}{\delta }_{N}^{2},{\Vert}{F}_{2}^{\prime }{{\Vert}}_{\mathcal{H}}{\leqslant}M\sqrt{N}{\delta }_{N})\hfill \\ \hfill & {\geqslant}{{\Pi}}^{\prime }\left({F}^{\prime }:{F}^{\prime }={F}_{1}^{\prime }+{F}_{2}^{\prime },\;{\Vert}{F}_{1}^{\prime }{{\Vert}}_{{\left({H}^{\kappa }\right)}^{{\ast}}}{\leqslant}Q\sqrt{N}{\delta }_{N}^{2},{\Vert}{F}_{2}^{\prime }{{\Vert}}_{\mathcal{H}}{\leqslant}{M}_{N}\right).\hfill \end{align*}$$

By the isoperimetric inequality for Gaussian processes Giné and Nickl (2016), theorem 2.6.12, the last probability is then lower bounded, using (A18), by

$$\begin{align*}\hfill & {\Phi}\left({{\Phi}}^{-1}\left[{{\Pi}}_{}^{\prime }\left({F}^{\prime }:{\Vert}{F}^{\prime }{{\Vert}}_{{\left({H}^{\kappa }\right)}^{{\ast}}}{\leqslant}Q\sqrt{N}{\delta }_{N}^{2}\right)\right]+{M}_{N}\right){\geqslant}{\Phi}\left({{\Phi}}^{-1}\left[{\mathrm{e}}^{-BN{\delta }_{N}^{2}}\right]+{M}_{N}\right)\hfill \\ \hfill & \;\;\;\;\;\;=1-{\mathrm{e}}^{-BN{\delta }_{N}^{2}},\hfill \end{align*}$$

concluding the proof.□

We conclude with the verification of the complexity bound (A7) for the sets ${\mathcal{A}}_{N}$.

Lemma 18. Let ${\mathcal{A}}_{N}$ be as in lemma 17 for some fixed M, Q > 0. Then,

$$\begin{equation*}\mathrm{log}N\left({\delta }_{N};{\mathcal{A}}_{N},h\right){\leqslant}CN{\delta }_{N}^{2},\end{equation*}$$

for some constant C > 0 (depending on σ, M, Q, α, β, γ, κ, d, S) and all N large enough.

Proof. If $F\in {\mathcal{A}}_{N}$, then F = F₁ + F₂ with ${\Vert}{F}_{1}{{\Vert}}_{{\left({H}^{\kappa }\right)}^{{\ast}}}{\leqslant}Q{\delta }_{N}$ and ${\Vert}{F}_{2}{{\Vert}}_{{H}^{\alpha }}{\leqslant}{M}^{\prime },$ the latter inequality following from the continuous imbedding of $\mathcal{H}$ into ${H}_{c}^{\alpha }$. Thus, recalling the metric entropy estimate (A14), if

$$\begin{equation*}\left\{{H}_{1},\dots ,\;{H}_{P}\right\}\subset {B}_{c}^{\alpha }\left({M}^{\prime }\right),\quad P{\leqslant}{\mathrm{e}}^{-q{\delta }_{N}^{-d/\left(\alpha +\kappa \right)}}={\mathrm{e}}^{-qN{\delta }_{N}^{2}},\;q{ >}0,\end{equation*}$$

is a δ_N-net with respect to ${\Vert}\cdot {{\Vert}}_{{\left({H}^{\kappa }\right)}^{{\ast}}}$, we can find H_i such that ${\Vert}{F}_{2}-{H}_{i}{{\Vert}}_{{\left({H}^{\kappa }\right)}^{{\ast}}}{\leqslant}{\delta }_{N}$. Then, using the second inequality in (A26) below and the local Lipschitz estimate (A2),

$$\begin{equation*}h\left({p}_{F},{H}_{i}\right)< sim {\Vert}\mathcal{G}\left(F\right)-\mathcal{G}\left({H}_{i}\right){{\Vert}}_{{L}^{2}\left(\mathcal{D}\right)}< sim \left(1+{\Vert}{F{\Vert}}_{{C}^{\beta }}^{\gamma }\vee {\Vert}{H}_{i}{{\Vert}}_{{C}^{\beta }}^{\gamma }\right){\Vert}F-{H}_{i}{{\Vert}}_{{\left({H}^{\kappa }\right)}^{{\ast}}}.\end{equation*}$$

Recalling that if $F\in {\mathcal{A}}_{N}$ then also ${\Vert}F{{\Vert}}_{{C}^{\beta }}{\leqslant}M$, and using the Sobolev imbedding of H^α into C^β to bound ${\Vert}{H}_{i}{{\Vert}}_{{C}^{\beta }}$, we then obtain

$$\begin{equation*}h\left({p}_{F},{H}_{i}\right)< sim {\Vert}F-{H}_{i}{{\Vert}}_{{\left({H}^{\kappa }\right)}^{{\ast}}}< sim {\Vert}F-{F}_{2}{{\Vert}}_{{\left({H}^{\kappa }\right)}^{{\ast}}}+{\Vert}{F}_{2}-{H}_{i}{{\Vert}}_{{\left({H}^{\kappa }\right)}^{{\ast}}}< sim {\delta }_{N}.\end{equation*}$$

It follows that {H₁, ..., H_P} also forms a q'δ_N-net for ${\mathcal{A}}_{N}$ in the Hellinger distance for some q' > 0, so that

$$\begin{equation*}\mathrm{log}N\left({\delta }_{N};{\mathcal{A}}_{N},h\right){\leqslant}\mathrm{log}N\left({\delta }_{N}/{q}^{\prime };{B}_{c}^{\alpha }\left(M\right),{\Vert}\cdot {{\Vert}}_{{\left({H}^{\kappa }\right)}^{{\ast}}}\right)< sim N{\delta }_{N}^{2}.\end{equation*}$$

□

We now derive contraction rates in L²-prediction risk in the inverse problem (A1), for the truncated Gaussian random series priors introduced in section 2.2.3. The proof again proceeds by an application of theorem 13.

Theorem 19. Let the forward map $\mathcal{G}$ satisfy (A2) and (A3) for given β, γ, κ ⩾ 0 and S > 0. For any α > β + d/2, let Π be the random series prior in (19), and let Π(⋅|Y^(N), X^(N)) be the resulting posterior distribution arising from observations (Y^(N), X^(N)) in (A1). Then, for each α₀ ⩾ α, any ${F}_{0}\in {H}_{K}^{{\alpha }_{0}}\left(\mathcal{O}\right)$ and any D > 0 there exists L > 0 large enough (depending on σ, F₀, D, α, β, γ, κ, S, d) such that, as N → ∞,

$$\begin{equation}{\Pi}\left(F:{\Vert}\mathcal{G}\left(F\right)-\mathcal{G}\left({F}_{0}\right){{\Vert}}_{{L}^{2}\left(\mathcal{D}\right)}{ >}L{\xi }_{N}\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right)={O}_{{P}_{{F}_{0}}^{N}}\left({\mathrm{e}}^{-DN{\xi }_{N}^{2}}\right),\end{equation} \tag{ A19 }$$

where ${\xi }_{N}={N}^{-\left({\alpha }_{0}+\kappa \right)/\left(2{\alpha }_{0}+2\kappa +d\right)}\mathrm{log}N$. Moreover, for ${\mathcal{H}}_{J}$ the finite-dimensional subspaces from (18) and ${J}_{N}\in \mathbb{N}$ such that ${2}^{{J}_{N}}\simeq {N}^{1/\left(2{\alpha }_{0}+2\kappa +d\right)}$, we also have that for sufficiently large M > 0 (depending on D, α, β, d):

$$\begin{equation}{\Pi}\left(F:F\in {\mathcal{H}}_{{J}_{N}},\;{\Vert}F{{\Vert}}_{{H}^{\alpha }}{\leqslant}M{2}^{{J}_{N}\alpha }N{\xi }_{N}^{2}\vert {Y}^{\left(N\right)},{X}^{\left(N\right)}\right)={O}_{{P}_{{F}_{0}}^{N}}\left({\mathrm{e}}^{-DN{\xi }_{N}^{2}}\right).\end{equation} \tag{ A20 }$$

We begin deriving a suitable small ball estimate in the L²-prediction risk.

Lemma 20. Let Π, F₀ and ξ_N be as in theorem 19. Then, for all sufficiently large q > 0 there exists A > 0 (depending on q, F₀, α, β, γ, κ, d) such that

$$\begin{equation}{\Pi}\left(F:{\Vert}\mathcal{G}\left(F\right)-\mathcal{G}\left({F}_{0}\right){{\Vert}}_{{L}^{2}\left(\mathcal{D}\right)}{\leqslant}q{\xi }_{N}\right)\gtrsim {\mathrm{e}}^{-AN{\xi }_{N}^{2}}\end{equation} \tag{ A21 }$$

for all N large enough.

Proof. For each $j\in \mathbb{N}$, denote by Π_j the Gaussian probability measure on the finite dimensional subspace ${\mathcal{H}}_{j}$ in (18) defined as after (19) with the series truncated at j. For ${J}_{N}\in \mathbb{N}:{2}^{{J}_{N}}\simeq {N}^{1/\left(2{\alpha }_{0}+2\kappa +d\right)}$, note

$$\begin{equation}{2}^{{J}_{N}d}\mathrm{log}{2}^{{J}_{N}d}\simeq {N}^{d/\left(2{\alpha }_{0}+2\kappa +d\right)}\mathrm{log}N=N{\xi }_{N}^{2},\end{equation} \tag{ A22 }$$

so that, recalling the properties (20) of the random truncation level J, for some s > 0,

$$\begin{equation*}\mathrm{Pr}\left(K={J}_{N}\right)\gtrsim {\mathrm{e}}^{-{2}^{{J}_{N}d}\mathrm{log}{2}^{{J}_{N}d}}{\geqslant}{\mathrm{e}}^{-sN{\xi }_{N}^{2}}\end{equation*}$$

for all N large enough. It follows

$$\begin{align*}\hfill {\Pi}\left(F:{\Vert}\mathcal{G}\left(F\right)-\mathcal{G}\left({F}_{0}\right){{\Vert}}_{{L}^{2}}{\leqslant}q{\xi }_{N}\right)& {\geqslant}{{\Pi}}_{{J}_{N}}\left(F:{\Vert}\mathcal{G}\left(F\right)-\mathcal{G}\left({F}_{0}\right){{\Vert}}_{{L}^{2}}{\leqslant}q{\xi }_{N}\right)\mathrm{Pr}\left(K={J}_{N}\right)\hfill \\ \hfill & \gtrsim {{\Pi}}_{{J}_{N}}\left(F:{\Vert}\mathcal{G}\left(F\right)-\mathcal{G}\left({F}_{0}\right){{\Vert}}_{{L}^{2}}{\leqslant}q{\xi }_{N}\right){\mathrm{e}}^{-sN{\xi }_{N}^{2}}.\hfill \end{align*}$$

Next, let

$$\begin{equation*}{P}_{{\mathcal{H}}_{{J}_{N}}}\left({F}_{0}\right)=\chi \sum _{\ell {\leqslant}{J}_{N},r\in {\mathcal{R}}_{\ell }}{\langle {F}_{0},{{\Psi}}_{\ell r}\rangle }_{{L}^{2}}{{\Psi}}_{\ell r}\end{equation*}$$

be the 'projection' of F₀ onto ${\mathcal{H}}_{{J}_{N}}$. Since ${F}_{0}\in {H}_{K}^{{\alpha }_{0}}\subset {C}^{\beta }$ by a Sobolev imbedding, it follows using (A2) and standard approximation properties of wavelets (B6),

$$\begin{equation*}{\Vert}\mathcal{G}\left({F}_{0}\right)-\mathcal{G}\left({P}_{{\mathcal{H}}_{{J}_{N}}}\left({F}_{0}\right)\right){{\Vert}}_{{L}^{2}\left(\mathcal{D}\right)}< sim {\Vert}{F}_{0}-{P}_{{\mathcal{H}}_{{J}_{N}}}\left({F}_{0}\right){{\Vert}}_{{\left({H}^{\kappa }\right)}^{{\ast}}}< sim {2}^{-{J}_{N}\left({\alpha }_{0}+\kappa \right)}={N}^{-\frac{{\alpha }_{0}+\kappa }{2{\alpha }_{0}+2\kappa +d}},\end{equation*}$$

which implies by the triangle inequality that

$$\begin{align*}\hfill & {{\Pi}}_{{J}_{N}}\left(F:{\Vert}\mathcal{G}\left(F\right)-\mathcal{G}\left({F}_{0}\right){{\Vert}}_{{L}^{2}}{\leqslant}q{\xi }_{N}\right)\hfill \\ \hfill & \quad \;\;\;\;{\geqslant}{{\Pi}}_{{J}_{N}}\left(F:{\Vert}\mathcal{G}\left(F\right)-\mathcal{G}\left({P}_{{\mathcal{H}}_{{J}_{N}}}\left({F}_{0}\right)\right){{\Vert}}_{{L}^{2}}{\leqslant}q{\xi }_{N}-{\Vert}\mathcal{G}\left({F}_{0}\right)-\mathcal{G}\left({P}_{{\mathcal{H}}_{{J}_{N}}}\left({F}_{0}\right)\right){{\Vert}}_{{L}^{2}}\right)\hfill \\ \hfill & \quad \;\;\;\;{\geqslant}{{\Pi}}_{{J}_{N}}\left(F:{\Vert}\mathcal{G}\left(F\right)-\mathcal{G}\left({P}_{{\mathcal{H}}_{{J}_{N}}}\left({F}_{0}\right)\right){{\Vert}}_{{L}^{2}}{\leqslant}{q}^{\prime }{\xi }_{N}\right).\hfill \end{align*}$$

Using again that H^α imbeds continuously into C^β as well as (A2) and (B5), we can lower bound the last probability by

$$\begin{align*}\hfill & {{\Pi}}_{{J}_{N}}\left(F:{\Vert}\mathcal{G}\left(F\right)-\mathcal{G}\left({P}_{{\mathcal{H}}_{{J}_{N}}}\left({F}_{0}\right)\right){{\Vert}}_{{L}^{2}\left(\mathcal{D}\right)}{\leqslant}{q}^{\prime }{\xi }_{N},{\Vert}F-{P}_{{\mathcal{H}}_{{J}_{N}}}\left({F}_{0}\right){{\Vert}}_{{H}^{\alpha }\left(\mathcal{O}\right)}{\leqslant}{\xi }_{N}\right)\hfill \\ \hfill & \quad \;\;\;\;{\geqslant}{{\Pi}}_{{J}_{N}}\left(F:{\Vert}F-{P}_{{\mathcal{H}}_{{J}_{N}}}\left({F}_{0}\right){{\Vert}}_{{\left({H}^{\kappa }\left(\mathcal{O}\right)\right)}^{{\ast}}}{\leqslant}{q}^{{\prime\prime}}{\xi }_{N},{\Vert}F-{P}_{{\mathcal{H}}_{{J}_{N}}}\left({F}_{0}\right){{\Vert}}_{{H}^{\alpha }\left(\mathcal{O}\right)}{\leqslant}{\xi }_{N}\right)\hfill \\ \hfill & \quad \;\;\;\;{\geqslant}{{\Pi}}_{{J}_{N}}\left(F:{\Vert}F-{P}_{{\mathcal{H}}_{{J}_{N}}}\left({F}_{0}\right){{\Vert}}_{{H}^{\alpha }\left(\mathcal{O}\right)}{\leqslant}q{\prime\prime\prime}{\xi }_{N}\right),\hfill \end{align*}$$

which, by corollary 2.6.18 in Giné and Nickl (2016) and in view of (B2) is further lower bounded by

$$\begin{equation*}{\mathrm{e}}^{-\frac{1}{2}{\Vert}{P}_{{\mathcal{H}}_{{J}_{N}}}{\left({F}_{0}\right){\Vert}}_{{\mathcal{H}}_{{J}_{N}}}^{2}}{{\Pi}}_{{J}_{N}}\left(F:{\Vert}F{{\Vert}}_{{H}^{\alpha }}{\leqslant}q{\prime\prime\prime}{\xi }_{N}\right){\geqslant}{\mathrm{e}}^{-{s}^{\prime }{\Vert}{F}_{0}{{\Vert}}_{{H}^{{\alpha }_{0}}}^{2}}{{\Pi}}_{{J}_{N}}\left(F:{\Vert}F{{\Vert}}_{{H}^{\alpha }}{\leqslant}q{\prime\prime\prime}{\xi }_{N}\right).\end{equation*}$$

Now since $f{\mapsto}\chi f,\chi \in {C}^{\infty }\left(\mathcal{O}\right)$, is continuous on ${H}^{\alpha }\left(\mathcal{O}\right)$,

$$\begin{align*}\hfill {{\Pi}}_{{J}_{N}}\left(F:{\Vert}F{{\Vert}}_{{H}^{\alpha }}{\leqslant}q{\prime\prime\prime}{\xi }_{N}\right)& =\mathrm{Pr}\left({{\Vert}\chi ,{\sum }_{\ell {\leqslant}{J}_{N},r\in {\mathcal{R}}_{\ell }},{2}^{-\ell \alpha },{F}_{\ell r},{{\Psi}}_{\ell r}{\Vert}}_{{H}^{\alpha }}{\leqslant}q{\prime\prime\prime}{\xi }_{N}\right)\hfill \\ \hfill & {\geqslant}\mathrm{Pr}\left({\sum }_{m=1}^{\mathrm{dim}\left({\mathcal{H}}_{{J}_{N}}\right)}{Z}_{m}^{2}{\leqslant}t{\xi }_{N}^{2}\right)\hfill \end{align*}$$

for some t > 0, where ${Z}_{m}\text{iid}{\sim }N\left(0,1\right)$, and where we have used the wavelet characterisation of the ${H}^{\alpha }\left({\mathbb{R}}^{d}\right)$ norm. To conclude, note that the last probability is greater than

$$\begin{align*}\mathrm{Pr}\left(\sqrt{\mathrm{dim}\left({\mathcal{H}}_{{J}_{N}}\right)}\underset{m{\leqslant}\mathrm{dim}\left({\mathcal{H}}_{{J}_{N}}\right)}{\mathrm{max}}\vert {Z}_{m}\vert {\leqslant}\sqrt{t}{\xi }_{N}\right)\hfill & {\geqslant}\mathrm{Pr}\left(\underset{m{\leqslant}\mathrm{dim}\left({\mathcal{H}}_{{J}_{N}}\right)}{\mathrm{max}}\vert {Z}_{m}\vert {\leqslant}{t}^{\prime }{N}^{-\frac{{\alpha }_{0}+\kappa }{2{\alpha }_{0}+2\kappa +d}}{N}^{-\frac{d/2}{2{\alpha }_{0}+2\kappa +d}}\right)\hfill \\ \hfill & =\prod _{m{\leqslant}\mathrm{dim}\left({\mathcal{H}}_{{J}_{N}}\right)}\mathrm{Pr}\left(\vert {Z}_{m}\vert {\leqslant}{t}^{\prime }{N}^{-\frac{{\alpha }_{0}+\kappa +d/2}{2{\alpha }_{0}+2\kappa +d}}\right).\hfill \end{align*}$$

Finally, a standard calculation shows that Pr(|Z₁| ⩽ t) ≳ t if t → 0, and hence the last product is lower bounded, for large N, by

$$\begin{equation*}{\left({t}^{\prime }{N}^{-\frac{{\alpha }_{0}+\kappa +d/2}{2{\alpha }_{0}+2\kappa +d}}\right)}^{\mathrm{dim}\left({\mathcal{H}}_{{J}_{N}}\right)}={\mathrm{e}}^{\mathrm{dim}\left({\mathcal{H}}_{{J}_{N}}\right)\mathrm{log}\left({t}^{\prime }{N}^{-\frac{{\alpha }_{0}+\kappa +d/2}{2{\alpha }_{0}+2+d}}\right)}{\geqslant}{\mathrm{e}}^{-{t}^{{\prime\prime}}{2}^{{J}_{N}d}\mathrm{log}N}={\mathrm{e}}^{-t{\prime\prime\prime}N{\xi }_{N}^{2}}.\end{equation*}$$

□

In the following lemma we construct suitable approximating sets, for which we check the excess mass condition (A6) and the complexity bound (A7) required in theorem 13.

Lemma 21. Let Π, ξ_N and J_N be as in theorem 8, and let ${\mathcal{H}}_{{J}_{N}}$ be the finite dimensional subspace defined in (18) with J = J_N. Define for each M > 0

$$\begin{equation}{\mathcal{A}}_{N}=\left\{F\in {\mathcal{H}}_{{J}_{N}},\;{\Vert}F{{\Vert}}_{{H}^{\alpha }}{\leqslant}M{2}^{{J}_{N}\alpha }N{\xi }_{N}^{2}\right\}.\end{equation} \tag{ A23 }$$

Then, for any B > 0 there exists M > 0 large enough (depending on B, α, β, d) such that, for sufficiently large N

$$\begin{equation}{\Pi}\left({\mathcal{A}}_{N}^{c}\right){\leqslant}2{\mathrm{e}}^{-BN{\xi }_{N}^{2}}.\end{equation} \tag{ A24 }$$

Moreover, for each fixed M > 0 and all N large enough

$$\begin{equation}\mathrm{log}N\left({\xi }_{N};{\mathcal{A}}_{N},h\right){\leqslant}CN{\xi }_{N}^{2}\end{equation} \tag{ A25 }$$

for some C > 0 (depending on σ, α, β, γ, κ, S, d).

Proof. Letting ${Z}_{m}\text{iid}{\sim }N\left(0,1\right)$, noting ${\Vert}{F{\Vert}}_{{H}^{\alpha }}^{2}{\leqslant}{2}^{2{J}_{N}\alpha }{\sum }_{\ell {\leqslant}{J}_{N},r\in {\mathcal{R}}_{\ell }}{F}_{\ell r}^{2}$ for all $F\in {\mathcal{H}}_{{J}_{N}}$ (cf. (B2)) and using (A22) and (20), we have for sufficiently large N

$$\begin{align*}\hfill {\Pi}\left({\mathcal{A}}_{N}^{c}\right)& {\leqslant}\mathrm{Pr}\left(J{ >}{J}_{N}\right)+\mathrm{Pr}\left(\sum _{\ell {\leqslant}J\wedge {J}_{N},r\in {\mathcal{R}}_{\ell }}{F}_{\ell r}^{2}{\leqslant}MN{\xi }_{N}^{2}\right)\hfill \\ \hfill & {\leqslant}{\mathrm{e}}^{-{2}^{{J}_{N}d}\mathrm{log}{2}^{{J}_{N}d}}+\mathrm{Pr}\left(\sum _{m{\leqslant}\mathrm{dim}\left({\mathcal{H}}_{{J}_{N}}\right)}{Z}_{m}^{2}{ >}MN{\xi }_{N}^{2}\right)\hfill \\ \hfill & {\leqslant}{\mathrm{e}}^{-BN{\xi }_{N}^{2}}+\mathrm{Pr}\left(\sum _{m{\leqslant}\mathrm{dim}\left({\mathcal{H}}_{{J}_{N}}\right)}\left({Z}_{m}^{2}-1\right){ >}\overline{M}N{\xi }_{N}^{2}\right)\hfill \end{align*}$$

for any constant $0{< }\overline{M}{< }{M}^{2}-1$, since $\mathrm{dim}\left({\mathcal{H}}_{{J}_{N}}\right)< sim {2}^{{J}_{N}d}\simeq {N}^{d/\left(2\alpha +2+d\right)}=o\left(N{\xi }_{N}^{2}\right)$. The bound (A24) then follows applying theorem 3.1.9 in Giné and Nickl (2016) to upper bound the last probability, for any B > and for sufficiently large M and $\overline{M}$, by

$$\begin{equation*}{\mathrm{e}}^{-\frac{{\overline{M}}^{2}{\left(N{\xi }_{N}^{2}\right)}^{2}}{4\mathrm{dim}\left({\mathcal{H}}_{{J}_{N}}\right)+\overline{M}N{\xi }_{N}^{2}}}{\leqslant}{\mathrm{e}}^{-BN{\xi }_{N}^{2}}.\end{equation*}$$

We proceed with the derivation of (A25). By choice of J_N, if $F\in {\mathcal{A}}_{N}$ then ${\Vert}F{{\Vert}}_{{H}^{\alpha }}^{2}< sim {N}^{\left(2\alpha \right)/\left(2\alpha +2\kappa +d\right)}N{\xi }_{N}^{2}.$ Hence, by the second inequality in (A26), using (A2) and the Sobolev imbedding of H^α into C^β, if ${F}_{1},{F}_{2}\in {\mathcal{A}}_{N}$ then

$$\begin{align*}\hfill h\left({p}_{{F}_{1}},{p}_{{F}_{2}}\right)& < sim {\Vert}\mathcal{G}\left({F}_{1}\right)-\mathcal{G}\left({F}_{2}\right){{\Vert}}_{{L}^{2}\left(\mathcal{D}\right)}\hfill \\ \hfill & < sim \left(1+{\left({N}^{\frac{\alpha }{2\alpha +2\kappa +d}}\sqrt{N}{\xi }_{N}\right)}^{\gamma }\right){\Vert}{F}_{1}-{F}_{2}{{\Vert}}_{{\left({H}^{\kappa }\right)}^{{\ast}}}\hfill \\ \hfill & < sim {N}^{\frac{\alpha \gamma }{2\alpha +2\kappa +d}}{\left(\sqrt{N}{\xi }_{N}\right)}^{\gamma }\sqrt{\sum _{\ell {\leqslant}{J}_{N},r\in {\mathcal{R}}_{\ell }}{\left({F}_{1,\ell r}-{F}_{2,\ell r}\right)}^{2}}.\hfill \end{align*}$$

Therefore, using the standard metric entropy estimate for balls ${B}_{{\mathbb{R}}^{p}}\left(r\right),\;r{ >}0$, in Euclidean spaces (Giné and Nickl (2016), proposition 4.3.34), we see that for N large enough

$$\begin{align*}\hfill \mathrm{log}N\left({\xi }_{N};{\mathcal{A}}_{N},h\right)& < sim \mathrm{log}N\left({\xi }_{N}{N}^{\frac{-\alpha \gamma }{2\alpha +2\kappa +d}}{\left(\sqrt{N}{\xi }_{N}\right)}^{-\gamma };{B}_{{\mathbb{R}}^{\mathrm{dim}\left({\mathcal{H}}_{{J}_{N}}\right)}}\left(M\sqrt{N}{\xi }_{N}\right),{\Vert}\cdot {{\Vert}}_{{\mathbb{R}}^{\mathrm{dim}\left({\mathcal{H}}_{{J}_{N}}\right)}}\right)\hfill \\ \hfill & {\leqslant}\mathrm{dim}\left({\mathcal{H}}_{{J}_{N}}\right)\mathrm{log}\frac{3M\sqrt{N}{\xi }_{N}}{{\xi }_{N}{N}^{\frac{\alpha \gamma }{2\alpha +2\kappa +d}}{\left(\sqrt{N}{\xi }_{N}\right)}^{-\gamma }}< sim N{\xi }_{N}^{2}.\hfill \end{align*}$$

□

In the following lemma due to Birgé (2004) we exploit the boundedness assumption (A3) on $\mathcal{G}$ to show the equivalence between the Hellinger distance appearing in the conclusion of theorem 13 and the L²-distance on the 'regression functions' $\mathcal{G}\left(F\right)$.

Lemma 22. Let the forward map $\mathcal{G}$ satisfy (A3) for some S > 0. Then, for all ${F}_{1},{F}_{2}\in \mathcal{F}$

$$\begin{equation}\frac{1-{\mathrm{e}}^{-{S}^{2}/\left(2{\sigma }^{2}\right)}}{4{S}^{2}}{\Vert}\mathcal{G}\left({F}_{1}\right)-\mathcal{G}{\left({F}_{2}\right){\Vert}}_{{L}^{2}\left(\mathcal{D}\right)}^{2}{\leqslant}{h}^{2}\left({p}_{{F}_{1}},{p}_{{F}_{2}}\right){\leqslant}\frac{1}{4{\sigma }^{2}}{\Vert}\mathcal{G}\left({F}_{1}\right)-\mathcal{G}\left({F}_{2}\right){{\Vert}}_{{L}^{2}\left(\mathcal{D}\right)}^{2}.\end{equation} \tag{ A26 }$$

Proof. Note ${h}^{2}\left({p}_{{F}_{1}},{p}_{{F}_{2}}\right)=2-2\rho \left({p}_{{F}_{1}},{p}_{{F}_{2}}\right)$, where

$$\begin{equation*}\rho \left({p}_{{F}_{1}},{p}_{{F}_{2}}\right){:=}{\int }_{\mathbb{R}{\times}\mathcal{D}}\sqrt{{p}_{{F}_{1}}\left(y,x\right){p}_{{F}_{2}}\left(y,x\right)}\mathrm{d}y\mathrm{d}x\end{equation*}$$

is the Hellinger affinity. Using the expression of the likelihood in (8) (with $\mathcal{D}$ instead of $\mathcal{O}$), the right hand side is seen to be equal to

$$\begin{align*}\hfill & {\int }_{\mathbb{R}{\times}\mathcal{D}}\frac{1}{\sqrt{2\pi {\sigma }^{2}}}{\mathrm{e}}^{-\left\{{\left[y-\mathcal{G}\left({F}_{1}\right)\left(x\right)\right]}^{2}-{\left[y-\mathcal{G}\left({F}_{2}\right)\left(x\right)\right]}^{2}\right\}/\left(4{\sigma }^{2}\right)}\mathrm{d}y\mathrm{d}x\hfill \\ \hfill & \quad \;\;\;\;={\int }_{\mathcal{D}}{\mathrm{e}}^{-\left\{{\left[\mathcal{G}\left({F}_{1}\right)\left(x\right)\right]}^{2}+{\left[\mathcal{G}\left({F}_{2}\right)\left(x\right)\right]}^{2}\right\}/\left(4{\sigma }^{2}\right)}\hfill \\ \hfill & \quad \;\;\;\;\quad {\times}\;\left[{\int }_{\mathbb{R}}\frac{{\mathrm{e}}^{-{y}^{2}/\left(2{\sigma }^{2}\right)}}{\sqrt{2\pi {\sigma }^{2}}}{\mathrm{e}}^{y\left[\mathcal{G}\left({F}_{1}\right)\left(x\right)+\mathcal{G}\left({F}_{2}\right)\left(x\right)\right]/\left(2{\sigma }^{2}\right)}\mathrm{d}y\right]\mathrm{d}x\hfill \\ \hfill & \quad \;\;\;\;={\int }_{\mathcal{D}}{\mathrm{e}}^{-\left\{{\left[\mathcal{G}\left({F}_{1}\right)\left(x\right)\right]}^{2}+{\left[\mathcal{G}\left({F}_{2}\right)\left(x\right)\right]}^{2}\right\}/\left(4{\sigma }^{2}\right)}{\mathrm{e}}^{{\left[\mathcal{G}\left({F}_{1}\right)\left(x\right)+\mathcal{G}\left({F}_{2}\right)\left(x\right)\right]}^{2}/\left(8{\sigma }^{2}\right)}\mathrm{d}x\hfill \end{align*}$$

having used that the moment generating function of Z ∼ N(0, σ²) satisfies $E{\mathrm{e}}^{tZ}={\mathrm{e}}^{{\sigma }^{2}{t}^{2}/2},\;t\in \mathbb{R}$. Thus, the latter integral equals

$$\begin{equation*}{\int }_{\mathcal{D}}{\mathrm{e}}^{-\left\{{\left[\mathcal{G}\left({F}_{1}\right)\left(x\right)\right]}^{2}+{\left[\mathcal{G}\left({F}_{2}\right)\left(x\right)\right]}^{2}-2\mathcal{G}\left({F}_{2}\right)\left(x\right)\mathcal{G}\left({F}_{2}\right)\left(x\right)\right\}/\left(8{\sigma }^{2}\right)}\mathrm{d}x={E}^{\mu }{\mathrm{e}}^{-{\left\{\mathcal{G}\left({F}_{1}\right)\left(X\right)-\mathcal{G}\left({F}_{2}\right)\left(X\right)\right\}}^{2}/\left(8{\sigma }^{2}\right)}.\end{equation*}$$

To derive the second inequality in (A26), we use Jensen's inequality to lower bound the expectation in the last line by

$$\begin{equation*}{\mathrm{e}}^{-{E}^{\mu }{\left\{\mathcal{G}\left({F}_{1}\right)\left(X\right)-\mathcal{G}\left({F}_{2}\right)\left(X\right)\right\}}^{2}/\left(8{\sigma }^{2}\right)}={\mathrm{e}}^{-{\Vert}\mathcal{G}\left({F}_{1}\right)-\mathcal{G}\left({F}_{2}\right){{\Vert}}_{{L}^{2}\left(\mathcal{D}\right)}^{2}/\left(8{\sigma }^{2}\right)}.\end{equation*}$$

Hence

$$\begin{equation*}{h}^{2}\left({p}_{{F}_{1}},{p}_{{F}_{2}}\right){\leqslant}2\left[1-{\mathrm{e}}^{-{\Vert}\mathcal{G}\left({F}_{1}\right)-\mathcal{G}\left({F}_{2}\right){{\Vert}}_{{L}^{2}\left(\mathcal{D}\right)}^{2}/\left(8{\sigma }^{2}\right)}\right],\end{equation*}$$

whereby the claim follows using the basic inequality 1 − e^−z/c ⩽ z/c, for all c, z > 0.

To deduce the first inequality we follow the proof of proposition 1 in Birgé (2004): note that for all 0 ⩽ z₁ < z₂

$$\begin{equation*}{\mathrm{e}}^{-{z}_{1}}{\leqslant}\frac{{z}_{1}}{{z}_{2}}{\mathrm{e}}^{-{z}_{2}}+\left(1-\frac{{z}_{1}}{{z}_{2}}\right)=\frac{{\mathrm{e}}^{-{z}_{2}}-1}{{z}_{2}}{z}_{1}+1.\end{equation*}$$

Then taking ${z}_{1}={\left\{\mathcal{G}\left({F}_{1}\right)\left(X\right)-\mathcal{G}\left({F}_{2}\right)\left(X\right)\right\}}^{2}/\left(8{\sigma }^{2}\right)$ and z₂ = S²/(2σ²),

$$\begin{equation*}{E}^{\mu }{\mathrm{e}}^{-{\left\{\mathcal{G}\left({F}_{1}\right)\left(X\right)-\mathcal{G}\left({F}_{2}\right)\left(X\right)\right\}}^{2}/\left(8{\sigma }^{2}\right)}{\leqslant}\frac{{\mathrm{e}}^{-{S}^{2}/\left(2{\sigma }^{2}\right)}-1}{4{S}^{2}}{\Vert}\mathcal{G}\left({F}_{1}\right)-\mathcal{G}{\left({F}_{2}\right){\Vert}}_{{L}^{2}\left(\mathcal{D}\right)}^{2}+1\end{equation*}$$

which in turn yields the result.□

The next lemma bounds the Kullback–Leibler divergences appearing in (A4) in terms of the L²-prediction risk.

Lemma 23. Let the observation Y_i in (A1) be generated by some fixed ${F}_{0}\in \mathcal{F}$. Then, for each $F\in \mathcal{F}$,

$$\begin{equation*}{E}_{{F}_{0}}^{1}\left[\mathrm{log}\frac{{p}_{{F}_{0}}\left({Y}_{1},{X}_{1}\right)}{{p}_{F}\left({Y}_{1},{X}_{1}\right)}\right]=\frac{1}{{2\sigma }^{2}}{\Vert}\mathcal{G}\left({F}_{0}\right)-\mathcal{G}{\left(F\right){\Vert}}_{{L}^{2}\left(\mathcal{D}\right)}^{2},\end{equation*}$$

and

$$\begin{equation*}{E}_{{F}_{0}}^{1}{\left[\mathrm{log}\frac{{p}_{{F}_{0}}\left({Y}_{1},{X}_{1}\right)}{{p}_{F}\left({Y}_{1},{X}_{1}\right)}\right]}^{2}{\leqslant}\frac{2\left({S}^{2}+{\sigma }^{2}\right)}{{\sigma }^{4}}{\Vert}\mathcal{G}\left({F}_{0}\right)-\mathcal{G}{\left(F\right){\Vert}}_{{L}^{2}\left(\mathcal{D}\right)}^{2}.\end{equation*}$$

Proof. If ${Y}_{1}=\mathcal{G}\left({F}_{0}\right)\left({X}_{1}\right)+\sigma {W}_{1},$ then

$$\begin{align*}\hfill & \mathrm{log}\frac{{p}_{{F}_{0}}\left({Y}_{1},{X}_{1}\right)}{{p}_{F}\left({Y}_{1},{X}_{1}\right)}\hfill \\ \hfill & \quad \;\;\;\;=-\frac{1}{2{\sigma }^{2}}\left\{{\left[\mathcal{G}\left({F}_{0}\right)\left({X}_{1}\right)+\sigma {W}_{1}-\mathcal{G}\left({F}_{0}\right)\left({X}_{1}\right)\right]}^{2}-{\left[\mathcal{G}\left({F}_{0}\right)\left({X}_{1}\right)+\sigma {W}_{1}-\mathcal{G}\left(F\right)\left({X}_{1}\right)\right]}^{2}\right\}\hfill \\ \hfill & \quad \;\;\;\;=\frac{1}{2{\sigma }^{2}}{\left\{\mathcal{G}\left({F}_{0}\right)\left({X}_{1}\right)-\mathcal{G}\left(F\right)\left({X}_{1}\right)\right\}}^{2}+\frac{1}{\sigma }{W}_{1}\left\{\mathcal{G}\left({F}_{0}\right)\left({X}_{1}\right)-\mathcal{G}\left(F\right)\left({X}_{1}\right)\right\}.\hfill \end{align*}$$

Hence, since EW₁ = 0 and X₁ ∼ μ,

$$\begin{align*}\hfill {E}_{{F}_{0}}^{1}\left[\mathrm{log}\frac{{p}_{{F}_{0}}\left({Y}_{1},{X}_{1}\right)}{{p}_{F}\left({Y}_{1},{X}_{1}\right)}\right]& ={E}^{\mu }\left[\frac{1}{2{\sigma }^{2}}{\left\{\mathcal{G}\left({F}_{0}\right)\left({X}_{1}\right)-\mathcal{G}\left(F\right)\left({X}_{1}\right)\right\}}^{2}\right]\hfill \\ \hfill & =\frac{1}{2{\sigma }^{2}}{\Vert}\mathcal{G}\left({F}_{0}\right)-\mathcal{G}{\left(F\right){\Vert}}_{{L}^{2}\left(\mathcal{D}\right)}^{2}.\hfill \end{align*}$$

On the other hand,

$$\begin{align*}{\left[\mathrm{log}\frac{{p}_{{F}_{0}}\left({Y}_{1},{X}_{1}\right)}{{p}_{F}\left({Y}_{1},{X}_{1}\right)}\right]}^{2}\hfill & ={\left[\frac{1}{2{\sigma }^{2}}{\left\{\mathcal{G}\left({F}_{0}\right)\left({X}_{1}\right)-\mathcal{G}\left(F\right)\left({X}_{1}\right)\right\}}^{2}+\frac{1}{\sigma }{W}_{1}\left\{\mathcal{G}\left({F}_{0}\right)\left({X}_{1}\right)-\mathcal{G}\left(F\right)\left({X}_{1}\right)\right\}\right]}^{2}\hfill \\ \hfill & {\leqslant}2{\left[\frac{1}{2{\sigma }^{2}}{\left\{\mathcal{G}\left({F}_{0}\right)\left({X}_{1}\right)-\mathcal{G}\left(F\right)\left({X}_{1}\right)\right\}}^{2}\right]}^{2}+2{\left[\frac{1}{\sigma }{W}_{1}\left\{\mathcal{G}\left({F}_{0}\right)\left({X}_{1}\right)-\mathcal{G}\left(F\right)\left({X}_{1}\right)\right\}\right]}^{2}\hfill \\ \hfill & {\leqslant}\frac{2{S}^{2}}{{\sigma }^{4}}{\left\{\mathcal{G}\left({F}_{0}\right)\left({X}_{1}\right)-\mathcal{G}\left(F\right)\left({X}_{1}\right)\right\}}^{2}+\frac{2}{{\sigma }^{2}}{W}_{1}^{2}{\left\{\mathcal{G}\left({F}_{0}\right)\left({X}_{1}\right)-\mathcal{G}\left(F\right)\left({X}_{1}\right)\right\}}^{2},\hfill \end{align*}$$

whence the second claim follows since $E{W}_{1}^{2}=1$.□