Generalized Sampling and Infinite-Dimensional Compressed Sensing

Abstract

We introduce and analyze a framework and corresponding method for compressed sensing in infinite dimensions. This extends the existing theory from finite-dimensional vector spaces to the case of separable Hilbert spaces. We explain why such a new theory is necessary by demonstrating that existing finite-dimensional techniques are ill suited for solving a number of key problems. This work stems from recent developments in generalized sampling theorems for classical (Nyquist rate) sampling that allows for reconstructions in arbitrary bases. A conclusion of this paper is that one can extend these ideas to allow for significant subsampling of sparse or compressible signals. Central to this work is the introduction of two novel concepts in sampling theory, the stable sampling rate and the balancing property, which specify how to appropriately discretize an infinite-dimensional problem.

Appendix

This appendix contains all the proofs not given so far. Before we do this, there are two results that will be crucial. The first is a due to Rudelson [54].

Lemma 11.1

(Rudelson) Let $\eta _1, \ldots , \eta _M \in {\mathbb {C}}^n$ and let $\varepsilon _1, \ldots \varepsilon _M$ be independent Bernoulli variables taking values $1,-1$ with probability 1 / 2. Then

$$\begin{aligned} {\mathbb {E}}\left( \left\| \sum _{i=1}^M \varepsilon _i \bar{\eta }_i \otimes \eta _i \right\| \right) \le \frac{3}{2}\sqrt{p} \max _{i \le M} \Vert \eta _i\Vert \sqrt{\left\| \sum _{i=1}^M \bar{\eta }_i \otimes \eta _i \right\| } \end{aligned}$$

where $p= \max \{ 2, 2\log (n) \}$.

Note that the original lemma in [54] does not apply in this case. Actually, we need the complex version proved in [61]. We will, however, still refer to it as Rudelson’s Lemma. The following theorem is also indispensable:

Theorem 11.2

(Talagrand [45, 59]) There exists a number K with the following property. Consider n independent random variables $X_i$ valued in a measurable space $\Omega .$ Let ${\mathcal {F}}$ be a (countable) class of measurable functions on $\Omega$ and consider the random variable $Z = \sup _{f \in {\mathcal {F}}}\sum _{i \le n} f(X_i).$ Let

$$\begin{aligned} S = \sup _{f \in {\mathcal {F}}}\Vert f\Vert _{\infty }, \quad V = \sup _{f \in {\mathcal {F}}} {\mathbb {E}}\left( \sum _{i \le n}f(X_i)^2 \right) \!. \end{aligned}$$

If ${\mathbb {E}}(f(X_i)) = 0$ for all $f \in {\mathcal {F}}$ and $i\le n$, then, for each $t > 0$, we have

$$\begin{aligned} {\mathbb {P}}(|Z - {\mathbb {E}}(Z)| \ge t) \le 3 \exp \left( -\frac{1}{K}\frac{t}{S} \log \left( 1 + \frac{tS}{V+S{\mathbb {E}}(\overline{Z})} \right) \right) \!, \end{aligned}$$

where $\overline{Z} = \sup _{f\in {\mathcal {F}}}|\sum _{i \le n} f(X_i)|$.

Note that we deliberately forgo the use of any vector/matrix Bernstein inequalities in the proofs that follow, and instead use Talagrand’s result. This allows for more flexibility in the infinite-dimensional setting.

We next present the proofs of Propositions 8.5 and 8.6. For this, it is useful to have a result about the existence of unique minimizers. The finite-dimensional version of the following proposition has become standard for showing existence of unique minimizers for finite-dimensional problems found in CS (see, e.g., [20, Lem. 2.1] or [34, Thm. 4.26]). Fortunately, the extension to infinite dimensions is rather straightforward:

Proposition 11.3

Let $U \in {\mathcal {B}}(l^2({\mathbb {N}}))$ be unitary and let $\Omega , \Delta \subset {\mathbb {N}}$ be such that $|\Omega |, |\Delta | < \infty$. Suppose that $x_0 \in {\mathcal {H}}$ with $\mathrm {supp}(x_0) = \Delta$ and consider the optimization problem

$$\begin{aligned} \inf _{\eta \in {\mathcal {H}}} \Vert \eta \Vert _{l^1} \ \text{ subject } \text{ to }\ P_{\Omega }U\eta = P_{\Omega }Ux_0. \end{aligned}$$

(11.1)

Suppose that there exists a vector $\rho \in {\mathcal {H}}$ such that

• (i) $\rho = U^*P_{\Omega }\eta$ for some $\eta \in {\mathcal {H}}$

• (ii) $\langle \rho , e_j \rangle = \langle \mathrm {sgn}(x_0), e_j \rangle$, $j \in \Delta$

• (iii) $|\langle \rho , e_j \rangle | < 1$, $j \notin \Delta ,$

and in addition that $P_{\Omega }UP_{\Delta }: P_{\Delta }{\mathcal {H}} \rightarrow P_{\Omega }{\mathcal {H}}$ has full rank, then $x_0$ is the unique minimizer of (11.1). If U and $x_0$ are real the converse is also true.

Proof

By assumption, there is a $\rho \in l^{\infty }({\mathbb {N}})$ such that $\rho = U^*P_{\Omega }y$ for some $y \in P_{\Omega }{\mathcal {H}}$ and $\Vert \rho \Vert _{l^{\infty }} \le 1$. Also, by (ii)

$$\begin{aligned} \mathrm {Re}\big (\langle P_{\Omega }UP_{\Delta }x_0, y\rangle \big ) = \mathrm {Re}\big (\langle x_0,P_{\Delta } \rho \rangle \big ) = \sum _{j \in \Delta } \mathrm {sign}\big (\langle x_0,e_j\rangle \big )\langle x_0,e_j\rangle = \Vert x_0\Vert _{l^1}. \end{aligned}$$

Thus, by using duality (recall Proposition 8.3), in particular the fact that $P_{\Omega }U : {\mathcal {H}} \rightarrow P_{\Omega }{\mathcal {H}}$ is onto (this follows since U is unitary) and that

$$\begin{aligned} \inf \{\Vert x\Vert _{l^1}: P_{\Omega }Ux = P_{\Omega }Ux_0\} = \sup \{\mathrm {Re}\big (\langle P_{\Omega }Ux_0, y \rangle \big ):\, \Vert U^*P_{\Omega }y\Vert _{l^{\infty }} \le 1\}, \end{aligned}$$

it follows that $x_0$ is a minimizer. But $\langle \rho ,e_j\rangle < 1$ for $j \notin \Delta$ so if $\xi$ is another minimizer then $\mathrm {supp}(\xi ) = \Delta .$ However, $P_{\Omega }UP_{\Delta }$ has full rank, so $\xi = x_0.$

As for the converse in the real case, suppose that $x_0$ is the unique minimizer. Then, for all sufficiently large n, $x_0$ is the unique minimizer to the finite-dimensional optimization problem $\inf \{\Vert x\Vert _{l^1}: x \in P_n{\mathcal {H}}, \, P_{\Omega }UP_nx = P_{\Omega }Ux_0\}.$ Proposition 11.3 is well known to be true in finite dimensions [34]. It follows that there is a $y_n$ such that, for $\rho _n = P_nU^*P_{\Omega } y_n,$ we have $\langle \rho _n, e_j \rangle < 1$ when $j \notin \Delta$ and $j \le n$, and $\langle \rho _n, e_j \rangle = \mathrm {sgn}(\langle x_0, e_j \rangle )$ for $j \in \Delta .$ It is easy to see that there is a constant $M < \infty$ such that $\Vert y_n\Vert _{l^{\infty }} \le M$ for all large n. Now we can define $\rho = U^* P_{\Omega } y_n.$ Then $\rho = \rho _n + P^{\perp }_nU^*P_{\Omega }y_n$, and thus $\rho$ satisfies the requirements (i), (ii) and (iii) for large n. $\square$

Proof of Proposition 8.5

Let $\alpha = |\Delta |$ and also $\omega = \{\omega _j\}_{j=1}^{\alpha }$, where $\omega _j \in {\mathbb {C}}$. Now define

$$\begin{aligned} V_{\omega } = I_{\Delta ^c} \oplus S_{\omega }: P_{\Delta }^{\perp }{\mathcal {H}}\oplus P_{\Delta } {\mathcal {H}} \rightarrow P_{\Delta }^{\perp }{\mathcal {H}}\oplus P_{\Delta }{\mathcal {H}}, \end{aligned}$$

(11.2)

where $S_{\omega } = \mathrm {diag}\left( \{\omega _j\}_{j=1}^{\alpha }\right)$ on $P_{\Delta }{\mathcal {H}}$ and $I_{\Delta ^c}$ is the identity on $P_{\Delta }^{\perp }{\mathcal {H}}$. Define $U(\omega ) = UV_{\omega }.$ Note that to prove the proposition it suffices to show that $V_{\omega }x$ is the unique minimizer of $\inf \{\Vert \eta \Vert _{l_1}: P_{\Omega }U\eta = P_{\Omega }U(\omega )x\}$ for all $\omega$, where

$$\begin{aligned} \omega \in \Lambda = \left\{ \left( \mathrm{{e}}^{i\theta _1}, \ldots , \mathrm{{e}}^{i\theta _{\alpha }}\right) \in {\mathbb {C}}^{\alpha }:\theta _j \in [0,2\pi ), 1 \le j \le \alpha \right\} . \end{aligned}$$

(11.3)

Indeed, if the assertion is true, Proposition 11.3 yields that every real $\tilde{x} \in l^2({\mathbb {N}})$ with $\mathrm {supp}(\tilde{x}) = \Delta$ is the unique minimizer of $\inf \{\Vert \eta \Vert _{l_1}: P_{\Omega }U\eta = P_{\Omega }U\tilde{x}\}$. Thus, for any $y \in l^2({\mathbb {N}})$ such that $\mathrm {supp}(y) = \Delta$ choose $\omega \in \Lambda$ and a real $\tilde{x} \in l^2({\mathbb {N}})$ such that $y = V_{\omega }\tilde{x}$. Then, by using the assertion above for $\tilde{x}$ we have proved the proposition.

To prove the assertion, note that if $\omega \in \Lambda ,$ then $V_{\omega }$ is clearly unitary and also an isometry on $l^1({\mathbb {N}}).$ Thus, it is easy to see that $V_{\omega }\zeta$ is a minimizer of $\inf \{\Vert \eta \Vert _{l_1}: P_{\Omega }U\eta = P_{\Omega }U(\omega )x\}$ if and only if $\zeta$ is a minimizer of $\inf \{\Vert \eta \Vert _{l_1}: P_{\Omega }U(\omega )\eta = P_{\Omega }U(\omega )x\}.$ We will therefore consider the latter minimization problem and show that x is the unique minimizer of that problem for all $\omega \in \Lambda .$ To do that, it suffices, by Proposition 11.3 and the fact that $U(\omega )$ is unitary to show that there exists a vector $\rho \in l^2({\mathbb {N}})$ such that

$$\begin{aligned} P_{\Omega ^c}U(\omega )\rho = 0, \quad P_{\Delta }\rho = \mathrm {sgn}(x), \quad \Vert P_{\Delta ^c}\rho \Vert _{l^{\infty }} < 1. \end{aligned}$$

(11.4)

Now, for $\epsilon > 0$ (we will specify the value of $\epsilon$ later), define the function $\varphi :\cup _{a\in \Lambda }{\mathcal {B}}(a, \epsilon ) \rightarrow \mathbb {R}_+,$ where ${\mathcal {B}}(a, \epsilon )$ denotes the $\epsilon$-ball around a, in the following way. Let

$$\begin{aligned} W = I_{\Delta } \oplus P_{\Omega ^c}UP_{\Delta ^c}: P_{\Delta }{\mathcal {H}} \oplus P_{\Delta ^c}{\mathcal {H}} \rightarrow P_{\Delta }{\mathcal {H}} \oplus P_{\Omega ^c}{\mathcal {H}}, \end{aligned}$$

and define

$$\begin{aligned} \varphi (\omega ) = \inf \{\Vert P_{\Delta ^c}\rho \Vert _{l^{\infty }}: W\rho = \iota ^*_{\Delta }\mathrm {sgn}(x) \oplus -P_{\Omega ^c}U(\omega )P_{\Delta }\iota ^*_{\Delta }\mathrm {sgn}(x)\}, \end{aligned}$$

where $\iota _{\Delta } : P_{\Delta }l^2({\mathbb {N}}) \rightarrow l^2({\mathbb {N}})$ is the inclusion operator. Then (11.4) is satisfied if and only if $\varphi (\omega ) < 1.$ Thus, to show (11.4) we must show that $\varphi (\omega ) < 1$ for all $\omega \in \Lambda .$

Suppose for the moment that $\epsilon$ is chosen such that $\varphi$ is defined on its domain. We will show that $\varphi$ is continuous. For this, it suffices to show that $\varphi$ is continuous on ${\mathcal {B}}(a,\epsilon )$ for $a \in \Lambda$. Note that, by the fact that ${\mathcal {B}}(a,\epsilon )$ is open it is enough to show that $\varphi$ is convex. To see that $\varphi$ is convex, let $\omega _1, \omega _2 \in {\mathcal {B}}(a, \epsilon )$ and $t \in (0,1).$ Also let $\xi , \eta \in l^2({\mathbb {N}})$ be such that

$$\begin{aligned} W\xi= & {} \iota ^*_{\Delta }\mathrm {sgn}(x) \oplus -P_{\Omega ^c}U(\omega _1)P_{\Delta }\iota ^*_{\Delta }\mathrm {sgn}(x), \\ W\eta= & {} \iota ^*_{\Delta }\mathrm {sgn}(x) \oplus -P_{\Omega ^c}U(\omega _2)P_{\Delta }\iota ^*_{\Delta }\mathrm {sgn}(x). \end{aligned}$$

Note that the existence of such vectors is guaranteed by the assumption that $\varphi$ is defined on its domain. Now

$$\begin{aligned} \varphi (t\omega _1 + (1-t)\omega _2) \le \Vert P_{\Delta ^c}(t\xi + (1-t)\eta )\Vert _{l^{\infty }} \le t\Vert P_{\Delta ^c}\xi \Vert _{l^{\infty }} + (1-t)\Vert P_{\Delta ^c}\eta \Vert _{l^{\infty }}. \end{aligned}$$

Thus, taking infimum on the right-hand side yields $\varphi (t\omega _1 + (1-t)\omega _2) \le t \varphi (\omega _1) + (1-t)\varphi (\omega _2),$ as required. Returning to the question of the domain of $\varphi ,$ note that if $(P_{\Omega }UP_{\Delta })^* P_{\Omega }UP_{\Delta }|_{P_{\Delta }l^2({\mathbb {N}})}$ is invertible, then $(P_{\Omega }U(\omega )P_{\Delta })^* P_{\Omega }U(\omega )P_{\Delta }|_{P_{\Delta }l^2({\mathbb {N}})}$ is invertible if $\Vert U(\tilde{\omega }) - U(\omega )\Vert$ is small and $\tilde{\omega } \in \Lambda$. Letting

$$\begin{aligned} \rho = U(\omega )^*P_{\Omega }U(\omega )P_{\Delta } ((P_{\Omega }U(\omega )P_{\Delta })^* P_{\Omega }U(\omega ) P_{\Delta }|_{P_{\Delta }l^2({\mathbb {N}})})^{-1}\mathrm {sgn}(x) \end{aligned}$$

we get

$$\begin{aligned} P_{\Omega ^c}UP_{\Delta ^c}\rho = - P_{\Omega ^c}U(\omega )P_{\Delta }\mathrm {sgn}(x). \end{aligned}$$

Thus, $\varphi$ is defined on its domain for small $\epsilon .$

Let $\Gamma$ denote the subset of all $\omega \in \Lambda$ such that x is the unique minimizer of $\inf \{\Vert \eta \Vert _{l_1}: P_{\Omega }U(\omega )\eta = P_{\Omega }U(\omega )x\}.$ Note that $\Gamma$ is closed. Indeed, if $\omega \in \overline{\Gamma }$ and $\{\omega _n\} \subset \Gamma$ is a sequence such that $\omega _n \rightarrow \omega$ then $\omega \in \Gamma .$ To see that, observe that since $\{U,\Omega , \Delta \}$ is weakly f stable, it follows that for $\xi \in l^2({\mathbb {N}})$ satisfying

$$\begin{aligned} \Vert \xi \Vert _{l^1} = \inf \{\Vert \eta \Vert _{l_1}: P_{\Omega }U(\omega )\eta = P_{\Omega }U(\omega )x\} \end{aligned}$$

we have

$$\begin{aligned} \Vert \xi - x\Vert _{l^1} \le f(\Vert \omega - \omega _n\Vert _{l^{\infty }}), \quad \forall \, n \in {\mathbb {N}}. \end{aligned}$$

Thus, $\xi = x$ and hence $\omega \in \Gamma .$

Note also that $\Gamma$ is open. Indeed, for if $\tilde{\omega } \in \Gamma$ then there exist $\rho \in {\mathcal {H}}$ such that $\rho$ satisfies (11.4) (with $\omega$ replaced by $\tilde{\omega }$) e.g., $\varphi (\tilde{\omega }) < 1.$ But, by continuity of $\varphi$ it follows that $\varphi$ is strictly less than one on a neighborhood of $\tilde{\omega }.$ Since $(P_{\Omega }UP_{\Delta })^* P_{\Omega }UP_{\Delta }|_{P_{\Delta }l^2({\mathbb {N}})}$ is invertible, it is easy to see that $P_{\Omega }U(\omega )P_{\Delta })^* P_{\Omega }U(\omega )P_{\Delta }|_{P_{\Delta }l^2({\mathbb {N}})}$ is invertible, for all $\omega \in \Lambda$. Thus it follows by Proposition 11.3 that (11.4) is satisfied for all $\omega \in \Lambda$ in a neighborhood of $\tilde{\omega }$ and hence $\Gamma$ is open.

The fact that $\Gamma$ is open and closed yields that either $\Gamma = \emptyset$ or $\Gamma = \Lambda$. The fact that $\{1,\ldots ,1\} \in \Gamma$ by assumption yields the theorem. $\square$

Proof of Proposition 8.6

Let $V_{\omega }$ and $\Lambda$ be defined as in (11.2) and (11.3), respectively. Suppose that $y \in l^2({\mathbb {N}})$ is real with $\mathrm {supp}(y) = \Delta .$ Then, by assumption, $V_{\omega }y$ is the unique minimizer of $\inf \{\Vert \eta \Vert _{l_1}: P_{\Omega }U\eta = P_{\Omega }UV_{\omega }y\},$ when $V_{\omega }$ is real. Thus, by Proposition 11.3 it follows that there exists a $\rho _{\omega } \in l^2({\mathbb {N}})$ such that

$$\begin{aligned} P_{\Omega ^c}U\rho _{\omega } = 0, \quad P_{\Delta }\rho _{\omega } = \mathrm {sgn}(V_{\omega }y), \quad \Vert P_{\Delta ^c}\rho _{\omega }\Vert _{l^{\infty }} < 1. \end{aligned}$$

(11.5)

Let $\beta = \max _{\omega \in \Lambda }\{\Vert P_{\Delta ^c}\rho _{\omega }\Vert _{l^{\infty }}, \omega \text { is real}\}.$ It is clear that $\beta < 1$. Thus, for every $y \in {\mathcal {H}}$ with $\mathrm {supp}(y) = \Delta$ there exists $\rho _{\omega } \in l^2({\mathbb {N}})$ satisfying (11.5) where $\Vert P_{\Delta ^c}\rho _{\omega }\Vert _{l^{\infty }} \le \beta$. It is now easy to show that (see the proof of Lemma 2.1 in [21]) there exists a constant $C >0$ (depending on $\beta$) such that, if $\xi \in l^2({\mathbb {N}})$, $\mathrm {supp}(\xi ) = \Delta$, is the unique minimizer of $\inf \{\Vert \eta \Vert _{l_1}: P_{\Omega }U\eta = P_{\Omega }U\xi \}$, $\zeta \in l^2({\mathbb {N}})$ and x is a minimizer of $\inf \{\Vert \eta \Vert _{l_1}: P_{\Omega }U\eta = P_{\Omega }U\zeta \}$ then $\Vert P_{\Delta ^c}x\Vert _{l_1} \le C\Vert \xi -\zeta \Vert _{l_1}.$ Thus, since

$$\begin{aligned} P_{\Omega }UP_{\Delta }(x-\xi ) = P_{\Omega }U(\zeta -\xi ) - P_{\Omega }UP_{\Delta ^c}x, \end{aligned}$$

and $(P_{\Omega }UP_{\Delta })^* P_{\Omega }UP_{\Delta } |_{P_{\Delta }{\mathcal {H}}}$ is invertible, the proposition follows. $\square$

Proof of Proposition 9.1

Without loss of generality, we may assume that $\Vert \eta \Vert = 1.$ Let $\{\delta _j\}_{j=1}^N$ be random Bernoulli variables with ${\mathbb {P}}(\delta _j = 1) = q.$ We will split the proof into two steps, where we will prove the finite-dimensional part of the proposition in Step I, and then tweak these ideas to fit the infinite-dimensional part of the proposition in Step II.

Step I: We start by noting that, since U is an isometry, we have

$$\begin{aligned} q^{-1}P_MP_{\Delta }^{\perp }E_{\Omega }P_{\Delta }\eta= & {} q^{-1}\sum _{j=1}^NP_MP_{\Delta }^{\perp }U^* \delta _j (e_j \otimes e_j) UP_{\Delta }\eta \nonumber \\= & {} q^{-1}\sum _{j=1}^NP_MP_{\Delta }^{\perp }U^* (\delta _j -q)(e_j \otimes e_j) UP_{\Delta }\eta \nonumber \\&+ P_MP_{\Delta }^{\perp }U^* P_N^{\perp } UP_{\Delta }\eta . \end{aligned}$$

(11.6)

Our goal is to eventually use Bernstein’s inequality, and the following is therefore a setup to do so. For $1 \le j \le N$, define the random variables

$$\begin{aligned} Y_j= & {} q^{-1}P_MP_{\Delta }^{\perp } U^* (\delta _j -q)(e_j \otimes e_j) UP_{\Delta }\eta , \nonumber \\ X^i_j= & {} \langle q^{-1}U^* (\delta _j -q)(e_j \otimes e_j) UP_{\Delta }\eta ,e_i \rangle , \quad i \in \Delta ^c \cap \{1,\ldots ,M\}. \end{aligned}$$

Thus, for $s>0$ it follows from (11.6) that

$$\begin{aligned}&{\mathbb {P}} \left( \left\| q^{-1}P_MP_{\Delta }^{\perp }E_{\Omega }P_{\Delta }\eta \right\| _{l^{\infty }} > s \right) = {\mathbb {P}}\left( \left\| \sum _{j=1}^N Y_j + P_MP_{\Delta }^{\perp }U^* P_N^{\perp }UP_{\Delta }\eta \right\| _{l^{\infty }} > s \right) \\&\quad \le \sum _{i \in \Delta ^c \cap \{1,\ldots ,M\}} {\mathbb {P}}\left( \left| \sum _{j=1}^N X_j^i + \langle P_MP_{\Delta }^{\perp }U^* P_N^{\perp } UP_{\Delta }\eta , e_i \rangle \right| > s \right) \\&\quad \le \sum _{i \in \Delta ^c \cap \{1,\ldots ,M\}} {\mathbb {P}}\left( \left| \sum _{j=1}^N X_j^i \right| > s - \Vert P_MP_{\Delta }^{\perp }U^* P_N UP_{\Delta }\Vert _{\mathrm {mr}}\right) \!, \end{aligned}$$

where we have used the fact that U is an isometry and hence

$$\begin{aligned} P_MP_{\Delta }^{\perp }U^* P_N UP_{\Delta } = -P_MP_{\Delta }^{\perp }U^* P_N^{\perp } UP_{\Delta }. \end{aligned}$$

Thus, by choosing $s = t + \Vert P_MP_{\Delta }^{\perp }U^* P_N UP_{\Delta }\Vert _{\mathrm {mr}}$ we deduce that

$$\begin{aligned}&{\mathbb {P}}\left( \left\| q^{-1}P_MP_{\Delta }^{\perp }E_{\Omega }P_{\Delta }\eta \right\| _{l^{\infty }} > t + \Vert P_MP_{\Delta }^{\perp }U^* P_N UP_{\Delta }\Vert _{\mathrm {mr}} \right) \nonumber \\&\quad \le \sum _{i \in \Delta ^c \cap \{1,\ldots ,M\}} {\mathbb {P}}\left( \left| \sum _{j=1}^N X_j^i \right| > t \right) \!. \end{aligned}$$

(11.7)

To estimate the right-hand side of (11.7), we shall use Bernstein’s inequality, and in order to do that, we need a couple of observations. First note that

$$\begin{aligned} {\mathbb {E}}\left( |X_j^i|^2\right)= & {} q^{-2} {\mathbb {E}}\left( |\langle UP_{\Delta } \eta , (\delta _j -q)(e_j \otimes e_j)Ue_i \rangle |^2\right) \\= & {} q^{-2} {\mathbb {E}}\left( (\delta _j-q)^2\right) |\langle UP_{\Delta }\eta ,e_j \rangle \langle Ue_i,e_j \rangle |^2 \\= & {} q^{-1}(1-q) |\langle UP_{\Delta }\eta ,e_j \rangle \langle Ue_i,e_j \rangle |^2, \quad i \in \Delta ^c \cap \{1,\ldots ,M\}. \end{aligned}$$

Thus

$$\begin{aligned} \sum _{j=1}^N {\mathbb {E}}\left( |X_j^i|^2\right) \le q^{-1}(1-q)\Vert \eta \Vert ^2\upsilon ^2(U) = q^{-1}(1-q)\upsilon ^2(U), \quad i \in \Delta ^c \cap \{1,\ldots ,M\}. \end{aligned}$$

(11.8)

Also, observe that

$$\begin{aligned} |X_j^i| = q^{-1}|(\delta _j -q)||\langle \eta , P_{\Delta }U^*(e_j \otimes e_j)Ue_i \rangle | \le \max \{(1-q)/q,1\}\upsilon ^2(U)\sqrt{|\Delta |}, \end{aligned}$$

(11.9)

for $1 \le j \le N$ and $i \in \Delta ^c \cap \{1,\ldots ,M\}.$ Now applying Bernstein’s inequality to $\mathrm {Re}(X_1^i), \ldots , \mathrm {Re}(X_N^i)$ and $\mathrm {Im}(X_1^i), \ldots , \mathrm {Im}(X_N^i)$, we get that

$$\begin{aligned}&{\mathbb {P}}\left( \left| \sum _{j=1}^N X_j^i\right| > t \right) \nonumber \\&\quad \le 4\exp \left( -\frac{t^2/4}{q^{-1}(1-q)\upsilon ^2(U) + \max \{q^{-1}(1-q),1\}\upsilon ^2(U)\sqrt{|\Delta |}t/3\sqrt{2}}\right) ,\nonumber \\ \end{aligned}$$

(11.10)

for all $i \in \Delta ^c \cap \{1,\ldots ,M\}.$ Thus, by invoking (11.10) and (11.7) it follows that

$$\begin{aligned} {\mathbb {P}} \left( \left\| q^{-1}P_MP_{\Delta }^{\perp }E_{\Omega }P_{\Delta }\eta \right\| _{l^{\infty }} > t + \Vert P_MP_{\Delta }^{\perp }U^* P_N UP_{\Delta }\Vert _{\mathrm {mr}} \right) \le \gamma \end{aligned}$$

when

$$\begin{aligned} q \ge \left( \frac{4}{t^2} + \frac{2\sqrt{2}}{3t}\sqrt{|\Delta |}\right) \log \left( \frac{4}{ \gamma }|\Delta ^c \cap \{1,\ldots ,M\}|\right) \upsilon ^2(U). \end{aligned}$$

The first part of the proposition now follows. The fact that the left-hand side of (9.3) is zero when $q=1$ is clear from (11.8) and (11.9).

Step II: To prove the second part of the proposition, we will use the same ideas; however, we are now faced with the problem that $P_{\Delta }^{\perp }E_{\Omega }P_{\Delta }\eta$ (contrary to $P_MP_{\Delta }^{\perp }E_{\Omega }P_{\Delta }\eta$) has infinitely many components. This is an obstacle since the proof of the bound on $P_MP_{\Delta }^{\perp }E_{\Omega }P_{\Delta }\eta$ was based on bounding the probability of the deviation of every component of $P_MP_{\Delta }^{\perp }E_{\Omega }P_{\Delta }\eta$, and thus, if there are infinitely many components to take care of, the task would be impossible. To overcome this obstacle, we proceed as follows. Note that, just as argued in the previous case in Step I, we have that

$$\begin{aligned} q^{-1}P_{\Delta }^{\perp }E_{\Omega }P_{\Delta }\eta= & {} \sum _{j=1}^N \widetilde{Y}_j + P_{\Delta }^{\perp }U^* P_N^{\perp }UP_{\Delta }\eta , \nonumber \\ \widetilde{Y}_j= & {} q^{-1}P_{\Delta }^{\perp } U^* (\delta _j -q)(e_j \otimes e_j) UP_{\Delta }\eta . \end{aligned}$$

(11.11)

Define (as we did above) the random variables

$$\begin{aligned} X^i_j = \langle q^{-1}U^* (\delta _j -q)(e_j \otimes e_j) UP_{\Delta }\eta ,e_i \rangle , \quad i \in \Delta ^c. \end{aligned}$$

Note that we now have infinitely many $X^i_j$’s. However, suppose for a moment that for every $t >0$ there exists a nonempty set $\Lambda _t \subset {\mathbb {N}}$ such that

$$\begin{aligned} {\mathbb {P}}\left( \sup _{i \in \Lambda _t}\left| \sum _{j=1}^N X^i_j\right| > t \, \right) = 0 \quad \left| \Delta ^c \setminus \Lambda _t\right| < \infty . \end{aligned}$$

(11.12)

Then, if that was the case, we would immediately get (by arguing as in Step I and using (11.11) and the assumption that $\Vert \eta \Vert = 1$) that

$$\begin{aligned}&{\mathbb {P}}\left( \left\| q^{-1}P_{\Delta }^{\perp }E_{\Omega }P_{\Delta }\eta \right\| _{l^{\infty }} > t + \Vert P_{\Delta }^{\perp }U^* P_N UP_{\Delta }\Vert _{\mathrm {mr}} \right) \nonumber \\&\quad = {\mathbb {P}}\left( \left\| \sum _{j=1}^N \widetilde{Y}_j + P_{\Delta }^{\perp }U^* P_N^{\perp }UP_{\Delta }\eta \right\| _{l^{\infty }} > t + \Vert P_{\Delta }^{\perp }U^* P_N UP_{\Delta }\Vert _{\mathrm {mr}} \right) \nonumber \\&\quad \le \sum _{i \in \left| \Delta ^c \setminus \Lambda _t\right| } {\mathbb {P}}\left( \left| \sum _{j=1}^N X_j^i \right| > t \right) \!, \end{aligned}$$

Thus, we could use the analysis provided above, via (11.10), and deduce that

$$\begin{aligned} {\mathbb {P}}\left( \left\| q^{-1}P_{\Delta }^{\perp }E_{\Omega }P_{\Delta } \eta \right\| _{l^{\infty }} > t + \Vert P_{\Delta }^{\perp }U^* P_N UP_{\Delta }\Vert _{\mathrm {mr}} \right) \le \gamma \end{aligned}$$

when

$$\begin{aligned} q \ge \left( \frac{4}{t^2} + \frac{2\sqrt{2}}{3t}\sqrt{|\Delta |}\right) \log \left( \frac{4}{ \gamma } \left| \Delta ^c \setminus \Lambda _t\right| \right) \upsilon ^2(U). \end{aligned}$$

(11.13)

Hence, if we could show the existence of $\Lambda _t$ and provide a bound on $\left| \Delta ^c \setminus \Lambda _t\right|$, we could appeal to (11.11) and (11.13) and complete the proof. To do that, define

$$\begin{aligned} \Lambda _t = \left\{ i \notin \Delta : {\mathbb {P}}\left( \left\| \sum _{j=1}^NP_{\Delta } U^*\delta _j(e_j \otimes e_j)Ue_i\right\| \le tq \right) = 1\right\} \!. \end{aligned}$$

Note that $(e_j \otimes e_j)Ue_i \rightarrow 0$ as $i \rightarrow \infty$ for all $j \le N$. Thus, $\Lambda _t \ne \emptyset .$ Moreover, we also immediately deduce that $\left| \Delta ^c \setminus \Lambda _t\right| < \infty .$ Note also that (11.12) follows by the fact that $X^i_j = \langle \eta ,q^{-1} P_{\Delta } U^*\delta _j(e_j \otimes e_j)Ue_i\rangle$ and the Cauchy–Schwarz inequality. With the existence of $\Lambda _t$ established, we now continue with the task of estimating $\left| \Delta ^c \setminus \Lambda _t\right| .$ Note that to estimate $\left| \Delta ^c \setminus \Lambda _t\right|$ we need information about the location of $\Delta$ which is not assumed. We only assume the knowledge of some $M \in {\mathbb {N}}$ such that $P_M \ge P_{\Delta }$. Thus (although an estimate of $\left| \Delta ^c \setminus \Lambda _t\right|$ would be sharper than what we will eventually obtain), we define

$$\begin{aligned} \tilde{\Lambda }_{q}(|\Delta |,M,t) = \left\{ i \in {\mathbb {N}}: \underset{\Gamma _2 \subset \{1,\ldots , N \}}{\underset{\Gamma _1 \subset \{1,\ldots , M \}, |\Gamma _1| = |\Delta |}{\max }} \left\| P_{\Gamma _1} U^*P_{\Gamma _2}Ue_i\right\| \le tq \right\} \!. \end{aligned}$$

Note that it is straightforward to show that $\tilde{\Lambda }_{q}(|\Delta |,M,t) \subset \Lambda _t.$ Also, $\tilde{\Lambda }_{q}(|\Delta |,M,t)$ depends only on known quantities. Observe that, clearly, for any $\Gamma _1 \subset \{1,\ldots , M \}$ and $\Gamma _2 \subset \{1,\ldots , N \}$ then $\left\| P_{\Gamma _1} U^*P_{\Gamma _2}Ue_i\right\| \rightarrow \infty$ as $i \rightarrow \infty .$ Thus, $|\Delta ^c \setminus \Lambda _{q}(|\Delta |,M,t)| < \infty$ and since $\Lambda _{q}(|\Delta |,M,t) \subset \Lambda _t$ it follows that

$$\begin{aligned} \left| \Delta ^c \setminus \Lambda _{q}(\Delta ,t)\right| \le \left| \left\{ i \in {\mathbb {N}}: \underset{\Gamma _2 \subset \{1,\ldots , N \}}{\underset{\Gamma _1 \subset \{1,\ldots , M \}, |\Gamma _1| = |\Delta |}{\max }} \left\| P_{\Gamma _1} U^*P_{\Gamma _2}Ue_i\right\| > tq \right\} \right| . \end{aligned}$$

This gives the second part of the proposition. The fact that the left-hand side of (9.4) is zero when $q=1$ is clear from (11.8) and (11.9). $\square$

Proof of Proposition 9.2

Without loss of generality, we may assume that $\Vert \eta \Vert = 1.$ Let $\{\delta _j\}_{j=1}^N$ be random Bernoulli variables with ${\mathbb {P}}(\delta _j = 1) = q.$ Also, for $k \in {\mathbb {N}},$ let $\xi _k = (UP_{\Delta })^*e_k.$ Observe that, since U is an isometry,

$$\begin{aligned} q^{-1}(P_{\Omega }UP_{\Delta })^* P_{\Omega }UP_{\Delta } = \sum _{k=1}^{N} q^{-1}\delta _k \xi _k\otimes \bar{\xi }_k, \quad P_{\Delta } = \sum _{k=1}^{\infty } \xi _k\otimes \bar{\xi }_k, \end{aligned}$$

(11.14)

and

$$\begin{aligned}&\left\| \left( \frac{1}{q}(P_{\Omega }UP_{\Delta })^* P_{\Omega }UP_{\Delta } -P_{\Delta }\right) \eta \right\| \le \left\| \left( \sum _{k=1}^{N} (q^{-1}\delta _k -1)\xi _k\otimes \bar{\xi }_k\right) \eta \right\| \nonumber \\&\quad + \Vert (P_{\Delta }U^* P_NUP_{\Delta } -P_{\Delta })\eta \Vert , \end{aligned}$$

(11.15)

where the infinite series in (11.14) converges in operator norm. To get the desired result, we first focus on obtaining bounds on $\Vert ( \sum _{k=1}^{N} (q^{-1}\delta _k -1)\xi _k\otimes \bar{\xi }_k)\eta \Vert$. The goal is to use Talagrand’s formula, and what follows is a setup for that. In particular, let $\zeta \in {\mathcal {H}}$ be a unit vector and denote the mapping ${\mathcal {H}} \ni \xi \mapsto \mathrm {Re}(\langle \xi , \zeta \rangle )$ by $\hat{\zeta }.$ Let ${\mathcal {F}}$ be a countable collection of unit vectors such that for any $\xi \in {\mathcal {H}}$ we have $\Vert \xi \Vert = \sup _{\zeta \in {\mathcal {F}}}\hat{\zeta }(\xi ),$ and now define

$$\begin{aligned} Z = \Vert X\Vert , \quad X = \sum _{k=1}^{N}Z_k, \quad Z_k = ((q^{-1}\delta _k -1)\xi _k\otimes \bar{\xi }_k)\eta . \end{aligned}$$

Note that the following is clear (and note how this immediately gives us the setup for Talagrand’s Theorem)

$$\begin{aligned} Z = \left\| \left( \sum _{k=1}^{N} (q^{-1}\delta _k -1)\xi _k\otimes \bar{\xi }_k\right) \eta \right\| = \sup _{\zeta \in {\mathcal {F}}}\hat{\zeta }\left( \sum _{k=1}^N Z_k\right) = \sup _{\zeta \in {\mathcal {F}}}\sum _{k=1}^N \hat{\zeta }(Z_k). \end{aligned}$$

To use Talagrand’s theorem, we must estimate the following quantities:

$$\begin{aligned} V = \sup _{\zeta \in {\mathcal {F}}} {\mathbb {E}}\left( \sum _{k=1}^N \hat{\zeta }(Z_{k})^2\right) , \quad S = \sup _{\zeta \in {\mathcal {F}}}\Vert \hat{\zeta }\Vert _{\infty }, \quad R = {\mathbb {E}}\left( \left\| \sum _{k=1}^N Z_{k}\right\| \right) . \end{aligned}$$

For V we have the following estimate:

$$\begin{aligned} \sup _{\zeta \in {\mathcal {F}}} {\mathbb {E}}\left( \sum _{k=1}^N \hat{\zeta }(Z_k)^2\right)\le & {} \sup _{\zeta \in {\mathcal {F}}} {\mathbb {E}}\left( \sum _{k \le N}\left( q^{-1}\delta _k - 1\right) ^2 | \langle \xi _k, \zeta \rangle |^2 |\langle \xi _k, \eta \rangle |^2 \right) \nonumber \\\le & {} q^{-1}(1-q) \sum _{k \le N}\Vert \xi _k\Vert ^2|\langle e_k, UP_{\Delta }\eta \rangle |^2\nonumber \\\le & {} q^{-1}(1-q)\upsilon ^2(U)|\Delta |, \end{aligned}$$

where we have used the fact that U is an isometry in the step going from the second to the third inequality. The S term can be estimated as follows. Note that

$$\begin{aligned} \hat{\zeta }(Z_k) = |\left( q^{-1}\delta _k - 1\right) |\langle \xi _k,\zeta \rangle ||\langle \xi _k,\eta \rangle | \le \max \{q^{-1}-1,1\} \upsilon ^2(U)|\Delta |, \quad k \le N, \end{aligned}$$

(11.16)

thus

$$\begin{aligned} S \le \max \{q^{-1}-1,1\} \upsilon ^2(U)|\Delta |. \end{aligned}$$

(11.17)

Finally, we can estimate R as follows:

$$\begin{aligned} {\mathbb {E}}\left( \left\| \sum _{k=1}^NZ_{k}\right\| ^2\right)= & {} \sum _{k=1}^N {\mathbb {E}}(\Vert Z_k\Vert ^2) + \sum _{k \ne j} {\mathbb {E}}(\langle Z_k,Z_j \rangle ) \nonumber \\\le & {} q^{-1}(1-q) \sum _{k \le N}\Vert \xi _k\Vert ^2|\langle e_k, UP_{\Delta }\eta \rangle |^2\nonumber \\\le & {} q^{-1}(1-q)\upsilon ^2(U)|\Delta |, \end{aligned}$$

again using the fact that U is an isometry. Therefore,

$$\begin{aligned} {\mathbb {E}}\left( \left\| \sum _{k \le N} Z_{k}\right\| \right) \le \sqrt{{\mathbb {E}}\left( \left\| \sum _{k \le N} Z_{k}\right\| ^2\right) } \le \sqrt{q^{-1}(1-q)\upsilon ^2(U)|\Delta |}. \end{aligned}$$

(11.18)

With the estimates on V,  S and R now established we may appeal to Theorem 11.2 and deduce that there is a constant $K >0$ such that, for $\theta > 0$,

$$\begin{aligned}&{\mathbb {P}}\left( \left\| \left( \sum _{k=1}^{N} (q^{-1}\delta _k -1)\xi _k\otimes \bar{\xi }_k\right) \eta \right\| \ge \theta + \sqrt{q^{-1}(1-q)\upsilon ^2(U)|\Delta |}\right) \nonumber \\&\quad \le 3\exp \left( -\frac{\theta }{K}\left( \max \{q^{-1}-1,1\} \upsilon ^2(U)|\Delta | \right) ^{-1} \log \left( 1 + \frac{\theta }{2} \right) \right) . \end{aligned}$$

(11.19)

provided q is chosen such that the right-hand side of (11.18) is bounded by 1 (this is guaranteed by the assumptions of the proposition). But by (11.15) it follows that for any $r > 0$, we have

$$\begin{aligned}&{\mathbb {P}}\left( \left\| \left( \frac{1}{q}(P_{\Omega }UP_{\Delta })^* P_{\Omega }UP_{\Delta } -P_{\Delta }\right) \eta \right\| \ge r \right) \nonumber \\&\quad \le {\mathbb {P}}\left( \left\| \left( \sum _{k=1}^{N} (q^{-1}\delta _k -1)\xi _k\otimes \bar{\xi }_k\right) \eta \right\| \ge r - \Vert (P_{\Delta }U^* P_NUP_{\Delta }-P_{\Delta })\Vert \right) .\qquad \qquad \end{aligned}$$

(11.20)

Therefore, by appealing to (11.20) and (11.19) we obtain that

$$\begin{aligned}&{\mathbb {P}}\left( \left\| \left( \frac{1}{q}(P_{\Omega }UP_{\Delta })^* P_{\Omega }UP_{\Delta } -P_{\Delta }\right) \eta \right\| \ge \theta + \sqrt{q^{-1}(1-q)\upsilon ^2(U)|\Delta |} + \Xi \right) \nonumber \\&\quad \le 3\exp \left( -\frac{\theta }{K}\left( \max \{q^{-1}-1,1\} \upsilon ^2(U)|\Delta | \right) ^{-1} \log \left( 1 + \frac{\theta }{2} \right) \right) , \end{aligned}$$

where $\Xi = \Vert (P_{\Delta }U^* P_NUP_{\Delta } -P_{\Delta })\Vert .$ Choosing $\theta = t/2$ yields the proposition. $\square$

Proof of Theorem 9.3

The proof is quite similar to the proof of Proposition 9.2. Let $\{\delta _j\}_{j=1}^N$ be random Bernoulli variables with ${\mathbb {P}}(\delta _j = 1) = \theta .$ Note that we may argue as in (11.14) and observe that

$$\begin{aligned}&\left\| \theta ^{-1}(P_{\Omega }UP_{\Delta })^* P_{\Omega }UP_{\Delta } -P_{\Delta }\right\| \le \left\| \sum _{k=1}^{N} (\theta ^{-1}\delta _k -1)\xi _k\otimes \bar{\xi }_k\right\| \nonumber \\&\quad + \left\| (P_{\Delta }U^* P_NUP_{\Delta } -P_{\Delta })\right\| , \end{aligned}$$

(11.21)

where $\xi _k = (UP_{\Delta })^*e_k$. To get the desired result, we first focus on getting bounds on $\Vert \sum _{k=1}^{N} (\theta ^{-1}\delta _k -1)\xi _k\otimes \bar{\xi }_k\Vert .$ As in the proof of Proposition 9.2, the goal is to use Talagrand’s thereom and the first step to do so is to estimate ${\mathbb {E}}\left( \Vert Z\Vert \right) ,$ where $Z = \sum _{k=1}^{N} (\theta ^{-1}\delta _k -1)\xi _k\otimes \bar{\xi }_k.$

Claim: We claim that

$$\begin{aligned} {\mathbb {E}}\left( \Vert Z\Vert \right) ^2 \le 48\max \{\log (|\Delta |),1\} \theta ^{-1}\upsilon ^2(U)|\Delta |, \end{aligned}$$

(11.22)

when

$$\begin{aligned} \theta \ge 18 \max \{\log (|\Delta |), 1\} \upsilon ^2(U)|\Delta |. \end{aligned}$$

To prove the claim, we simply rework the techniques used in [54]. This is now standard and has also been used in [19, 61]. We start by letting $\tilde{\delta }= \{\tilde{\delta }_k\}_{k=1}^N$ be independent copies of $\delta = \{\delta _k\}_{k=1}^N.$ Then

$$\begin{aligned} {\mathbb {E}}_{\delta }\left( \Vert Z\Vert \right)= & {} {\mathbb {E}}_{\delta }\left( \left\| Z - {\mathbb {E}}_{\tilde{\delta }}\left( \sum _{k=1}^N \left( \theta ^{-1}\tilde{\delta }_k-1\right) \xi _k\otimes \bar{\xi }_k \right) \right\| \right) \nonumber \\\le & {} {\mathbb {E}}_{\delta }\left( {\mathbb {E}}_{\tilde{\delta }}\left( \left\| Z - \sum _{k=1}^N \left( \theta ^{-1}\tilde{\delta }_k-1\right) \xi _k\otimes \bar{\xi }_k \right\| \right) \right) , \end{aligned}$$

(11.23)

by Jensen’s inequality. Let $\varepsilon = \{\varepsilon _j\}_{j=1}^N$ be a sequence of Bernoulli variables taking values $\pm 1$ with probability 1 / 2. Then, by (11.23), symmetry, Fubini’s Theorem and the triangle inequality, it follows that

$$\begin{aligned} {\mathbb {E}}_{\delta }\left( \Vert Z\Vert \right)\le & {} {\mathbb {E}}_{\varepsilon }\left( {\mathbb {E}}_{\delta } \left( {\mathbb {E}}_{\tilde{\delta }}\left( \left\| \sum _{k=1}^N \varepsilon _k \left( \theta ^{-1}\delta _k - \theta ^{-1}\tilde{\delta }_k\right) \xi _k\otimes \bar{\xi }_k \right\| \right) \right) \right) \nonumber \\\le & {} 2 {\mathbb {E}}_{\delta } \left( {\mathbb {E}}_{\varepsilon }\left( \left\| \sum _{k=1}^N \varepsilon _k \theta ^{-1}\delta _k \xi _k\otimes \bar{\xi }_k \right\| \right) \right) . \end{aligned}$$

(11.24)

Now, by Lemma 11.1 we get that

$$\begin{aligned}&{\mathbb {E}}_{\varepsilon }\left( \left\| \sum _{k=1}^N \varepsilon _k \theta ^{-1}\delta _k \xi _k\otimes \bar{\xi }_k \right\| \right) \nonumber \\&\quad \le 3\sqrt{\max \{2\log (|\Delta |),2\}\theta ^{-1}}\max _{1\le k \le N}\Vert \xi _k\Vert \sqrt{ \left\| \sum _{k=1}^N\theta ^{-1}\delta _k \xi _k\otimes \bar{\xi }_k\right\| }.\quad \end{aligned}$$

(11.25)

And hence, by using (11.24) and (11.25), it follows that

$$\begin{aligned} {\mathbb {E}}_{\delta }\left( \Vert Z\Vert \right) \le 3\sqrt{\max \{2\log (|\Delta |),2\}\theta ^{-1} \upsilon ^2(U)|\Delta |}\sqrt{{\mathbb {E}}_{\delta }\left( \left\| Z + \sum _{k=1}^N \xi _k\otimes \bar{\xi }_k\right\| \right) }. \end{aligned}$$

Thus, by using the easy calculus fact that if $r >0$, $c\le 1$ and $r \le c\sqrt{r+1}$ then $r \le c(1+\sqrt{5})/2,$ and the fact that U is an isometry (so that $\Vert \sum _{k=1}^N \xi _k\otimes \bar{\xi }_k\Vert \le 1$), it is easy to see that the claim follows.

To be able to use Talagrand’s formula, there are now some preparations that have to be done. First write

$$\begin{aligned} Z = \sum _{k=1}^{N}Z_k, \quad Z_k =\left( \theta ^{-1}\delta _k-1\right) \xi _k\otimes \bar{\xi }_k. \end{aligned}$$

Clearly, since Z is self-adjoint, we have that $\Vert Z\Vert = \sup _{\eta \in {\mathcal {F}}}|\langle Z\eta ,\eta \rangle |,$ where $\mathcal {G}$ is a countable set of unit vectors. For $\eta \in \mathcal {G},$ let the mappings ${\mathcal {B}}({\mathcal {H}}) \ni T \mapsto \langle T\eta ,\eta \rangle$ and ${\mathcal {B}}({\mathcal {H}}) \ni T \mapsto -\langle T\eta ,\eta \rangle$ be denoted by $\hat{\eta }_1$ and $\hat{\eta }_2$, respectively. Letting ${\mathcal {F}}$ denote the family of all these mappings we have that $\Vert Z\Vert = \sup _{\hat{\eta }_i \in {\mathcal {F}}} \sum _{k \le N} \hat{\eta }_i(Z_k)$, and the setup for Talagrand’s theorem is complete.

For $k = 1,\ldots ,N$ note that

$$\begin{aligned} |\hat{\eta }_i(Z_k)| = \left| \left( \theta ^{-1}\delta _k-1\right) \right| |\langle \left( \xi _k\otimes \bar{\xi }_k\right) \eta ,\eta \rangle | \le \theta ^{-1}\Vert \xi \Vert ^2. \end{aligned}$$

Thus, after restricting $\hat{\eta }_i$ to the ball of radius $\theta ^{-1}\max _{k\le N}\Vert \xi _k\Vert ^2$ it follows that

$$\begin{aligned} S = \sup _{\eta _i \in {\mathcal {F}}}\Vert \hat{\eta }_i\Vert _{\infty } \le \theta ^{-1}\max _{k\le N}\Vert \xi _k\Vert ^2 \le \theta ^{-1}\upsilon ^2(U)|\Delta |. \end{aligned}$$

(11.26)

Also, note that

$$\begin{aligned} V= & {} \sup _{\hat{\eta }_i\in {\mathcal {F}}} {\mathbb {E}} \left( \sum _{k \le N}\hat{\eta }(Z_k)^2\right) \le \sup _{\hat{\eta }\in {\mathcal {F}}} {\mathbb {E}}\left( \sum _{k \le N} \left( \theta ^{-1}\delta _k-1\right) ^2|\langle \xi _k,\eta \rangle |^4 \right) \nonumber \\\le & {} \max _{k\le N}\Vert \xi _k\Vert ^2\left( \theta ^{-1}-1\right) \sup _{\hat{\eta }\in {\mathcal {F}}} \sum _{k\le N}|\langle e_k,UP_{\Delta }\eta \rangle |^2 \nonumber \\\le & {} \left( \theta ^{-1}-1\right) \max _{k\le N}\Vert \xi _k\Vert ^2 \le \left( \theta ^{-1}-1\right) \upsilon ^2(U)|\Delta |, \end{aligned}$$

(11.27)

where the third inequality follows from the fact that U is an isometry. It follows by Talagrand’s inequality (Theorem 11.2), the earlier claim (and requiring that the right hand side of (11.22) is bounded by one, which is guarantied by the assumption of the theorem), the first part of the assumed (9.6), (11.26) and (11.27), that there is a constant $K >0$ such that for $t > 0$

$$\begin{aligned}&{\mathbb {P}}\left( \left\| \sum _{k=1}^{N} (\theta ^{-1}\delta _k -1)\xi _k\otimes \bar{\xi }_k\right\| \ge t + 48\log (|\Delta |) \theta ^{-1}\upsilon ^2(U)|\Delta |\right) \nonumber \\&\quad \le 3\exp \left( -\frac{t}{K}(\theta ^{-1}\upsilon ^2(U)|\Delta |)^{-1} \log \left( 1 + \frac{t}{2} \right) \right) . \end{aligned}$$

(11.28)

But by (11.21) we have

$$\begin{aligned}&{\mathbb {P}}\left( \left\| \frac{1}{\theta }(P_{\Omega }UP_{\Delta })^* P_{\Omega }UP_{\Delta } -P_{\Delta }\right\| \ge r \right) \nonumber \\&\quad \le {\mathbb {P}}\left( \left\| \sum _{k=1}^{N} (\theta ^{-1}\delta _k -1)\xi _k\otimes \bar{\xi }_k\right\| \ge r - \Vert (P_{\Delta }U^* P_NUP_{\Delta }-P_{\Delta })\Vert \right) .\qquad \qquad \end{aligned}$$

(11.29)

for any $r > 0$. Therefore, by appealing to (11.29) and (11.28) we obtain

$$\begin{aligned}&{\mathbb {P}}\left( \left\| \frac{1}{\theta }(P_{\Omega }UP_{\Delta })^* P_{\Omega }UP_{\Delta } -P_{\Delta }\right\| \ge t + 48\log (|\Delta |) \theta ^{-1}\upsilon ^2(U)|\Delta | + \Xi \right) \\&\quad \le 3\exp \left( -\frac{t}{K}(\theta ^{-1}\upsilon ^2(U)|\Delta |)^{-1} \log \left( 1 + \frac{t}{ 2} \right) \right) , \quad \Xi = \Vert (P_{\Delta }U^* P_NUP_{\Delta }. -P_{\Delta })\Vert . \end{aligned}$$

for $t > 0$. Choosing $t = \frac{1}{2\gamma }$ yields the first part of the theorem. The last statement of the theorem is clear. $\square$