Convergence to diagonal form of block Jacobi-type methods

Abstract

We provide sufficient conditions for the general sequential block Jacobi-type method to converge to the diagonal form for cyclic pivot strategies which are weakly equivalent to the column-cyclic strategy. Given a block-matrix partition $(A_{ij})$ of a square matrix $\mathbf {A}$, the paper analyzes the iterative process of the form $\mathbf {A}^{(k+1)} = [\mathbf {P}^{(k)}]^*\,\mathbf {A}^{(k)}\,\mathbf {Q}^{(k)}$, $k\ge 0$, $\mathbf {A}^{(0)}=\mathbf {A}$, where $\mathbf {P}^{(k)}$ and $\mathbf {Q}^{(k)}$ are elementary block matrices which differ from the identity matrix in four blocks, two diagonal and the two corresponding off-diagonal blocks. In our analysis of convergence a promising new tool is used, namely, the theory of block Jacobi operators. Typical applications lie in proving the global convergence of block Jacobi-type methods for solving standard and generalized eigenvalue and singular value problems.

Appendix

Figure 2 (Fig. 3) shows behavior of the zero pattern of $\mathbf {F}^{(k)}$ ($\mathbf {G}^{(k)}$) for the first cycle. In the second cycle the roles of $\mathbf {F}^{(k)}$ and $\mathbf {G}^{(k)}$ are interchanged since $\mathbf {F}^{(M)}$ ($\mathbf {G}^{(M)}$) is strictly block lower- (upper-) triangular. Hence, these figures give insight into the zero patterns of $\mathbf {F}^{(k)}$ and $\mathbf {G}^{(k)}$ for all $k\ge 0$.

Fig. 2

First cycle applied to $\mathbf {F}^{(0)}$. The blocks colored gray need not be zero. The dark gray are pivot blocks

Fig. 3

First cycle applied to $\mathbf {G}^{(0)}$

1.1 Proof of Lemma 4.8

As mentioned earlier, we can assume that every second equivalence in the inserted sequence is $\mathop {\sim }\limits ^{s}$ and the others are $\sim$. Let $r+1$ be the length of the chain linking $\fancyscript{O}=\fancyscript{O}_0$ and $\fancyscript{O}'= \fancyscript{O}_r$. Then we have two possibilities:

$$\begin{aligned} \fancyscript{O}_0\sim \fancyscript{O}_1\mathop {\sim }\limits ^{s}\fancyscript{O}_2&\sim \ \cdots \ \left\{ \begin{array}{ll} \sim \fancyscript{O}_r,\ \text{ if } \text{ r } \text{= } \text{2d+1 }\\ \mathop {\sim }\limits ^{s}\fancyscript{O}_r,\ \text{ if } \text{ r } \text{= } \text{2d } \end{array} \right. \text{ and } \nonumber \\ \fancyscript{O}_0\mathop {\sim }\limits ^{s}\fancyscript{O}_1\sim \fancyscript{O}_2&\mathop {\sim }\limits ^{s}&\cdots \ \left\{ \begin{array}{l} \sim \fancyscript{O}_r,\ \text{ if } \text{ r } \text{= } \text{2d }\\ \mathop {\sim }\limits ^{s} \fancyscript{O}_r,\ \text{ if } \,r = 2d-1 \end{array} \right. . \end{aligned}$$

(i) The proof uses induction with respect to $d$. For $d=0$, we have just one possibility: $r=1$ and $\fancyscript{O}=\fancyscript{O}_0\sim \fancyscript{O}_1=\fancyscript{O}'$. The assertion assumes the form $\Vert { \large J}\!_{{\fancyscript{O}}'}^{\,[1]} { \large J}\!_{{\fancyscript{O}}'}^{\,[2]}\Vert \le c$. Lemma 4.4(i) implies ${ \large J}\!_{{\fancyscript{O}}'}^{\,[1]}={ \large J}\!_{{\fancyscript{O}}}^{\,[1]}$ and ${ \large J}\!_{{\fancyscript{O}}'}^{\,[2]}={ \large J}\!_{{\fancyscript{O}}}^{\,[2]}$, where ${ \large J}\!_{{\fancyscript{O}}}^{\,[1]}$ denotes the Jacobi operator composed by the same Jacobi annihilators as ${ \large J}\!_{{\fancyscript{O}}'}^{\,[1]}$, and similar holds for ${ \large J}\!_{{\fancyscript{O}}}^{\,[2]}$. Hence, the assertion is implied by the assumption $\Vert { \large J}\!_{{\fancyscript{O}}}{ }\large {J}_{{\fancyscript{O}}}\Vert _2 \le c$.

For $d=1$, we have four possibilities: $\fancyscript{O}\mathop {\sim }\limits ^{s}\fancyscript{O}'$, $\fancyscript{O}\sim \fancyscript{O}_1\mathop {\sim }\limits ^{s}\fancyscript{O}'$, $\fancyscript{O}\mathop {\sim }\limits ^{s}\fancyscript{O}_1\sim \fancyscript{O}'$ and $\fancyscript{O}\sim \fancyscript{O}_1\mathop {\sim }\limits ^{s}\fancyscript{O}_2\sim \fancyscript{O}'$. For all these cases, the assertion assumes the form $\chi \equiv \Vert { \large J}\!_{{\fancyscript{O}}'}^{\,[1]} { \large J}\!_{{\fancyscript{O}}'}^{\,[2]}{ \large J}\!_{{\fancyscript{O}}'}^{\,[3]}\Vert _2 \le c$.

In the first case, $\fancyscript{O}\mathop {\sim }\limits ^{s} \fancyscript{O}'$, so we can write ${ \large J}\!_{{\fancyscript{O}}'}^{\,[t]}=\varTheta _2^{[t]}\varTheta _1^{[t]},\ { \large J}\!_{{\fancyscript{O}}}^{\,[t]}=\varTheta _1^{[t]}\varTheta _2^{[t]},\ t=1,2,3$ where $\varTheta _1^{[t]}$ and $\varTheta _2^{[t]}$ are made of the Jacobi annihilators which build ${ \large J}\!_{{\fancyscript{O}}'}^{\,[t]}$. Using Proposition 4.2, we have $\chi =\Vert \varTheta _2^{[1]}\varTheta _1^{[1]}\varTheta _2^{[2]}\varTheta _1^{[2]}\varTheta _2^{[3]}\varTheta _1^{[3]}\Vert _2 \le \Vert (\varTheta _1^{[1]}\varTheta _2^{[2]})(\varTheta _1^{[2]}\varTheta _2^{[3]})\Vert _2$. Because each parenthesis represents one Jacobi operator made of the Jacobi annihilators from $\Upsilon _{\pi }$ and defined by the ordering $\fancyscript{O}$, the assumption $\Vert { \large J}\!_{{\fancyscript{O}}}{ }\large {J}_{{\fancyscript{O}}}\Vert _2 \le c$ implies $\chi \le c^2<c$.

In the second case, we have ${ \large J}\!_{{\fancyscript{O}}_1}^{\,[t]}={ \large J}\!_{{\fancyscript{O}}}^{\,[t]}$ for $t=1,2,3$, where ${ \large J}\!_{{\fancyscript{O}}}^{\,[t]}$ denotes the Jacobi operator composed by the same Jacobi annihilators as ${ \large J}\!_{{\fancyscript{O}}_1}^{\,[t]}$. Hence, $\Vert { \large J}\!_{{\fancyscript{O}}_1}^{\;[t_1]}{ \large J}\!_{{\fancyscript{O}}_1}^{\,[t_2]}\Vert _2 \le c,\ t_1,t_2\in \{1,2,3\}$. The next equivalence relation is $\mathop {\sim }\limits ^{s}$, so we can write ${ \large J}\!_{{\fancyscript{O}}'}^{\,[t]}=\varTheta _2^{[t]}\varTheta _1^{[t]}$, ${ \large J}\!_{{\fancyscript{O}}_1}^{\,[t]}=\varTheta _1^{[t]}\varTheta _2^{[t]}$ for $t=1,2,3$. Here $\varTheta _1^{[t]}$ and $\varTheta _2^{[t]}$ are composed of the Jacobi annihilators from ${ \large J}\!_{{\fancyscript{O}}'}^{\,[t]}$ and are not necessarily the same as earlier. The rest of the proof is the same as earlier.

In the third case, we first note that ${ \large J}\!_{{\fancyscript{O}}_1}^{\,[t]}={ \large J}\!_{{\fancyscript{O}}'}^{\,[t]}$ for $t=1,2,3$. For each $t$, we start from ${ \large J}\!_{{\fancyscript{O}}'}^{\,[t]}$, rearrange the Jacobi annihilators, and obtain ${ \large J}\!_{{\fancyscript{O}}_1}^{\,[t]}$. Then we continue as in the first case. We conclude that $\chi = \Vert { \large J}\!_{{\fancyscript{O}}_1}^{\,[1]} { \large J}\!_{{\fancyscript{O}}_1}^{\,[2]}{ \large J}\!_{{\fancyscript{O}}_1}^{\,[3]}\Vert _2 \le c$.

Finally, in the fourth case, we start making conclusions as in the third case, and then we complete the analysis as in the second case.

The Induction Hypothesis. We assume that the assertion of the lemma holds provided the chain contains $d-1$ shift-equivalences, where $d\ge 1$.

The Induction Step. Let the chain contain $d$ shift-equivalences. We divide it into two parts: the head and the tail, where the tail is the right-most part of the chain. The head contains the remaining part of the chain. The form of the tail can be either $\sim \fancyscript{O}_{r-1}\mathop {\sim }\limits ^{s}\fancyscript{O}'$ or $\mathop {\sim }\limits ^{s}\fancyscript{O}_{r-1}\sim \fancyscript{O}'$.

Let us consider the first case. By the induction hypothesis, the relation

$$\begin{aligned} \Vert { \large J}\!_{{\fancyscript{O}}_{r-1}}^{\,[1]} { \large J}\!_{{\fancyscript{O}}_{r-1}}^{\,[2]}\cdots { \large J}\!_{{\fancyscript{O}}_{r-1}}^{\,[d+1]}\Vert _2 \le c\ \end{aligned}$$

(7.1)

holds if the Jacobi operators ${ \large J}\!_{{\fancyscript{O}}_{r-1}}^{\,[t]}$ are made of the Jacobi annihilators from $\Upsilon _{\pi }$ and are defined by $\fancyscript{O}_{r-1}$. Let us denote the left-hand side of the inequality (4.13) by $\chi$. We have to show that $\chi \le c$.

Since $\fancyscript{O}_{r-1}\mathop {\sim }\limits ^{s}\fancyscript{O}'$, we have ${ \large J}\!_{{\fancyscript{O}}'}^{\,[t]}=\varTheta _2^{[t]}\varTheta _1^{[t]}$, ${ \large J}\!_{{\fancyscript{O}}_{r-1}}^{\,[t]}=\varTheta _1^{[t]}\varTheta _2^{[t]}$, $t=1,\ \ldots \ ,d+2$, for some $\varTheta _1^{[t]}$ and $\varTheta _2^{[t]}$ which are made of the Jacobi annihilators from ${ \large J}\!_{{\fancyscript{O}}'}^{\,[t]}$. We have

$$\begin{aligned} \chi&= \Vert \varTheta _2^{[1]}\varTheta _1^{[1]}\varTheta _2^{[2]}\varTheta _1^{[2]}\cdots \varTheta _2^{[d+1]}\varTheta _1^{[d+1]} \varTheta _2^{[d+2]}\varTheta _1^{[d+2]}\Vert _2\\&\le \Vert (\varTheta _1^{[1]}\varTheta _2^{[2]})(\varTheta _1^{[2]}\varTheta _2^{[3]})\cdots (\varTheta _1^{[d]}\varTheta _2^{[d+1]})(\varTheta _1^{[d+1]}\varTheta _2^{[d+2]})\Vert _2\ \nonumber \\&= \ \Vert { }\large {J}_{{\fancyscript{O}}_{r-1}}^{\,[1]} { }\large {J}_{{\fancyscript{O}}_{r-1}}^{\,[2]}\cdots { }\large {J}_{{\fancyscript{O}}_{r-1}}^{\,[d+1]}\Vert _2. \end{aligned}$$

Here each   ${ }\large {J}_{{\fancyscript{O}}_{r-1}}^{\,[t]}=\varTheta _1^{[t]}\varTheta _2^{[t+1]}$, $1\le t\le d+1$ is the Jacobi operator built of the Jacobi annihilators appearing in ${ \large J}\!_{{\fancyscript{O}}'}^{\,[t]}$, $1\le t\le d+2$ and defined by $\fancyscript{O}_{r-1}$. Since the Jacobi annihilators are from $\Upsilon _{\pi }$, we can use the induction hypothesis (7.1) to conclude that $\chi \le c$.

If the tail has the form $\mathop {\sim }\limits ^{s}\fancyscript{O}_{r-1}\sim \fancyscript{O}'$, the proof is quite similar. One first notes that $\chi = \Vert { \large J}\!_{{\fancyscript{O}}_{r-1}}^{\,[1]} { \large J}\!_{{\fancyscript{O}}_{r-1}}^{\,[2]}\cdots { \large J}\!_{{\fancyscript{O}}_{r-1}}^{\,[d+2]}\Vert _2$, where for each $t$, ${ \large J}\!_{{\fancyscript{O}}_{r-1}}^{\,[t]}$ has the Jacobi annihilators from ${ \large J}\!_{{\fancyscript{O}}'}^{\,[t]}$, only rearranged according to $\fancyscript{O}_{r-1}$. The rest of the proof is as in the first case, but the roles of ${ \large J}\!_{{\fancyscript{O}}'}^{\,[t]}$ and ${ \large J}\!_{{\fancyscript{O}}_{r-1}}^{\,[t]}$ are replaced by ${ \large J}\!_{{\fancyscript{O}}_{r-1}}^{\,[t]}$ and ${ \large J}\!_{{\fancyscript{O}}_{r-2}}^{\,[t]}$, respectively.

(ii) The proof is very similar to the proof of the assertion (i).