Gearhart–Koshy acceleration for affine subspaces
Introduction
Our setting is a real Hilbert space equipped with inner-product and induced norm . Consider closed affine subspaces , and suppose Given , we study the best approximation problem In this work, our focus is the case in which the nearest point projectors onto the individual spaces, , are accessible. Recall that the projector onto is the operator given by The method of cyclic projections is an iterative procedure for solving (1) (i.e., for computing ) by using only the individual projection operators . Although originally studied when are linear subspaces [11], [12], the following affine variant readily follows from translation properties of the projector.
Theorem 1.1 The Method of Cyclic Projections Let be closed affine subspaces of with . Then, for each ,
The convergence rate of the sequence in Theorem 1.1 can be related to the angle between the subspaces. Recall that the (Friederichs) angle between two closed subspaces and is the angle in whose cosine is given by The following result provides a bound on the convergence rate based on this quantity.
Theorem 1.2 [9, Corollary 9.34] Let be closed affine subspaces of with . For , let denote the linear subspace parallel to . Then, for each , where the constant is given by
When , Theorem 1.2 establishes -linear convergence of the method of cyclic projections. This is easily seen to be the case, for instance, when for all . In the setting with , this characterisation can be further refined: if and only if is closed (which always holds in finite dimensions) in which case convergence is linear, else and the rate of convergence is arbitrarily slow [1], [4].
Let denote the cyclic projections operator given by . In an attempt to accelerate the method of cyclic projections, Gearhart & Koshy [10] proposed the following scheme which iterates by performing an exact line search to choose to nearest point to in the affine span of . When are linear subspaces, it can be shown (see Section 3) that the step size can be computed using the expression Since it only requires vector arithmetic, evaluating this expression comes with relatively low computational cost. Moreover, Gearhart & Koshy’s scheme gives the following refinement of the upper-bound provided by Theorem 1.2 in (2).
Theorem 1.3 Gearhart–Koshy [10] Let be closed affine subspaces of with . For each sequence generated by Algorithm1, there exists a sequence such that where the constant is given by (3).
Although Theorem 1.3 still holds for affine subspaces, the efficient expression for provided by (4) is only valid for linear subspaces (this will be explained more precisely Section 3). Thus, in the affine case, it is no longer obvious how to efficiently apply the scheme.
In this work, we address the aforementioned problem by deriving an alternative expression for (4) which still holds in the affine case and still only requires vector arithmetic for its evaluation. Our key insight is the observation that (4) implicitly relies on the fact that the zero vector is always feasible for linear subspaces. The remainder of this work is structured as follows: In Section 2, we collect the necessary preliminaries. In Section 3, we discuss Gearhart & Koshy’s derivation of (4) and, in Section 4, we provide an alternative formula which still holds in the affine setting. In Section 5, we discuss some implications for nonlinear fixed iterations and finally, in Section 6, we provide computational examples.
Section snippets
Preliminaries
Let be a nonempty subset of . Recall that the (nearest point) projector onto is the operator defined by It is well-known (see, for instance, [9, 3.5]) that is a well-defined operator whenever is closed and convex. Further, the definition in (5) also implies the translation formula where . The following proposition collects important properties of projectors for use in the subsequence sections.
Proposition 2.1 Properties of Projectors Let be a nonempty,
Gearhart–Koshy acceleration for linear subspaces
In this section, we recall the derivation of Gearhart & Koshy’s scheme for linear subspaces [10]. This serves to both introduce the scheme, and to highlighting the immediate difficulty with extending the result to affine spaces.
Denote . Using this notation, the method of cyclic projection (as discussed in Theorem 1.1) generates a sequence according to the fixed-point iteration Gearhart & Koshy’s scheme attempts to accelerate convergence by instead defining the
Gearhart–Koshy acceleration for affine subspaces
In this section, we derive an alternate expression for the step size in Gearhart & Koshy’s scheme which is still valid for affine subspaces and which can be explicitly computed without knowledge of an intersection point (unlike the expression in (15)). To this end, let denote the operator
Lemma 4.1 Let be closed affine subspaces of with . For , let denote the linear subspace parallel to . If , then the solution of (8) is
Extensions to firmly nonexpansive operators
The orthogonality condition (18) was a key ingredient in the proof of Lemma 4.1. In this section, we investigate what remains true without this property. Our focus will be the following class of operators which generalise affine projectors.
Definition 5.1 An operator is firmly quasi-nonexpansive if for all and .
It is straightforward to check that the inequality (23) is equivalent to requiring As a consequence of Proposition 2.1
Computational examples
In this section, we provide numerical examples to demonstrate the results from the previous sections. Our presentation will focus on the comparison between the method of cycling projections (Theorem 1.2) and its accelerated counterpart (Theorem 4.1). However analogous conclusions apply for the other methods considered in this paper. All computations were performed in Python 3 on a machine running Ubuntu 18.04 with an Intel Core i7-8665U and 16 GB of memory.
Since the bound in Theorems 1.2 & 4.1
Acknowledgements
This work is supported in part by DE200100063 from the Australian Research Council . The author would like to thank Janosch Rieger for discussions relating to [13] which initiated this work, and the anonymous referee and editor for their helpful comments and suggestions.
References (14)
- et al.
Finding best approximation pairs relative to two closed convex sets in Hilbert spaces
J. Approx. Theory
(2004) - et al.
Acceleration schemes for the method of alternating projections
J. Comput. Appl. Math.
(1989) - et al.
The method of cyclic projections for closed convex sets in Hilbert space
Contemp. Math.
(1997) - et al.
Convex Analysis and Monotone Operator Theory in Hilbert Spaces
(2017) - et al.
Characterizing arbitrarily slow convergence in the method of alternating projections
Int. Trans. Oper. Res.
(2009) - et al.
Accelerating the convergence of the method of alternating projections
Trans. Amer. Math. Soc.
(2003) - et al.
A cyclic Douglas–Rachford iteration scheme
J. Optim. Theory Appl.
(2014)
Cited by (4)
GENERALIZED GEARHART-KOSHY ACCELERATION FOR THE KACZMARZ METHOD
2023, Mathematics of ComputationA LEARNING-ENHANCED PROJECTION METHOD for SOLVING CONVEX FEASIBILITY PROBLEMS
2022, Discrete and Continuous Dynamical Systems - Series B