Abstract
For large systems of linear equations, iterative methods provide attractive solution techniques. We describe the applicability and convergence of iterative methods of Krylov subspace type for an important class of symmetric and indefinite matrix problems, namely augmented (or KKT) systems. Specifically, we consider preconditioned minimum residual methods and discuss indefinite versus positive definite preconditioning. For a natural choice of starting vector we prove that when the definite and indenfinite preconditioners are related in the obvious way, MINRES (which is applicable in the case of positive definite preconditioning) and full GMRES (which is applicable in the case of indefinite preconditioning) give residual vectors with identical Euclidean norm at each iteration. Moreover, we show that the convergence of both methods is related to a system of normal equations for which the LSQR algorithm can be employed. As a side result, we give a rare example of a non-trivial normal(1) matrix where the corresponding inner product is explicitly known: a conjugate gradient method therefore exists and can be employed in this case.
Similar content being viewed by others
References
W. Arnoldi,The principle of minimized iterations in the solution of the matrix eigenvalue problem, Quart. Appl. Math., 9 (1951), pp. 17–29.
Å. Björck,Iterative refinement of linear least squares solutions, BIT, 7 (1967), pp. 257–278.
R. Chandra,Conjugate Gradient Methods for Partial Differential Equations, PhD thesis, Yale University Res. Rep. 129, 1978.
I. S. Duff,The solution of augmented systems, in Numerical Analysis 1993, D.F. Griffiths and G.A. Watson, eds., Longman, London, 1994, pp. 40–55.
S. Eisenstat,Some observations on the generalized conjugate gradient method, in Numerical Methods, V. Pereyra and A. Reinoza, eds., Lecture Notes in Mathematics, Volume 1005, Springer-Verlag, Berlin, 1983, pp. 99–107.
H. C. Elman and D. J. Silvester,Fast nonsymmetric iterations and preconditioning for Navier-Stokes equations, SIAM J. Sci. Comput., 17 (1996), pp. 33–46.
V. Faber and T. Manteuffel,Necessary and sufficient conditions for the existence of a conjugate gradient method, SIAM J. Numer. Anal. 21 (1984), pp. 352–362.
B. Fischer,Polynomial Based Iteration Methods for Symmetric Linear Systems, Wiley-Teubner Series in Advances in Numerical Mathematics, Wiley-Teubner, Chichester, 1996.
M. Fortin and F. Brezzi,Mixed and Hybrid Finite Elements Methods, Springer-Verlag, Berlin, 1991.
R. W. Freund and N. M. Nachtigal,QMR: a quasi-minimal residual method for non-Hermitian linear systems, Numer. Math., 60 (1991), pp. 315–339.
R. W. Freund and N. M. Nachtigal,A new Krylov-subspace method for symmetric indefinite linear systems, in Proceedings of the 14th IMACS World Congress on Computational and Applied Mathematics, W. F. Ames, ed., IMACS, 1994, pp. 1253–1256.
G. H. Golub and C. F., Van Loan,Matrix Computations, 2nd ed., Johns Hopkins University Press, London, 1989.
C. Lanczos,An iteration method for the solution of the eigenvalue problem of linear differential and integral operators, J. Res. Nat. Bur. Stand., 45 (1950), pp. 255–282.
N. M. Nachtigal, S. C. Reddy and L. N. Trefethen,How fast are nonsymmetric matrix iterations?, SIAM J. Matrix Anal. Appl., 13 (1992), pp. 778–795.
C. C. Paige and M. A. Saunders,Solution of sparse indefinite, systems of linear equations, SIAM J. Numer. Anal., 12 (1975), pp. 617–629.
C. C. Paige and M. A. Saunders,LSQR: An algorithm for sparse linear equations and sparse least squares, ACM Trans. Math. Software 8 (1982), pp. 43–71.
Y. Saad and M. H. Schultz,GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems, SIAM J. Sci. Stat. Comput., 7 (1986), pp. 856–869.
M. A. Saunders,Solution of sparse rectangular systems using LSQR and Craig, BIT, 35 (1995), pp. 588–604.
D. J. Silvester and A. J. Wathen,Fast iterative solution of stabilised Stokes systems part II: using general block preconditioners, SIAM J. Numer. Anal., 31 (1994), pp. 1352–1367.
G. Szegö,Orthogonal Polynomials, AMS Colloquium Publications XXIII (revised ed.), AMS, New York, 1959.
Author information
Authors and Affiliations
Additional information
Communicated by Robert S. Schreiber
This work was supported by British Council/German Academic Exchange Service Research Collaboration Project 465 and NATO Collaborative Research Grant CRG 960782
Rights and permissions
About this article
Cite this article
Fischer, B., Ramage, A., Silvester, D.J. et al. Minimum residual methods for augmented systems. Bit Numer Math 38, 527–543 (1998). https://doi.org/10.1007/BF02510258
Received:
Issue Date:
DOI: https://doi.org/10.1007/BF02510258