Abstract
An important class of optimization problems involve minimizing a cost function on a Lie group. In the case where the Lie group is non-compact there is no natural choice of a Riemannian metric and it is not possible to apply recent results on the optimization of functions on Riemannian manifolds. In this paper the invariant structure of a Lie group is exploited to provide a strong interpretation of a Newton iteration on a general Lie group. The paper unifies several previous algorithms proposed in the literature in a single theoretical framework. Local asymptotic quadratic convergence is proved for the algorithms considered.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Absil, P.-A., Mahony, R., Sebulchre, R. and VanDooren, P., (2002). A Grassman-Rayleigh quotient iteration for computing invariant subspaces. SIAM Review 44(1), 57–73.
Boothby, W.M. (1986), An Introduction to Differentiable Manifolds and Riemannian Geometry. Academic Press, London.
Botsaris, C.A. (1978), Differential gradient methods. Journal of Mathematical Analysis and Applications 63, 177–198.
Botsaris, C.A. (1981a), A class of differential descent methods for constrained optimization. Journal Mathematical Analysis and Applications 79, 96–112.
Botsaris, C.A. (1981b), Constrained optimization along geodesics. Journal Mathematical Analysis and Applications 79, 295–306.
Brockett, R.W. (1989), Least squares matching problems. Linear Algebra and its Applications, 122-124, 761–777.
Brockett, R.W. (1993), Differential geometry and the design of gradient algorithms. Proceedings of Symposia in Pure Mathematics 54, 69–92.
Bruyne, F. De, Anderson, B.D.O., Gevers, M. and Linard, N. (1999), Gradient expressions for a closed-loop identification scheme with a tailor-made parametrization. Automatica, 35(11), 1867–1871.
Chu, M.T., (1988). On the continuous realization of iterative processes. SIAM Review 30(3), 375–387.
Chu, M.T. and Driessel, K.R. (1990), The projected gradient method for least squares matrix approximations with spectral constraints. SIAM Journal of Numerical Analysis 27(4), 1050–1060.
Dehaene, J. (1995), Continuous-time matrix algorithms systolic algorithms and adaptive neural networks. Ph.D. thesis, Faculteit Toegepaste Wetenschappen, Department Elektrotechniek-ESAT, Katholieke Universiteit Leuven.
Deift, P., Nanda, T. and Tomei, C., (1983). Ordinary differential equations for the symmetric eigenvalue problem. SIAM Journal of Numerical Analysis 20(1), 1–22.
Dennis, J.E. and Schnabel, R.B. (1983), Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Prentice-Hall Series in Computational Mathematics. Prentice-Hall, Englewood Cliffs, NJ.
do Carmo, M.P. (1992), Riemannian Geometry. Mathematics: Theory and Applications. Birkhäuser, Boston, MA.
Edelman, A., Arias, T.A. and Smith, S.T. (1998), The geometry of algorithms with orthogonality constraints. SIAM Journal on Matrix Analysis and Applications 20(2), 303–353.
Flashka, H. (1974), The Toda lattice, II. Existence of integrals. Physical Review b 9(4), 1924–1925.
Fleming, W. (1977), Functions of Several Variables. Undergraduate Texts in Mathematics, Springer, NY.
Fletcher, R. (1996), Practical Methods of Optimization, 2nd edn. Wiley, Oxford.
Gabay, D. (1982), Minimizing a differentiable function over a differentiable manifold, Journal of Optimization Theory and Applications 37(2), 177–219.
Gevers, M.R. and Li, G. (1993), Parametrizations in control, estimation and filtering problems: accuracy aspects. In: Communications in Control Engineering. Springer, NY.
Helgason, S. (1978), Differential Geometry, Lie Groups and Symmetric Spaces. Academic Press, NY.
Helmke, U. and Moore, J.B. (1994), Optimization and dynamical systems. In: Communications and Control Engineering. Springer, London.
Liu, W.Q., Yan, W.Y., Sreeram, V. and Teo, K.L. (1996), Gradient flow approach to LQ cost improvement for simultaneous stabilization problem, Optimal Control Applications and Methods 17, 367–375.
Madievski, A.G., Anderson, B.D.O. and Gevers, M.R. (1994), Optimum realizations of sampled data controllers for FWL sensitivity minimization, Automatica 31(3), 367–379.
Mahony, R.E. (1994), Optimization algorithms on homogeneous spaces: with applications in linear systems theory. Ph.D. thesis, Department of Systems Engineering, Canberra, Australia.
Mahony, R.E. (1996), The constrained Newton method on a Lie-group and the symmetric eivenvalue problem. Linear Algebra and its Applications 248, 67–89.
Manton, J. (2001), Optimisation algorithms exploiting unitary constraints, IEEE Transactions on Signal Processing. Accepted for publication.
Moore, J.B., Mahony, R.E. and Helmke, U. (1994), Numerical gradient algorithms for eigenvalue and singular value calculations. SIAM Journal of Matrix Analysis 15(3), 881–902.
Nomizu (1954), Invariant affine connections on homogeneous spaces. American Journal of Mathematics 76, 33–65.
Owren, B. and Welfert, B. (2000), The Newton method on Lie groups. BIT 40(1), 121–145.
Perkins, J.E., Helmke, U. and Moore, J.B. (1990), Balanced realizations via gradient flow techniques Systems and Control Letters 14, 369–380.
Smith, S. (1994), Optimization techniques on Riemannian manifolds. Fields Institute Communications 3, 113–136. Proceedings Fields Inst. Workshop on Hamiltonian and Gradient Flows, Algorithms and Control.
Smith, S.T. (1993), Geometric optimization methods for adaptive filtering. Ph.D. thesis, Division of Applied Science.
Tseng, C.H., Teo, K.L., Cantoni, A. and Zang, Z. (1998), Iterative methods for optimal envelope-constrained filter design using gradient flow approach. Proceedings of Optimization Techniques and Applications 1, 534–541.
Udriste, C. (1994), Convex Functions and Optimization Methods on Riemannian Manifolds. Kluwer Academic Publishers, Dordrecht.
Varadarajan, V.S. (1984), Lie groups, Lie algebras and their representations. In: Graduate Texts in Mathematics. Springer, NY.
Warner, F.W. (1983), Foundations of differentiable manifolds and Lie groups. Graduate Texts in Mathematics. Springer, NY.
Watkins, D.S. and Elsner, L. (1988), Self similar flows. Linear Algebra and its Applications 110, 213–242.
Yan, W.-Y., Helmke, U. and Moore, J.B. (1994), Global analysis of Oja's flow for a neural networks. IEEE Transactions on Neural Networks, 5(5).
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Mahony, R., Manton, J.H. The Geometry of the Newton Method on Non-Compact Lie Groups. Journal of Global Optimization 23, 309–327 (2002). https://doi.org/10.1023/A:1016586831090
Issue Date:
DOI: https://doi.org/10.1023/A:1016586831090