Optimal conditioning of self-scaling variable Metric algorithms

Abstract

Variable Metric Methods are “Newton—Raphson-like” algorithms for unconstrained minimization in which the inverse Hessian is replaced by an approximation, inferred from previous gradients and updated at each iteration. During the past decade various approaches have been used to derive general classes of such algorithms having the common properties of being Conjugate Directions methods and having “quadratic termination”. Observed differences in actual performance of such methods motivated recent attempts to identify variable metric algorithms having additional properties that may be significant in practical situations (e.g. nonquadratic functions, inaccurate linesearch, etc.). The SSVM algorithms, introduced by this first author, are such methods that among their other properties, they automatically compensate for poor scaling of the objective function. This paper presents some new theoretical results identifying a subclass of SSVM algorithms that have the additional property of minimizing a sharp bound on the condition number of the inverse Hessian approximation at each iteration. Reducing this condition number is important for decreasing the roundoff error. The theoretical properties of this subclass are explored and two of its special cases are tested numerically in comparison with other SSVM algorithms.