doiSerbia

Multiplicative parameters in gradient descent methods

Stanimirović Predrag (Department of Mathematics, Faculty of Science and Mathematics, Niš)
Miladinović Marko (Department of Mathematics, Faculty of Science and Mathematics, Niš)
Đorđević Snežana (Department of Mathematics, Faculty of Science and Mathematics, Niš)

We introduced an algorithm for unconstrained optimization based on the reduction of the modified Newton method with line search into a gradient descent method. Main idea used in the algorithm construction is approximation of Hessian by a diagonal matrix. The step length calculation algorithm is based on the Taylor's development in two successive iterative points and the backtracking line search procedure.

Keywords: Line search, gradient descent methods, Newton method, backtracking line search, convergence rate