Line Search Guidelines

Three Line Search Guidelines are included in this repo, Armijo, Goldstein and Wolfe respectively. Python version implementation.

Armijo

Let $d^k$ refers to a direction of descent in $x^k$, if: $$ f(x^k + \alpha d^k)\leq f(x^k) + c_1\alpha\nabla f(x^k)^\text Td^k $$ then we said step length $\alpha$ satisfies Armijo Guidelines.

Goldstein

Let $d^k$ refers to a direction of descent in $x^k$, if: $$ f(x^k + \alpha d^k)\leq f(x^k) + c\alpha\nabla f(x^k)^\text Td^k\ f(x^k + \alpha d^k)\geq f(x^k) + (1-c)\alpha\nabla f(x^k)^\text Td^k $$ then we said step length $\alpha$ satisfies Goldstein Guidelines, while $c\in(0, \frac12)$.

Wolfe

Let $d^k$ refers to a direction of descent in $x^k$, if: $$ f(x^k + \alpha d^k)\leq f(x^k) + c_1\alpha\nabla f(x^k)^\text Td^k\ \nabla f(x^k+\alpha d^k)^\text Td^k\geq c_2\nabla f(x^k)^\text T d^k $$ then we said step length $\alpha$ satisfies Wolfe Guidelines, while $c_1, c_2\in(0, \frac12)$ and $c_1<c_2$.

GitHub

View Github