Broyden–Fletcher–Goldfarb–Shanno algorithm

Common card · No. 207884

In numerical optimization, the Broyden–Fletcher–Goldfarb–Shanno (BFGS) algorithm is an iterative method for solving unconstrained nonlinear optimization problems. Like the related Davidon–Fletcher–Powell method, BFGS determines the descent direction by preconditioning the gradient with curvature information. It does so by gradually improving an approximation to the Hessian matrix of the loss function, obtained only from gradient evaluations (or approximate gradient evaluations) via a generalized secant method. Since the updates of the BFGS curvature matrix do not require matrix inversion, its computational complexity is only O ( n 2 ) {\displaystyle {\mathcal {O}}(n^{2})} , compared to O ( n 3...

Read on Wikipedia