[SciPy-Dev] minimizer benchmark

[Gael Varoquaux]

On Mon, Nov 18, 2013 at 03:20:13PM +0000, Jacob Stevenson wrote:
> The results are interesting with L-BFGS-B outperforming all the others 
> by a significant margin in both total time and total number of function 
> calls.

That seems right. BFGS/LBFGS should in general be prefered to other
approaches if your problem is smooth without noisy gradient and you know
nothing about the Hessian. A piece of warning: the clear cut win of
L-BFGS will depend a bit on the function to optimize. Rosenbrock is
amongst the nastiest function to optimize. On easer function (eg
quadratic or close to quadratc) results will differ.

By the way, a somewhat lengthy discussion on this, with benchmark
scripts, can be found on
http://scipy-lectures.github.io/advanced/mathematical_optimization/index.html

G