[SciPy-Dev] minimizer benchmark

[Jacob Stevenson]

Hi Everyone, it seemed to me that the folder scipy/optimize/benchmarks/ 
was rather lonely, with only one file in it, so I wrote a script which 
benchmarks the scipy minimizers.  The script simpy runs all the 
optimizers on the Rosenbrock function and prints various information.  
Here's the output sorted by minimization time

Optimizer benchmark on the Rosenbrock function sorted by time
        L-BFGS-B pass nfev    6 njev    0 nhev    0 time 0.000691891
             TNC pass nfev   13 njev    0 nhev    0 time 0.00105786
       Newton-CG pass nfev   17 njev   27 nhev   11 time 0.00400996
           SLSQP pass nfev   41 njev   27 nhev    0 time 0.00410509
       trust-ncg pass nfev   18 njev   16 nhev   15 time 0.00415802
          dogleg pass nfev   16 njev   14 nhev   13 time 0.00426602
              CG pass nfev   63 njev   63 nhev    0 time 0.0065279
            BFGS pass nfev   44 njev   44 nhev    0 time 0.0070231
          Powell pass nfev  524 njev    0 nhev    0 time 0.0262001
          COBYLA fail nfev 1000 njev    0 nhev    0 time 0.026603

The results are interesting with L-BFGS-B outperforming all the others 
by a significant margin in both total time and total number of function 
calls.

I have not submitted a pull request because I have no idea how to fit 
what I've done into an existing benchmarking framework (if there is 
one).  I will submit the pull request if there is interest.  In my 
opinion it would be really useful to have a long set of benchmarks to 
see how all the minimizers perform on different types of minimization 
problems.

Here is the script in my scipy fork

https://github.com/js850/scipy/blob/benchmarks/scipy/optimize/benchmarks/bench_optimizers.py

Best wishes,
Jake