[SciPy-dev] Patching fmin_l_bfgs_b in scipy.optimize

[jason-sage at creativetrax.com]

Gilles Rochefort wrote:
>
> Hi,
>
> I had some troubles getting a stable version of fmin_l_bfgs_b. I got a 
> segmentation fault
> as soon as I turned a different value than -1 to iprint argument.
>
> As far as I understood, this iprint parameter is directly related to 
> some fortran printing/writing logs.
> It prints iteration, norm of gradient on screen and also in a file 
> called iterate.dat .. Even recompiling
> the whole stuff from lapack, atlas etc.. (using gfortran only) up to 
> scipy doesn't solve the problem.
>
> Because I definitely gave up the idea to fix the bug by recompiling,  
> I decided to patch the way python
> interacts with lbfgsb fortran procedure. So, I disabled the fortran 
> iprint and add some iprint functionnality
> in pure python code directly in lbfgsb.py (scipy/optimize).  Doing so, 
> I get rid of the useless iterate.dat file too.
>
> While patching, I followed by adding some few things :
> - an optional stopping rule based on maximum number of iterations. I 
> think this is more usefull than
> the number of function evaluations because the fortran code performs a 
> linesearch procedure inside the lbfgsb ones.
> - a callback procedure which allows to inspect solution, criterion, 
> and gradient at current iterate.
>
> Maybe my patch should be usefull for someone,
> so I decided to share it with the scipy community.
>

Thanks for sharing.  As a general rule, I like having callback functions
inside of these types of procedures, if only for educational purposes
(as long as the callback check is negligible if no callback is
provided).  Along these lines, I notice that several times you use
"!=None" in an if statement.  It is faster to use "is not None" versus
"!= None".  I think this has to do with there being only one None
object, so a memory reference comparison (done with "is") is sufficient
and faster.

If a python wizard knows better, please correct me!

In [1]: a=lambda x: x

In [2]: %timeit a is not None
10000000 loops, best of 3: 113 ns per loop

In [3]: %timeit a != None
10000000 loops, best of 3: 146 ns per loop

In [4]: a=None

In [5]: %timeit a is not None
10000000 loops, best of 3: 99.2 ns per loop

In [6]: %timeit a != None
10000000 loops, best of 3: 147 ns per loop

Thanks,

Jason