[SciPy-User] Sometimes fmin_l_bfgs_b tests NaN parameters and then fails to converge

[josef.pktd at gmail.com]

2010/12/30 Yury V. Zaytsev <yury at shurup.com>:
> Dear Scipy experts,
>
> I am implementing a proof of concept code for a bounded optimization
> problem using Scipy optimization module and particularly fmin_l_bfgs_b
> bounded optimizer.
>
> The problem is that my code runs fine on one machine, but on another one
> at some point the optimizer passes NaN's as parameter values and then
> after some time fails to converge normally (it terminates, but with an
> ABNORMAL_TERMINATION_IN_LNSRCH error message).
>
> I am very confused about that and puzzled to which extent I can trust
> the results if they are so much system dependent. I would appreciate if
> the developers could tell me whether this problem is reproducible and
> what is the reason for its occurrence. If I can provide any additional
> information which would be helpful in diagnosing the issue, please bear
> with me.
>
> An extended description of my setup and machines and some test code to
> reproduce the problem are presented below:
>
> ---
>
> I have two machines:
>
> 1) Development machine: Ubuntu Hardy / 32 bit
> Linux mypride 2.6.24-28-generic #1 SMP Wed Nov 24 09:30:14 UTC 2010 i686 GNU/Linux
>
> 2) Test machine: Ubuntu Lucid / 64 bit
> Linux davis 2.6.32-27-generic #49-Ubuntu SMP Thu Dec 2 00:51:09 UTC 2010 x86_64 GNU/Linux
>
> The versions on Python and Numpy/Scipy stack on both machines are
> identical.
>
> I run ActiveState Python 2.7.1 installed in a virtualenv, where latest
> versions of Numpy and Scipy are installed (using pip install numpy / pip
> install scipy, scipy needs a patch that I fished out of the svn in order
> to compile). The tests pass on both machines with an exception for
> nakagami distribution on (1) that I don't care about. The pre-requisites
> were automatically installed as follows:
>
> $ sudo apt-get build-dep python-numpy python-scipy
>
> The patch, test logs, test script and optimization logs are attached.
>
> Please note, that on machine (1) it does 5 extra steps in between trying
> out NaN parameters for some reason. In this case, the optimization
> converges, but it is not always true for my bigger problem. However, you
> see already the problem of having extra iterations taking time which can
> lead to problems when the number of parameters is 100+.
>
> I have literally compared the numbers that come out of my bigger
> optimization and they are slightly different, although the difference is
> like in 10-th significant digit or so, that's why I didn't attribute any
> special meaning to it.
>
> The test script actually implements the simplest function I could think
> of (it gives inf if x <= 0 and x + 1 if x > 0, so basically the
> optimizer has to get as close to zero as it can).

I get on Windows 32
(array([  3.03873549e-08]), array([ 1.00000003]), {'warnflag': 0,
'task': 'CONVERGENCE: REL_REDUCTION_OF_F <= FACTR*EPSMCH', 'grad':
array([ 0.99999999]), 'funcalls': 16})

But your function has a discontinuity, and I wouldn't expect a bfgs
method to produce anything useful since the method assumes smoothness,
 as far as I know.

Josef

>
> One obvious difference between the machines is the bitness, but also,
> the environment on 32-bit machine is older, so this could be a compiler
> or library problem. That's why I am seeking for confirmation from other
> users of Scipy possibly running on completely different sets of
> libraries and compilers.
>
> Thanks!
>
> --
> Sincerely yours,
> Yury V. Zaytsev
>
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
>