[SciPy-User] fmin_l_bfgs_b issues with small numbers

[Warren Weckesser]

On Thu, Dec 15, 2011 at 7:07 PM, Daniel Arribas-Bel <darribas at asu.edu>wrote:

> Hi all!
>
> [I hope this hasn't been asked before on the list, I haven't found
> anything similar at least.]
>
> I have a simple optimization problem that involves two very small arrays
> of dimension (2, 2) and (2, 1):
>
> aa = np.array([[ 0.00030763, -0.00011521], \
>         [ 0.00093007, -0.00015189]])
> a = np.array([[  2.54854751e-05], [ -3.93219333e-05]])
>
>
> The function to optimize is:
>
> optim_par = lambda par: np.sum((np.dot(aa, np.array([par, par**2])) -
> a)**2)
>
> Now, if I just run optimize.fmin_l_bfgs_b with the default parameters:
>
>> start = [0.0]
>> bounds=[(-1.0,1.0)]
>> ll = op.fmin_l_bfgs_b(optim_par, start, approx_grad = True, bounds =
>> bounds)
>>
>
> I get a parameter of exactly 0.0, same as the starting value.
>
> If instead I either modify the parameters 'pgtol' and 'factr':
>
> l = op.fmin_l_bfgs_b(optim_par, start, approx_grad = True, bounds =
>> bounds, pgtol = 1e-50, factr = 10.0)
>>
>
> or simply re-scale the arrays multiplying them by 10,000 and run it as
> initially, the parameter I get is in either case -0.0296399132248. The
> attached script runs the three options.
>
> A couple of questions about this:
>
>    - Is this caused because of the way the optimizer handles relatively
>    small numbers or am I missing something else?
>    - I want this to run inside an app that will take different datasets
>    even though the arrays 'a' and 'aa' will always be of that shape. Which of
>    the two solutions would you recommend as more stable, robust and fast? Why?
>
> Any other comment/suggestion is of course most welcome. Thank you very
> much in advance,
>

Hi Daniel,

I can reproduce the behavior that you see with fmin_l_bfgs_b, but I don't
know why it is behaving that way.

Do you have a compelling reason for using fmin_l_bfgs_b?  I get a much more
accurate answer if I used fminbound:

In [12]: fminbound(optim_par, -1, 1)
Out[12]: -0.0089143209088738753

In [13]: fminbound(optim_par, -1, 1, xtol=1e-10)
Out[13]: -0.0089129582673392344


Warren
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20111216/f57da18a/attachment.html>