[Numpy-discussion] `allclose` vs `assert_allclose`

[josef.pktd at gmail.com]

On Thu, Jul 17, 2014 at 4:07 PM, <josef.pktd at gmail.com> wrote:

>
>
>
> On Wed, Jul 16, 2014 at 9:52 AM, Nathaniel Smith <njs at pobox.com> wrote:
>
>> On 16 Jul 2014 10:26, "Tony Yu" <tsyu80 at gmail.com> wrote:
>> >
>> > Is there any reason why the defaults for `allclose` and
>> `assert_allclose` differ? This makes debugging a broken test much more
>> difficult. More importantly, using an absolute tolerance of 0 causes
>> failures for some common cases. For example, if two values are very close
>> to zero, a test will fail:
>> >
>> >     np.testing.assert_allclose(0, 1e-14)
>> >
>> > Git blame suggests the change was made in the following commit, but I
>> guess that change only reverted to the original behavior.
>> >
>> >
>> https://github.com/numpy/numpy/commit/f43223479f917e404e724e6a3df27aa701e6d6bf
>> >
>> > It seems like the defaults for  `allclose` and `assert_allclose` should
>> match, and an absolute tolerance of 0 is probably not ideal. I guess this
>> is a pretty big behavioral change, but the current default for
>> `assert_allclose` doesn't seem ideal.
>>
>> What you say makes sense to me, and loosening the default tolerances
>> won't break any existing tests. (And I'm not too worried about people who
>> were counting on getting 1e-7 instead of 1e-5 or whatever... if it matters
>> that much to you exactly what tolerance you test, you should be setting the
>> tolerance explicitly!) I vote that unless someone comes up with some
>> terrible objection in the next few days then you should submit a PR :-)
>>
>
> If you mean by this to add atol=1e-8 as default, then I'm against it.
>
> At least it will change the meaning of many of our tests in statsmodels.
>
> I'm using rtol to check for correct 1e-15 or 1e-30, which would be
> completely swamped if you change the default atol=0.
> Adding atol=0 to all assert_allclose that currently use only rtol is a lot
> of work.
> I think I almost never use a default rtol, but I often leave atol at the
> default = 0.
>
> If we have zeros, then I don't think it's too much work to decide whether
> this should be atol=1e-20, or 1e-8.
>

copied from
http://mail.scipy.org/pipermail/numpy-discussion/2014-July/070639.html
since I didn't get any messages here

This is a compelling use-case, but there are also lots of compelling
usecases that want some non-zero atol (i.e., comparing stuff to 0).
Saying that allclose is for one of those use cases and assert_allclose
is for the other is... not a very felicitious API design, I think. So
we really should do *something*.

Are there really any cases where you want non-zero atol= that don't
involve comparing something against a 'desired' value of zero? It's a
little wacky, but I'm wondering if we ought to change the rule (for
all versions of allclose) to

if desired == 0:
    tol = atol
else:
    tol = rtol * desired

In particular, means that np.allclose(x, 1e-30) would reject x values
of 0 or 2e-30, but np.allclose(x, 0) will accept x == 1e-30 or 2e-30.

-n


That's much too confusing.
I don't know what the usecases for np.allclose are since I don't have any.

assert_allclose is one of our (statsmodels) most frequently used numpy
function

this is not informative:

`np.allclose(x, 1e-30)`


since there are keywords
either np.assert_allclose(x, atol=1e-30)
if I want to be "close" to zero
or

np.assert_allclose(x, rtol=1e-11, atol=1e-25)

if we have a mix of large numbers and "zeros" in an array.

Making the behavior of assert_allclose depending on whether desired is
exactly zero or 1e-20 looks too difficult to remember, and which desired I
use would depend on what I get out of R or Stata.

atol=1e-8 is not close to zero in most cases in my experience.


The numpy.testing assert functions are some of the most useful functions in
numpy, and heavily used "code".

Josef



>
> Josef
>
>
>
>> -n
>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20140718/76ae5624/attachment.html>