Possible bug (was Re: numpy, overflow, inf, ieee, and rich comparison)

[Huaiyu Zhu]

Tim,

Thank you for the explanations.  We actually agree more than it appears:

- Full IEEE 754 support in Python is good, which should allow the user to
  decide when to use raised flags and when to use Python exceptions.

- If the user default can be changed with one line of code, it does not
  matter if Python default use exceptions, even if that's not strict
  compliance in words.  The "factory default" might as well set for the
  benefit of newbies.

- Without IEEE 754 support, or with just partial support, f.p. computation
  is a mess.

Following are some contentious issues:

I am not so sure that even with 754, there would still be a great variety.
I see you giving one example: sqrt(-1) in 1.5.2.  But wasn't that caused by
extra Python checking?  I say this because errno==0 in C.  Maybe I'm missing
something.  Or, do you mean that glibc itself is deficient, because it
should have set the flag?

My worry concerns the decision to unify 2.0 f.p. behavior in a specific way
before full IEEE 754 support.  I understand that you say this does not count
as a changed behavior because the previous one is accidental.  But I don't
see the particular fixed subset chosen as any better than any other. I agree
that I do have a choice of using -lieee, though.

It is interesting to know that someone actually relies on OverflowError to
detect underflow.  But why not let them continue that as well, given that
all of these would be available in the future anyway?

> At this point we favor forcing consistency over maintaining platform
> accidents, no matter how attached people have gotten to them; else we'll
> be stuck with them forever.

I do not understand.  Once there is full IEEE 754, everyone would have a
choice, which pretty much includes all the things people are doing now.
Before that happens, why consistency accross platform is deemed more
important than consistency accross releases?  My naive thinking is that
people port more from 1.5.2 to 2.0 than from one platform to another, isn't
it?  :-)

Following does not have much new to say, just some clarification.

You are right to emphasise that Kahan does not favor any subset of IEEE 754:

> >   They are called "Exceptions" because to any policy for handling them,
> >   imposed in advance upon all programmers by the computer system, some
> >   programmers will have good reason to take exception.

Yes.  And that's what's happening. :-)

> His paper "How JAVA's Floating-Point Hurts Everyone Everywhere" lists his 5
> main gripes about Java in bullet points at the top of page 3.  The paper
> runs on to 80 pages.  You somehow managed to miss 79.89 of them <0.4 wink>,
> reducing it to a bogus claim about Java that Kahan didn't make (Java has
> neither "Trap" nor "Flag" style fp exception signaling).

Thank you for correcting me on this one.  I did "glance over" all the pages.
I was somewhat puzzled by his decriptions, so I made a quick test: 1/0, and
got Exception in thread "main" java.lang.ArithmeticException: / by zero

But now I see that 1/0.0 is Infinity.  So is Math.exp(777).  And so on.
So that teaches me to test, test, test, ...

> > He told "A cautionary Tale of the Ariane 5",
> 
> Which has nothing to do with Java -- the software in question was written in
> Ada, which (unlike Java) has "Trap" style.  Why he put it in a paper about
> Java is a bit of a mystery; I write it off to rhetorical excess.

You are right it does not apply to Java.  I guess he meant that as a warning
that changing to the other side might be equally bad.

> > So do we want every Python program to bomb whenever there is a floating
> > point value outside bound and is not trapped by try/except?
> 
> By default, yes, Guido does, but for overflow, not underflow.

That's the harsh reality I have to live with, I suppose. <0.9wink>

>  If you read
> the following section of Kahan's paper, you should have noticed that he is
> *just* as critical of just returning Infs and NaNs (which is all Java does,
> and which is the accidental Python behavior you're arguing we keep in glibc)
> as he is of just trapping!  His conclusion:  "Without Traps nor Flags, Java’
> s floating-point is Dangerous":
> 
>     Without flags, detecting rare creations of Inf and NaN before they
>     disappear requires programmed tests and branches that, besides
>     duplicating tests already performed by the hardware, slow down the
>     program and impel a programmer to make decisions prematurely in many
>     cases. Worse, a plethora of tests and branches undermines a program’s
>     modularity, clarity and concurrency.
> 
> I agree with him there wholeheartedly:  without the 754-mandated flags,
> *and* without raising exceptions, the subset of 754 remaining sucks.

That is true, but all these problems apply to "without raising flags",
whether with or without "raising exceptions".  Silent overflow forces
programmers to do extra checking if they need to find them.  Mandatory
exception forces programmer to do extra checking if they don't want them to
stop the program.  Which is more common?  It depends on programming style.
In my programs, intentional NaN is far more than NaN by mistake.

As to whether it is dangerous, I would say that if an overflow is really an
error, good programming style requires that it be detected much earlier than
the actual overflow.  I agree this cannot be expected of everyone, though.

> >> ironically, 754 is hardest to sell to those who could benefit from
> >> it the most.
> 
> > So far the only concern from number-crunchers voiced here is that it might
> > allow sqrt(-1)==0, which is not the case.
> 
> Gimme a break!  In the very msg Paul Dubois said that, he also said "Some
> people use Inf but most people want the code to STOP so they can find out
> where the INFS started.  Otherwise, two hours later you have big arrays of
> Infs and no idea how it happened.".  I'm starting to drop my assumption of
> good faith on your part:  what you remember and how you paraphrase (whether
> it's Paul or Kahan) is extremely selective, beyond normal bounds of
> zeal-inspired distortion.

Well, I was talking about whether anyone has shown a concern to oppose 754.
I was not talking about whether exception is prefered to flags.  See the
line I was replying to?  Actually, thinking about it, Paul was not talking
about concerns over 754, either.  So there appears zero resistence to 754
from number crunchers here.

> > Obviously we agree on the benefit of having a choice.  But we
> > have different perspective on what choice means.  Your want a choice
> > to force user to take explicit action, I want a choice to ignore these
> > events.
> 
> No.  I want exactly the choices 754 lays out:  user-settable sticky flags
> and user-maskable exceptions.  We're not getting that for 2.0.  And we're
> *never* getting it if 2.0 doesn't do *something* to start disabusing people
> of the notion that their historical numeric platform accidents in Python
> will be faithfully preserved forever more.

Let me make it clear that we agree on all essential points about full 754
support.  The difference only exists in the period before that.  Since both
the 1.5.2 accidental behaviors and the one you want to unify to are just
subsets of the full support, it does not appear to me that disabling several
of them is a prerequisite for the full support.

> Kinda, except that each of the 5 754-defined traps must be individually
> maskable.  754 spells out what's needed.  A PEP will be required to spell
> out the Python- and C-level APIs.  I've corresponded (just -- no time now) a
> little with Kevin Jacobs about that offline.

Great!  If the true IEEE 754 support could come soon, all my gripe is moot.
So I'll stop disturbing you on the good work.  :-)

OK, I see that you want consistency above anything else before that good day
comes.  Well, consistency is good.  Whether the price is too high is a
subjective judgement.  So I'll just use the -lieee hack for myself, then.

[On presubstitution] 

Ok, I found it.  It looks good.

[On whether 754 default uses flags or exceptions] 

I agree with you: the system default matters very little as long as users
can set their own defaults easily.
 
> I expect we'll have to make InvalidOperationError a subclass of ValueError
> for that reason, when 754 support materializes.

That'll be handy.

> > My favorite is returning 1j, as MatPy does.
> 
> Then you should use cmath.sqrt instead of math.sqrt. 

Thanks.

Huaiyu