Possible bug (was Re: numpy, overflow, inf, ieee, and rich comparison)

[Tim Peters]

[Tim]
> ...
> We cannot consider this to be a bug since Python has had no *intended*
> behavior whatsoever wrt 754 gimmicks.  We can and do consider gripes
> about these accidents to be feature requests.

[Huaiyu Zhu]
> Yes it varies widely in Python.  But isn't that precisely because Python
> does not support IEEE?

Yes, but that's darned near tautological <0.5 wink>.

> If Python does not actively undermine IEEE on platforms that have it,
> would it still vary widely across these platforms?

Yes.  Your own -lieee example showed that math.sqrt(-1) computes not only a
non-754 result, but a senseless result, on your platform under 1.5.2.  If I
show you a platform, how are you going to decide whether it "has" IEEE?  754
is a very involved std, and it's still rare to find a platform C that
supports it correctly.  On platform C's that do support it correctly, I know
of none that do so without requiring platform-specific tricks.  Python has
*nothing* portable it can rely on, and its mass of utterly non-754-aware
code building on haphazard C implementations adds up to the random
x-platform crap we have today.

...
[T]
> The patch to stop setting -lieee was contributed by a Python user who
> claimed it fixed bugs on *their* platform.  That's "the reason".
> We don't want to screw them either.

[H]
> Q: What bug?  Which platform?  Maybe there is another way that has less
>    impact on others?  How to find out?

I didn't check it in, and you can search the CVS archives as easily as I can
(I'm not inclined to, since I don't think it can affect the outcome of
this).  Jeremy Hylton may, or may not, remember more about it (IIRC, he said
he checked it in).

> ...
> If there is a way to fix the bug on that other platform while still keep
> -lieee on Linux, would that be acceptable?

No:  Guido *wants* overflow to raise OverflowError by default, and *wants*
sqrt(-1.0) to raise ValueError by default.  Python's x-platform fp numerics
today are an utter mess, and the best we can do for 2.0 is to make them as
consistent as we have time to make them, despite that we know it will break
some of your code (but you can still link with -lieee if you feel you must),
and that Mark Favas has confirmed it will break code under both his Solaris
and Tru64 Unix systems (but for a reason opposite of yours:  he's been
getting OverflowError on underflow since 1.5.2, and we're taking that away
from him, and he'll have *no* way to go back).  At this point we favor
forcing consistency over maintaining platform accidents, no matter how
attached people have gotten to them; else we'll be stuck with them forever.

[oh some of Kahan's papers, at http://http.cs.berkeley.edu/~wkahan/]
> Interesting! I've glanced through some of Prof Kahan's writings.

He is an entertaining writer, although his rhetoric is often more extreme
than either of ours <wink>.

> It appears that he favors raising flags that can be checked optionally,
and
> he is against mandatary raising exceptions.

Have you read the 754 std?  He's in favor of the 754 std.

[quotes snipped, except for my favorite, which I've paraphrased many times
 over the years on c.l.py]
> ...
>   They are called "Exceptions" because to any policy for handling them,
>   imposed in advance upon all programmers by the computer system, some
>   programmers will have good reason to take exception.

> ...
> From these quotes and others I understand that he wants f.p. exceptions to
> be merely messanges, not errors that force programmers to take extra
> actions, unless the programmer choose to.  So what aspect of Java did he
> think is deficient?  It is the "Trap" style exception that mandate a users
> action.

His paper "How JAVA's Floating-Point Hurts Everyone Everywhere" lists his 5
main gripes about Java in bullet points at the top of page 3.  The paper
runs on to 80 pages.  You somehow managed to miss 79.89 of them <0.4 wink>,
reducing it to a bogus claim about Java that Kahan didn't make (Java has
neither "Trap" nor "Flag" style fp exception signaling).

> He told "A cautionary Tale of the Ariane 5",

Which has nothing to do with Java -- the software in question was written in
Ada, which (unlike Java) has "Trap" style.  Why he put it in a paper about
Java is a bit of a mystery; I write it off to rhetorical excess.

> the European rocket that mulfunctioned after lift off.  It turned out
> that a piece of software produced an unimportant exception that is
> not caught by a handler and caused debugging data to be dumped to a
> critical memory location.  This would have been avoided if the exception
> merely raised a flag and returned a NaN, and the program continued.

Actually, that was impossible to do, and he didn't suggest that.  What he
wrote is

    Had overflow merely obeyed the IEEE 754 default policy, the
    recalibration software would have raised a flag and delivered an
    invalid result both to be ignored by the motor guidance programs,
    and the Ariane 5 would have pursued its intended trajectory.

Note that he didn't say NaN.  He couldn't, because this was overflow in a
conversion from Float to Integer.  Despite his cheery optimism here, 754 is
approximately useless in this case, since there is no generally accepted
"invalid result" in integer formats (and 754 doesn't define one either --
the result of overflow in converting floats to ints is ill-defined in 754;
754 doesn't even require that it set the overflow flag).

> So do we want every Python program to bomb whenever there is a floating
> point value outside bound and is not trapped by try/except?

By default, yes, Guido does, but for overflow, not underflow.  If you read
the following section of Kahan's paper, you should have noticed that he is
*just* as critical of just returning Infs and NaNs (which is all Java does,
and which is the accidental Python behavior you're arguing we keep in glibc)
as he is of just trapping!  His conclusion:  "Without Traps nor Flags, Java’
s floating-point is Dangerous":

    Without flags, detecting rare creations of Inf and NaN before they
    disappear requires programmed tests and branches that, besides
    duplicating tests already performed by the hardware, slow down the
    program and impel a programmer to make decisions prematurely in many
    cases. Worse, a plethora of tests and branches undermines a program’s
    modularity, clarity and concurrency.

I agree with him there wholeheartedly:  without the 754-mandated flags,
*and* without raising exceptions, the subset of 754 remaining sucks.

>> ironically, 754 is hardest to sell to those who could benefit from
>> it the most.

> So far the only concern from number-crunchers voiced here is that it might
> allow sqrt(-1)==0, which is not the case.

Gimme a break!  In the very msg Paul Dubois said that, he also said "Some
people use Inf but most people want the code to STOP so they can find out
where the INFS started.  Otherwise, two hours later you have big arrays of
Infs and no idea how it happened.".  I'm starting to drop my assumption of
good faith on your part:  what you remember and how you paraphrase (whether
it's Paul or Kahan) is extremely selective, beyond normal bounds of
zeal-inspired distortion.

> I was a fortran programmer before.  I switched to using IEEE ten years
> ago as soon as it was available on the machine I was using.  So there
> are different opinions.

Certainly, and good 754 support will cater to most of them.  I don't believe
your opinion (or mine, for that matter) reflects anything even close to the
majority view here, though.  As I said in my reply to Paul, my observations
(based on 15 years on the vendor side of the fp biz) matched his exactly
(based on his experience with hundreds of numeric code authors):  a vast
majority still want divide-by-0, invalid operation, and overflow to raise an
exception the vast majority of the time.

> ...
> Obviously we agree on the benefit of having a choice.  But we
> have different perspective on what choice means.  Your want a choice
> to force user to take explicit action, I want a choice to ignore these
> events.

No.  I want exactly the choices 754 lays out:  user-settable sticky flags
and user-maskable exceptions.  We're not getting that for 2.0.  And we're
*never* getting it if 2.0 doesn't do *something* to start disabusing people
of the notion that their historical numeric platform accidents in Python
will be faithfully preserved forever more.

> The ideal solution might be to have a sys.fp_exception class.  Each
> exception would set a flag like fp_exception.overflow so that user can
> check them.  There could also be a variable like fp_exception.force_trap,
> which when set, could make each f.p. exception to produce a Python
> exception.  This would confirm to IEEE 754 and give both of us a choice
> we want.

Kinda, except that each of the 5 754-defined traps must be individually
maskable.  754 spells out what's needed.  A PEP will be required to spell
out the Python- and C-level APIs.  I've corresponded (just -- no time now) a
little with Kevin Jacobs about that offline.

> Now that we are in feature freeze for 2.0, we can't get both choices.
> Can we adopt the one that does not break behaviors on previous official
> release, even if that's only by accident?

Not as far as Guido or I are concerned, no.  You can link with -lieee
yourself if you must, although I can't recommend perpetuating the x-platform
confusion here.  We want to say that, e.g., math.exp works substantially the
same way on *all* platforms under 2.0, not repeat the same decade-old "umm,
don't know, and no way to guess -- try it on your platform and see what it
does" yet again.  I don't expect 2.0 will actually achieve that on all
platforms, but it should on the primary ones.

> ...
> So now I know why people reported that NaN and Inf were broken on HPUX
> and Windows and some others.  If Python had simply ignored the flags ...

Then we would have broken 754 subsets in slightly subtler ways, and still
wildly varying across platforms.

> ...
> Could you please point a URL to "presubstitution"?  In the paragraphs I
> quoted the only mention of presubstitute is with NaNs and Inf, etc.

Sorry, I don't save URLs, and don't recall whether this was in a paper or on
David Hough's "Numeric Interest" mailing list in the early 90's.  A Google
search on "presubstitution Kahan" turns up 4 promising-looking hits I'll
leave you to pursue.  The basic idea is that the programmer gets to specify,
via library calls (whatever), which particular fp value each 754 exceptional
case returns.  So, e.g., if it made sense for some part of the app, the
programmer could say they wanted +- pi returned instead +- Inf, if and
whenever overflow happened.  Then set it back to +- Inf (or anything else
they needed) later.  This would be best if supported in HW directly, of
course.

> ...
> You are not confirming to 754 until you allow all the exceptions to go
> through with well defined values as default.

I know that, but don't care.  As I said, enabling overflow + invalid + div0
by default is the only scheme that can be *sold*.  If you can find two NumPy
users in favor of disabling them all by default, I'll be mildly astonished.
754 did itself enormous harm by insisting on that trivial point (it's
trivial because, with proper support, anyone can change the defaults to
anything they like with one line of code):  I worked in the supercomputer
biz at the time 754 was adopted, and for a decade after, and the "non stop"
defaults were *universally* opposed by customers.  Scared the hell out of
language stds committees too, which goes a lot farther than the 754 folks
would like to admit toward explaining why language committees stayed away
from 754 bindings in droves.  C99 is 15 years(!) after the fact.

> Raising Python exceptions on some of them does not confirm to 754, as
> the default IEEE 754 behavior on f.p. exception is not a Python exception,
> but just a raised flag.

See above.

> Before that happens, what is the motive to make IEEE less usable for those
> who have it (pretty much every one nowadays), even if just by accident?

I think I've repeated the reasons to death already, and you've repeated to
death that you'll never agree.  So no more from me on that.

> ...
> But just to clarify some wordings you quoted in another post: I meant that
> sqrt(-1)==0 is insane, whether silent or not.

I agree.

> Raising an exception is sane,

Not according to 754's default rules, which you wanted me to take as sacred
just a few paragraphs ago <wink>.

> although OverflowError is not so reasonable.

I'd say it's insane.

> Raising a ValueError is reasonable.

I expect we'll have to make InvalidOperationError a subclass of ValueError
for that reason, when 754 support materializes.

> But just returning NaN without a fuss is even better.

You're going to find that's a very lonely position.

> My favorite is returning 1j, as MatPy does.

Then you should use cmath.sqrt instead of math.sqrt.  Even $3 business
calculators have a sqrt key these days, and you do not want to be on the
other end of email when trying to explain to a businessperson why their
crappy std deviation program returned a complex number <0.1 wink>.

> if-it-ain't-broke-don't-fix-it-ly y'rs

it-it-weren't-broke-we-wouldn't-ly y'rs  - tim