[Python-Dev] Numerical robustness, IEEE etc.

[Nick Maclaren]

"Tim Peters" <tim.peters at gmail.com> wrote:
>
> I suspect Nick spends way too much time reading standards ;-)

God help me, YES!  And in trying to get them improved.  Both of which
are very bad for my blood pressure :-(

My real interest is in portable, robust programming - I DON'T abuse
the terms to mean bitwise identical, but that is by the way - and I
delved in here trying to write a jiffy little bit of just such code
as part of a course example.  BANG!!!  It failed in both respects on
the first two systems I tried on, and it wasn't my code that was wrong.

The killer is that standards are the nearest to a roadmap for portability,
especially portability and robustness.  If you have non-conforming code,
and it goes bananas, the compiler vendor will refuse to do anything, no
matter how clearly it is a bug in the compiler or library.  What is
worse is that there is an incentive for the leading vendors (see below)
to implement down to the standard, even when it is easier to do better.
And this is happening in this area.

> What he said is:
> 
>     If you look at floatobject.c, you will find it solid with constructions
>     that make limited sense in C99 but next to no sense in C89.
> 
> And, in fact, C89 truly defines almost nothing about floating-point
> semantics or pragmatics.  Nevertheless, if a thing "works" under gcc
> and under MS C, then "it works" for something like 99.9% of Python's
> users, and competitive pressures are huge for other compiler vendors
> to play along with those two.

Yup, though you mean gcc on an x86/AMD64/EM64T system, and 99.9% is a
rhetorical exaggeration - but one of the failures WAS on one of those! 

> I don't know what specifically Nick had in mind, and join the chorus
> asking for specifics.

That is why I wanted to:

   a) Read the decimal stuff and play around with the module
and:
   b) Write a sandbox and sort out my obvious errors
and:
   c) Write a PEP describing the issue and proposals

BEFORE going into details.  The devil is in the details, and I wanted
to leave him sleeping until I had lined up my howitzers ....

> I _expect_ he's got a keen eye for genuine
> coding bugs here, but also expect I'd consider many technically
> dubious bits of FP code to be fine under the "de facto standard"
> dodge.

Actually, I tried to explain that I don't have many objections to the
coding of the relevant files - whoever wrote them and I have a LOT of
common attitudes :-) And I have been strongly into de facto standards
for over 30 years, so am happy with them.  Yes, I have found a couple of
bugs, but not ones worth fixing (e.g. there is a use of x != x where
PyISNAN should be used, and a redundant test for an already excluded
case, but what the hell?)  My main objection is that they invoke C
behaviour in many places, and that is (a) mostly unspecified in C, (b)
numerically insane in C99 and (c) broken in practice.

> So, sure, everything we do is undefined, but, no, we don't really care
> :-)  If a non-trivial 100%-guaranteed-by-the-standard-to-work C
> program exists, I don't think I've seen it.

I can prove that none exists, though I would have to trawl over
SC22WG14 messages to prove it.  I spent a LONG time trying to get
"undefined" defined and used consistently (let alone sanely) in C, and
failed dismally.

> BTW, Nick, are you aware of Python's fpectl module?  That's
> user-contributed code that attempts to catch overflow, div-by-0, and
> invalid operation on 754 boxes and transform them  into raising a
> Python-level FloatingPointError exception.  Changes were made all over
> the place to try to support this at the time.  Every time you see a
> PyFPE_START_PROTECT or PyFPE_END_PROTECT macro in Python's C code,
> that's the system it's trying to play nicely with.  "Normally", those
> macros have empty expansions.

Aware of, yes.  Have looked at, no.  I have already beaten my head
against that area and already knew the issues.  I have even implemented
run-time systems that got it right, and that is NOT pretty.

> fpectl is no longer built by default, because repeated attempts failed
> to locate any but "ya, I played with it once, I think" users, and the
> masses of platform-specific #ifdef'ery in fpectlmodule.c were
> suffering fatal bit-rot.  No users + no maintainers means I expect
> it's likely that module will go away in the foreseeable future.  You'd
> probably hate its _approach_ to this anyway ;-)

Oh, yes, I know that problem.  You would be AMAZED at how many 'working'
programs blow up when I turn it on on systems that I manage - not
excluding Python itself (integer overflow) :-)  And, no, I don't hate
that approach, because it is one of the plausible ones; not good, but
what can you do?



Regards,
Nick Maclaren,
University of Cambridge Computing Service,
New Museums Site, Pembroke Street, Cambridge CB2 3QH, England.
Email:  nmm1 at cam.ac.uk
Tel.:  +44 1223 334761    Fax:  +44 1223 334679