Some Q&A with Kahan on IEEE 754

[Kevin Jacobs]

Huaiyu Zhu <huaiyu_zhu at yahoo.com> wrote:
> In the past few days I asked Prof Kahan several questions about IEEE 754.

I'm glad to see that Dr. Kahan's opinions are in line with what _everyone_
here wants.  I am drafting a PEP and reference implementation that embodies
these ideas and goals:

  1) Support for setting and testing special IEEE 754 floating-point values
     (i.e. NaNS, NaNQ, Inf, denormals).

  2) Flexible delivery and recovery from floating point traps:

	These will be deliverable as Python exceptions, handled by a
	user-specified Python function or queried by sticky status bits. 
	All options will be setable, maskable, and mergeable at runtime.

  3) A reasonable default set of IEEE 754 traps enabled.

	Current plan: Invalid OP, Divide-by-zero, and overflow.

  4) Rounding mode and precision control if supported by the platform.

  5) Thread specific handling of traps with creation-time inheritance of
     settings.

  6) Integration with Numeric Python, MatPy, and other common numerical
     extensions via sensible additions to the Python and C APIs.
 
  6) A practical software-only emulation of some of these features for
     non-IEEE754 systems that doesn't kill performance.

  7) Available in the Python 2.1 time-frame, assuming consensus can be
     reached, problems solved, and damsels saved.

Of the many issues to be resolved, here are a few things to consider:

  1) Handling of signaling NaNs.  This can get expensive since it involves a
     test every time a floatobject is used or requires mucking about with
     signal handlers.

  2) Speaking of signal handlers, Tim Peters stated that returns are not
     allowed from sIGFPE handlers.  I have yet to see this restriction in
     any platform documentation.  What is the deal?

  3) Thread support -- Python threading will have to know about
     per-thread floating point data-structures.  There are also implications
     for uthreads.

  4) I am going to need help implementing support for many of the commonly
     used Python platforms.  I have access to Windows 95/98/NT/2000,
     Compaq/Digital Unix, AIX (Power3), IRIX, Linux (i386+glibc2.1+), and
     Solaris.  Edward Jason Riedy has offered to help with AIX, Linux,
     Solaris and Unicos/mk.  We still need volunteers for the following:

	Windows
	OS/2
	BeOS
	VxWorks
	OpenVMS
	QNX
	
	Others?  Speak now!


Just a few comments:

> [Huaiyu Zhu]
>> 2. Enable IEEE whenever possible.  Each f.p. exception produces a Python
>>    exception, which would halt the program unless trapped and handled.

[Dr. Kahan]
>      This is an option worth integrating with a debugger,  but it must not
>      be the default.  Do you really wish to see Python programmers trying
>      to program their own trap handlers?

Actually, I do want to see Python programmers programming their own trap
handlers... in Python!  It'll be slow, but very much in line with the
dynamic nature of Python.  ;)


> [Huaiyu Zhu]
>> 4. Enable IEEE whenever possible.  Let a special module contain all the
>>    flags, without changing program flow.  Programers could choose to
>>    examine these flags or choose to ignore them.

[Dr. Kahan]
>      Yes; and provide quick and convenient ways to save-and-restore and
>      merge exception flags.  Other facilities should be provided too, but
>      I cannot spare the time just now to discuss them in detail.

I think the problem here is in the definition of "Enable IEEE".  Your
meaning seems to be "Enable IEEE" == "Enable all/some IEEE 754 traps". My
meaning, and I believe the standard one is "use features available in
conforming IEEE 754 implementations." This is, by necessity, a superset of
the generic C floating point support (with a few notable, but minor
exceptions).

> [Huaiyu Zhu]
>> The current problem with IEEE 754 support in Python is that, at least as
>> it is perceived, there is no general way to turn it on on every
>> 754-enabled platform and expect to obtain the same kind of signalling,
>> etc.

Yes and no.  No, there is no general way to support IEEE 754 in Python.  Its
still a bloody mess in terms of APIs.  However, I think there is much we can
do to provide as much as possible, for as many platforms as possible, in the
near future.

> [Huaiyu Zhu]
>> Another problem is that the leading authorities of the language plan to
>> unify the behavior in such a way as to raise exceptions for Overflow,
>> Divide-by-Zero, Invalid Operation, while letting through Underflow and
>> Inexact Operations. They consider this as an interim step toward full
>> IEEE 754 support.  Since Python exceptions have to be caught, this would
>> make life extremely difficult for vectorized computations during this
>> period.

In the short term, i.e. Python 2.0, yes.  Its simply not feasible to do all
that is necessary (not the least of which write a PEP, implement it and test
for n platforms) before Python 2.0 goes out.  While important, this isn't
important enough to even consider delaying 2.0.


I look forward to working with you to address these issues (then I can get
back to working on statistical extensions to Python -- my real goal for all
this silliness!)


Regards,
-Kevin Jacobs

-- 
----------->  Kevin Jacobs  <-----------|------->  (216) 778-8211  <--------
Informatics Consultant                  | Department of Epidemiology
Primary mail:   jacobs at theopalgroup.com |   & Biostatistics
Alternate mail: jacobs at darwin.cwru.edu  | Case Western Reserve University
----------------------------------------------------------------------------