pickle broken: can't handle NaN or Infinity under win32

[Grant Edwards]

I finally figured out why one of my apps sometimes fails under
Win32 when it always works fine under Linux: Under Win32, the
pickle module only works with a subset of floating point
values.  In particular the if you try to dump/load an infinity
or nan value, the load operation chokes:

  
Under Linux:

  $ python
  Python 2.3.4 (#2, Feb  9 2005, 14:22:48) 
  [GCC 3.4.1 (Mandrakelinux 10.1 3.4.1-4mdk)] on linux2
  Type "help", "copyright", "credits" or "license" for more information.
  >>
  
  $ python pickletest.py
  (inf, nan) (inf, nan)

  
Under Win32:

  $ python
  ActivePython 2.3.4 Build 233 (ActiveState Corp.) based on
  Python 2.3.4 (#53, Oct 18 2004, 20:35:07) [MSC v.1200 32 bit (Intel)] on win32
  Type "help", "copyright", "credits" or "license" for more information.
  >>
  
  $ python pickletest.py
  Traceback (most recent call last):
    File "pickletest.py", line 8, in ?
      d = pickle.loads(s)
    File "C:\PYTHON23\lib\pickle.py", line 1394, in loads
      return Unpickler(file).load()
    File "C:\PYTHON23\lib\pickle.py", line 872, in load
      dispatch[key](self)
    File "C:\PYTHON23\lib\pickle.py", line 968, in load_float
      self.append(float(self.readline()[:-1]))
  ValueError: invalid literal for float(): 1.#INF

I realize that this is probably due to underlying brokenness in
the Win32 libc implimentation, but should the pickle module
hide such platform-dependancies from the user?

Best case, it would be nice if pickle could handle all floats
in a portable way.

Worst case, shouldn't the pickle module documentation mention
that pickling floats non-portable or only partially implimented?

On a more immediate note, are there hooks in pickle to allow
the user to handle types that pickle can't deal with?  Or, do I
have to throw out pickle and write something from scratch?

[NaN and Infinity are prefectly valid (and extremely useful)
floating point values, and not using them would require huge
complexity increases in my apps (not using them would probably
at least triple the amount of code required in some cases).]

-- 
Grant Edwards                   grante             Yow!  Yow!
                                  at               
                               visi.com