pickle broken: can't handle NaN or Infinity under win32

[Scott David Daniels]

Grant Edwards wrote:
> I finally figured out why one of my apps sometimes fails under
> Win32 when it always works fine under Linux: Under Win32, the
> pickle module only works with a subset of floating point
> values.  In particular the if you try to dump/load an infinity
> or nan value, the load operation chokes:
There is no completely portable way to do this.  Any single platform
can have a solution, but (since the C standards don't address how
NaNs and Infs are represented) there is not a good portable way to do
the pickle / unpickle.  It is nice the exception is raised, since at
one point it was not (and a simple 1.0 was returned).
See explanations in article 654866:

http://sourceforge.net/tracker/index.php?func=detail&aid=714733&group_id=5470&atid=105470

>   $ python pickletest.py
>   Traceback (most recent call last): ...
>     File "C:\PYTHON23\lib\pickle.py", line 968, in load_float
>       self.append(float(self.readline()[:-1]))
>   ValueError: invalid literal for float(): 1.#INF

> I realize that this is probably due to underlying brokenness in
> the Win32 libc implimentation, but should the pickle module
> hide such platform-dependancies from the user?
As mentioned above, there is no C standard-accessible way to
predictably build or represent NaNs, negative zeroes, or Infinities.

> [NaN and Infinity are prefectly valid (and extremely useful)
> floating point values, and not using them would require huge
> complexity increases in my apps (not using them would probably
> at least triple the amount of code required in some cases).]

You could check to see if the Python 2.5 pickling does a better
job.  Otherwise, you've got your work cut out for you.

-Scott David Daniels
Scott.Daniels at Acm.Org