[Python-Dev] PyMarshal_WriteLongToFile: writes 'long' or 32-bitinteger?

[Tim Peters]

[GregS]
> It isn't only bytecodes. As I mentioned before, marshal byte streams are
> specified as portable. Therefore, they are used all the time for passing
> over the network, for persistence mechanisms, etc.
>
> Having a marshal stream that is *not* cross-platform is just plain busted.
>
> So... you can silently truncate to 32-bits or you can raise an exception.

But in either of those cases it's not cross-platform == it's just plain
busted.

> You cannot, however, marshal a 64-bit value when the definition of an
> "marshal integer" is only 32.

The current code is insane.  Long-term the visible distinction between
(Python) ints and longs should vanish.  Short-term the PyInt_Check case of
w_object should see whether the int fits in 4 bytes, and if so write it as a
C long, and if not write the thing as a Python long (that's the only way to
keep the marshal portable!).  Any code keying off the SIZEOF_LONG symbol is
highly suspect, piling another layer of "convenient but wrong" assumptions
on top of the current layer.  Code in marshal should *never* try to exploit
that C longs happen to be bigger than 4 bytes on the current platform.
PyMarshal_WriteLongToFile should be changed to raise an exception if passed
something that doesn't fit in 4 bytes, and should be declared obsolete.  The
Python core should be changed now never to call PyMarshal_WriteLongToFile
(in answer to Trent's original question, the intent of that code can't be
distinguished from what it does, which is to blindly write exactly 4 bytes
no matter what).

Most of this crap is in support of a bad assumption about mtimes, and,
sorry, but it sure looks to me like the .pyc header format is "just plain
busted" too.  Let's bite the bullet and fix the header -- then all the rest
of this will be clearly seen as the broken hackery it really is.