Bad marshal data

[Alex Martelli]

Michael McGarry <michael.mcgarry at gmail.com> wrote:

> I am using the marshal module in python to save a data structure to a
> file. It does not appear to be portable. The data is saved on a Linux
> machine. Loading that same data on a Mac gives me the bad marshal data
> message.

If you're using identical versions of Python and exactly the same bits
in both cases, this should not happen.  For example, I run on Linux,
with Python 2.4.1:

>>> foo=open('foo','wb')
>>> marshal.dump({1:23, 2:[3,4]}, foo)
>>> foo.close()

Then I scp foo to my Mac, and there, again with Python 2.4.1:

>>> import marshal
>>> marshal.load(open('foo', 'rb'))
{1: 23, 2: [3, 4]}

It would be OK if either version was, say, 2.4.2 (compatibility is
guaranteed across bugfix-only releases), but not, e.g., 2.3.5 (no
guarantee across minor-releases).


> Is this data really not portable or I am doing something wrong here. I
> thought the whole point of using Python and similar cross platform
> scripting languages to write once run everywhere, does that not hold
> for data too?

If you need portability across Python releases, you can use pickle.
marshal is a lower-level module, intended for simple (essentially
elementary) data, to be serialized with min size, max speed.

> Marshal should save the data in a readable text format, but I guess it
> does not.

It most definitely should *NOT* use "readable text format", as that
would make it impossible to minimize size and maximize speed.  pickle
uses an (unreadable but portable) text format by default, but the binary
format (and even more, the mode 2, most-modern pickle format) are
generally better, unless you need portability to very old Python
releases.  (sharing binary data across machines is no big deal anyway;
and you can always b64-encode it if you must send it by email...).


Alex