[Python-Dev] PEP 3154 - pickle protocol 4

[Xavier Morel]

On 2011-08-12, at 12:58 , Antoine Pitrou wrote:
> Current protocol versions export object sizes for various built-in types
> (str, bytes) as 32-bit ints.  This forbids serialization of large data
> [1]_. New opcodes are required to support very large bytes and str
> objects.
How about changing object sizes to be 64b always? Too much overhead for the
common case (which might be smaller pickled objects)? Or a slightly more
devious scheme (e.g. tag-bit, untagged is 31b size, tagged is 63), which
would not require adding opcodes for that?

> Also, dedicated set support
> could help remove the current impossibility of pickling
> self-referential sets [2]_.

Is there really no possibility of fix recursive pickling once
and for all? Dedicated optcodes for resource consumption
purposes (and to match those of other build-in types) is
still a good idea, but being able to pickle arbitrary
recursive structures would be even better would it not?

And if specific (new) opcodes are required to handle recursive
pickling correctly, that's the occasion.