[Python-Dev] Pickle implementation questions

[Tim Peters]

[Bruce Christensen]
> So just to be clear, is it something like this?

I hope you've read PEP 307:

    http://www.python.org/dev/peps/pep-0307/

That's where __reduce_ex__ was introduced (along with all the rest of
pickle protocol 2).

> class object:
>     def __reduce__(self):
>         return copy_reg._reduce_ex(self, -1)
>
>     def __reduce_ex__(self, protocol):
>         return copy_reg._reduce_ex(self, protocol)

The implementation is more like:

class object:
    def __common_reduce__(self, proto=0):

        if self.__class__.__reduce__ is not object.__reduce__:
            # The class overrode __reduce__, so call the override.
            # From PEP 307:
            #     The 'object' class implements both __reduce__ and
            #     __reduce_ex__;  however, if a subclass overrides __reduce__
            #     but not __reduce_ex__,  the __reduce_ex__ implementation
            #     detects this and calls   __reduce__.
            return self.__reduce__()

        elif proto < 2:
            return copy_reg._reduce_ex(self, proto)

        else:
            # about 130 lines of C code exploiting proto 2

    __reduce__ = __reduce_ex__ = __common_reduce__

> Does _reduce_ex's behavior actually change depending on the specified protocol
> version? The only difference that I can see or think of is that an assert causes it to
> fail if the protocol is >= 2.

That's right.  As above, the object reduce methods never call
copy_reg._reduce_ex() when proto >= 2.

Note that __reduce_ex__ doesn't exist for the _benefit_ of object:  it
was introduced in protocol 2 for the benefit of user classes that want
to exploit protocol-specific pickle opcodes in their own __reduce__
methods.  They couldn't do that using the old-time __reduce__ because
__reduce__ wasn't passed the protocol version.

copy_reg._reduce_ex exists only because Guido got fatally weary of
writing mountains of C code, so left what "should be" a rarely-taken
path coded in Python.

>>>  - What does copy_reg.constructor() do?

>> It does this:
>>
>> def constructor(object):
>>     if not callable(object):
>>         raise TypeError("constructors must be callable")

> So it is part of the public interface? It's exported in __all__, but it appears
> that it's undocumented.

It's documented in the Library Reference Manual, in the `copy_reg` docs:

"""
constructor(object)

Declares object to be a valid constructor. If object is not callable
(and hence not valid as a constructor), raises TypeError.
"""

Unfortunately, while all the "safe for unpickling?" gimmicks (of which
this is one -- see PEP 307 again) were abandoned in Python 2.3, the
docs and code comments still haven't entirely caught up.
copy_reg.constructor() exists now only for backward compatibility (old
code may still call it, but it no longer has any real use).