__reduce__(1) vs __reduce__(2)

[Ziga Seilnacht]

Kirill Simonov wrote:
> Could someone explain why __reduce__(2) works for files while
> __reduce__(1) doesn't?

I think it is a bug. Both should raise an error.
__reduce__ and __reduce_ex__ are part of the pickle protocol.
Files are not meant to be pickable, since they are already
persistent. With protocol 0 and 1, an error is raised when
you try to pickle a builtin object that doesn't have a custom
__reduce__() method. However, protocol 2 changed that, and
now almost all objects can be pickled, but not necessarily
unpickled to their original state. Files are one such example:
you can pickle them, but the unpickled object won't be useful;
you will get uninitialised file object. Basically, the result
is equivalent to the following:
unpickled = file.__new__(file)

[snipped]
> What is a correct procedure of getting state and restoring Python objects?

You should never call __reduce__() with the protocol argument.
It works only for objects that don't override the default
__reduce__() method, and is possible only because
object.__reduce_ex__ and object.__reduce__ represent the same
underlying function. For example, sets override this method,
and don't expect the protocol argument:

>>> s = set([1,2,3])
>>> s.__reduce__(2)
Traceback (most recent call last):
  ...
TypeError: __reduce__() takes no arguments (1 given)

The correct way is to try to call obj.__reduce_ex__(protocol)
first, and fall back to obj.__reduce__() if that method is not
available. Example:

    if hasattr(obj, '__reduce_ex__'):
        state_tuple = obj.__reduce_ex__(2)
    else:
        state_tuple = obj.__reduce__()

For more details check the pickle protocol documentation:
http://docs.python.org/lib/pickle-protocol.html

as well as Extensions to the pickle protocol PEP:
http://www.python.org/dev/peps/pep-0307/

> -- 
> xi

Ziga