[ python-Bugs-1742889 ] Pickling of exceptions broken

[SourceForge.net]

Bugs item #1742889, was opened at 2007-06-25 14:43
Message generated for change (Comment added) made by dcjim
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1742889&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Jim Fulton (dcjim)
Assigned to: Nobody/Anonymous (nobody)
Summary: Pickling of exceptions broken

Initial Comment:

Exceptions with required initialization arguments can't be unpickled:

>>> class E(Exception):
...     def __init__(self, x):
...         self.x = x
... 
>>> import pickle
>>> e = E(1)
>>> p = pickle.dumps(e, 1)
>>> pickle.loads(p)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/python/2.5.1/lib/python2.5/pickle.py", line 1374, in loads
    return Unpickler(file).load()
  File "/usr/local/python/2.5.1/lib/python2.5/pickle.py", line 858, in load
    dispatch[key](self)
  File "/usr/local/python/2.5.1/lib/python2.5/pickle.py", line 1133, in load_reduce
    value = func(*args)
TypeError: __init__() takes exactly 2 arguments (1 given)

This is because __reduce__ defined in exceptions.c returns the type and the args variable, which an exception subclass might not populate.

Also, the reduce implementation doesn't properly serialize the message attribute.

I assume that the need for a custom reduce is due to the micro-optimization to store arge and message in C slots.  Is this really necessary?


----------------------------------------------------------------------

>Comment By: Jim Fulton (dcjim)
Date: 2007-06-25 15:57

Message:
Logged In: YES 
user_id=73023
Originator: YES

I'll note that I think the right thing to do is to:

- Take args and message out of the C struct.

- inherit the default reduce behavior from object.


----------------------------------------------------------------------

Comment By: Jim Fulton (dcjim)
Date: 2007-06-25 15:53

Message:
Logged In: YES 
user_id=73023
Originator: YES

I'm not aware of any such contract. Can you point to anything in writing?

See for example:

file:///home/jim/Documentation/Python-Docs-2.4.1/tut/node10.html#SECTION0010500000000000000000

which teaches people to create custom exceptions that:

- don't set args ro message and

- won't be unpicklable in Python 2.5.

Also, as I mentioned, the reduce implementation doesn't preserve the
message, so even if that was the contract, the contract is broken.


----------------------------------------------------------------------

Comment By: Georg Brandl (gbrandl)
Date: 2007-06-25 15:26

Message:
Logged In: YES 
user_id=849994
Originator: NO

AFAIR we were told that filling args and message is part of the exception
contract...

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1742889&group_id=5470