[Python-Dev] r88147 - in python/branches/py3k: Misc/NEWS Modules/_pickle.c Tools/scripts/find_recursionlimit.py

["Martin v. Löwis"]

> I am still curious why a previous exception changed pickle behavior, and
> only in 3.2, but I would rather you fix another bug than speeding much
> time to get me up to speed on the intricacies of _pickle ;-).

IIUC, the code change made pickle actually aware of the exception,
rather than just setting it in the thread state, but then happily
declaring that pickle succeeded (with what would turn out to be
incorrect data).

As for why an explicit exception breaks the reporting, and omitting
it makes it report the exception correctly:

the report that it gave wasn't actually correct. I got

raceback (most recent call last):
  File "a.py", line 4, in <module>
    for i in range(100):
RuntimeError: maximum recursion depth exceeded while pickling an object

So the exception is reported on the range call, or the for loop.
After the change, we get

Traceback (most recent call last):
  File "a.py", line 7, in <module>
    _pickle.Pickler(io.BytesIO(), protocol=-1).dump(l)
RuntimeError: maximum recursion depth exceeded while pickling an object

So it appears that the interpreter would actually pick up the exception
set by pickle, and attribute it to the for loop. When you add an
explicit raise, this raise will clear the stack overflow exception,
and set the new exception. So the error manages to pass silently,
without being explicitly silenced.

I wonder whether we could sprinkle more exception-set? checks in
the interpreter loop, at least in debug mode.

It's a design flaw in CPython that there are two ways to report
an exception: either through the thread state, or through the return
value. I don't think this flaw can be fully fixed. However, I
wonder whether static analysis of the C code could produce better
detection of this kind of bug.

Regards,
Martin