[New-bugs-announce] [issue28925] Confusing exception from cPickle on reduce failure

[Raoul Gough]

New submission from Raoul Gough:

Description
===========

Running the attached example program with Python 2.7.12 produces the output below. The demo deliberately raises a user-defined exception during the unpickling process, but the problem is that this exception does not propagate out of the unpickle call. Instead, it gets converted into a TypeError which is confusing and does not help identify the original problem.

INFO:root:Creating BadReduce object
INFO:root:Pickling
INFO:root:Unpickling
INFO:root:Raising exception "BadReduce init failed"
Traceback (most recent call last):
  File "cpickle_reduce_failure.py", line 48, in <module>
    main()
  File "cpickle_reduce_failure.py", line 41, in main
    pickler.loads(s1)
  File "cpickle_reduce_failure.py", line 27, in __init__
    raise exception
TypeError: __init__() takes exactly 2 arguments (4 given)

If you change the demo to use the Python pickle module, i.e. "import pickle as pickler", it produces the expected output below:

INFO:root:Creating BadReduce object
INFO:root:Pickling
INFO:root:Unpickling
INFO:root:Raising exception "BadReduce init failed"
INFO:root:Got MyException "BadReduce init failed"
INFO:root:Done

Analysis
========

The following code in Modules/cPickle.c in the function Instance_New (around https://github.com/python/cpython/blob/2.7/Modules/cPickle.c#L3917) does a PyErr_Restore with the exception type MyException, as raised from BadReduce.__init__, but it replaces the exception value with a tuple (original_exception, cls, args):

        PyObject *tp, *v, *tb, *tmp_value;

        PyErr_Fetch(&tp, &v, &tb);
        tmp_value = v;
        /* NULL occurs when there was a KeyboardInterrupt */
        if (tmp_value == NULL)
            tmp_value = Py_None;
        if ((r = PyTuple_Pack(3, tmp_value, cls, args))) {
            Py_XDECREF(v);
            v=r;
        }
        PyErr_Restore(tp,v,tb);

Later, PyErr_NormalizeException attempts to convert the exception value (the tuple) to the original exception type. This fails because MyException.__init__() can't handle the multiple arguments. This is what produces the TypeError "__init__() takes exactly 2 arguments (4 given)"


Proposed Fix
============

I think it would be better if Instance_New did the PyErr_NormalizeException itself instead of allowing it to be done lazily. If the normalize works, i.e. the exception type accepts the extra arguments, the behaviour would be almost unchanged - the only difference being the PyErr_NormalizeException happens earlier. However, if the normalize fails, Instance_New can restore the original exception unchanged. That means that we lose the cls, args information in this case, but not the original exception.

----------
components: Library (Lib)
files: cpickle_reduce_failure.py
messages: 282795
nosy: raoul_gough_baml
priority: normal
severity: normal
status: open
title: Confusing exception from cPickle on reduce failure
type: behavior
versions: Python 2.7
Added file: http://bugs.python.org/file45822/cpickle_reduce_failure.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28925>
_______________________________________