UnpicklingError: NEWOBJ class argument isn't a type object

[dieter]

skunkwerk <skunkwerk at gmail.com> writes:

> Hi,
>   I'm using a custom pickler that replaces any un-pickleable objects (such as sockets or files) with a string representation of them, based on the code from Shane Hathaway here:
> http://stackoverflow.com/questions/4080688/python-pickling-a-dict-with-some-unpicklable-items
>
> It works most of the time, but when I try to unpickle a Django HttpResponse, I get the following error:
> UnpicklingError: NEWOBJ class argument isn't a type object
>
> I have no clue what the error actually means.

The pickling protocol uses a form of bytecode which is executed
during the unpickling to reconstruct the python objects based
on their state found in the pickle alongside the bytecode.

"NEWOBJ" is executed in response to such a bytecode operation.
It expects to get a type as a parameter but in your case,
it gets something else.


> If it pickles okay, why should it not be able to unpickle?  Any ideas?

It is by principle impossible for the pickler to garantee
that an unpickler will later succeed: the pickler does not know
which classes/types are available for the unpickler.

In your special case, the pickler could probably
detect that unpickling will fail - but when
an aim cannot be achieved completely this may provide
motivation to abandon it as a whole - and not put much effort
into a partial achievement.
I have seen many cases where pickling succeeded but
unpickling failed and in principle the pickler could have
already predicted the failure (under the assumption that
the unpickler sees the same classes/types as the pickler).


If it is important for you to get Django HttpResponses
successfully unpickled then you likely need to guide
their pickling process better.

Maybe (as an alternative), you can extract the relevant information
from the "HttpResponse" and pickle that instead of
the response itself?