UnpicklingError: NEWOBJ class argument isn't a type object

[Peter Otten]

skunkwerk wrote:

> On Monday, July 8, 2013 12:45:55 AM UTC-7, Peter Otten wrote:
>> skunkwerk wrote:
>> 
>> 
>> 
>> > Hi,
>> 
>> >   I'm using a custom pickler that replaces any un-pickleable objects
>> >   (such
>> 
>> >   as sockets or files) with a string representation of them, based on
>> >   the
>> 
>> >   code from Shane Hathaway here:
>> 
>> > http://stackoverflow.com/questions/4080688/python-pickling-a-dict-with-
>> 
>> some-unpicklable-items
>> 
>> > 
>> 
>> > It works most of the time, but when I try to unpickle a Django
>> 
>> > HttpResponse, I get the following error: UnpicklingError: NEWOBJ class
>> 
>> > argument isn't a type object
>> 
>> > 
>> 
>> > I have no clue what the error actually means.  If it pickles okay, why
>> 
>> > should it not be able to unpickle?  Any ideas?
>> 
>> 
>> 
>> A simple way to provoke the error is to rebind the name referring to the
>> 
>> class of the pickled object:
>> 
>> 
>> 
>> >>> import cPickle
>> 
>> >>> class A(object): pass
>> 
>> ...
>> 
>> >>> p = cPickle.dumps(A(), -1)
>> 
>> >>> cPickle.loads(p)
>> 
>> <__main__.A object at 0x7fce7bb58c50>
>> 
>> >>> A = 42
>> 
>> >>> cPickle.loads(p)
>> 
>> Traceback (most recent call last):
>> 
>>   File "<stdin>", line 1, in <module>
>> 
>> cPickle.UnpicklingError: NEWOBJ class argument isn't a type object
>> 
>> 
>> 
>> You may be doing something to that effect.
> 
> Hey Peter,
>   I tried unpickling even from another file with no other code in it, but
>   came up with the same error - so I don't think it's a rebinding issue.
> 
> But I got the error to disappear when I removed the "hasattr(obj,
> '__getstate__')" from this line of code in the persistent_id function: if
> not hasattr(obj, '__getstate__') and isinstance(obj,(basestring, bool,
> int, long, float, complex, tuple, list, set, dict)):
>     return ["filtered:%s" % type(obj)]
> 
> When I do that, I get a few more FilteredObjects in the result, for things
> like: <class 'django.core.handlers.wsgi.WSGIRequest'>
> <class 'MySQLdb.connections.Connection'>
> 
> I figured these classes must have __getstate__ methods which leads to them
> being pickled without a persistent_id (it turns out they actually have
> __repr__ methods).
> 
> So these classes get pickled fine, but run into problems when trying to
> unpickle them.  I understand why ImportErrors would happen if the
> necessary modules haven't been loaded, but this NEWOBJ error is still kind
> of mystifying. I guess I just won't pickle any classes for now, if
> unpickling them is going to be dicey.
> 
> thanks for the help guys,
> imran

Maybe you can find the problem by temporarily switching from cPickle to the 
pickle module which produces a slightly more helpful traceback:

>>> import cPickle, pickle
>>> class A(object): pass
... 
>>> p = cPickle.dumps(A(), -1)
>>> A = 42
>>> cPickle.loads(p)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
cPickle.UnpicklingError: NEWOBJ class argument isn't a type object
>>> pickle.loads(p)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/pickle.py", line 1382, in loads
    return Unpickler(file).load()
  File "/usr/lib/python2.7/pickle.py", line 858, in load
    dispatch[key](self)
  File "/usr/lib/python2.7/pickle.py", line 1083, in load_newobj
    obj = cls.__new__(cls, *args)
TypeError: int.__new__(X): X is not a type object (int)