in a pickle

[duncan smith]

On 06/03/2019 16:14, duncan smith wrote:
> Hello,
>       I've been trying to figure out why one of my classes can be
> pickled but not unpickled. (I realise the problem is probably with the
> pickling, but I get the error when I attempt to unpickle.)
> 
> A relatively minimal example is pasted below.
> 
> 
>>>> import pickle
>>>> class test(dict):
> 	def __init__(self, keys, shape=None):
> 		self.shape = shape
> 		for key in keys:
> 			self[key] = None
> 
> 	def __setitem__(self, key, val):
> 		print (self.shape)
> 		dict.__setitem__(self, key, val)
> 
> 		
>>>> x = test([1,2,3])
> None
> None
> None
>>>> s = pickle.dumps(x)
>>>> y = pickle.loads(s)
> Traceback (most recent call last):
>   File "<pyshell#114>", line 1, in <module>
>     y = pickle.loads(s)
>   File "<pyshell#111>", line 8, in __setitem__
>     print (self.shape)
> AttributeError: 'test' object has no attribute 'shape'
> 
> 
> I have DUCkDuckGo'ed the issue and have tinkered with __getnewargs__ and
> __getnewargs_ex__ without being able to figure out exactly what's going
> on. If I comment out the print call, then it seems to be fine. I'd
> appreciate any pointers to the underlying problem. I have one or two
> other things I can do to try to isolate the issue further, but I think
> the example is perhaps small enough that someone in the know could spot
> the problem at a glance. Cheers.
> 
> Duncan
> 

OK, this seems to  be a "won't fix" bug dating back to 2003
(https://bugs.python.org/issue826897). The workaround,


class DictPlus(dict):
  def __init__(self, *args, **kwargs):
    self.extra_thing = ExtraThingClass()
    dict.__init__(self, *args, **kwargs)
  def __setitem__(self, k, v):
    try:
      do_something_with(self.extra_thing, k, v)
    except AttributeError:
      self.extra_thing = ExtraThingClass()
      do_something_with(self.extra_thing, k, v)
    dict.__setitem__(self, k, v)
  def __setstate__(self, adict):
    pass


doesn't work around the problem for me because I need the actual value
of self.shape from the original instance. But I only need it for sanity
checking, and under the assumption that the original instance was valid,
I don't need to do this when unpickling. I haven't managed to find a
workaround that exploits that (yet?). Cheers.

Duncan