pickling subclasses of dict/list

[Greg Chapman]

On Sat, 03 Jul 2004 14:37:28 -0400, Edward Loper
<edloper at gradient.cis.upenn.edu> wrote:

>I'm having trouble pickling subclasses of dict when they contain cycles. 
>
>Note that pickling dict's that contain cycles works just fine, as does 
>pickling classes that contain cycles in their instance variables.  E.g.:
>
>     >>> d = {}
>     >>> d[1] = d
>     >>> print d
>     >>> pickle.dump(d, open('d.pickle', 'w'))
>     >>> print pickle.load(open('d.pickle'))
>     {1: {...}}
>
>I tried several things with __getstate__, __reduce__, etc., but couldn't 
>get this to work.  Is there some magic that I'm missing?  What's the 
>best way to get around this?  (And should I file this as a bug in 
>pickle?  Or am I just not seeing the right way to do it?)
>

I think the answer to this problem is to use the extended __reduce__ API
documented here:

http://www.python.org/peps/pep-0307.html

Your subclass's __reduce__ can return a dictitems (or listitems) iterator, the
items of which will get pickled after the subclass instance is memoized, thus
avoiding the infinite recursion.  The following works for your simple example:

>>> class D(dict):
...     def __reduce__(self):
...         return (D, (), None, None, self.iteritems())
...
>>> d = D()
>>> d[1] = d
>>> pickle.dumps(d)
'c__main__\nD\np0\n(tRp1\nI1\ng1\ns.'
>>> s = _
>>> dd = pickle.loads(s)
>>> dd.items()
[(1, {1: {...}})]

---
Greg Chapman