problems with shelve(), collections.defaultdict, self

[Ian Kelly]

On Fri, Feb 10, 2012 at 7:48 PM, 7stud <7stud at excite.com> wrote:
> But I cannot get a class that inherits from collections.defaultdict to
> shelve itself:
>
>
> import collections as c
> import shelve
>
> class Dog(c.defaultdict):
>    def __init__(self):
>        super().__init__(int, Joe=0)
>        print('****', self)
>
>    def save(self):
>        my_shelve = shelve.open('data22.shelve')
>        my_shelve['dd'] = self
>        my_shelve.close()
>
>    def load(self):
>        my_shelve = shelve.open('data22.shelve')
>        data = my_shelve['dd']
>        my_shelve.close()
>
>        print(data)
>
>
> d = Dog()
> d.save()
> d.load()
>
> --output:--
>
> **** defaultdict(<class 'int'>, {'Joe': 30})
> Traceback (most recent call last):
>  File "/Library/Frameworks/Python.framework/Versions/3.2/lib/
> python3.2/shelve.py", line 111, in __getitem__
>    value = self.cache[key]
> KeyError: 'dd'
>
> During handling of the above exception, another exception occurred:
>
> Traceback (most recent call last):
>  File "3.py", line 95, in <module>
>    d.load()
>  File "3.py", line 87, in load
>    data = my_shelve['dd']
>  File "/Library/Frameworks/Python.framework/Versions/3.2/lib/
> python3.2/shelve.py", line 114, in __getitem__
>    value = Unpickler(f).load()
> TypeError: __init__() takes exactly 1 positional argument (2 given)
>
>
>
> I deleted all *.shelve.db files between program runs.  I can't figure
> out what I'm doing wrong.

The problem is that defaultdict defines a custom __reduce__ method
which is used by the pickle protocol to determine how the object
should be reconstructed.  It uses this to reconstruct the defaultdict
with the same default factory, by calling the type with a single
argument of the default factory.  Since your subclass's __init__
method takes no arguments, this results in the error you see.

There are a couple of ways you could fix this.  The first would be to
change the signature of the __init__ method to take an optional
argument accepting the default factory instead of hard-coding it, like
this:

    def __init__(self, default_factory=int):
        super().__init__(default_factory, Joe=0)

The other would be to just fix the __reduce__ method to not pass the
default factory to your initializer, since it is hard-coded.  That
would look like this:

    def __reduce__(self):
        return (type(self), ())

Cheers,
Ian