writing pickle function

[Peter Otten]

Steve Holden wrote:

> Peter Otten wrote:
>> perfreem at gmail.com wrote:
>> 
>>> i am using nested defaultdict from collections and i would like to
>>> write it as a pickle object to a file. when i try:
>>>
>>> from collections import defaultdict
>>> x = defaultdict(lambda: defaultdict(list))
>>>
>>> and then try to write to a pickle file, it says:
>>>
>>> TypeError: can't pickle function objects
>>>
>>> is there a way around this? it's simply a dictionary that i want to
>>> write to file.. this works no problems with ordinary dicts.
>> 
>> The error message is misleading. You can pickle your defaultdict if you
>> use a "normal" function instead of the lambda:
>> 
>> $ cat pickle_defaultdict.py
>> import sys
>> from collections import defaultdict
>> from cPickle import dumps, loads
>> 
>> if "--lambda" in sys.argv:
>>     make_inner = lambda: defaultdict(list)
>> else:
>>     def make_inner():
>>         return defaultdict(list)
>> 
>> d = defaultdict(make_inner)
>> d[1][2].append(42)
>> e = loads(dumps(d))
>> print e == d
>> 
>> $ python pickle_defaultdict.py
>> True
>> $ python pickle_defaultdict.py --lambda
>> Traceback (most recent call last):
>>   File "pickle_defaultdict.py", line 13, in <module>
>>     e = loads(dumps(d))
>>   File "/usr/lib/python2.5/copy_reg.py", line 69, in _reduce_ex
>>     raise TypeError, "can't pickle %s objects" % base.__name__
>> TypeError: can't pickle function objects
>> 
> But can you unpickle the objects you pickle? I believe that functions
> are references relative to the modules they come from, and so won't be
> found unless the same module can be imported at unpickle time. Could be
> wrong, haven't read the source, but I understand that functions work the
> same as classes.

Indeed, what is stored in the pickle is the module and function name, not
the byte code:

>>> from cPickle import dumps, loads
>>> def f(): print "original"
...
>>> s = dumps(f)
>>> def f(): print "updated"
...
>>> g = loads(s)
>>> g()
updated

That limits pickle as an exchange format to installations that provide
compatible versions of the pickled classes and functions. 

Do you think that would be a problem for the OP?

Peter