Is there anything that pickle + copy_reg cannot serialize?

Maurice LING mauriceling at acm.org
Thu Dec 8 13:17:10 EST 2005


> Since copy_reg lets you specify arbitrary code to serialize arbitrary 
> objects, you shouldn't run into any single object that you cannot 
> serialize to a pickle.

In http://www.effbot.org/librarybook/pickle.htm, it specifically 
mentions that code objects cannot be pickled and require the use of 
copy_reg, as follows:

import copy_reg
import pickle, marshal, types

#
# register a pickle handler for code objects

def code_unpickler(data):
     return marshal.loads(data)

def code_pickler(code):
     return code_unpickler, (marshal.dumps(code),)

copy_reg.pickle(types.CodeType, code_pickler, code_unpickler)

#
# try it out

CODE = """
print "suppose he's got a pointed stick"
"""

code = compile(CODE, "<string>", "exec")

exec code
exec pickle.loads(pickle.dumps(code))


I cannot understand 2 things, which I seek assistance for:
1. Is code object the only thing can cannot be pickled (less facing 
recursion limits)?
2. In the above example, how copy_reg works with pickle?

Thanks and Cheers
Maurice

> 
> However, both pickle implementations are recursive, so you will be 
> limited by the amount of memory you can allocate for your stack.  By 
> default, this will limit you to something like object graphs 333 edges 
> deep or so (if I'm counting stack frames correctly).  Note that this 
> does not mean you cannot serialize more than 333 objects at a time, 
> merely that if it takes 333 or more steps to go from the first object to 
> any other object in the graph (using the traversal order pickle uses), 
> the pickling will fail.  You can raise this limit, to a point, with 
> sys.setrecursionlimit().
> 
> Jean-Paul



More information about the Python-list mailing list