Is there anything that pickle + copy_reg cannot serialize?

Jean-Paul Calderone exarkun at divmod.com
Thu Dec 8 11:26:18 EST 2005


On Thu, 08 Dec 2005 22:42:32 +0800, Maurice LING <mauriceling at acm.org> wrote:
>Hi,
>
>I need to look into serialization for python objects, including codes,
>recursive types etc etc. Currently, I have no idea exactly what needs to
>be serialized, so my scope is to be as wide as possible.
>
>I understand that marshal is extended by pickle to serialize class
>instances, shared elements, and recursive data structures
>(http://www.effbot.org/librarybook/pickle.htm) but cannot handle code
>types. pickle can be used together with copy_reg and marshal to
>serialize code types as well
>(http://www.effbot.org/librarybook/copy-reg.htm).
>
>So my question will be, are there anything that pickle/copy_reg/marshal
>combination cannot serialize? If so, what are the workarounds?

Since copy_reg lets you specify arbitrary code to serialize arbitrary 
objects, you shouldn't run into any single object that you cannot 
serialize to a pickle.

However, both pickle implementations are recursive, so you will be 
limited by the amount of memory you can allocate for your stack.  By 
default, this will limit you to something like object graphs 333 edges 
deep or so (if I'm counting stack frames correctly).  Note that this 
does not mean you cannot serialize more than 333 objects at a time, 
merely that if it takes 333 or more steps to go from the first object 
to any other object in the graph (using the traversal order pickle 
uses), the pickling will fail.  You can raise this limit, to a point, 
with sys.setrecursionlimit().

Jean-Paul



More information about the Python-list mailing list