pickle complexity limit?

[Martin v. Löwis]

"Mark Hahn" <mark at hahnca.com> writes:

> I don't understand how this could happen with pickle.  Isn't it supposed to
> stop when it runs into an object it has already pickled?  

Yes, and it does.

> Couldn't it have been coded in a stackless way?

Yes, but it doesn't. Please look at save_inst implementation; it ends
with

        save(stuff)
        write(BUILD)

This is where the recursion happens. You run into the limit only if
you can find a path through your object graph that does not visit the
same object twice.

> What do I do now?   I don't want to recompile with a deeper stack as this
> will obviously only help in limited cases, not solve the real problem.

Read the pickle code, understand why it is recursive, and implement an
alternative that isn't recursive, yet preserves the original semantics
in terms of pickling order and sequence in which methods are called on
objects. When you are done, please contribute your changes back, to
sf.net/projects/python.

Regards,
Martin