pickling generators

[Jeremy Bowers]

On Sat, 22 Mar 2003 17:54:50 +0100, Paul Boehm wrote:
> this is not suitable for our engine, we need to HARD restore state, not
> try to recreate it.

Ahhh, there really isn't any such thing. The very act of pickling and
unpickling is "recreating state". It's not like it just loads a binary
image into memory.

A lower-tech, but significantly more bullet-proof, is to instantiate a
class around each generator and store all data the generator might call
"local" in that class, then pickle the class normally. Your generators are
quite likely either glorified loops, or initialization + glorified loops,
and this can be made to work, with minimal additional state variables to
help the generators over the parts they should skip. (Sometimes you can
also write non-destructive initializers (if not hasattr(self, "name"):
self.name = "initial name") so that even if the generators re-do that part
the first time they run after unpickling it won't hurt anything.)

It may not seem initially like it will work, but it can almost certainly
be made to do so with some creativity. (I'd even expect that you'd reap
some unexpected benefits in other areas as a result of such careful
refactoring.)

Frankly, I'm surprised that the solution Gerrit Holl posted works at all.
I wouldn't expect anything better to come along anytime soon. You're going
to have to do some work to make this go. Bleh. ;-)