Pickling generators, why not? (was Re: Scheme style and Python style [was: Re: Typed Python?]

[Ville Vainio]

>>>>> "Francois" == François Pinard <pinard at iro.umontreal.ca> writes:

    Francois> Suddenly backtracking out of a complex involvement for
    Francois> restarting in a new direction, for the above project at
    Francois> least, requires a lot of state resumption, and doing
    Francois> this cleanly in Python implies a rather strict and
    Francois> burdening discipline about where and how the state
    Francois> information is kept.

Would such a problem be solvable if you could pickle a "running"
generator? In fact, why is it not possible to pickle a generator that
is hygienic enough to not mutate nonlocal objects?

def mkgen():
  x = 1
  while 1:
    x += 1
    yield x

g = mkgen()
g.next()
g.next()

# x is 3 in generator

pickle.dump(g, open("a.tmp","w'))

g.next()
g.next()

g2 = pickle.load(open("a.tmp"))

g2.next()   # yields 4 


Is this what you were trying to achieve with continuations? I don't
really have an intuitive grasp of the continuation idea, it seems
messy compared to generators, so I'm trying to learn here...

Mutating nonlocals (in the gtor, or between pickling and unpickling)
would obviously hose this construct, but many generators should avoid
it in the first place. Referenced global objects would still be
referenced normally (because it happens on source code level), the
behaviour would just not be that of "saved state"...

So, is there some deep philosophical (as opposed to implementation
detail) flaw in the picklable generator idea?

-- 
Ville Vainio   http://tinyurl.com/2prnb