[Python-ideas] Copy (and/or pickle) generators

[Antoine Pitrou]

On Wed, 20 Jun 2018 12:15:18 -0400
Yury Selivanov <yselivanov.ml at gmail.com>
wrote:
> 
> > Finally, another comment made the point that there wasn't a strong use case given for it. With the data science libraries that have sprung up around Python in the intervening years, I believe there now is one.  
> 
> As Guido has pointed out, pickling generators would require proper
> pickling of the entire frame stack (otherwise generators that use
> "global" or "nonlocal" won't unpickle correctly).

Depends what level of automatic (magic?) correctness you're expecting.
A generator is conceptually an iterator expressed in a different syntax.
If you define an iterator object, it will probably get pickling for
free, yet pickling it won't bother serializing the global variables that
are accessed from its __next__() and send() methods.

A generator needn't be different: you mainly have to be careful to
serialize its module's __name__, so that you can lookup the frame's
global dict by module name when the generator is recreated.

By contrast, closure variables would be an issue.  But a first
implementation could simply refuse to pickle generators that access an
enclosing local state.

Regards

Antoine.