PEP 255: Simple Generators

[Tim Peters]

[Greg Copeland]
> So what happens when you have two threads calling a generator.  I
> guess they are distinct...

That depends on whether you created a distinct generator for each thread, or
shared a common one.  If Threads A and B both do

for x in g():
    yadda(x)

it may or may not be thread-safe, depending on how g was implemented; if it
sticks to using local vrbls, it probably will be; if it mucks around
mutating globals without benefit of synchronization constructs, probably
not.

If thread A alone instead does:

iter = g()

and iter is visible to both A and B, and then they both do

while 1:
    x = iter.next()
    yadda(x)

then they're sharing a single iterator object, and an exception is the most
likely outcome:  while an instance of a generator-function is running, it's
a detected error to try to invoke it again before it yields (hmm! I should
add that to the PEP).  If the threads manage to avoid that, they'll see
disjoint subsequences of g()'s full sequence (which may well be a *feature*
if synchronized properly:  "thread safe" is relative to what an app is
trying to accomplish, it's not an absolute thing).

If you stick to the for-loop spelling, it's easy to write thread-safe
generators.  For example, the tree-traversal generators in the PEP can be
used simultaneously by multiple for-loops in multiple threads without
problems.  Despite persistent rumors to the contrary <wink>, generators
*are* functions, and each invocation gets its own local variables and local
instruction pointer and local stack space etc.