Thread vs. generator problem

[Robert Kern]

Paul Rubin wrote:
> As I understand it, generators are supposed to run til they hit a
> yield statement:
> 
>    import time
>    def f():
>      print 1
>      time.sleep(3)
>      for i in range(2,5):
>         yield i
> 
>    for k in f():
>       print k
> 
> prints "1" immediately, sleeps for 3 seconds, then prints 2, 3, and 4
> without pausing, as expected.  When I try to do it in a separate thread:
> 
>     import time, itertools
>     def remote_iterate(iterator, cachesize=5):
>         # run iterator in a separate thread and yield its values
>         q = Queue.Queue(cachesize)
>         def f():
>             print 'thread started'
>             for x in iterator:
>                 q.put(x)
>         threading.Thread(target=f).start()
>         while True:
>             yield q.get()
> 
>     g = remote_iterate(itertools.count)
>     print 'zzz...'
>     time.sleep(3)
>     print 'hi'
>     for i in range(5):
>         print g.next()
> 
> I'd expect to see 'thread started' immediately, then 'zzz...', then a 3
> second pause, then 'hi', then the numbers 0..4.  Instead, the thread
> doesn't start until the 3 second pause has ended.

My 10-second analysis is that *none* the body of a generator runs until a value
is requested from it.

In [3]: def f():
   ...:     print 'Starting f'
   ...:     for i in range(3):
   ...:         yield i
   ...:
   ...:

In [4]: g = f()

In [5]: for i in g:
   ...:     print i
   ...:
   ...:
Starting f
0
1
2

In your first example, you instantiate the generator and then iterate over it
immediately; in your second, you separate the two things. I don't think threads
have anything to do with it.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco