Turning f(callback) into a generator

[Jeff Epler]

On Wed, Dec 03, 2003 at 06:07:08PM -0800, Jimmy Retzlaff wrote:
> My solution used threads. I'd love to know if someone comes up with a
> solution not involving threads. Here's a simple example of my
> thread-based approach:
[...]

In standard Python, this is the only approach that can work.  It's
possible that Stackless Python might provide a solution that doesn't
involve OS threads, but I don't have any actual experience with
Stackless.

One problem with your code is the use of a sentinel value.  Another
poster suggested using multiple queues and a timeout to return
exceptions or end-of-queue.  Instead, you should use a queue but insert
easily distinguishable items in it.  For instance:
    None: end of iterator, raise StopIteration
    (0, a, b): exception in f, raise a, None, b (re-raise exception)
    (1, blah): callback got blah, return it from next()


import sys, Queue, threading

class CallbackGenerator:
    def __init__(self, func, pre=(), post=()):
        self.func = func
        if type(pre) is not tuple: pre = (pre,)
        if type(post) is not tuple: post = (post,)
        self.args = pre + (self.cb,) + post
        self.queue = Queue.Queue(maxsize=1)
        self.thread = threading.Thread(target=self.go)
        self.thread.start()

    def __iter__(self): return self

    def next(self):
        item = self.queue.get()
        if item is None:
            raise StopIteration
        if item[0] == 0:
            raise item[1], None, item[2]
        else:
            return item[1]

    def go(self):
        try:
            self.func(*self.args)
        except:
            info = sys.exc_info()
            self.queue.put((0, info[1], info[2]))
        else:
            self.queue.put(None)

    def cb(self, *args):
        self.queue.put((1, args))

# Example 1: os.path.walk -> generator
import os
for i in CallbackGenerator(os.path.walk, "/tmp", None):
    print i

# Example 2: Prints 1, 2 then shows a traceback
def f(cb):
    cb(1)
    cb(2)
    1/0

for i in CallbackGenerator(f):
    print i