Callbacks to generators

[Robert Brewer]

Dave Benjamin wrote:
> Is there a straightforward way to create a generator from a 
> function that
> takes a callback? For instance, I have a function called "process":
> 
> def process(text, on_token):
>     ...
>     
> For each token that "process" finds in "text", it creates a 
> token object,
> "token", and calls "on_token(token)".
> 
> Now, suppose I wanted to create a generator based on this 
> function. I tried
> to do the following:
> 
> def process_iter(text):
>     def on_token(token):
>         yield token
>     process(text, on_token)
> 
> However, rather than create a generator as I had hoped, this function
> doesn't return anything. I guess it creates a bunch of 
> singleton generators,
> one per token, and throws them away. In any case, is there a 
> way to do what
> I am trying to do here without resorting to threads?
> 
> The reason I am trying to do this is that I'd still like to 
> be able to use
> "process" in code for Python 2.1, which does not support 
> generators. I'd
> like to avoid implementing the entire thing twice or basing 
> the callback
> version on the generator version.

You're not the first to tackle this problem:

http://mail.python.org/pipermail/python-list/2003-December/197828.html

...but no solution forthcoming.


FuManChu