Turning a callback function into a generator

[Kirk McDonald]

Alex Martelli wrote:
> Lawrence D'Oliveiro <ldo at geek-central.gen.new_zealand> wrote:
> 
> 
>>In article <44a89559 at nntp0.pdx.net>,
>> Kirk McDonald <kirklin.mcdonald at gmail.com> wrote:
>>
>>
>>>I want to somehow, in some way, provide an iteration interface to this
>>>function. Thoughts?
>>
>>Run it in a separate thread/process?
> 
> 
> Sounds best to me.  Specifically, given (e.g.) something like the OP's
> 
> def func(callback):
>      for i in [1, 2, 3, 4, 5]:
>          callback(i)
> 
> we might have a wrapper such as:
> 
> import thread
> import Queue
> 
> def wrap_cb_into_gen(func):
>     q = Queue.Queue(1)           # maximum size of 1
>     def callback(item):
>         q.put(item)
>     all_done_sentinel = object()
>     def thread_skeleton():
>         func(callback)
>         q.put(all_done_sentinel)
>     thread.start_new_thread(thread_skeleton, ())
>     while True:
>         item = q.get()
>         if item is all_done_sentinel: break
>         yield item
> 
> 
> Of course, there are lighter-weight options than a length-1 Queue for
> the purpose of synchronizing these two threads, but I always tend to
> prefer Queue (for its simplicity, generality, solidity) to other
> synchronization structures and architectures, unless there is some very
> strong reason to do otherwise in a specific case.  Also, I normally use
> module threading rather than the lower-level module thread -- here,
> clearly, either one will do just fine.
> 
> 
> Alex

Threads are probably a weak point of mine. I have little experience with 
them, and so I am inclined to paranoia while using them. What 
implications does this have for "func"? In my case, it is a function in 
an extension module. Does it have to be thread safe? If it isn't, what 
limitations does this impose on the use of this wrapper?

All that aside, I read the previous threads but it took until just now 
to understand how this works. :-) I'll probably have a go at 
implementing this in D, as that will be more convenient for the library...

-Kirk McDonald