multiprocessing: pool with blocking queue

[ryles]

On Jul 2, 11:09 am, masher <vertesp... at gmail.com> wrote:
> My questions, then, is: Is there a more elegant/pythonic way of doing
> what I am trying to do with the current Pool class?

Another thing you might try is to subclass Pool and add an apply_async
() wrapper which would wait for _taskqueue.qsize() to reach the
desired size. You would probably do this wait in a loop with a small
sleep. This approach would avoid needing a second Queue, but you would
also add some delay to your producer due to the sleep (something
you're not currently worried about). The minimum sleep may be
something like 1 ms (it's really system dependent), but the time it
takes for a thread blocked on a mutex to wake up is often more on the
order of microseconds, which you have with your blocking queue.

I doubt this offers you much satisfaction, though.

> If the verdict is no, I'll be happy to file a bug report.

Yeah, I think it's a worth a try.