alternative to JoinableQueue's please

Raymond Hettinger python at rcn.com
Fri Jun 26 17:44:16 EDT 2009


[Filipe Fernandes]
> The reasons for using JoinableQueue I think are obvious.  I want to
> block the main processing using queue.join() until the tasks that have
> been placed on the queue have been finished by the worker processes.
>
> I can't be the only one experiencing this (besides Brian)... are there
> others who ran into this?  Are there work arounds (besides a
> home-brewed one) ?

Before Queue.task_done() and Queue.task_join() were added, other
idioms were used.

One technique is to use a second queue to track incomplete tasks.

# adding work
unfinished_tasks.put(None)
q.put(task)


# doing work
t = q.get()
f(t)
unfinished_tasks.get()


# waiting for unfinished tasks to complete
while unfinished_tasks.qsize():
    sleep(0.1)



Raymond




More information about the Python-list mailing list