use of Queue

[Fredrik Lundh]

Alexandru Mosoi wrote:

>> how is Queue intended to be used? I found the following code in python
>> manual, but I don't understand how to stop consumers after all items
>> have been produced. I tried different approaches but all of them
>> seemed incorrect (race, deadlock or duplicating queue functionality)
>>
>>     def worker():
>>         while True:
>>             item = q.get()
>>             do_work(item)
>>             q.task_done()
>>
>>     q = Queue()
>>     for i in range(num_worker_threads):
>>          t = Thread(target=worker)
>>          t.setDaemon(True)
>>          t.start()
>>
>>     for item in source():
>>         q.put(item)
>>
>>     q.join()       # block until all tasks are done
> 
> 
> ok. I think I figured it out :). let me know what you think
> 
> global num_tasks, num_done, queue
> num_tasks = 0
> num_done = 0
> queue = Queue()
> 
> # producer
> num_tasks += 1
> for i in items:
>   num_tasks += 1
>   queue.put(i)

what's the point of using a thread-safe queue if you're going to use a 
non-thread-safe counter?  if you want to write broken code, you can do 
that in a lot fewer lines ;-)

as others have mentioned, you can use sentinels:

     http://effbot.org/librarybook/queue.htm

or, in Python 2.5 and later, the task_done/join pattern shown here:

     http://docs.python.org/lib/QueueObjects.html

</F>