use of Queue

[Iain King]

On Aug 27, 1:17 pm, Alexandru  Mosoi <brtz... at gmail.com> wrote:
> On Aug 27, 12:45 pm, Alexandru  Mosoi <brtz... at gmail.com> wrote:
>
>
>
> > how is Queue intended to be used? I found the following code in python
> > manual, but I don't understand how to stop consumers after all items
> > have been produced. I tried different approaches but all of them
> > seemed incorrect (race, deadlock or duplicating queue functionality)
>
> >     def worker():
> >         while True:
> >             item = q.get()
> >             do_work(item)
> >             q.task_done()
>
> >     q = Queue()
> >     for i in range(num_worker_threads):
> >          t = Thread(target=worker)
> >          t.setDaemon(True)
> >          t.start()
>
> >     for item in source():
> >         q.put(item)
>
> >     q.join()       # block until all tasks are done
>
> ok. I think I figured it out :). let me know what you think
>
> global num_tasks, num_done, queue
> num_tasks = 0
> num_done = 0
> queue = Queue()
>
> # producer
> num_tasks += 1
> for i in items:
>   num_tasks += 1
>   queue.put(i)
>
> num_tasks -= 1
> if num_tasks == num_done:
>   queue.put(None)
>
> # consumer
> while True:
>   i = queue.get()
>   if i is None:
>     queue.put(None)
>     break
>
>   # do stuff
>
>   num_done += 1
>   if num_done == num_tasks:
>     queue.put(None)
>     break

Are you sure you want to put the final exit code in the consumer?
Shouldn't the producer place a None on the queue when it knows it's
finished?  The way you have it, the producer could make 1 item, it
could get consumed, and the consumer exit before the producer makes
item 2.

Iain