Not fully understanding the role of Queue.task_done()

[Fredrik Lundh]

Martin DeMello wrote:

> I'm writing a cluster monitor, that collects information from a set of
> machines and logs it to a database
> 
> In the interests of not hammering the db unnecessarily, I'm
> considering the following
> 1. A series of independent "monitor" threads that collect information
> over TCP from the cluster of machines, and write it to a queue
> 2. A "logger" thread that empties the queue every second or so and
> inserts the collected information to the db via a single insert
> statement

why are you using a queue for this case, btw?  why not just use a plain list

     L = []
     lock = threading.Lock()

and add stuff using append in the monitor threads

     with lock:
         L.append(item)

and regularily reset the list in the logger thread

     with lock:
         data = L[:]
         L[:] = [] # clear the list
     for item in data:
         ... insert into database ...

(list append and assignments to global variables are atomic in CPython, 
so you can eliminate the lock by being a bit clever, but that's probably
better left for a non-premature optimization pass).

</F>