[Tutor] Threads (was: demon / app I/F better way ?

[Terry Carroll]

On Sun, 14 Nov 2004, Kent Johnson wrote:

> I'm starting to think there is a cultural bias against threads in the 
> python community.

I think there's a bias against threads in the tutorial mailing list, which 
is kind of understandable.  It's a big leap in writing and debugging.  I 
suspect that most programmers who are looking for help, and who are 
proficient enough to use threads, would probably be hitting the 
comp.lang.python newsgroup, rather than python-tutor.

I'm toying with using threads for an app I have in mind.  The Queue module 
you point out looks like just the ticket for what I'm thinking of.

One question I do have on threads, though.  In my app, a finite number 
(12-50) requests including URLs would be put on the queue, and multiple 
(2-4) consumer threads would pick things off the queue to process.

But what's the mechanism to tell those consumer threads that there will be 
nothing more to consume?  It's not just an empty queue, because that could 
happen whenever the consumers outpace the producer.  

I was toying with a special-purpose element to add to the queue, as a 
sentinel to say "that's it, show's over; you can shut down now!"  But that 
would take care only of the thread that happens to pull that element off 
the queue; the other threads would just see an empty queue and have no 
basis for knowing that this is the big final empty.  

I suppose I could ass that "shutdown element" multiple times, once for 
each consumer thread spawned, but that seems so inelegant.

Another technique would be to pass, in addition to the queue, a Shutdown
flag, so that when the queue is empty and the shutdown flag is set, the
thread shouts down.  But the problem with this is that Queue.empty()  is
documented as not reliable.  I suppose I could do a get with a timeout,
and check the Shutdown flag; but I can imagine there are synch issues 
there.

What's the ordinary way of telling a thread that it's no longer needed?