Thread limits?

[François Pinard]

[Tim Peters]

> If you think you need 1,000 threads, you're solving the wrong problem or
> solving the right one in a doomed way <wink>.  Use the Queue (std library)
> module to enter requests for work.  Spin off a small number of threads, each
> structured like so:

>     while 1:
>         request = your_queue_object.get()  # sleeps until queue is non-empty
>         request.deal_with_it()   # do whatever work it asks for

This gave me a few conceptual problems.

Suppose I'm done with the main application (that is, it distributed all the
work it had to) and I want to send each serving thread (N of them) a command
so it terminates.  At first, one could just queue N termination requests
in the queue, right after all the real work requests.  We rely on the fact
each thread will never process more than one termination request :-).

May the main application just terminate then, or should it first wait to
see that the request queue got empty (meaning that all server threads
terminated)?  In the later case, is there a preferred way to do so?

But the real problem is that I read somewhere than the queued-ness (first-in
first-out behaviour) of the Queue class is not guaranteed.  So, it may
well happen that server threads receive the termination request early,
and all terminate prematurely, leaving the queue forever non-empty.

This is why I'm staying away from the Queue class for now, until I find some
elegant way to handle the termination of the server threads.  Any opinion
or suggestion you might have is welcome, of course! :-)

-- 
François Pinard   http://www.iro.umontreal.ca/~pinard