thread limit in python

[Peter Hansen]

Bryan Olson wrote:
> Peter Hansen wrote:
>  > Probably, but I haven't yet seen anyone ask the real important question.
>  >  What possible use could you have for more than 1000 *simultaneously
>  > active* threads?  There are very likely several alternative approaches
>  > that will fit your use case and have better characteristics (e.g. higher
>  > performance, simpler code, safer architecture, etc).
> 
> Threading systems have come a long way in the last decade or so,
> and they're still advancing. 64-bit, multi-core processors make
> mega-threading even more attractive.
> 
> To read why zillions of threads are good, see:
>   http://www.usenix.org/events/hotos03/tech/vonbehren.html

Judging by the abstract alone, that article promotes "user-level" 
threads, not OS threads, and in any case appears (with a quick scan) to 
be saying nothing more than that threads are not *inherently* a bad 
idea, just that current implementations suffer from various problems 
which they believe could be fixed.

My question was in the context of the OP's situation.  What possible use 
for 1000 OS threads could he have?  (Not "could his requirements be 
answered with a non-existent proposed improved-performance 
implementation of threads?")

> Prediction: Ten years from now, someone will ask that same
> "What possible use..." question, except the number of threads
> will be a million.

Perhaps.  And in ten years it will still be a valid question whenever 
the context is not fully known.  Is the OP's situation IO-bound, 
CPU-bound, or just an experiment to see how many threads he can pile on 
the machine at one time?  The fact that these threads are all sleeping 
implies the latter, though what he posted could have been a contrived 
example.  I'm interested in the real requirements, and whether more than 
1000 threads in this day and age (not some imaginary future) might not 
be a poor approach.

(For the record, Bryan, I am a strong proponent of thread systems for 
many of the reasons the authors of that article put forth.  None of 
which means my question was without merit.)

-Peter