[Tutor] Breaking threads

[Daniel Yoo]

On Mon, 30 Apr 2001 kromag@nsacom.net wrote:

> I am working my way through "Programming Python" from the begining (I have 
> asked too many silly questions! :-). Here is a non-silly question:

No problem.


> import thread, time 
> 
> glarf=open('\windows\desktop\goat.txt', 'w')
> 
> def counter(myId, count):
>     for i in range(count):
> 	mutex.acquire()
> #	time.sleep(1)
> 	glarf.write('[%s]=> %s' % (myId, i))
> 	mutex.release()
> 		
> mutex = thread.allocate_lock()
> for i in range(10000):
> 	thread.start_new_thread(counter, (i, 3))

> It starts to write 10,000 counts to goat.txt, then dies with the following:
> 
> 
> Traceback (most recent call last):
>   File "cornfedbeef.py", line 14, in ?
>     thread.start_new_thread(counter, (i, 3))
> thread.error: can't start new thread
> 
> It works fine in iterations of 10, 100 and 1000. Why does it puke at the 
> 10000 mark?


Python's thread support is based on what the system underneath provides
us.  Unfortunately, not all platforms support a threaded model well.  
What probably happened was that the system got flooded by too many
threads.

Many systems have a hard time supporting even 1000 threads.  According to
one group:


"It has been our experience that in real world use, Windows NT is
incapable of running more than 1000 simultaneous threads at a time"

http://www.lyris.com/about/company/whitepapers/lm_extreme/secondgen.html



On the Linux side, the story is even more grim: you're limited to 256
threads at a time:

http://pauillac.inria.fr/~xleroy/linuxthreads/faq.html


So this is something that will probably need to be fixed.  At the moment
though, try to avoid writing programs that abuse the threading system.  
*grin*