threads and memory

[Lee Leahu]

Hi List,

I am trying to write a simple threaded application which will simulate 1000 connections to a remote service in order to "stress test" that the remote service can handle that many connections.

However, I have encountered the following error after I have started my 381st thread:

---------------------------------------------------------------------------
Traceback (most recent call last):
  File "./test.py", line 39, in ?
    current.start()
  File "/usr/lib/python2.4/threading.py", line 416, in start
    _start_new_thread(self.__bootstrap, ())
thread.error: can't start new thread
---------------------------------------------------------------------------

The command 'ps -eo pid,%cpu,%mem,rss,sz,vsz' gives at the point of error, the following information:
---------------------
pid: 5448
%cpu: 33.5
%mem: 0.6
rss: 9492
sz: 782729
vsz: 3130916
---------------------

I have 1.5GB of total ram in my computer, as evident by the command 'free':
---------------------
             total       used       free     shared    buffers     cached
Mem:       1553128    1314152     238976          0          0     357392
-/+ buffers/cache:     956760     596368
Swap:       977248       7964     969284
---------------------

An 'strace' of the script reveals the following:
NOTE the line that reads 'ENOMEM (Cannot allocate memory).
--------------------------------------------------------
futex(0x80b2b08, FUTEX_WAKE, 1)         = 1
select(0, NULL, NULL, NULL, {0, 0})     = 0 (Timeout)
write(1, "381\n", 4381
)                    = 4
futex(0x80b2b08, FUTEX_WAKE, 1)         = 0
futex(0x80b2b08, FUTEX_WAKE, 1)         = 0
futex(0x80581f8, FUTEX_WAKE, 1)         = 0
mmap2(NULL, 8392704, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory)
write(2, "Traceback (most recent call last"..., 35Traceback (most recent call last):
) = 35
open("./test.py", O_RDONLY|O_LARGEFILE) = 383
write(2, "  File \"./test.py\", line 40, in "..., 34  File "./test.py", line 40, in ?
) = 34
fstat64(383, {st_mode=S_IFREG|0755, st_size=1037, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7f65000
read(383, "#!/usr/bin/python\n\nimport time\nf"..., 4096) = 1037
write(2, "    ", 4    )                     = 4
write(2, "current.start()\n", 16current.start()
--------------------------------------------------------

Is there a way to accomplish what I am trying to do, perhaps in a more memory-friendly way?

My source code is attached.

Please advise.

Thanks!

-- 
Lee Leahu                           RICIS, Inc.
Internet Technology Specialist      866-RICIS-77 Toll Free Voice (US)
lee at ricis.com                       708-444-2690 Voice (International)
http://www.ricis.com/               866-99-RICIS Toll Free Fax (US)
                                    708-444-2697 Fax (International)

RICIS, Inc. is a member of the Public Safety Alliance Group

This email and any attachments that are included in it have been scanned
for malicious or inappropriate content and are believed to be safe.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: test.py
Type: text/x-python
Size: 1093 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-list/attachments/20060206/2b82a78d/attachment.py>