questions regarding stack size use for multi-threaded python programs

[Eyal Gordon]

Hi,

background:
we are using python 2.4.3 on CentOS 5.3 with many threads - and our shell's
default stack size limit is set to 10240KB (i.e. ~10MB).

we noticed that python's Threading module appears to create threads with
this value as their stack size (we ran a sample program that creates 10
threads and measured its virtual memory size, then reduced the stack size
limit of the shell to 5120KB - and saw that the program's virtual memory
size was reduced by ~50MBs).

the problem:
our program uses numerous threads, and thus the virtual memory size gets to
be very large. we would like to reduce the size of the stack to reduce this
size. we were looking for information about recommendation for the stack
size to use, but found none.

questions:
1. is there some rule-of-thumb for the recommended stack size for python
programs of various sorts?

2. is there a way for us, at runtime (from inside the code or outside the
process), to find how much of a thread's stack we are using (in KB or some
other size units)?

3. when we define local objects - do the objects themselves get allocated on
the stack - or are they allocated on the heap and only references to them
are kept on the stack?

4. would the size of the stacks (which are probably not really allocated by
the linux virtual memory sub-system, unless used) have a noticeable
performance effect on a python program? same question regarding the use of a
large number of threads?

thanks,
Eyal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20091109/0c95532c/attachment.html>