[Mailman-Users] Python process size grows 30x in 8 hours (memory leak?)

[Tim Bell]

Back at the beginning of this thread, Fletcher Cocquyt wrote:

> Config: 
> Solaris 10 x86
> Python 2.5.2
> Mailman 2.1.9 (8 Incoming queue runners - the leak rate increases with this)
> SpamAssassin 3.2.5
> 
> At this point I am looking for ways to isolate the suspected memory leak - I
> am looking at using dtrace: http://blogs.sun.com/sanjeevb/date/200506
> 
> Any other tips appreciated!

With Solaris 10, you can interpose the libumem library when starting those python processes.
This gives you different malloc()/free() allocators including extra instrumentation that is
low enough in overhead to run in a production environment, and (when combined with mdb) a 
powerful set of debugging tools.

Set LD_PRELOAD, UMEM_DEBUG, UMEM_LOGGING environment variables in the parent process before 
starting python so they will inherit the settings.  If you have to, you could replace 'python' 
with a script that sets what you want in the environment and then runs the python executable.

I know this will be looking at the lower, native layers of the problem, and you may not see the 
upper (python) part of the stack very well, but libumem has been a big help to me so I thought 
I would mention it.

Here are two references... there are many more if you start searching:

Identifying Memory Management Bugs Within Applications
Using the libumem Library
http://access1.sun.com/techarticles/libumem.html

Solaris Modular Debugger Guide
http://docs.sun.com/db/doc/806-6545

Hope this helps - this is too long, so I'll stop now.

Tim