Searching for solution: Severe memory leak, HELP!

[Steve Holden]

"Randy Young" <randall_young at hotmail.com> wrote in message
news:8338c7cc.0301180618.5c67da85 at posting.google.com...
> Sirs,
>
> I have a very large Python application squarely in my lap, that's
> leaking almost 20mb of main memory an hour!
>
> I spent most of last night looking through various postings, and not
> surprisingly, I'm wondering if I have a "circular reference" at work.
> Specifically, something happening with the heavy use of the "exec"
> library call.
>
> Here's a code snippet, can anyone see anything grievously wrong? I
> understand that we may need to look at WHAT is being "exec'd", but I
> thought this might be a place to start...
>
>
> ---- CODE BEGINS
>
>
> #import the module, then execute the probe
>   where = 'import'
>   win32event.WaitForSingleObject(self.iSemaphore, win32event.INFINITE)
>     try:
>       exec ("import %s" % probeInfo['ImportName'])
>     except Exception, e:
>       ret = "Import error on " + str(probeInfo['ImportName']) + " " +
> str(e)
>       self.ExceptOut(e, 'Exception importing Probe ID ' +
> str(probeInfo['ID']))
>     win32event.ReleaseSemaphore(self.iSemaphore, 1)
>
>     if (ret == None):
>       where = 'exec'
>       try:
> s = "ret = %s.%s (probeInfo)" % (probeInfo['ImportName'],
> probeInfo['Invoke'])
>         exec (s)
>
> # ... much more yada-yada ...
>
> ---- CODE ENDS
>
> As you can see there are "exec(s)" all over the place in the module,
> one mistake would be magnified many times! I'm trying to understand if
> I should be forcing a garbage collection or some other obvious thing?
>
> Thanks in advance for the consideration of a reply.
>

Well, firstly it might be better to use

    __import__(...)

than

    exec "import ..."

Also note, by the way, that exec is a statement and not a function, so the
parentheses are redundant.

What are you trying to import? It's conceivable that failed imports of a
badly-constructed extension might leak memory. Also note that if you leave
pointers to the stack frame lying about in "except" clauses that can cause
leakage.

Just a few random thoughts.

regards
--
Steve Holden                                  http://www.holdenweb.com/
Python Web Programming                 http://pydish.holdenweb.com/pwp/
Bring your musical instrument to PyCon!    http://www.python.org/pycon/