[Numpy-discussion] Python memory management issues using, Linux. Maybe Numpy, related.

[Jeffrey Spencer]

    Further isolated the error and it seems to come in from the sum function in numpy. I will probe further later but the code below causes the memory usage to go up and up. If you change sum below to some other function there are no issues.

    from numpy import arange, sum

    for x in range(10000000):
             inhibVal = sum(arange(15))

    Cheers,
    Jeff

    Message: 3
    Date: Sun, 22 May 2011 09:32:10 -0500
    From: Bruce Southey<bsouthey at gmail.com>
    Subject: Re: [Numpy-discussion] Python memory management issues using
    	Linux. Maybe Numpy, related.
    To: Discussion of Numerical Python<numpy-discussion at scipy.org>
    Message-ID:<BANLkTinAuFp0o6OTwvP=GDhBKNkhkwH-vg at mail.gmail.com>
    Content-Type: text/plain; charset=ISO-8859-1

    On Sun, May 22, 2011 at 8:42 AM, Ralf Gommers
    <ralf.gommers at googlemail.com>  wrote:

>     >
>     >
>     >  On Sun, May 22, 2011 at 2:54 PM, Jeffrey Spencer<jeffspencerd at gmail.com>
>     >  wrote:
>>     >>
>>     >>  Sorry. I attached here the class and script to run it. It is just a
>>     >>  minimalistic example of what I'm really doing that exhibits the same
>>     >>  behavior. All the script does is run at.createSpecific after instantiating
>>     >>  the class. I have seen this exhibited on many other cases but it runs fine
>>     >>  in Windows XP. Here is what the script does:
>>     >>
>>     >>  import MemoryLeak
>>     >>  at = MemoryLeak.Model()
>>     >>  at.createSpecific()
>>     >>
>     >  That manages to eat up all my cpu cycles for about 5 minutes, but no leak
>     >  (on OS X). I also don't see anything in the code that can potentially give
>     >  problems. I suggest you strip this down a lot more so you can isolate the
>     >  problem.
>     >
>     >  If in your original code you were making a lot of figures in one of those
>     >  for loops without explicitly closing them, that could perhaps be a problem.
>     >  But you removed all I/O and matplotlib related code already (except for
>     >  imports and some keywords).
>     >
>     >  Ralf
>     >
>     >  _______________________________________________
>     >  NumPy-Discussion mailing list
>     >  NumPy-Discussion at scipy.org
>     >  http://mail.scipy.org/mailman/listinfo/numpy-discussion
>     >
    A good OS is likely to keep Python module in memory just because it
    knows that it might be requested again. However, from the little I
    know (too much LWN), if Linux needs that memory it will reclaim that
    memory only when it needs it or part of some other process.

    Measuring memory usage using simple tools (like ps or top) are very
    far from sufficient, so exactly how do you know that the memory has
    not been released?

    Bruce



-- 
________________________
Jeffrey Spencer
jeffspencerd at gmail.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20110523/e57a5ea6/attachment.html>