best way to discover this process's current memory usage, cross-platform?

Alex Martelli aleax at mail.comcast.net
Mon Nov 14 21:29:34 EST 2005


Having fixed a memory leak (not the leak of a Python reference, some
other stuff I wasn't properly freeing in certain cases) in a C-coded
extension I maintain, I need a way to test that the leak is indeed
fixed.  Being in a hurry, I originally used a q&d hack...:


if sys.platform in ('linux2', 'darwin'):
  def _memsize():
    """ this function tries to return a measurement of how much memory
        this process is consuming, in some arbitrary unit (if it doesn't
        manage to, it returns 0).
    """
    gc.collect()
    try:
      x = int(os.popen('ps -p %d -o vsz|tail -1' % os.getpid()).read())
    except:
      x = 0
    return x
else:
  def _memsize():
    return 0

Having a _memsize() function available, the test then does:
    before = _memsize()
    # a lot of repeated executions of code that should not consume
    # any net memory, but used to when the leak was there
    after = _memsize()
and checks that after==before.

However, that _memsize is just too much of a hack, and I really want to
clean it up.  It's also not cross-platform enough.  Besides, I got a bug
report from a user on a Linux platform different from those I had tested
myself, and it boils down to the fact that once in a while on his
machine it turns our that after is before+4 (for any large number of
repetitions of the code in the above comment) -- I'm not sure what the
unit of measure is supposed to be (maybe blocks of 512 byte, with a page
size of 2048? whatever...), but clearly an extra page is getting used
somewhere.

So, I thought I'd turn to the "wisdom of crowds"... how would YOU guys
go about adding to your automated regression tests one that checks that
a certain memory leak has not recurred, as cross-platform as feasible?
In particular, how would you code _memsize() "cross-platformly"?  (I can
easily use C rather than Python if needed, adding it as an auxiliary
function for testing purposes to my existing extension).


TIA,

Alex



More information about the Python-list mailing list