Tracking down memory leaks?

MKoool mohankhurana at gmail.com
Sun Feb 12 18:54:05 EST 2006


> How big is the file you are reading in? If it is (say) 400 MB, then it is
> hardly surprising that you will be using 500MB of RAM. If the file is 25K,
> that's another story.

Actually, I am downloading the matrix data from a file on a server on
the net using urllib2, and then I am running several basic stats on it
using some functions that i get from matplotlib.  Most are statistical
functions I run on standard vectors, such as standard deviation, mean,
median, etc.  I do then loop through various matrix items, and then
based on a set of criteria, I attempt to perform a sort of linear
regression model using a few loops on the vectors.

> How are you storing your data while you are processing it? I'd be looking
> for hidden duplicates.

I am storing basically everything as a set of vectors.  For example, I
would have one vector for my X-axis, time.  The other variables are the
number of units sold and the total aggregate revenue from selling all
units.

I am wondering if it's actually urllib2 that is messing me up.  It
could be matplotlib as well, although I doubt it since I do not use
matplotlib unless the statistical significance test I produce indicates
a high level of strength (very rare), indicating to me that the company
has a "winning" product.




More information about the Python-list mailing list