[Tutor] Memory Problem Continued

[Danny Yoo]

On Mon, 7 Jul 2003 DORSEY_EDMUND_K@LILLY.COM wrote:

> I did a little more testing and I realized I'm having a different problem.
>  The problem seems to be in assinging data to each index.  Calculating the
> indices doesn't eat any memory.  Sorry about that.
> Each call self._newData[index] = self._data[n] eats up some memory.

Hi Dorsey,


Sure; the problem is that keeping the values in memory is what's eating
your memory.  *grin*

Python's numbers are objects --- this is important, because each new
integer or floating-point number will take up a little more space than you
might expect.



> Prior to this function I initialized self._newData by saying
>
> newData = [0] * self._numVoxelsOrig
>
> Then I proceeded to fill up newData.  Everytime I put something into a
> different index it uses up some more memory.  Any ideas as why it would
> do this??


Out of curiosity: how large is _numVoxelsOrig?

> #self._numVoxelsOrig has a value of around 20 million

Since your vector of numers is very large, you may want to try saving
space by using a more "homogenous" data structure, like the vector and
matrix data types provided by the Numeric Python project.  numpy's matrix
classes are optimized to hold numbers efficiently.

    http://pfdubois.com/numpy/

Modifying the code to use numpy should be fairly nonintrusive: I think all
you need to do is modify the initialization of newData, perhaps like this:

    newData = Numeric.zeros(self._numVoxelsOrig)

Just wondering: do you expect your newData to be very sparse, or will most
of your values be nonzero?

Good luck to you!