Memory problem

[John Machin]

Yi Xing wrote:
> Thanks! I just found that that I have no problem with
> x=[[10.0]*2560*2560]*500, but x=range(1*2560*2560*30) doesn't work.
>

range(1*2560*2560*30) is creating a list of 196M *unique* ints.
Assuming 32-bit ints and pointers: that's 4 bytes each for the value, 4
for the type pointer, 4 for the refcount and  4 for the actual list
element (a pointer to the 12-byte object). so that's one chunk of
4x196M = 786MB of contiguous list, plus 196M chunks each whatever size
gets allocated for a request of 12 bytes. Let's guess at 16. So the
total memory you need is 3920M.

Now let's look at [[10.0]*2560*2560]*500.
Firstly that creates a tiny list [10.0]. then you create a list that
contains 2560*2560 = 6.5 M references to that *one* object containing
10.0. That's 26MB. Then you make a list of 500 references to that big
list. This new list costs you 2000 bytes. Total required: about 26.2MB.
The minute you start having non-unique numbers instead of 10.0, this
all falls apart.

In any case, your above comparison is nothing at all to do with the
solution that you need, which as already explained will involve
array.array or numpy.

What you now need to do is answer the questions about your pagefile
etc.

Cheers,
John