[SciPy-user] Large matrix support

[Alan McIntyre]

Dinesh,

I don't have a Windows machine to try it on, but on Linux I didn't get
any issues until I actually ran out of memory and virtual memory
(750MB total).   Constructing an array in the same way as your code,
30 million randomly placed entries was enough to push me over 750MB,
even though the array itself should only take up about 150M (that's an
estimate based on an array with fewer filled slots).

So even though you might have memory for the array itself, the other
stuff that gets instantiated during the construction might be pushing
you over the top. But that's just a guess, since I don't know if
there's something different going on in Windows (I can't imagine what
that might be, though).  Can you watch the working set for your
process as it's doing the construction?  It could also help to know
where the actual errors occur for you, if it's possible to tell.

Cheers,
Alan