[SciPy-user] pairwise difference between large sets of arrays

[Roger Herikstad]

Hi all, I am trying to cluster sets of arrays, typically consisting of ~
10,000 arrays, each about 1600 points long, using the Pycluster package (
http://bonsai.ims.u-tokyo.ac.jp/%7Emdehoon/software/cluster/software.htm#pycluster),
but my problem is that I can't seem to create the diffence matrix
needed.
Using the zeros functions to preallocate the space, it raises a ValueError
saying dimensions too large. Now, I realise this might not be strictly
relevant to this list, but I was wondering if anyone knew what the limits
are for creating arrays like this? Is it an allocation error, or some other
restriction in the numpy package? Does anyone know of alternative ways of
achieving this clustering that does not require preallocation of such a
large matrix in python?

~ Thanks

~ Roger
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20080130/1ab2e4d3/attachment.html>