[Numpy-discussion] corrcoeff on big matrix

[CL WU]

Hi, group,
              I have a big "Float64" matrix (42x22300) and I want to get 
its correlation coefficient matrix, but I got the error as the following:

 >>> data.shape
(42, 22300)
 >>> mlab.corrcoef(data)
Traceback (most recent call last):
  File "<interactive input>", line 1, in ?
  File "C:\Python23\Lib\site-packages\numarray\linear_algebra\mlab.py", 
line 300, in corrcoef
    c = cov(x, y)
  File "C:\Python23\Lib\site-packages\numarray\linear_algebra\mlab.py", 
line 294, in cov
    val = squeeze(dot(transpose(m),conjugate(y)) / fact)
  File "C:\Python23\Lib\site-packages\numarray\numarraycore.py", line 
1150, in dot
    return ufunc.innerproduct(array1, _gen.swapaxes(array2, -1, -2))
  File "C:\Python23\Lib\site-packages\numarray\ufunc.py", line 2047, in 
innerproduct
    r = a.__class__(shape=adots+bdots, type=rtype)
ValueError: new_memory: invalid region size: -633294592.

I suspect corrcoef function can not handle such a big matrix. If so, 
what is the upper limit for array size? How can I get around this 
problem in numarray?

BTW, I am using numarray 0.9/python 2.3.3 on win2kSP4

Thanks.

Chunlei