[SciPy-user] Handle large array

[Tan Tran]

Hello,

I'm trying to do some & like this 

xx = (d[:,0:1] == 0) & (d[:,2:3] == 2) & (d[:, 1:2]==1) & (d[:, 1:2]==2)

If d is small, 19 columns and about 5000 rows, the code runs fine. But if I have large data like d has about 40k rows, I got error message: MemoryError

I tried to make separate variable but still have problem when trying to & them
aa = d[:,0:1] == 0
bb =  d[:,2:3] == 2
cc = d[:, 1:2]==1
dd = d[:, 1:2]==2

xx = aa & bb & cc & dd <-- MemoryError's here

Have anybody seen this problem before? How to play with large data?

Thanks,



      
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20090116/aaa34823/attachment.html>