[Numpy-discussion] advanced indexing bug with huge arrays?

[Sturla Molden]

Den 23.01.2012 22:08, skrev Christoph Gohlke:
>
> Maybe this explains the win-amd64 behavior: There are a couple of places
> in mtrand where array indices and sizes are C long instead of npy_intp,
> for example in the randint function:
>
> <https://github.com/numpy/numpy/blob/master/numpy/random/mtrand/mtrand.pyx#L863>
>
>

Both i and length could overflow here. It should overflow on allocation 
of more than 2 GB.

There is also a lot of C longs in the internal state (line 55-105), as 
well as the other functions.

Producing 2 GB of random ints twice fails:

 >>> import numpy as np
 >>> np.random.randint(5000000,size=(2*1024**3,))
array([0, 0, 0, ..., 0, 0, 0])
 >>> np.random.randint(5000000,size=(2*1024**3,))

Traceback (most recent call last):
   File "<pyshell#3>", line 1, in <module>
     np.random.randint(5000000,size=(2*1024**3,))
   File "mtrand.pyx", line 881, in mtrand.RandomState.randint 
(numpy\random\mtrand\mtrand.c:6040)
MemoryError
 >>>


Sturla