[Numpy-discussion] advanced indexing bug with huge arrays?

[David Warde-Farley]

On Tue, Jan 24, 2012 at 06:00:05AM +0100, Sturla Molden wrote:
> Den 23.01.2012 22:08, skrev Christoph Gohlke:
> >
> > Maybe this explains the win-amd64 behavior: There are a couple of places
> > in mtrand where array indices and sizes are C long instead of npy_intp,
> > for example in the randint function:
> >
> > <https://github.com/numpy/numpy/blob/master/numpy/random/mtrand/mtrand.pyx#L863>
> >
> >
> 
> Both i and length could overflow here. It should overflow on allocation 
> of more than 2 GB.
> 
> There is also a lot of C longs in the internal state (line 55-105), as 
> well as the other functions.
> 
> Producing 2 GB of random ints twice fails:

Sturla, since you seem to have access to Win64 machines, do you suppose you
could try this code:

>>> a = numpy.ones((1, 972))
>>> b = numpy.zeros((4993210,), dtype=int)
>>> c = a[b]

and verify that there's a whole lot of 0s in the matrix, specifically,

>>> c[574519:].sum()
356.0
>>> c[574520:].sum()
0.0

is the case on Linux 64-bit; is it the case on Windows 64?

Thanks a lot,

David