[Numpy-discussion] Unexpected behavior with numpy array

[Anne Archibald]

On 03/02/2008, Damian Eads <eads at soe.ucsc.edu> wrote:
> Good day,
>
> Reversing a 1-dimensional array in numpy is simple,
>
>     A = A[:,:,-1]     .
>
> However A is a new array referring to the old one and is no longer
> contiguous.
>
> While trying to reverse an array in place and keep it contiguous, I
> encountered some weird behavior. The reason for keeping it contiguous is
> the array must be passed to an old C function I have, which expects the
> buffer to be in row major order and contiguous. I am using lots of
> memory so I want to minimize copying and allocation of new arrays.

The short answer is that reversing an array in-place requires a
certain amount of care, in C - you have to explicitly walk through
swapping element i and element n-i, using a temporary variable.
Getting numpy to exchange the elements in-place is going to be a real
pain. I suggest trying a copying method first, and only getting
fancier if it's too slow.

>  >>> A=numpy.arange(0,10)
>  >>> A
> array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
>  >>> A[::-1]
> array([9, 8, 7, 6, 5, 4, 3, 2, 1, 0])
>  >>> A[:] = A[::-1]
>  >>> A
> array([9, 8, 7, 6, 5, 5, 6, 7, 8, 9])
>  >>>
>
> Is there any way to perform assignments of the following form
>
>       X[sliceI] = expression involving X
>                   with dimensions that are
>                   compatible with the left
>                   hand side
>
> without causing the odd behavior I mentioned. If not, it might be
> helpful to throw an exception when both the LHS and the RHS of an
> assignment reference an array slice of the same variable?

This is, odd as it seems, intended behaviour. Sometimes it's really
useful to use the same array on the LHS and RHS: for example

A[:-1]=A[1:]

You just need to know how the operation is done under the hood (the
arrays are iterated over in index order). (Actually, I'm not totally
sure this is specified - under some circumstances numpy may iterate
over dimensions in an order based on stride and/or size; whether this
can affect the result of an operation like the above I'll have to
think about.) In any case, much code depends on this.

Tricks for getting the array backwards without copying... hmm. Well,
you might be able to fill in the array in an unconventional order:

A = N.arange(n-1,-1,-1)
A=A[::-1]
N.sin(A,A) # or whatever

Now the reversal of A is a perfectly normal C array (though numpy may
not realize this; don't trust the flags, check the strides and sizes).

Or you could just write a little C function inplace_reverse(A); if
you're already linking to C this shouldn't add too much complexity to
your project.

Or you could do it explicitly in numpy:
for i in xrange(n/2):
    t = A[i]
    A[i]=A[n-i]
    A[n-i]=t
In the likely case that this is too slow, you can do the copying in
blocks, small enough that memory consumption is moderate but large
enough that the python overhead is not too much.

Finally, I realize that digging around in legacy code can be
miserable, but it is often not really very difficult to make a C
function handle strided data - the whole principle of numpy is that
compiled code really just needs to know the start address, data type,
and the spacing and length of an array along each dimension.

Anne