[Numpy-discussion] "expected a single-segment buffer object"

[Robert Kern]

On Wed, Jul 9, 2008 at 18:55, Anne Archibald <peridot.faceted at gmail.com> wrote:
> Hi,
>
> When trying to construct an ndarray, I sometimes run into the
> more-or-less mystifying error "expected a single-segment buffer
> object":
>
> Out[54]: (0, 16, 8)
> In [55]: A=np.zeros(2); A=A[np.newaxis,...];
> np.ndarray(strides=A.strides,shape=A.shape,buffer=A,dtype=A.dtype)
> ---------------------------------------------------------------------------
> <type 'exceptions.TypeError'>             Traceback (most recent call last)
>
> /home/peridot/<ipython console> in <module>()
>
> <type 'exceptions.TypeError'>: expected a single-segment buffer object
>
> In [56]: A.strides
> Out[56]: (0, 8)
>
> That is, when I try to construct an ndarray based on an array with a
> zero stride, I get this mystifying error. Zero-strided arrays appear
> naturally when one uses newaxis, but they are valuable in their own
> right (for example for broadcasting purposes). So it's a bit awkward
> to have this error appearing when one tries to feed them to
> ndarray.__new__ as a buffer. I can, I think, work around it by
> removing all axes with stride zero:
>
> def bufferize(A):
>   idx = []
>   for v in A.strides:
>       if v==0:
>           idx.append(0)
>       else:
>           idx.append(slice(None,None,None))
>   return A[tuple(idx)]
>
> Is there any reason for this restriction?

Yes, the buffer interface, at least the subset that ndarray()
consumes, requires that all of the data be contiguous in memory.
array_as_buffer() checks for that using PyArray_ISONE_SEGMENT(), which
looks like this:

#define PyArray_ISONESEGMENT(m) (PyArray_NDIM(m) == 0 ||                      \
                                 PyArray_CHKFLAGS(m, NPY_CONTIGUOUS) ||       \
                                 PyArray_CHKFLAGS(m, NPY_FORTRAN))

Trying to get a buffer object from anything that is neither C- or
Fortran-contiguous will fail. E.g.

In [1]: from numpy import *

In [2]: A = arange(10)

In [3]: B = A[::2]

In [4]: ndarray(strides=B.strides, shape=B.shape, buffer=B, dtype=B.dtype)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)

/Users/rkern/today/<ipython console> in <module>()

TypeError: expected a single-segment buffer object


What is the use case, here? One rarely has to use the ndarray
constructor by itself. For example, the result you seem to want from
the call you make above can be done just fine with .view().

In [8]: C = B.view(ndarray)

In [9]: C
Out[9]: array([0, 2, 4, 6, 8])

In [10]: B
Out[10]: array([0, 2, 4, 6, 8])

In [11]: C is B
Out[11]: False

In [12]: B[0] = 10

In [13]: C
Out[13]: array([10,  2,  4,  6,  8])

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
 -- Umberto Eco