[Numpy-discussion] Initializing array from buffer

[Andrea Arteaga]

Hello.
Using the numpy.frombuffer function [1] one can initialize a numpy array
using an existing python object that implements the buffer protocol [2].
This is great, but currently this function supports only 1D buffers, even
if the provided buffer is multidimensional and it exposes all information
about its structure (shape, strides, data type).

Apparently, one can extract every kind of buffer information out of a
buffer of a numpy array (pointer, number of dimensions, shape, strides,
suboffsets,...), but the other way around is only partially implemented:
providing a multidimensional buffer does not mean being able of creating a
numpy array the uses that buffer with the desired structure.

My use case is the following: we have a some 3D arrays in our C++
framework. The ordering of the elements in these arrays is neither C nor
Fortran style: it might be IJK (i.e. C style, 3rd dimension contiguous in
memory), KJI (i.e. Fortran style, first dimension contiguous) or, e.g. IKJ.
Moreover we put some padding to optimize aligned access. This kind of
memory structure cannot be just expressed as 'C' or 'Fortran', but it can
be perfectly expressed using the Python buffer protocol by providing the
shape and the strides. We would like to export this structure to a numpy
array that should be able of accessing the same memory locations in a
consistent way and make some operations like initializing the content or
plotting it.

Is this currently possible?
If not, is it planned to implement such a feature?

==========

Maybe just to clarify I could show an example entirely in python. Assume a
in a 2D numpy array:

a = np.ones((10,20))

It contains information about its structure which can be portably accessed
using its data member:

a.data.format == 'd'
a.data.ndim == 2
a.data.shape == (10,20)
a.data.strides == (160,8)

Unfortunately, when initializing an array b from this buffer, the structure
of the buffer is "downgraded" to unidimensional shape:

b = np.frombuffer(a.data)

b.ndim == 1
b.shape == (200,)
b.strides == (8,)

I wished b had the same multi-dimensional structure of a.

(This is of course a very simple example. In my use case I would initialize
b with my own buffer instead of that of another numpy array).

Best regards

[1]
http://docs.scipy.org/doc/numpy/reference/generated/numpy.frombuffer.html
[2] https://docs.python.org/3/c-api/buffer.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20141116/e5ef03f3/attachment.html>