[Numpy-discussion] PyBUF_SIMPLE requests
Stefan Krah
stefan-usenet at bytereef.org
Sat Jul 16 06:42:36 EDT 2011
Hello,
I'm working on the completion of the PEP-3118 (buffer protocol) implementation
in Python core.
I was wondering how a PyBUF_SIMPLE buffer request sent to an object with
a complex structure should behave. The current Python C-API documentation
for a PyBUF_SIMPLE buffer request says:
"This is the default flag. The returned buffer exposes a read-only memory
area. The format of data is assumed to be raw unsigned bytes, without any
particular structure. This is a “stand-alone” flag constant. It never needs
to be ‘|’d to the others. The exporter will raise an error if it cannot
provide such a contiguous buffer of bytes."
To me a memory view "without any particular structure" means that the
returned buffer should always expose the whole memory area.
But NumPy behaves differently (see below for the printbuffer() hack):
Python 3.3.0a0 (default:1dd6908df8f5, Jul 16 2011, 11:16:00)
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from numpy import *
>>> x = ndarray(buffer=bytearray([1,2,3,4,5,6,7,8,9,10]), shape=[10], strides=[-1], dtype="B", offset=9)
>>> x
array([10, 9, 8, 7, 6, 5, 4, 3, 2, 1], dtype='uint8')
>>> x.printbuffer()
PyBUF_FULL:
obj: 0x7f527e946240
buf: 0x7f5281268c79
buf[0]: 10
len: 10
itemsize: 1
readonly: 0
ndim: 1
format: B
shape: 10
strides: -1
PyBUF_SIMPLE:
obj: 0x7f527e946240
buf: 0x7f5281268c79
buf[0]: 10
len: 10
itemsize: 1
readonly: 0
ndim: 0
format: (null)
shape:
strides:
The PyBUF_FULL request returns a reasonable buffer, because consumers that
follow the strides will behave correctly.
I do not understand the PyBUF_SIMPLE result. According to the C-API docs
a consumer would be allowed to access buf[9], which would be invalid.
Or is a consumer supposed to observe ndim = 0 and treat the result
as a scalar? If so, there is a problem with this approach. For example,
consumers like the struct module do not care about ndim and would walk
right out of the array.
Stefan Krah
printbuffer() hack:
============================================================================
diff --git a/numpy/core/src/multiarray/methods.c b/numpy/core/src/multiarray/methods.c
index 68f697a..1cd260a 100644
--- a/numpy/core/src/multiarray/methods.c
+++ b/numpy/core/src/multiarray/methods.c
@@ -2234,6 +2234,51 @@ array_newbyteorder(PyArrayObject *self, PyObject *args)
}
+static int
+do_print(PyObject *self, int flags, const char *flagstring)
+{
+ Py_buffer view;
+ int i;
+
+ if (PyObject_GetBuffer(self, &view, flags) < 0)
+ return -1;
+
+ fprintf(stderr, "%s:\n\n", flagstring);
+ fprintf(stderr, "obj: %p\n", view.obj);
+ fprintf(stderr, "buf: %p\n", view.buf);
+ fprintf(stderr, "buf[0]: %d\n", ((char *)(view.buf))[0]);
+ fprintf(stderr, "len: %zd\n", view.len);
+ fprintf(stderr, "itemsize: %zd\n", view.itemsize);
+ fprintf(stderr, "readonly: %d\n", view. readonly);
+ fprintf(stderr, "ndim: %d\n", view.ndim);
+ fprintf(stderr, "format: %s\n", view.format);
+
+ fprintf(stderr, "shape: ");
+ for (i = 0; i < view.ndim; i++)
+ fprintf(stderr, "%zd ", view.shape[i]);
+ fprintf(stderr, "\n");
+
+ fprintf(stderr, "strides: ");
+ for (i = 0; i < view.ndim; i++)
+ fprintf(stderr, "%zd ", view.strides[i]);
+ fprintf(stderr, "\n\n");
+
+ PyBuffer_Release(&view);
+
+ return 0;
+}
+
+static PyObject *
+array_print_getbuf(PyObject *self, PyObject *dummy)
+{
+ if (do_print(self, PyBUF_FULL, "PyBUF_FULL") < 0)
+ return NULL;
+ if (do_print(self, PyBUF_SIMPLE, "PyBUF_SIMPLE") < 0)
+ return NULL;
+
+ Py_RETURN_NONE;
+}
+
NPY_NO_EXPORT PyMethodDef array_methods[] = {
/* for subtypes */
@@ -2426,6 +2471,9 @@ NPY_NO_EXPORT PyMethodDef array_methods[] = {
{"view",
(PyCFunction)array_view,
METH_VARARGS | METH_KEYWORDS, NULL},
+ {"printbuffer",
+ (PyCFunction)array_print_getbuf,
+ METH_NOARGS, NULL},
{NULL, NULL, 0, NULL} /* sentinel */
};
More information about the NumPy-Discussion
mailing list