Python's buffer interface

[Randall Hopper]

Robin Boerdijk:
 |Randall Hopper:
 |
 |>      From a commentary I read, it looks very useful.  I'd like to learn
 |> more about the it -- in particular, how to write conforming C interfaces.
 |>
 |>      However, I didn't find any mention of this in the docs.  Did I miss
 |> something?  Or have the docs not caught up with it.
 |
 |>From previous discussions on this newsgroup, it appears that the buffer
 |interface is rather controversial and that may be the reason why it hasn't
 |been properly documented yet. My impression is that part of the
 |controversion comes from the confusion between the buffer *interface* and
 |buffer *objects*.

Thanks for detailed response!  Lots of good info in your post.

I'm frequently coming up with cases where I'd like to use Python to pass
large chunks of data (geometry lists, images, etc.) between different
subsystems without incurring copy overhead.  Rather than translating these
two subsystems' data formats inside the wrappers to glue them together,
passing the metadata through Python with a large canonical data buffer
in-tow seems to make more sense.

 | The buffer interface entry is a pointer to a structure of four
   functions:
 |
 | static PyBufferProcs string_as_buffer = {
 |   (getreadbufferproc)string_buffer_getreadbuf,
 |   (getwritebufferproc)string_buffer_getwritebuf,
 |   (getsegcountproc)string_buffer_getsegcount,
 |   (getcharbufferproc)string_buffer_getcharbuf,
 | };
...
 | PyTypeObject PyString_Type = {
...
 |   &string_as_buffer, /*tp_as_buffer*/
 | };

I think this answers my main question.  It appears you have to define a
full user-defined type in the C wrappers to yield a class that implements
the buffer interface.

I was hoping to be able to define a few methods on a wrapped C/C++ class
(e.g. like __getreadbuffer__, __setwritebuffer__, __getsegcount__) to have
an object that conforms the buffer interface.

Intuitively, I think it might be more flexible if the buffer interface was
defined at the Python level rather than the C level.  Then classes could
implement the buffer interface without a formal Python type definition in
C.  The wrappers could call up to get the "callable objects" in the buffer
interface and just use them.  If a buffer method was a reference to a C
function, it could be blazingly fast.  If it happened to be a Python
expression (callable object, lambda expression, etc.), it might be slower,
but it would still work.  Either way, no formal Python type definition in C
would be required.  And as a side benefit, you could define classes
implementing buffer interfaces in pure Python.

(What I'm leaning toward is something that is easily implemented in wrapper
generators like SWIG.  When generating wrappers for C code, SWIG doesn't
generate Python classes in C, but rather generates Python classes in
Python, handing off to C inside the Python class.)

Randall

--
Randall Hopper
aa8vb at yahoo.com