[Python-Dev] PEP 296 - The Buffer Problem

[Thomas Heller]

From: "Scott Gilbert" <xscottg@yahoo.com>
> --- Thomas Heller <thomas.heller@ion-tof.com> wrote:
> > What if we would 'fix' the buffer interface?
> > 
> 
> This gets us part of the way there, but still has shortcomings.  For one I,
> and people more significant than me, would still need a type that
> implemented the bytes object behavior.

Sure, the extension of the buffer interface is only part of the
picture. The bytes type is still needed as well.

The extension I proposed is motivated by these thoughts:

It would enable some of Python's builtin objects to
expose the interface extension by supplying two
trivial functions for each in the extended tp_as_buffer slot.

The new functions expose a 'safe buffer interface', where
there are guarantees about the lifetime of the pointer. So
your bytes object can be a view of these builtin objects
as well.

It dismisses the segment count of the normal buffer interface.

>   Everything but efficient pickling
> _could_ be done with third party extensions, but ignoring pickling (which I
> don't want to do), then we'd still have several significant third parties
> reinventing the same wheel.  To me at least, this feels like a battery that
> should be included.
> 

I don't think my proposal prevents this.

> 
> > Extend the PyBufferProcs structure by new fields:
> > 
> >     typedef size_t (*getlargereadbufferproc)(PyObject *, void **);
> >     typedef size_t (*getlargewritebufferproc)(PyObject *, void **);
> > 
> 
> How would you designate failure/exceptions?  size_t is unsigned everywhere
> I can find it, so it can't return a negative number on failure.  I guess
> the void** could be filled in with NULL.
> 

Details, not yet fleshed out completely. Store NULL in the void **,
use ptrdiff_t instead of size_t, or something else. Or return
((size_t)-1) on failure. Or return -1 on failure, and fill out
an size_t pointer:
  typedef int (*getlargereadwritebufferproc(PyObject *, size_t *, void **);


> > Python strings, unicode strings, mmap objects, and maybe other types
> > would expose the large buffer interface, but the array type would
> > *not*. We could also change the name from 'large buffer interface'
> > to something more sensible, currently I don't have a better name.

Maybe it should be renamed 'safe buffer interface extension' instead
of 'large buffer interface' (it could be large as well)?

> 
> I've been trying to keep the proposal as unintrusive as possible while
> still implementing the functionality needed.  Adding more flags/members to
> PyObjects and modifying string, unicode, mmap, ... feels like a more
> intrusive change to me.  I'm open to the idea, but I'm not ready to retract
> the current proposal.  Then there is still the problem of needing something
> like a bytes object as mentioned above.

The advantage (IMO) is that it defines a new protocol to get the
pointer to the internal byte array on objects instead of
requiring that these objects are instances of a special type
or subtype thereof.

> 
> __________________________________________________
> Do You Yahoo!?

No, I google. ;-)

Thomas