[Python-Dev] Understanding the buffer API

[Jeff Allen]

On 08/08/2012 11:47, Stefan Krah wrote:
> Nick Coghlan<ncoghlan at gmail.com>  wrote:
>> It does place a constraint on consumers that they can't assume those
>> fields will be NULL just because they didn't ask for them, but I'm
>> struggling to think of any reason why a client would actually *check*
>> that instead of just assuming it.
> Can we continue this discussion some other time, perhaps after 3.3 is out?
> I'd like to respond, but need a bit more time to think about it than I have
> right now (for this issue).
Those who contributed to the design of it through discussion here may be 
interested in how this has turned out in Jython. Although Jython is 
still at a 2.7 alpha, the buffer API has proved itself in a few parts of 
the core now and feels reasonably solid. It works for bytes in one 
dimension. There's a bit of description here:
http://wiki.python.org/jython/BufferProtocol

Long story short, I took the route of providing all information, which 
makes the navigational parts of the flags argument unequivocally a 
statement of what navigation the client is assuming will be sufficient. 
(The exception if thrown says explicitly that it won't be enough.) It 
follows that if two clients want a view on the same object, an exporter 
can safely give them the same one. Buffers take care of export counting 
for the exporter (as in the bytearray resize lock), and buffers can give 
you a sliced view of themselves without help from the exporter. The 
innards of memoryview are much simpler for all this and enable it to 
implement slicing (as in CPython 3.3) in one dimension. There may be 
ideas worth stealing here if the CPython buffer is revisited.

N dimensional arrays and indirect addressing, while supported in 
principle, have no implementation. I'm fairly sure multi-byte items, as 
a way to export arrays of other types, makes no sense in Java where type 
security is strict and a parallel but type-safe approach will be needed.

Jeff Allen