[Numpy-discussion] Buffer interface PEP

[Zachary Pincus]

Hi,

I have a specific question and then a general question, and some  
minor issues for clarification.

Specifically, regarding the arguments to getbufferproc:
> 166 format
> 167    address of a format-string (following extended struct
> 168    syntax) indicating what is in each element of
> 169    of memory.  The number of elements is len / itemsize,
> 170    where itemsize is the number of bytes implied by the format.
> 171    NULL if not needed in which case format is "B" for
> 172    unsigned bytes.  The memory for this string must not
> 173    be freed by the consumer --- it is managed by the exporter.

Is this saying that either NULL or a pointer to "B" can be supplied  
by getbufferproc to indicate to the caller that the array is unsigned  
bytes? If so, is there a specific reason to put the (minor)  
complexity of handling this case in the caller's hands, instead of  
dealing with it internally to getbufferproc? In either case, the  
wording is a bit unclear, I think.

The general question is that there are several other instances where  
getbufferproc is allowed to return ambiguous information which must  
be handled on the client side. For example, C-contiguous data can be  
indicated either by a NULL strides pointer or a pointer to a properly- 
constructed strides array. Clients that can't handle C-contiguous  
data (contrived example, I know there is a function to deal with  
that) would then need to check both for NULL *and* inside the strides  
array if not null, before properly deciding that the data isn't  
usable them. Similarly, the suboffsets can be either all negative or  
NULL to indicate the same thing.

Might it be more appropriate to specify only one canonical behavior  
in these cases? Otherwise clients which don't do all the checks on  
the data might not properly interoperate with providers which format  
these values in the alternate manner.


Also, some typos, and places additional clarification could help:

> 253 PYBUF_STRIDES (strides and isptr)
Should 'isptr' be 'suboffsets'?

> 75 of a larger array can be described without copying the data.   T
Dangling 'T'.

> 279 Get the buffer and optional information variables about the  
> buffer.
> 280 Return an object-specific view object (which may be simply a
> 281 borrowed reference to the object itself).
This phrasing (and similar phrasing elsewhere) is somewhat opaque to  
me. What's an "object-specific view object"?

> 287 Call this function to tell obj that you are done with your "view"
Similarly, the 'view' concept and terminology should be defined more  
clearly in the preamble.

> 333 The struct string-syntax is missing some characters to fully
> 334 implement data-format descriptions already available elsewhere (in
> 335 ctypes and NumPy for example).  Here are the proposed additions:
Is the following table just the additions? If so, it might be good to  
show the full spec, and flag the specific additions. If not, then the  
additions should be flagged.

> 341 't'               bit (number before states how many bits)
vs.
> 372 According to the struct-module, a number can preceed a character
> 373 code to specify how many of that type there are.  The
I'm confused -- could this be phrased more clearly? Does '5t' refer  
to a field 5-bits wide, or 5-one bit fields? Is 'ttttt' allowed? If  
so, is it equivalent to or different from '5t'?

> 378 Functions should be added to ctypes to create a ctypes object from
> 379 a struct description, and add long-double, and ucs-2 to ctypes.
Very cool.

In general, the logic of the 'locking mechanism' should be described  
at a high level at some point. It's described in nitty-gritty  
details, but at least I would have appreciated a bit more of a  
discussion about the general how and why -- this would be helpful to  
clients trying to use the locking mechanism properly.

Thanks to Travis and everyone else involved for your work on this  
cogent and sorely-needed PEP.

Zach



On Mar 27, 2007, at 12:42 PM, Travis Oliphant wrote:

>
> Hi all,
>
> I'm having a good discussion with Carl Banks and Greg Ewing over on
> python-dev about the buffer interface. We are converging on a pretty
> good design, IMHO.   If anybody wants to participate in the  
> discussion,
> please read the PEP (there are a few things that still need to change)
> and add your two cents over on python-dev (you can read it through the
> gmane gateway and participate without signing up).
>
> The PEP is stored in numpy/doc/pep_buffer.txt on the SVN tree for  
> NumPy
>
> Best regards,
>
> -Travis
>
>
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion