[Python-Dev] Extended Buffer Interface/Protocol

[Greg Ewing]

Travis Oliphant wrote:

> I'm talking about arrays of pointers to other arrays:
> 
> i.e. if somebody defined in C
> 
> float B[10][20]
> 
> then B would B an array of pointers to arrays of floats.

No, it wouldn't, it would be a contiguously stored
2-dimensional array of floats. An array of pointers
would be

   float *B[10];

followed by code to allocate 10 arrays of 20 floats
each and initialise B to point to them.

> Yes, I saw that.  But, it could actually be supported, in general.

Certainly it *can* be supported, but the question is
how many different format variations it's reasonable
to expect the consumer of the data to be able to deal
with. Because each variation requires the consumer to
use different code to access the data, if it wants to
avoid making a copy.

> else if (ndims == 3)
> 
>      var = (float ***)segments
>      for (i=0; i<shape[0]; i++)
>          for (j=0; j<shape[1]; j++)
>              for (k=0; j<shape[2]; k++)
>               # process var[i][j][k]

This assumes that the 3-dimensional case is using
the array-of-pointers implementation at all levels.
But there are other possibilities, e.g. a 1d array
of pointers to contiguous 2d arrays, or a contiguous
2d array of pointers to 1d arrays. It's hard to
deal with all of those using a common piece of code.

I can imagine cases like that coming up in practice.
For example, an image object might store its data
as four blocks of memory for R, G, B and A planes,
each of which is a contiguous 2d array with shape
and stride -- but you want to view it as a 3d
array byte[plane][x][y].

(Actually you'd probably *prefer* to view it as
byte[x][y][plane], which would make things even
more difficult...)

> I was thinking more of a C-iterator, like NumPy provides.  This can be 
> very efficient (as long as the loop is not in Python).
> 
> It sure provides a nice abstraction that lets you deal with 
> discontiguous arrays as if they were contiguous, though.

Something like that might be useful.

-- 
Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | Carpe post meridiem!          	  |
Christchurch, New Zealand	   | (I'm not a morning person.)          |
greg.ewing at canterbury.ac.nz	   +--------------------------------------+