[Python-Dev] Allocation of shape and strides fields in Py_buffer

[Travis Oliphant]

Greg Ewing wrote:
> Antoine Pitrou wrote:
> 
>> Why should it need two? Why couldn't the embedded Py_buffer fullfill 
>> all the
>> needs of the memoryview object? 
> 
> Two things here:
> 
>   1) The memoryview should *not* be holding onto a Py_buffer
>      in between calls to its getitem and setitem methods. It
>      should request one from the underlying object when needed
>      and release it again as soon as possible.
>

This is actually a different design than the PEP calls for.  From the PEP:

    This is functionally similar to the current buffer object except a
reference to base is kept and the memory view is not re-grabbed.
Thus, this memory view object holds on to the memory of base until it
is deleted.

I'm open to this changing, but it is the current PEP.


>   2) The "second" Py_buffer referred to above only needs to
>      be materialized when someone makes a GetBuffer request on
>      the memoryview itself. It's not needed for Python getitem
>      and setitem calls. (The implementation might choose to
>      implement these by creating a temporary Py_buffer, but
>      again, it would only last as long as the call.)

The memoryview object will need to store some information for 
re-calculating strides, shape, and sub-offsets for consumers.

> 
>> If the memoryview can't be a relatively thin
>> object-oriented wrapper around a Py_buffer, then this all screams 
>> failure to me.
> 
> It shouldn't be a wrapper around a Py_buffer, it should be a
> wrapper around the buffer *interface* of the underlying object.
> 

This is a different object than what was proposed, but I'm not opposed 
to it.

> It sounds to me like whoever wrote the memoryview implementation
> didn't understand how the buffer interface is meant to be used.
> That doesn't mean there's anything wrong with the buffer interface.
> 
> I have some doubts myself about whether it needs to be as
> complicated as it is, but I think the basic idea is sound:
> that Py_buffer objects are ephemeral, to be obtained when
> needed and not kept for any longer than necessary.
> 

I'm all for simplifying as much as possible.  There are some things I 
understand very well (like how strides and shape information can be 
shared with views), but others that I'm trying to understand better 
(like whether holding on to a view or re-grabbing the view is better).

I think I'm leaning toward the re-grabbing concept.   I'm all for 
improving the memoryview object, but let's not confuse that effort with 
the buffer API implementation.

I do not think we need to worry about changes to the memoryview object, 
because I doubt anything outside of the standard library is using it yet.


-Travis