[Python-Dev] PEP-298: buffer interface widening too much?

[Martin v. Löwis]

Todd Miller <jmiller@stsci.edu> writes:

> I think that kind of backwards compatability just paves the wave for
> extensions to stay broken and complicates the object structures as
> well. I guess PEP-298 does create the possibility for creating correct
> extensions,  I just hate the thought of 5 getpointer calls where there
> are now 3 and should be 2.

I think this is the core of your objection. However, notice that all
types implementing have to be modified in either proposal, and that
adding the two getpointer functions should be trivial in most cases:
- if the type has read/write memory, implement them both the same
  way;
- if the type has read-only memory, set the write pointer to NULL
- make the getpointer function a wrapper around the existing
  getpointer, requesting only a single segment and simultaneously
  locking the buffer.

The overhead implementation-wise is really minimal; all the hard work
will come from the locking code - see Thomas' patch to the string
object for an example. We could even provide a Generic function which
delegates the new get functions to the existing ones, accessing
segment zero.

> Sounds good to me.  I hope you implement it!  If PEP-286 were here
> now, a simplified PEP-298 would be possible. 

I now recall what the issue is: you can't assume that modifying the
argument tuple object to record resources is feasible, as this breaks
apply (where in every case the user allocates a tuple, and where this
is a plain old tuple).

So I'm now envisioning a strategy where a thread-local
(i.e. thread-state associated) stack is maintained - on argument tuple
allocation, a frame is put on this stack, and all resources go into
that frame, on tuple decrefing, the frame is cleared. There would be
three kinds of resources: objects, void*, and locked buffers. For
efficiency, this stack and its frames would be a flat memory block.

In any case, I'm still -0 on PEP 298 as it stands: it should deal with
much more aspects of buffers (and I now consider argument parsing to
be one of them), when those things are added, I'll support the PEP -
your objection that this adds too much API is of no concern to me.

Regards,
Martin