[Python-Dev] PEP-298: buffer interface widening too much?
Todd Miller
jmiller@stsci.edu
Sat, 14 Dec 2002 13:45:55 -0500
I've been following the PEP-298 discussion a little, because I fear
eventually IAGNI. But when I look at the proposed changes, I can't
help but get the feeling that the "too wide" buffer interface just got
wider. By "too wide", I am referring to the extraneous segmentation
and character buffer aspects of the existing interface. Adding three
more calls for the locking problem bugs me. Doesn't anything ever get
simpler?
Looking at the proposed changes, the only thing that really seems
needed for the bufferprocs is the release call and a flag bit. Assuming
the release call is not implemented, the acquired buffer is never
locked or unlocked, and we're no worse off than we are now: it's the
BO's problem. Assuming the release call is present, the acquired
buffer should *always* be locked and later released. The existing
wrapper functions (plus one for release) can hide/implement this, making
it straight forward to provide correct extensions: just implement a
locking extension. It seems to me that the existing (3!) calls for
acquiring the buffer pointer can do just fine. Except...
In the case of existing extensions, there will certainly be no call to
the release function. So for an extension which is slow to update to
the revised protocol, there are negative consequences: buffers which
are never unlocked. But so what? These consequences seem less dire
(if more probable) than the existing consequences of "moving buffers."
It would be a bigger problem if PEP-298 wound up fiddling with
reference counts. In that case, un-released buffers would also become
memory leaks. But is it really necessary to mess with the reference
counts? I'm having a hard time imagining a situation in which the
reference count for a buffer goes to zero while it is locked which is
not a bug somewhere else. What am I missing?
Another possibility is to implement a "release buffer pointer" wrapper
function which *does nothing* for a Python release 2.3 and change
nothing else. Let extensions which use buffers adopt usage of the
wrapper and the locking protocol, with no immediate benefit. Then
implement locking for real in Python 2.4. That way, the number of
non-conforming extensions can be minimized, but the buffer API can be
kept simple.
So, in the end, what I'm wondering is whether there's a simpler middle
ground between dropping the PEP and implementing it as it stands now.
Are there compromises to be found between guaranteeing correctness,
backwards compatability, and growing complexity? Is there another
trade between backwards compatability and complexity which is tolerable
for the short term with both simplicity and correctness available in the
longer term?
Todd