[Python-checkins] r63987 - peps/trunk/pep-3118.txt
travis.oliphant
python-checkins at python.org
Fri Jun 6 18:33:51 CEST 2008
Author: travis.oliphant
Date: Fri Jun 6 18:33:50 2008
New Revision: 63987
Log:
Remove locking scheme from PEP 3118 buffer protocol. See issue 3046 in python bug tracker.
Modified:
peps/trunk/pep-3118.txt
Modified: peps/trunk/pep-3118.txt
==============================================================================
--- peps/trunk/pep-3118.txt (original)
+++ peps/trunk/pep-3118.txt Fri Jun 6 18:33:50 2008
@@ -147,16 +147,16 @@
releasebufferproc bf_releasebuffer;
} PyBufferProcs;
+Both of these routines are optional for a type object
+
::
typedef int (*getbufferproc)(PyObject *obj, PyBuffer *view, int flags)
This function returns ``0`` on success and ``-1`` on failure (and raises an
error). The first variable is the "exporting" object. The second
-argument is the address to a bufferinfo structure. If view is ``NULL``,
-then no information is returned but a lock on the memory is still
-obtained. In this case, the corresponding releasebuffer should also
-be called with ``NULL``.
+argument is the address to a bufferinfo structure. Both arguments must
+never be NULL.
The third argument indicates what kind of buffer the consumer is
prepared to deal with and therefore what kind of buffer the exporter
@@ -178,13 +178,16 @@
structure (with defaults or NULLs if nothing else is requested). The
PyBuffer_FillInfo function can be used for simple cases.
+
+Access flags
+------------
+
Some flags are useful for requesting a specific kind of memory
segment, while others indicate to the exporter what kind of
information the consumer can deal with. If certain information is not
asked for by the consumer, but the exporter cannot share its memory
without that information, then a ``PyErr_BufferError`` should be raised.
-
``PyBUF_SIMPLE``
This is the default flag state (0). The returned buffer may or may
@@ -198,29 +201,6 @@
The returned buffer must be writable. If it is not writable,
then raise an error.
-``PyBUF_LOCK``
-
- This flag is used to request a lock (either a read-lock or an
- exclusive write lock) on the data-area of the object before the
- buffer is returned. If the lock cannot be obtained, then an error
- should be raised. In conjunction with the PyBUF_WRITABLE flag, the
- PyBUF_LOCK flag creates four different access modes on the
- data-area of the buffer memory: read, read-with-write-lock, write,
- and exclusive write. The access modes are
-
- ================ ================= ==========================
- flags Description Other Requests Can
- ================ ================= ==========================
- neither read read and write
- WRITABLE write read and write
- LOCK locked read read but not write
- WRITABLE | LOCK exclusive write neither read nor write
- ================ ================= ==========================
-
- Care should be taken not to LOCK the buffer unless it is really
- necessary (especially the exclusive write lock) as it makes the
- object unable to share its memory until the lock is released.
-
``PyBUF_FORMAT``
The returned buffer must have true format information if this flag
@@ -256,7 +236,6 @@
All of these flags imply PyBUF_STRIDES and guarantee that the
strides buffer info structure will be filled in correctly.
-
``PyBUF_INDIRECT`` (implies ``PyBUF_STRIDES``)
The returned buffer must have suboffsets information (which can be
@@ -271,29 +250,21 @@
| ``PyBUF_CONTIG`` (``PyBUF_ND | PyBUF_WRITABLE``)
| ``PyBUF_CONTIG_RO`` (``PyBUF_ND``)
- | ``PyBUF_CONTIG_LCK`` (``PyBUF_ND | PyBUF_LOCK``)
- | ``PyBUF_CONTIG_XLCK`` (``PyBUF_ND | PyBUF_WRITABLE | PyBUF_LOCK``)
Multi-dimensional using strides but aligned
| ``PyBUF_STRIDED`` (``PyBUF_STRIDES | PyBUF_WRITABLE``)
| ``PyBUF_STRIDED_RO`` (``PyBUF_STRIDES``)
- | ``PyBUF_STRIDED_LCK`` (``PyBUF_STRIDES | PyBUF_LOCK``)
- | ``PyBUF_STRIDED_XLCK`` (``PyBUF_STRIDES | PyBUF_LOCK | PyBUF_WRITABLE``)
Multi-dimensional using strides and not necessarily aligned
| ``PyBUF_RECORDS`` (``PyBUF_STRIDES | PyBUF_WRITABLE | PyBUF_FORMAT``)
| ``PyBUF_RECORDS_RO`` (``PyBUF_STRIDES | PyBUF_FORMAT``)
- | ``PyBUF_RECORDS_LCK`` (``PyBUF_STRIDES | PyBUF_LOCK | PyBUF_FORMAT``)
- | ``PyBUF_RECORDS_XLCK`` (``PyBUF_STRIDES | PyBUF_LOCK | PyBUF_FORMAT | PyBUF_WRITABLE``)
Multi-dimensional using sub-offsets
| ``PyBUF_FULL`` (``PyBUF_INDIRECT | PyBUF_WRITABLE | PyBUF_FORMAT``)
| ``PyBUF_FULL_RO`` (``PyBUF_INDIRECT | PyBUF_FORMAT``)
- | ``PyBUF_FULL_LCK`` (``PyBUF_INDIRECT | PyBUF_LOCK | PyBUF_FORMAT``)
- | ``PyBUF_FULL_XLCK`` (``PyBUF_INDIRECT | PyBUF_LOCK | PyBUF_FORMAT | PyBUF_WRITABLE``)
Thus, the consumer simply wanting a contiguous chunk of bytes from
the object would use ``PyBUF_SIMPLE``, while a consumer that understands
@@ -307,6 +278,10 @@
buffer info structure correctly according to the provided flags if a
contiguous chunk of "unsigned bytes" is all that can be exported.
+
+The Py_buffer struct
+--------------------
+
The bufferinfo structure is::
struct bufferinfo {
@@ -322,14 +297,15 @@
void *internal;
} Py_buffer;
-Before calling the bf_getbuffer function, the bufferinfo structure can be
-filled with whatever. Upon return from bf_getbuffer, the bufferinfo
-structure is filled in with relevant information about the buffer.
-This same bufferinfo structure must be passed to bf_releasebuffer (if
-available) when the consumer is done with the memory. The caller is
-responsible for keeping a reference to obj until releasebuffer is
-called (i.e. the call to bf_getbuffer does not alter the reference
-count of obj).
+Before calling the bf_getbuffer function, the bufferinfo structure can
+be filled with whatever, but the ``buf`` field must be NULL when
+requesting a new buffer. Upon return from bf_getbuffer, the
+bufferinfo structure is filled in with relevant information about the
+buffer. This same bufferinfo structure must be passed to
+bf_releasebuffer (if available) when the consumer is done with the
+memory. The caller is responsible for keeping a reference to obj until
+releasebuffer is called (i.e. the call to bf_getbuffer does not alter
+the reference count of obj).
The members of the bufferinfo structure are:
@@ -343,14 +319,7 @@
``readonly``
an integer variable to hold whether or not the memory is readonly.
- 1 means the memory is readonly, zero means the memory is writable,
- -1 means the memory was read "locked" when this Py_buffer
- structure was filled-in therefore should be unlocked when this
- Py_buffer structure is "released." A -2 means this Py_buffer
- structure has an exclusive-write lock on the memory. This should
- be unlocked when the Py_buffer structure is released. The concept
- of locking is not supported by all objects that expose the buffer
- protocol.
+ 1 means the memory is readonly, zero means the memory is writable.
``format``
a NULL-terminated format-string (following the struct-style syntax
@@ -359,7 +328,7 @@
is the number of bytes implied by the format. This can be NULL which
implies standard unsigned bytes ("B").
-``ndims``
+``ndim``
a variable storing the number of dimensions the memory represents.
Must be >=0. A value of 0 means that shape and strides and suboffsets
must be ``NULL`` (i.e. the memory represents a scalar).
@@ -440,21 +409,25 @@
when releasebuffer is called.
+Releasing the buffer
+--------------------
+
The same bufferinfo struct should be used in the release-buffer
-interface call. The caller is responsible for the memory of the
-Py_buffer structure itself.
+interface call. The caller is responsible for the memory of the
+Py_buffer structure itself.
+
+::
-``typedef void (*releasebufferproc)(PyObject *obj, Py_buffer *view)``
- Callers of getbufferproc must make sure that this function is
- called when memory previously acquired from the object is no
- longer needed. The exporter of the interface must make sure that
- any memory pointed to in the bufferinfo structure remains valid
- until releasebuffer is called.
+ typedef void (*releasebufferproc)(PyObject *obj, Py_buffer *view)
- Both of these routines are optional for a type object
+Callers of getbufferproc must make sure that this function is called
+when memory previously acquired from the object is no longer needed.
+The exporter of the interface must make sure that any memory pointed
+to in the bufferinfo structure remains valid until releasebuffer is
+called.
- If the bf_releasebuffer function is not provided (i.e. it is NULL),
- then it does not ever need to be called.
+If the bf_releasebuffer function is not provided (i.e. it is NULL),
+then it does not ever need to be called.
Exporters will need to define a bf_releasebuffer function if they can
re-allocate their memory, strides, shape, suboffsets, or format
@@ -520,10 +493,6 @@
Thus, this memory view object holds on to the memory of base until it
is deleted.
-The bf_getbuffer and bf_releasebuffer for this object increments and
-decrements the lock on base, but passes on the contents of view to the
-caller.
-
This memory-view object will support multi-dimensional slicing and be
the first object provided with Python to do so. Slices of the
memory-view object are other memory-view objects with the same base
@@ -571,7 +540,7 @@
::
PyObject * PyMemoryView_GetContiguous(PyObject *obj, int buffertype,
- char fort)
+ char fortran)
Return a memoryview object to a contiguous chunk of memory represented
by obj. If a copy must be made (because the memory pointed to by obj
@@ -586,9 +555,8 @@
can use PyBUF_UPDATEIFCOPY to ensure that a a writable temporary
contiguous buffer is returned. The contents of this contiguous buffer
will be copied back into the original object after the memoryview
-object is deleted as long as the original object is writable and
-allows applying a read-write lock. If this is not allowed by
-the original object, then a BufferError is raised.
+object is deleted as long as the original object is writable. If this
+is not allowed by the original object, then a BufferError is raised.
If the object is multi-dimensional, then if fortran is 'F', the first
dimension of the underlying array will vary the fastest in the buffer.
@@ -597,9 +565,7 @@
you will get whatever the object decides is more efficient. If a copy
is made, then the memory must be freed by calling ``PyMem_Free``.
-You receive a new reference to the memoryview object which will also
-hold a lock on the objects data area (this lock will be released when
-the memoryview object is deleted).
+You receive a new reference to the memoryview object.
::
@@ -816,12 +782,15 @@
adding the C-API and the two functions to the existing buffer
protocol.
-The proposed locking mechanism relies entirely on the exporter object
-to not invalidate any of the memory pointed to by the buffer structure
-until a corresponding releasebuffer is called. If it wants to be able
-to change its own shape and/or strides arrays, then it needs to create
-memory for these in the bufferinfo structure and copy information
-over.
+Previous versions of this PEP proposed a read/write locking scheme,
+but it was later perceived as a) too complicated for common simple use
+cases that do not require any locking and b) too simple for use cases
+that required concurrent read/write access to a buffer with changing,
+short-living locks. It is therefore left to users to implement their
+own specific locking scheme around buffer objects if they require
+consistent views across concurrent read/write access. A future PEP
+may be proposed which includes a separate locking API after some
+experience with these user-schemes is obtained
The sharing of strided memory and suboffsets is new and can be seen as
a modification of the multiple-segment interface. It is motivated by
@@ -860,8 +829,6 @@
this proposal but will welcome any help.
-
-
Examples
========
@@ -943,7 +910,7 @@
}
/* No releasebuffer is necessary because the memory will never
- be re-allocated so the locking mechanism is not needed
+ be re-allocated
*/
Ex. 3
More information about the Python-checkins
mailing list