[Python-Dev] PEP 296 - The Buffer Problem

[Thomas Heller]

Let me ask some questions and about platforms with 32-bit
integers and 64-bit longs:

>     2. Alignment of the allocated byte array is whatever is promised by the
>     platform implementation of malloc.

On these platforms, does malloc() accept an unsigned long argument
for the requested size?

> [...]
>     8. The bytes object keeps track of the length of its data with a Python
>     LONG_LONG type.
> [...]
>     From Python scripting, the bytes object will be subscriptable with longs
>     so the 32 bit int limit can be avoided.

How is indexing done in C? Can you index these byte arrays by longs?

>     9. The bytes object can be constructed at the Python scripting level by
>     passing an int/long to the bytes constructor with the number of bytes to
>     allocate.  For example:
> 
>        b = bytes(100000) # alloc 100K bytes
> 
>     The constructor can also take another bytes object.  This will be useful
>     for the implementation of unpickling, and in converting a read-write bytes
>     object into a read-only one.  An optional second argument will be used to
>     designate creation of a readonly bytes object.
> 
>     10. From the C API, the bytes object can be allocated using any of the
>     following signatures:
> 
>        PyObject* PyBytes_FromLength(LONG_LONG len, int readonly);
>        PyObject* PyBytes_FromPointer(void* ptr, LONG_LONG len, int readonly
>                 void (*dest)(void *ptr, void *user), void* user);

Hm, if 'bytes' is a new style class, these functions should
require a 'PyObject *type' parameter as well. OTOH, new style
classes are usually created by calling their *type*, so you
should describe the signature of the byte type's tp_call.
(It may be possible to supply variations of the above functions
for convenience as well.)

>     The array module supports the creation of an array of bytes, but it does
>     not provide a C API for supplying pointers and destructors to extension
>     supplied memory.  This makes it unusable for constructing objects out of
>     shared memory, or memory that has special alignment or locking for things
>     like DMA transfers.  Also, the array object does not currently pickle.
>     Finally since the array object allows its contents to grow, via the extend
>     method, the pointer can be changed if the GIL is not held while using it.
...or if any code is executed which may change the array object, even
if the GIL is held!

Thomas