ctypes And The WACAH Principle

[eryk sun]

On Wed, Aug 10, 2016 at 1:45 AM, Lawrence D’Oliveiro
<lawrencedo99 at gmail.com> wrote:
>
> Ah, I wish... I had a look at the documentation for the buffer interface and memory views,
> and found no way to make use of them from pure Python. Basically, the paragraph I quoted
> above is wrong in just about every important detail.

ctypes classes have from_buffer and from_buffer_copy methods that use
the buffer protocol. For example, for a read-only buffer you can use
from_buffer_copy:

    >>> b = bytes(b'1234')
    >>> a = (ctypes.c_char * 3).from_buffer_copy(b, 1)
    >>> a[:]
    b'234'

from_buffer requires a writable buffer, a bit more explanation, and a
warning for Python 2. Switch to a bytearray to allow using
from_buffer:

    >>> b = bytearray(b'1234')
    >>> a = (ctypes.c_char * 3).from_buffer(b, 1)
    >>> a[:]
    b'234'

The memory is shared:

    >>> b[:] = b[::-1]
    >>> a[:]
    b'321'

from_buffer in Python 3 uses a memoryview to reference the source
object and exported buffer, which it stores in the _objects dict of
the ctypes data instance:

    >>> type(a._objects['ffffffff'])
    <class 'memoryview'>
    >>> a._objects['ffffffff'].obj
    bytearray(b'4321')

Because of the existing export, the bytearray cannot be resized (up or down):

    >>> del b[0]
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    BufferError: Existing exports of data: object cannot be re-sized

On the other hand, from_buffer in Python 2 naively uses
PyObject_AsWriteBuffer and only keeps a reference to the source
object. Thus there's no guarantee that the memory will remain valid.
Let's demonstrate this with a segfault:

    >>> b = bytearray(2**20)
    >>> a = (ctypes.c_char * 2**20).from_buffer(b)
    >>> b[-1] = 'z'
    >>> a[-1]
    'z'
    >>> del b[:]
    >>> a[-1]
    Segmentation fault (core dumped)