Who owns the memory in ctypes?

[eryk sun]

On Tue, Nov 15, 2016 at 3:15 AM, Cem Karan <cfkaran2 at gmail.com> wrote:
> if my C code allocates memory via GC_MALLOC() (the standard call for allocating memory
> in the garbage collector), and I access some object via ctypes in python, will the python
> garbage collector assume that it owns it and attempt to dispose of it when it goes out of
> scope?

ctypes objects own only the memory that they allocate. Inspect the
_b_needsfree_ attribute to determine whether a ctypes object owns the
referenced memory.

For example:

This array object owns a 12-byte buffer for the array elements:

    >>> arr = (ctypes.c_uint * 3)(0, 1, 2)
    >>> arr._b_needsfree_
    1

This pointer object owns an 8-byte buffer for the 64-bit target address:

    >>> p = ctypes.POINTER(ctypes.c_uint * 3)(arr)
    >>> p._b_needsfree_
    1

The following new array object created by dereferencing the pointer
does not own the 12-byte buffer:

    >>> ref = p[0]
    >>> ref[:]
    [0, 1, 2]
    >>> ref._b_needsfree_
    0

However, it does have a reference chain back to the original array:

    >>> ref._b_base_ is p
    True
    >>> p._objects['1'] is arr
    True

On the other hand, if you use the from_address() class method, the
resulting object is a dangling reference to memory that it doesn't own
and for which there's no supporting reference chain to the owning
object.

For example:

    >>> arr = (ctypes.c_uint * 2**24)()
    >>> arr[-1] = 42
    >>> ref = type(arr).from_address(ctypes.addressof(arr))
    >>> ref[-1]
    42

    >>> ref._b_base_ is None
    True
    >>> ref._objects is None
    True

2**24 bytes is a big allocation that uses mmap (Unix) instead of the
heap. Thus accessing ref[-1] causes a segfault after arr is
deallocated:

    >>> del arr
    >>> ref[-1]
    Segmentation fault (core dumped)