ctypes And The WACAH Principle

[Lawrence D’Oliveiro]

GvR has said “we’re all consenting adults here”. He was referring to the “public/private/protected” access-control business that most object-oriented languages require you to go through. But there is another area where I think this principle applies to Python, at least in part: low-level access to arbitrary memory.

For the most part, Python tries to avoid you having to deal with this. One important exception is the wonderful and useful ctypes module <https://docs.python.org/3/library/ctypes.html>, which by its very nature, requires you to worry about such things. But what’s so nice about ctypes is that you can wrap up such concerns in a Pythonic, high-level layer that others can easily use without knowing that such machinations are going on behind the scenes. However, precisely because of the lack of access control, the abstraction is leaky, so users can penetrate it if they feel they have a need to, and are prepared to face the consequences--so WACAH very much applies here.

When creating ctypes wrappers for libraries, one often comes across a call to the effect of “create object referencing given data”. That is, the caller supplies (and manages) the block of memory holding the data. For example, the Cairo graphics library has cairo_image_surface_create_for_data <https://www.cairographics.org/manual/cairo-Image-Surfaces.html#cairo-image-surface-create-for-data>, which treats the supplied block of memory as an array of pixels. Cairo also thoughtfully provides additional mechanisms to coordinate between its accesses, and those of the caller, to that block of memory.

For the most part, Python doesn’t make it easy to allocate an object and get access to its internal memory representation. One interesting exception is array.array objects <https://docs.python.org/3/library/array.html>, which have the buffer_info() method, directly returning both the address and length of the underlying memory block. The documentation for buffer_info() says:

    When using array objects from code written in C or C++ (the only way to
    effectively make use of this information), it makes more sense to use the
    buffer interface supported by array objects.

Ah, I wish... I had a look at the documentation for the buffer interface and memory views, and found no way to make use of them from pure Python. Basically, the paragraph I quoted above is wrong in just about every important detail.

So, for now, if I want to use Python-allocated memory with lower-level APIs via ctypes, the array module remains the way to go.