Buffers and pointers (Py3.4)

[Neal Becker]

When I worked on talking to some hardware in the past, the hardware was 
directly mapped to memory, and then numpy mmap worked great.  Maybe not the 
same hardware interface you have.

Rob Gaddi wrote:

> So long as we're all feeling so concerned about speed lately...
> 
> I've got a piece of hardware that allows me to call a C library that
> calls an ioctl() that kicks off a DMA transfer and moves data between
> process memory and external hardware.  I would like a way to turn an
> arbitrary object implementing the buffer protocol into a pointer so that
> I can pass that pointer to the library to be passed to the hardware to
> stream my data.
> 
> The obvious question on your minds is why on earth am I doing this.
> Example case: my hardware is being used as a bridge to talk to a 16
> channel transient digitizer, acquiring data at 500 ksps per channel.  I
> would like my Python application to be able to acquire, say, 10M samples
> per channel.  I can have a list of 16 10M NumPy arrays, and every 5 ms I
> can request a single DMA transfer to grab all the new data that's been
> acquired and push it, zero-copy, into my arrays before the hardware
> FIFOs get full and I start losing data.
> 
> If I knew the buffers would always be NumPy arrays I could use
> ndarray.ctypes.data to get the pointer.  Always ctypes buffers, I could
> just use ctypes.addressof.  In either case I can flip them into
> memoryviews, but nothing seems to give me a pointer from that.
> Bytearrays, array.arrarys..., I've got nothin'.
> 
> If I were writing this as a C extension, getting that information from
> any buffer object would be trivial, but that changes my project from a
> pure Python wrapper using ctypes to a mixed language project with
> compile dependencies and mental headaches.
> 
>    buffertype = c_uint8 * size
>    return addressof(buffertype.from_buffer(buf, offset))
> 
> works but is inefficient and woefully inelegant.  I saw some discussion
> on the ctypes bug tracker https://bugs.python.org/issue27274 about this
> nearly a year ago, but no actual action.
> 
> I really want to take all of this low-level horribleness, wrap it up
> into a higher-level API, and never be subject to the gory details again.
>   This pointer thing is about the last hangup.  Ideas?
>