Buffers and pointers (Py3.4)

[Rob Gaddi]

On 04/17/2017 11:47 AM, Chris Angelico wrote:
> On Tue, Apr 18, 2017 at 3:58 AM, Rob Gaddi
> <rgaddi at highlandtechnology.invalid> wrote:
>> If I were writing this as a C extension, getting that information from any
>> buffer object would be trivial, but that changes my project from a pure
>> Python wrapper using ctypes to a mixed language project with compile
>> dependencies and mental headaches.
>>
>>   buffertype = c_uint8 * size
>>   return addressof(buffertype.from_buffer(buf, offset))
>>
>> works but is inefficient and woefully inelegant.
>
> Have you considered Cython? It can massively simplify a lot of the
> work in doing this kind of thing.
>
> ChrisA
>

I did, and it definitely seems easier than grunting it up from pure C, 
but it still adds a compile step and extra tools, and I found the 
documentation on working with arbitrary memory to be lacking.  I know C, 
I know Python, I'm definitely unclear as to how to use Cython to split 
the difference.  Somewhere I've got the book and really need to get 
around to reading it.

I actually got to an implementation that isn't awful.  Creating new 
objects that represent each DMA transfer (and wrap a Structure to be 
passed to the library) takes about 9 us per object, including a call to:

   def setbuffer(self, buf, start=0, end=None):
     """Set the local memory.

     Parameters
        buf - The local memory, an object implementing the buffer
              protocol such as a numpy.ndarray or a bytearray.  Can
              be None, which invalidates the transfer.

        start - The index of the first buffer element to transfer.
                Defaults to 0.

        end - The index of the last buffer element to transfer.  Defaults
              to None for len(buf).
     """

     if buf is None:
       self._localmem = None
       self._desc.ptr = 0
       return

     basemv = memoryview(buf)
     mv = basemv[start:end]

     if not mv.contiguous:
       raise ValueError("noncontiguous buffer")
     self._localmem = mv
     self._desc.size = mv.nbytes
     self._desc.ptr = addressof(c_uint8.from_buffer(mv))

This could obviously be made faster by dropping some of the safety 
checks and flexibility, but I feel like the speed hit is worth not 
getting segfaults down the line because someone gets the clever idea to 
stride the array.  Plus the objects can be persistent, just calling 
setbuffer to update for the new destination, which takes a lot of that 
time back off.

-- 
Rob Gaddi, Highland Technology -- www.highlandtechnology.com
Email address domain is currently out of order.  See above to fix.