[Python-Dev] marshal (was:Buffer interface in abstract.c? )

[David Ascher]

On Sat, 14 Aug 1999, Greg Stein wrote:

> Having class instances respond to the buffer interface is interesting,
> but until more code attempts to *use* the interface, I'm not quite sure
> of the utility...

Well, here's an example from my work today.  Maybe someone can suggest an
alternative that I haven't seen.

I'm using buffer objects to pass pointers to structs back and forth
between Python and Windows (Win32's GUI scheme involves sending messages
to functions with, oftentimes, addresses of structs as arguments, and
expect the called function to modify the struct directly -- similarly, I
must call Win32 functions w/ pointers to memory that Windows will modify,
and be able to read the modified memory). With 'raw' buffer object
manipulation (after exposing the PyBuffer_FromMemoryReadWrite call to
Python), this works fine [*].  So far, no instances.

I also have a class which allows the user to describe the buffer memory
layout in a natural way given the C struct, and manipulate the buffer
layout w/ getattr/setattr.  For example:

class Win32MenuItemStruct(AutoStruct):
    #
    # for each slot, specify type (maps to a struct.pack specifier),
    # name (for setattr/getattr behavior) and optional defaults.
    #
    table = [(UINT, 'cbSize', AutoStruct.sizeOfStruct),
             (UINT, 'fMask', MIIM_STRING | MIIM_TYPE | MIIM_ID),
             (UINT, 'fType', MFT_STRING),
             (UINT, 'fState', MFS_ENABLED),
             (UINT, 'wID', None),
             (HANDLE, 'hSubMenu', 0),
             (HANDLE, 'hbmpChecked', 0),
             (HANDLE, 'hbmpUnchecked', 0),
             (DWORD, 'dwItemData', 0),
             (LPSTR, 'name', None),
             (UINT, 'cch', 0)]

AutoStruct has machinery which allows setting of buffer slices by slot
name, conversion of numeric types, etc.  This is working well.

The only hitch is that to send the buffer to the SWIG'ed function call, I
have three options, none ideal:

   1) define a __str__ method which makes a string of the buffer and pass
      that to the function which expects an "s#" argument.  This send
      a copy of the data, not the address.  As a result, this works
      well for structs which I create from scratch as long as I don't need
      to see any changes that Windows might have performed on the memory.

   2) send the instance but make up my own 'get-the-instance-as-buffer'
      API -- complicates extension module code.

   3) send the buffer attribute of the instance instead of the instance --
      complicates Python code, and the C code isn't trivial because there
      is no 'buffer' typecode for PyArg_ParseTuple().

If I could define an 

  def __aswritebuffer__

and if there was a PyArg_ParseTuple() typecode associated with read/write
buffers (I nominate 'w'!), I believe things would be simpler -- I could
then send the instance, specify in the PyArgParse_Tuple that I want a
pointer to memory, and I'd be golden.

What did I miss?

--david

[*] I feel naughty modifying random bits of memory from Python, but Bill
    Gates made me do it!