[C++-sig] boost::python: marshalling buffers

[Jim Bosch]

On 04/18/2012 01:57 PM, Christoph Rupp wrote:
> Hi,
>
> i have a C++ class that i'm wrapping with boost::python:
>
> class Foo {
>    // ...
>    const char *get_buffer();
>    size_t get_size() const;
>    // ...
> };
>
> I want to send this class through a thrift interface (which uses the
> write() method to send the data to a file or a socket). For a few days now
> i'm trying to implement this. By now i understand that i have to implement
> the buffer protocol, and i know that i can create a buffer with
> boost::python (read-only access is sufficient):
>
> inline object as_buffer(const Foo&f) {
>    return object (handle<>  (PyBuffer_FromMemory ((void*)f.get_buffer(),
> f.get_size())));
> }
>
> (This is based on an email from 2009 that i found in this mailing list's
> archive.)
>
> But i do not know how to glue it together to form a full module. This is
> how the module looks right now:
>
> BOOST_PYTHON_MODULE(Foo)
> {
>    class_<Foo, boost::noncopyable>("Foo",
>            init<int32_t, bool>())
>      .def("add", afn)
>      .def("finalize",&Foo::finalize)
>      .def("empty",&Foo::empty)
>      .def("clear",&Foo::clear)
>    ;
> }
>
> How can i include that buffer conversion?
>

Sorry about the long delay in a reply, but I'm afraid that right now it 
looks *really* tricky (possibly impossible) to add the buffer protocol 
to a Boost.Python class.  Python doesn't have a special named methods 
for the buffer protocol:

http://stackoverflow.com/questions/2079272/which-methods-implement-the-buffer-interface-in-python

And Boost.Python doesn't give you access to the PyTypeObject structure 
where you'd be able to add the PyBufferProcs slots.

I think you have two options:

  - Wrap your buffer-getter function with another name and find a way to 
get your serialization code to use that.

  - Wrap this class class using the raw Python C-API and use 
Boost.Python custom lvalue converters to make it work with any 
Boost.Python-wrapped code you have.  You could use Boost.Python to wrap 
all the member functions, for instance, and add them to the Python C-API 
class's dict.

Neither option is great, but hopefully one will allow you to move forward.

HTH!

Jim