[C++-sig] Some questions when making a C++ project as python externsions

[Jim Bosch]

On 09/07/2010 12:19 AM, Binbin Shen wrote:
> Hi,
> I am new to boost.python, and want to provide python interface of a
> large lab application written in C++. I have written some wrappers and
> make some classes works in python, but also encounter some problems
> that cannot be solved by Googling, and I do not want to modify the
> source of the C++ application, some of the problems are as follows:
>

Here's an obvious-only-once-you've-seen-it trick that applies to a lot 
of your questions:  if a C++ function isn't in a form where you can wrap 
it with Boost.Python, write a new C++ function that calls it and wrap 
that.  This even works for member functions; if the first argument of a 
free function is a class type, it can be wrapped in Python as a member 
function of that class.


> 1. Some C++ functions have return the result by parameters like this:
> void add(std::wstring&  a, const wstring b) {
>      a+=b;
> }
>>>> a = unicode()
>>>> b = u"test"
>>>> add(a, b)               # error when compiling, unicode can not convert to wstring automatically
>
> I should pass a python unicode into this function and return by "a" implicitly.
>

Unicode may be a little tricky; I don't think Boost.Python handles it at 
all automatically.  You may have to use the Python C-API a bit (others 
on this list may know more).  You'll probably want to make one argument 
of your C++ wrapper function a boost::python::object; you can then 
extract a PyObject* with the .ptr() member function 
(boost::python::object is basically a fancy smart pointer for 
PyObject*).  You can then return a boost::python::object containing a 
real Python unicode object.

Without the unicode part, turning a result-by-arg function into a 
result-by-return is of course easy to solve with a one-line C++ wrapper 
function (and one of the wrapper-generators, Py++, can do this 
automatically for you in some cases).


> 2. Some C++ functions show pass a "void*" pointer that pointing a
> pre-allocated memory into the function, like this:
> void read(void *memory, int size, int count)
> {
>        // copy something into memory
> }
>
> How can I make a void pointer in python and pass it to read()?
>

That depends on what you want that void pointer points to.  Is it a 
wrapped C++ class, something that maps to a Python built-in type (or a 
numpy array), or is it just raw, unallocated memory?

What do you want to do with it in Python?  Just pass it to another 
wrapped C++ function?

There are solutions for all of these, but they all require some work.


> 3. Some virtual function with default arguments, like this:
> class foo {
> public:
>      virtual int add(int x, int y = 10) { return x+y; }
> };
> and the wrapper looks like:
> struct fooWrap {
>      int add(int x, int y) {
>          if (override add_ = this->get_override("add")
>              return add_(x, y);
>          return this->foo::add(x, y);
>      }
>      int default add(int x, int y) {
>          return this->foo::add(x, y);
>      }
> }
> and
>    .def("add",&foo::add,&fooWrap::default_add);
> but when dealing with overloads, it looks like:
> BOOST_PYTHON_MEMBER_FUNCTION_OVERLOADS(add_overloads, add, 1, 2)
> and
>     .def("add",&foo::add, add_overloads())
>
> How can I write these two together?

Not sure; I just haven't used the OVERLOADS macros much.  Hopefully 
someone else can answer.

>
> 4. Some functions have variable length parameter:
> std::wstring format(const wchar_t* strFormat, ...) {
>      // use va_list, va_start, va_end...
> }
>
> how boost.python deal with it?
>

It can't directly.  You'll have to write a C++ wrapper function.  You 
may want to look into boost::python::raw_function (see the reference 
documentation), which lets you write a Python function with a variable 
number of arguments.


> 5. The last question is that I do not quiet understand the policies,
> :-(, is there any detail documents introduce them:
> with_custodian_and_ward: Ties lifetimes of the arguments
> with_custodian_and_ward_postcall: Ties lifetimes of the arguments and results
> return_internal_reference: Ties lifetime of one argument to that of result
> return_value_policy<T>  with T one of:
> reference_existing_object: naive (dangerous) approach object
> copy_const_reference: Boost.Python v1 approach
> copy_non_const_reference:
> manage_new_object: Adopt a pointer and hold the instance
> return_by_value
> return_by_reference
> return_opaque_pointer
>
> I use boost 1.42 with gcc 4.4.3 in Gentoo Linux.
>
> Any advice would be gratefully received.
>

The ones you'll probably use most often are:

- return_internal_reference: Main use is when a class returns a pointer 
or reference to a data member.  It keeps the Python object wrapping the 
class instance alive as long as the Python object wrapping the data 
member is alive.

- return_value_policy + copy_const_reference or 
copy_non_const_reference: Instead of wrapping a reference, invoke the 
copy constructor and return the new object.

- manage_new_object: assumes the return value was allocated by the new 
operator and invokes delete when Python is done with it.

When you return by value or smart pointer, there's generally no need to 
use call policies, and Boost.Python is prettying good at throwing up a 
compile error when you need to use them.


Hope that helps!

Jim Bosch