[SciPy-user] FYI: C++ Extensions for Python

[Prabhu Ramachandran]

>>>>> "SS" == Stefan Seefeld <sseefeld at art.ca> writes:

    SS> Just to balance this a bit: I must agree that the
    SS> documentation is very sparse, but otherwise it's a wonderful

I agree that it is an excellent tool.

[...]
    SS> extremely powerful yet easy to write (as an example: is there
    SS> any other binding that lets you define classes in C++ and
    SS> derive from them in python ?)

Yes, SWIG with the "directors" option.  Support for directors has been
around for atleast a year now!  SWIG seems to have a bad reputation
from its 1.1 era, the new versions are really powerful and support a
large subset of C++.

I admire boost.python and tried very hard to use it for my work.
Unfortunately it did not work out.  I needed to wrap close to 80 C++
classes so hand wrapping them was out of question.  I had to use Pyste
and fixed a bunch of bugs with that.  The biggest problem for me was
boost.python wrappers take *forever* to build with GCC.  In the end, I
was dealing with raw pointers in my C++ code and for reasons too
complex to discuss here that caused problems that were not trivial to
handle without writing code by hand.  At that point I had spent close
to 3 weeks on trying to get wrappers working and finally gave up.

I then tried SWIG and it worked really well.  Getting working wrappers
took very little time and then improving on that (there were bugs that
were fixed etc.) took a some time but eventually it worked
beautifully.  In all I spent less time on SWIG than boost.python and
it did the job well.  SWIG wrapper code is ANSI C so it takes close to
a tenth of the time to build the wrappers.  For a fairly trivial
example I made some simple comparisons between the two and presented
them last year at SciPy'03.  You can find the code and slides here:

 http://www.ae.iitm.ac.in/~prabhu/research/papers/swig_boost.zip

Currently, I use SWIG to build Python wrappers for about 80 classes.
Classes can be subclassed in Python.  Documentation from doxygen
comments in the C++ code is extracted and inserted into the shadow
Python docstrings using a script[1] that parses the XML generated by
doxygen and generates the necessary SWIG interface file.  Building the
wrappers takes about 10-15 minutes to build on an outdated machine.
Generating the wrappers takes around 500 lines of SWIG interface code.

Of course, SWIG also has support for so many other languages (atleast
10 other languages apart from Python!).

On the downside, SWIG does not yet support nested classes.  AFAIK it
does not support expression templates.  However, wrapping code that
uses expression templates might not be very common.  I use templates
but quite sparingly so that is not a problem for me.  The other
potential issue is that SWIG uses proxy classes to represent the
Python classes, whereas boost.python generates everything in the
extension module.  In reality this works both ways for SWIG, because
the proxy classes are pure Python you can do some neat magic there.
However, some folks have reported slowdowns because of this.  I don't
have a problem with this though.  I initially thought that the size of
the Python proxy files would cause a lot of bloat and problems.  In
practice, I have not had any prolems.

Also, it was quite easy (a days effort) to make SWIG2 and weave work
well together, so I can do inline C++ from Python on any of my wrapped
code.  All of these make SWIG incredibly attractive for my work.

Before you think otherwise, I'd like to mention that I think
boost.python is an excellent tool!  It has its place and I would think
there are some things that it would do better than any other tool.

Its just that SWIG is often underestimated and for me SWIG has worked
*really* nicely.  For the most part it is true that SWIG "just works".


cheers,
prabhu

[1] http://www.ae.iitm.ac.in/~prabhu/software/python.html and
    http://www.ae.iitm.ac.in/~prabhu/software/code/python/doxy2swig.py