Wrapping a C library in Python

[John Hunter]

>>>>> "Michael" == Michael Loritsch <loritsch at gmail.com> writes:

    Roy> For example, all of these functions return an error code
    Roy> (typically just errno passed along, but not always).  They
    Roy> all accept as one of their arguments a pointer to someplace
    Roy> to store their result.  I want to change all that to
    Roy> returning the result directly and throwing exceptions.

    Roy> I also want to mutate some of the return types.  A common way
    Roy> these functions return a set of values is a pair of arrays of
    Roy> strings, forming key-value pairs.  In Python, it would make
    Roy> sense to return this as a dictionary.

SWIG can do all this - see the section on typemaps and call policies
in the SWIG manual.

For example, it is easy to tell SWIG that pointers are used for output

  void somefunc(double *OUTPUT, double *OUTPUT)

will be called from python like

  x, y = o.somefunc()

And you need to do no more than add the one declaration line.  INPUT,
OUTPUT and INOUT are special tokens that SWIG recognizes and applies
translation rules too.  You can define your own such tokens to do more
complicated things (like turning a double *array into a python list,
etc)

    Michael> I recommend both boost::python
    Michael> (http://www.boost.org/libs/python/doc/index.html) and
    Michael> ctypes (http://starship.python.net/crew/theller/ctypes/),
    Michael> but for very different reasons.

    Michael> If wrapped code speed and production of a binary is a
    Michael> goal in creating your python extensions, use
    Michael> boost::python.  The price you pay for creating an fast
    Michael> binary python extension is coding your translation from
    Michael> python in C/C++ in C/C++ (I suppose you could get around
    Michael> this by creating a boost::python module, and then
    Michael> wrapping it with a python module to do the type
    Michael> translation).  And in boost::python, C++ to python
    Michael> exception translation is supported.

I don't fully agree with this.  I've been working on a wrapper for
antigrain, a C++ library that makes heavy use of templates.  I started
off using pyste (a boost::python generator) and boost.  It worked
reasonably well - pyste is not being actively maintained right but you
can usually work around the limitations by writing boost code where
you need to.  I was reasonably happy, until I saw my *.so files
ballooning.  After wrapping a small fraction of the library, and
having instantiated only a few of the many templates I ultimately
wanted, my extension files were at 20MB, which is *much larger* than
the agg library or moderately sophisticated applications built around
it.  And I still had *a lot* left to expose!

I started over in SWIG.  First, SWIG has excellent support for C++ and
templates, and in many cases could auto-wrap and entire header with

%include "someheader.h"

Second, by the time I had the SWIG wrapping to a comparable point that
the boost wrapping was at when I started over, I had only a 500K
extension module.  Same functionality, 40x smaller.  Since ultimately
I may want to distribute my software in compiled form (eg a windows
installer) this is an important difference.  Third, the compile times
were much shorter in SWIG.  Fourth, SWIG produces c and cxx files as
its output, which you can distribute with your app (user doesn't need
to have SWIG to compile).  This is not true for boost.

In a nutshell, for wrapping a large C++ library (not the original
poster's question, I know), I found SWIG more suitable for the reasons
above.  I don't want to slam boost - I think it is an awesome package
-- but you should be aware of these potential problems.  The ease of
wrapping agg was comparable in boost and SWIG.

Where boost (and pycxx) really shines above SWIG is when you want to
write functions and methods yourself that interact with python objects
-- for example if you were writing a python extension largely from
scratch rather than wrapping an existing library.  The ability to use
friendly boost C++ classes like dict and list that manage memory for
you and have a pythonic feel is great.

JDH