[SciPy-dev] Inclusion of cython code in scipy

[Brian Granger]

On Thu, Apr 24, 2008 at 10:25 AM, Francesc Altet <faltet at carabos.com> wrote:
> A Thursday 24 April 2008, Brian Granger escrigué:
>
> > > Then why not start a project to get SWIG to do away with the shadow
>  > > module?
>  >
>  > I don't have a passion for actually working seriously on these
>  > projects - I am lazy and simply want to use them :)
>  >
>  > >  Lets take a simple case of someone wanting to handle a growing
>  > >  collection of say a million particles and do something to them.
>  > > How do you do that in cython/pyrex and get the performance of C and
>  > > interface to numpy?  Worse, even if it were possible, you'll still
>  > > need to know something about allocating memory in C and
>  > > manipulating pointers.  I can do that with C++ and SWIG today.
>  >
>  > C++ STL containers are truly great for things like this.  I would
>  > write a simple c++ header file that defines the Particle class,
>  > create an std::vector<Particle> to hold them and wrap the whole thing
>  > into Cython.  I have already done stuff like this (templated c++ with
>  > STL) with cython and it works great.  Furthermore, you end up with an
>  > actual C/Python extension type that 1) is super fast 2) has its own C
>  > api that can be called from other C extensions.
>  >
>  > Sure SWIG can do this too.  If you don't mind the extra pure python
>  > layer that SWIG has, then it is really a matter of personal
>  > preference and experience.  But, if performance is really important
>  > and you don't want the extra python layer, SWIG doesn't cut it.  The
>  > one place where Cython does not do well with C++ code currently is if
>  > you have a very large C++ API (100s of classes).  Then SWIG is the
>  > way to go.
>
>  Can you develop a bit more in detail why Cython should not be adequate
>  for wrapping 100s of C++ classes?  Just curious.

Sure.  With Cython, you have to wrap each C++ class by hand.  It is
not a lot of coding, but to do it well, it does take time.  Whereas
for SWIG, a lot of that can be automated with a good .i file.  To make
matters worse, because Cython does not have _native_ support for
generating template instances, you currently have to wrap each
template instance by hand.  If you have classes that are templated
over multiple things, each of which are templated over multiple things
themselves, you end up having an exponential explosion in the number
of hand wrapped classes you need to handle.  For some of this, simple
code generation techniques would work fine, but for other things, it
would be quite painful.  It really depends on how patient you are :)

Brian

>  Thanks,
>
>
>  --
>  >0,0<   Francesc Altet     http://www.carabos.com/
>  V   V   Cárabos Coop. V.   Enjoy Data
>   "-"
>  _______________________________________________
>
>
> Scipy-dev mailing list
>  Scipy-dev at scipy.org
>  http://projects.scipy.org/mailman/listinfo/scipy-dev
>