[SciPy-dev] Inclusion of cython code in scipy

[Brian Granger]

>  Writing a substantial amount of code in a special-purpose language
>  like Cython is always risky.  Will Cython exist in 5,10,15 years?

At some level, I agree with this assessment.  I think it would be a
bad decision if there was a strong chance that cython would vanish
while scipy/numpy was still using it actively.  But I think this is
unlikely and here's why:


1) The high-profile scientific computing+python projects that are
written using it (or pyrex):

mpi4py (mpi binding in python - currently being rewritten to use cython)
pytables (hdf5 python bindings)
sage
numpy.random

RIght now, the worst case scenario is if cython became unmaintained.
But, with the large number of people depending on it, I think the
community would simply pick up the slack - the beauty of open source.

2) Cython is unique in what it does.

All the other solutions for making python code fast (swig, f2py,
ctypes, boost, etc.) still require spending lots of time writing low
level C/C++/fortran code.  Sure, many of us have the skills to do
that, but the reality is that few of us have the _time_ to write that
code.  If I had to write the ipython distributed array package in
C/C++ and then wrap it into python it simply _would_not_happen_
(because of time constraints).


>  I have little doubt that in 10 years I'll still be able to compile the
>  C/C++ codes I develop today.  Furthermore, if Cython/SWIG/SciPy/NumPy
>  disappeared overnight I'd still have a decent library to work with.
>  IMO using Cython for one-off application or wrappers is fine, but
>  large-scale Cython implementation is probably a bad long-term
>  approach.

True, a C/C++ library would live on _if_ these projects vanished.  But
I don't think you have made a strong argument that these projects will
in fact vanish.  Furthermore, I think your actions say the opposite.
You have invested a huge amount of time working on scipy.sparse (which
is greatly appreciated).  I could be wrong, but I don't think you
would do that if you really thought scipy was going to die soon :)

Another relevant point:  recently I have spent quite a lot of time
wrapping templated C++ code with Cython (a 1 million LOC parallel c++
electromagnetics code).  This code uses the STL extensively and Cython
has worked really well.  If someone really wants to use cython in the
same way you have used swig, that is completely possible - you still
end up with the nice C++ library underneath.

The big difference though between cython and swig is that the top
level layer in swig is slow pure python code.  In cython, you get fast
C/python extension types that can be called from other C/C++ extension
code without going through a slow python layer.  That is a huge
difference.

Brian

>  I don't mean to dismiss the merits of Cython.  It certainly has a
>  niche and might very well become the "right" way to interface Python
>  to compiled code.  OTOH I find the Cython proponents to be a bit
>  overzealous, so I'm wary of signing on just yet :)
>
>  --
>  Nathan Bell wnbell at gmail.com
>  http://graphics.cs.uiuc.edu/~wnbell/
>
>
> _______________________________________________
>  Scipy-dev mailing list
>  Scipy-dev at scipy.org
>  http://projects.scipy.org/mailman/listinfo/scipy-dev
>