[SciPy-dev] Inclusion of cython code in scipy

[Anne Archibald]

Hi,

What do people think about including cython code in scipy?

Pros:
* Easy way to write faster code
* Convenient way to interface to compiled code

Cons:
* Yet another interfacing setup
* Yet another language readers of the code need to understand
* Additional dependency for developers
* cython is in flux and drastic improvements in its handling of numpy
arrays are hoped for
* Requires support in build tools (distutils/makefiles/scons magic)

For one data point, a quick and nasty cythonization of my polynomial
interpolation code got something like a 4x speedup for large arrays of
evaluation points. This is a big improvement but far from the 20-100x
cython claims for raw python. I can't judge the behaviour for shorter
arrays without doing a more reasonable job of converting to cython,
but by looking at the (horrible) generated C code, the inner loops are
raw C with no python objects at all. (The 4x figure also uses gcc's
-march and -O options.) I take this as evidence that numpy's arrays
are doing a fairly good job of handing off the heavy lifting to
compiled code.

I can definitely see that if a little work went into cython's support
of numpy arrays specifically the code could get a lot cleaner: things
like c[i*k+j] could become c[i,k] with support for arbitrary strides
and (possibly) type conversion. So maybe it doesn't make sense to
start producing lots of cython code until that happens.

Anne