[SciPy-dev] cython in scipy?

[Fernando Perez]

On Wed, Oct 8, 2008 at 9:38 PM, Anne Archibald
<peridot.faceted at gmail.com> wrote:
> Hi,
>
> I've just been trying to clean up scipy.stats.vonmises.cdf: it's
> currently badly broken (tickets 747, 748, and 749). I can fix the
> outright bugs, but it also has distressingly large roundoff error
> under some circumstances. I'd like to simply implement the standard
> algorithm (Hill 1977) for evaluating it. Unfortunately, it involves
> evaluating a series with a number of terms that depends on the
> parameter, so vectorizing a python version (which is already written)
> is going to be a nuisance. On the other hand, it would be very easy to
> translate this into cython and get C performance out of it. So: is it
> okay to include cython code in scipy? What is the best way to go about
> this? Include the cython source in setup.py somehow? Check the
> generated C into svn and run cython by hand when I remember to?

This is the kind of very simple code I've used in the past (back in
the pyrex days, same thing for cython):

# Make this usable by people who don't have pyrex installed (I've committed
# the generated C sources to SVN).
try:
    from Pyrex.Distutils import build_ext
    has_pyrex = True
    cmdclass  = {'build_ext': build_ext}
except ImportError:
    has_pyrex = False
    cmdclass  = {}

# Set up key extension
key_sources = ['mwadap/key.pyx']
if not has_pyrex:
    key_sources = [s.replace('.pyx','.c') for s in key_sources]
keyx = Extension('mwadap.key',key_sources,
                 include_dirs = [numpy_include_dir])

###

Instead of triggering the use of the included C on the absence of
Cython, you could just trigger on the absence of the C sources to fall
back on the .pyx files (or even timestamp differences if you want).
This way, you can just nuke the C files with a little utility script
whenever you want and Cython will regenerate them for you at build
time.  Since they're checked into SVN, next  time you commit the
updated version will go back up.

The choice of whether to trigger the rebuild simply on timestamp
differences (a la .py/.pyc) or by manual removal of the C is up to
you, but the resulting workflow is pretty straightforward in any case.

Cheers,

f