[Cython] import-free setuptools integration

[Nils Werner]

I have implemented some very simple setuptools integration that circumvents
the requirement to `import cython` in `setup.py` entirely, doing away with
the
century-old problem of having to importing requirements during installation:

<https://github.com/cython/cython/pull/477>

This implementation is inspired by CFFI's setuptools integration:
<https://cffi.readthedocs.org/en/latest/cdef.html#distutils-setuptools>.

They do not require you to `import cffi` but simply define an additional
keyword
argument `cffi_modules` for `setup()`. This additional keyword argument
_does not
raise errors when cffi is not installed yet_ but can be used once cffi is
there
(as defined in `setup_requires`). Later setuptools will call cffi and have
it do
whatever it wants with the contents of the argument.

Cython can do the exact same. Below you have the usual `setup.py`, with the
chicken and egg `import` problem and a few `Extensions` that need to be
`cythonize`d:

    import setuptools
    from Cython.Build import cythonize  # this will fail if cython is not
present prior to running `pip install this`
    from setuptools.extension import Extension

    extensions = [
        Extension("fib", ["fib.pyx"]),
        Extension("fib2", ["fib2.pyx"]),
    ]

    setuptools.setup(
        name='example',
        version="1.0.0",
        packages=setuptools.find_packages(),
        setup_requires=['cython'],
        install_requires=['cython'],
        ext_modules=cythonize(extensions),
    )

With the changes I am proposing the usual

    ext_modules=cythonize(extensions),

can be replaced with

    cython_modules=extensions,

removing the need for `from Cython.Build import cythonize` and solving the
import problem:

    import setuptools
    from setuptools.extension import Extension

    extensions = [
        Extension("fib", ["fib.pyx"]),
        Extension("fib2", ["fib2.pyx"]),
    ]

    setuptools.setup(
        name='example',
        version="1.0.0",
        setup_requires=['cython'],  # we must later require a Cython
version that has this kind of integration
        install_requires=['cython'],
        cython_modules=extensions,
    )

This allows a nicer automated installation of tools that depend on cython
and
also allows end users to keep their `setup.py` cleaner and leaner.

Additionally, I have also extended `cythonize()` a bit and thus allow for
definitions like:

    import setuptools
    from setuptools.extension import Extension

    extensions = [
        Extension("fib", ["fib.c"]),
        Extension("fib2", ["fib2.c"]),
    ]

    setuptools.setup(
        name='example',
        version="1.0.0",
        extras_require={'cython': ['cython']},
        ext_modules=extensions,
        cython_modules=extensions,
    )

This will automatically compile pyx->c if cython is installed and a
re-compilation is
needed. Otherwise it will merely compile c->so.

This allows devs to ship the generated C-code (as you suggest per your
docs) but
takes the heuristic to find out whether to compile pyx->C->so or only C->so
completely out of `setup.py` and under control of cython.

The change is written to be backwards compatible:

 - Using `cython_modules` is entirely optional and not using it is
sideeffect-free
 - Having `cythonize()` internally automatically compile *.pyx files when
*.c files
   are provided is disabled by default and must be enabled using the
`replace_extension`
   keyword argument (`cython_modules` enables that option).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cython-devel/attachments/20160205/3f6d21e6/attachment-0001.html>