[Cython] Static checker for cython extension dependencies?

[Robert Bradshaw]

We write metadata in the generated C files for just this reason. You
can fall back to something like
https://gist.github.com/robertwb/25ab9838cc2b9b21eed646834cf4a108 if
cython is not available.


On Fri, Jun 10, 2016 at 10:55 AM, Nathan Goldbaum <nathan12343 at gmail.com> wrote:
> The reason we haven't done that is we would like our setup.py script to be
> runnable without cython being installed. I think cythonize is being invoked
> (or something similar to it) by setuptools, using a feature added in
> setuptools 18.0:
> https://setuptools.readthedocs.io/en/latest/history.html#id60
>
> Is there a way to use cythonize for this build workflow without importing it
> at the top-level in our setup.py file?
>
> FWIW, our setup.py file is here:
> https://bitbucket.org/yt_analysis/yt/src/yt/setup.py?at=yt&fileviewer=file-view-default
>
> On Fri, Jun 10, 2016 at 12:49 PM, Robert Bradshaw <robertwb at gmail.com>
> wrote:
>>
>> You should be using cythonize rather than listing and maintaining the
>> Extension definitions themselves.
>>
>>
>> http://docs.cython.org/src/quickstart/build.html#building-a-cython-module-using-distutils
>> https://github.com/cython/cython/wiki/enhancements-distutils_preprocessing
>>
>> On Fri, Jun 10, 2016 at 9:18 AM, Nathan Goldbaum <nathan12343 at gmail.com>
>> wrote:
>> > Hi all,
>> >
>> > I'm working on a pretty large python/cython codebase (yt) that has many
>> > interdependent C extensions written in cython.
>> >
>> > I've found it to be pretty hit or miss to depend on contributors to
>> > manually
>> > update cython dependency information in our setup.py file. The
>> > dependency
>> > information seems to only be used by setuptools to trigger
>> > recompilation,
>> > but critically setuptools will happily compile a C extension for the
>> > first
>> > time if the depends information is incomplete. This means that during
>> > development, if I update a cython routine, there's no way to ensure that
>> > other cython routines that cimport the one I modified will be recompiled
>> > unless I manually ensure the depends information is updated whenever
>> > cython
>> > code gains or loses a cimport.
>> >
>> > To make that more concrete, here's a pull request I just made to yt that
>> > adds missing dependencies for a cython header. Without this pull
>> > request,
>> > setuptools fails to recompile these routines when selection_routines.pxd
>> > changes, causing a build failure.
>> >
>> > https://bitbucket.org/yt_analysis/yt/pull-requests/2220
>> >
>> > I think it should be possible to write a test for this by defining the
>> > dependency information outside of setup.py and parsing grep and looking
>> > for
>> > all cython files that cimport other cython files defined inside yt.
>> > However,
>> > before I do that, I'm curious whether anyone has done something similar,
>> > or
>> > if there is some other way of forcing the dependency information to be
>> > complete on the first compilation, rather than just for subsequent
>> > incremental recompilations during development.
>> >
>> > Thanks for your help!
>> >
>> > -Nathan
>> >
>> > _______________________________________________
>> > cython-devel mailing list
>> > cython-devel at python.org
>> > https://mail.python.org/mailman/listinfo/cython-devel
>> >
>> _______________________________________________
>> cython-devel mailing list
>> cython-devel at python.org
>> https://mail.python.org/mailman/listinfo/cython-devel
>
>
>
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> https://mail.python.org/mailman/listinfo/cython-devel
>