[Distutils] setuptools special case Pyrex and break Cython

[Phillip J. Eby]

At 08:00 PM 9/5/2007 +0200, Stefan Behnel wrote:

>Phillip J. Eby wrote:
> > At 08:30 AM 9/5/2007 +0200, Stefan Behnel wrote:
> >> > Perhaps you'd care to produce a patch to implement that "cleaner step"?
> >> > It's not at all obvious to me how to do that without introducing
> >> > instability that would be unsuitable for an 0.6cN release.
> >>
> >> One way of implementing the above change would be to move the
> >> replacement code
> >> into build_ext rather than Extension. Something like the (untested)
> >> build_ext-patcher.py I attached. Note the type check that tests for
> >> build_ext
> >> being subclassed.
> >
> > You're illustrating my point.  It's easy to hand-wave about how it
> > should be done, but not so easy to actually *do*.  Did you look at where
> > all the .sources attribute gets used?  How the build_ext command can get
> > called by other commands?
>
>Sort of. Do you doubt it should work that way?

I know that other commands use the .sources of build_ext indirectly, 
at a time when run() has not yet been called.  That means that your 
approach may change those other commands' behavior in non-obvious 
ways.  That's why I'm not going to do something like that in an 0.6cN release.


> > You're also ignoring my larger point: the current mechanism allows you
> > to write setup scripts that *don't* need to subclass build_ext.  A
> > setuptools-based setup script just refers to '.pyx' files, and
> > everything else happens automatically.
>
>The script I posted doesn't change that. Read it.

Actually, now that I've read it in detail, I see that it's also 
broken in other ways, such as confusing modules and classes.

But it's also broken in the way I said: it *requires* subclassing of 
build_ext in setup.py, which the current mechanism does not.



> > And if you need to be able to distinguish between Cython-specific and
> > Pyrex-specific files, why are you using the same file extension?
>
>We don't distinguish, it's the same language. But users will have to install
>both of them

If it's the "same language" then, *why* do they need this?  If Pyrex 
is still required, then clearly it's *not* the "same language".


>You're actually forgetting the point you stressed most. If we prevent users
>from installing Cython and Pyrex at the same time, they will have to start
>modifying setup.py scripts. So it's pushing the burden on them.

Actually, I think you need to decide whether you are really providing 
an extended Pyrex compiler.  If so, then use .pyx as the extension 
and include a Pyrex-compatible build_ext module, and then there is no 
need to have both installed on the same system, and everything works 
splendidly for everyone.

If on the other hand you are *not* providing a Pyrex replacement and 
must co-exist, then use a different file extension (.cy, perhaps?) 
and you get exactly the behavior you wanted -- i.e., setuptools 
doesn't touch your files, and explicit action is required to change 
the file extensions.

And there is still a third solution: provide your own Extension class 
for users to import, since they will have to import from Cython in 
order to use your extended build_ext class anyway, nothing stops them 
from importing the Extension type too.  (Just have your 
Extension.__init__ save the sources and restore them after calling 
the superclass; it'll then work whether the patched version is present or not.)