[Distutils] [Fwd: Re: Annoucing distribute project]

[David Cournapeau]

Guido van Rossum wrote:
>
> Can you think of a specific way to evolve distutils to provide the API
> you would like to see, without (for the time being) losing backwards
> compatibility?
>   

Here is what I did with numscons: numscons is called through distutils
scons command, which is called during the build dance. By default, the
scons command does nothing, but you add scons scripts through a
distutils command, and in that case only scons does something. Those
hooks do not break anything in numpy.distutils. The best proof is that
the default build system is still numpy.distutils, and numscons is not
in numpy sources (only the hooks are). Some people build with numscons,
other - the majority - use the current system. Same source tree, same
setup.py files (except for C extensions of course; they have to be
synchronised, but the common part between the two build could easily be
factorized in a common_setup I think).

That's the idea I was suggesting and may be applicable to python as a whole:
    - design a small self contained build system, which does nothing but
is extensible (there is a lot of prior art: vellum, rake/rant, etc...),
and "hook" it into distutils through a new distutils command. People
smarter than me will enjoy this part, and will hopefully come up with
something nice.
    - The new command would not nothing if you don't use the new
features: you don't break any existing setup.py
    - Another thing would be to gradually move all the configuration
from code into a separate module in a sane way: putting the flags in
config files, not in the code, make an API to discover and reuse those
settings, etc... Basically, all the arcane stuff. This would be
independant of distutils, but would be callable from distutils (by the
new command and the current distutils). A good way to design the API
here would be to make sure it work in distutils and in e.g. scons (so a
python extension builder in scons could be easily done importing this
module only: all the compiler specific flags, etc...). It would be a
distutils.sysconfig on steroids, which works for every platform -
including windows with VS. So that every other python build tool, being
vellum, scons, whatever, could access those. I believe this can be made
general enough to be usable by almost any tool. At least I started
something toward this direction for scons, and if it works for scons, it
should work for a lot of tools.

That would enable refactoring something like 1/3 of current distutils
code, and provide a better build tool for people who use the new 'stuff'.

I realize this is really vague at that point, but up to today, I never
thought about what could be applied to python in general from my work
numscons. If you think the above makes any sense, I will think more
about this,

cheers,

David