[Distutils] thoughts on distutils 1 & 2

[Stefan Seefeld]

has wrote:

> - Eliminate DU1's "Swiss Army" tendencies. Separate the build, install 
> and register procedures for higher cohesion and lower coupling. This 
> will make it much easier to refactor design of each in turn.

I'd like to second that.

Example: compilation of extension modules.

Scons is aiming at providing an abstraction layer for portable compilation.
DU2 should at least allow to just delegate compilation of extension modules to scons.
(and as I said previously, I think anything that doesn't allow to
wrap traditional build systems based on 'make' and possibly the autotools
is 'not good enough' in as a general solution).


> - Every Python module should be distributed, managed and used as a 
> single folder containing ALL resources relating to that module: 
> sub-modules, extensions, documentation (bundled, generated, etc.), 
> tests, examples, etc. (Note: this can be done without affecting 
> backwards-compatibility, which is important.) Similar idea to OS X's 
> package scheme, where all resources for [e.g.] an application are 
> bundled in a single folder, but less formal (no need to hide package 
> contents from user).

are you really talking about 'package' here when you say 'module' ?
I don't think that mandating modules to be self contained is a good
idea. Often modules only 'make sense' in the context of the package
that contains them. Also, are you talking about how to distribute
packages, or about the layout of the installed files ?
I don't think DU2 should mandate any particular layout for the target
installation. It may well suggest layout of the files inside the
(not yet installed) package.

> - Question: is there any reason why modules should not be installable 
> via simple drag-n-drop (GUI) or mv (CLI)? A standard policy of "the 
> package IS the module" (see above) would allow a good chunk of both 
> existing and proposed DU "features" to be gotten rid of completely 
> without any loss of "functionality", greatly simplifying both build and 
> install procedures.

Again, I don't think it is DU2's role to impose anything concerning
the target layout. This is often platform dependent anyways.

> --Replace current system where user must explicitly state what they want 
> included with one where user need only state what they want excluded. 

That depends on how much control users want over the process. I believe
both are equally valid, and should be supported (similar in spirit to
the MANIFEST.in syntax 'include' and 'exclude')

> -- In particular, removing most DU involvment from build procedures 
> would allow developers to use their own development/build systems much 
> more easily.

yes !! Though that's more easily said than done: a minimum of collaboration
between the two is required, at least the adherence to some conventions.

> - Installation and compilation should be separate procedures. Python 
> already compiles .py files to .pyc on demand; is there any reason why 
> .c/.so files couldn't be treated the same? Have a standard 'src' folder 
> containing source files, and have Python's module mechanism look in/for 
> that as part of its search operation when looking for a missing module; 
> c.f. Python's automatic rebuilding of .pyc files from .py files when 
> former isn't found. (Q. How would this folder's contents need to be 
> represented to Python?)

As a starting point, the whole 'build_ext' mechanism should be re-evaluated.
The current 'Extension' mechanism is by far not abstract enough. Either
the build_ext or the Extension class should be made polymorphic to wrap
any external build system that could be used (make, scons, jam, ...)

> - What else may setup.py scripts do apart from install modules (2) and 
> build extensions (3)?

* building documentation (that, too, is highly domain specific. From
   Latex over Docbook to doxygen...)

* running unit tests

> - Remove metadata from setup.py and modules.

I don't quite agree in general. What metadata are we talking about
anyways ? There's metadata that is to be provided to the packager
backends, i.e. a package description of some sort. Some of these
can be generated automatically (such as MANIFEST.in -> MANIFEST,
build / host platform, etc.), others have to be explicitely provided
(maintainer address, package description).

Having 'all metadata' lumped together brings us back to the 'swiss
army knife' syndrome.

> - Improve version control. Junk current "operators" scheme (=, <, >, >=, 
> <=) as both unnecessarily complex and inadequate (i.e. stating module X 
> requires module Y (>= 1.0) is useless in practice as it's impossible to 
> predict _future_ compatibility). Metadata should support 'Backwards 
> Compatibility' (optional) value indicating earliest version of the 
> module that current version is backwards-compatible with. Dependencies 
> list should declare name and version of each required package 
> (specifically, the version used as package was developed and released). 

Good idea, though this issue highly depends on the packager backend used.

> - Make it easier to have multiple installed versions of a module. 

That, too, isn't really an DU2 issue, or is it ?

> - Reject PEP 262 (installed packages database). Complex, fragile, 
> duplication of information, single point of failure reminiscent of 
> Windows Registry. Exploit the filesystem instead - any info a separate 
> db system would provide should already be available from each module's 
> metadata.

I don't quite agree. I couldn't live without rpm these days.