[Distutils] Improving distutils vs redesigning it (was people want CPAN)

[Tarek Ziadé]

On Thu, Nov 12, 2009 at 1:57 PM, David Cournapeau <cournape at gmail.com> wrote:
[...]
> The signature does not really matter I think. Since those parameters
> depends on user customization (through config files or command line
> flags), it could be something like:
>
> class FinalizedOptions:
>    def get_build_dir_library(self):
>        pass
>
>    def get_build_dir_src(self):
>        pass
>
>    def get_prefix(self):
>
>    ...
>
> And a function like
>
> def get_finalized_options():
>    # compute the finalized options
>    # Cache it if expensive
>    ...
>    return FinalizedOptions()
>
> could be used to obtain those options.

I was not referring to the finalized options of any commands, but to
the build paths you are trying to get.
e.g. like for the install paths, we can extract from the build
commands, some schemes that can be queried
the same way:

_BUILD_SCHEMES = {
  'unix' :
    {'buildlib': '$base/xxx'
     ...}
  ..
}

and then:

get_build_paths(scheme, vars)


[...]
>> A command performs something, and should not be used to set up an *environment*.
>
> I am not sure I understand *environment* in that context. If you mean
> options, then each command has its set of options, and yes, that's one
> of the fundamental issue I often complain about in distutils. The
> build_clib/install interaction I described is typical, and we do this
> kind of stuff in pretty much every command.

By environment I mean that a command should be a standalone element
that does something,
and is used by distutils with this pattern:

>>> cmd = commandA(distribution)
>>> cmd.ensure_finalized()
>>> cmd.run()

If at some point you need to look to some of its finalized options in
another commandB, it means that
- either commandA is doing more than it is supposed to do: it sets an
*environement* that is reused by others
- either commandB should be a subcommand of commandA
- either commandA doesn't let you do what you want, and you are
building a competiting command

>
>> Do you agree that this change will help you, and is possible without
>> replacing Distutils ?
>
> It would definitely be helpful to get access to any finalized command
> option from anywhere at anytime, especially if it is supported for all
> the python versions we support (2.4 and above - I don't know what
> version distribute intends to support).

The schemes currently, go as far as Python 2.2.

>
> Ideally, it would solve a fair shair amount of issues if this was done
> at the UI level as well, that is something like
>
> python setup.py build --compiler=mingw -> use mingw compiler for every
> compiled code

As a matter of fact, having a "compiler" option in build, that would be reused
by build_ext and other build_ commands, where discussed lately in an issue

> But that's one of the thing which does not make much sense changing in
> distutils, though: it would break almost anyone's workflow.

Why that ? How this would break someone's workflow, since all the changes
we have discussed so far is moving code out of commands to have new APIs
(and have those commands call it), and modifying some command options.

Now there's another pattern emerging in this discussion : paths
options like --prefix can obviously
be shared by several commands.

So, based on the change that makes it possible to get some install and
build schemes,
what about having global options that will be used to built a global
environment dictionary containing paths:

$ python setup.py --var1 --var2 --var3 cmd1 cmd2 cmd3


then, var1, and var2  etc.. are put in a "vars" dict, that is passed
to get_paths(scheme, vars).
And a global "options" dict is filled in the Distribution object
(where scheme is a key for the current platform):

Distribution.options = get_paths(scheme, vars)

A further improvement would be to be able to register a function that
is called when the options dict is flled,
allowing a third party code to work on the options. The existing
commands in distutils would be changed accordingly: they'll look into
the global options dict for finalized options, and would fallback
using the old pattern (local options + finalization)

Tarek