[Distutils] People want CPAN

[David Cournapeau]

Tarek Ziadé <ziade.tarek <at> gmail.com> writes:

> 
> Also, notice that we are in the process of adding a new python module
> in the stdlib, called
> "sysconfig", that will contains all installation paths scheme for all
> supported platforms.

I don't think we are talking about the same thing. If the pb was just missing
API, we could have just added it to numpy.distutils. We override pretty much
every command of distutils and setuptools anyway.

I needed to obtain the install prefix, be it the default one, or the one set
through --prefix, or the current directory for in place build/develop install.
I need to know it when building C libraries, in build_clib.

AFAIK, there is only one way to get the information: since the --prefix is only
known once install.finalize_options() has run, you need to call the method from
build_clib. 

Naive code could be something like

class MyBuildClib(build_clib):
    def run(self):
        install_cmd = self.distribution.get_command_obj("install")
        if not install_cmd.finalized == 1:
            install_cmd.finalize_options()

        if self.inplace == 1:
            top = "."
        else:
            top = install_cmd.install_libbase
        # top is the top directory where libraries should be installed
        build_clib.run(self)

It is already quite ugly, but it looks like it could work. It actually does in
the simple case, like python setup.py install. But if you call python setup.py
build_clib install --prefix=something, it does not. The error message is a
typical example of distutils style: "error: must supply either prefix/exec-
prefix/home or install-base/install-platbase -- not both". The only way I
managed to make this work is to replace:

    install_cmd = self.distribution.get_command_obj("install"))                                  

by 

    install_cmd = copy.copy(self.distribution.get_command_obj("install"))                                  

That's not an API problem, or a documentation problem. That's a fundamental
issue because --prefix is only known from install, and when you start running
commands in an order different than expected by distutils, you get weird errors
(the above error is actually almost sensical if you really know distutils
code). I passed over the fact that in some conditions that elude me ATM and are
platform specific, install_libbase does not exist and raises an attribute
error.

If you have a configure / build / install like the vast majority of build
systems out there, this problem disappears. I don't see how the problem can be
fixed without touching how commands work.

Moreover, that's a typical example where the only way to be robust is to check
that every attribute you are using actually exist before. At that point, you
really want to run back to m4, perl, autogenerated makefiles and shell
programming :)

> I am still waiting for you comment on the solution to remove the
> compile code from build_ext
> and have it at the Extension level, with an option to add new
> compilers in a  more easier way.

I will try to document how scons does it. I think the basic idea could be
reused in distutils.

> If there's someone from the Numpy project that can help in isolating patches
> againts Distutils trunk in our issue tracker, I'd be more than happy
> to reduce the gap between
> the two projects.

If that was not clear, I am that guy. I have been the main
maintainer of numpy.distutils and of numpy/scipy build infrastructure for some
time now,

David