[Distutils] Hacks and bugs for distutilizing wxPython

[Greg Ward]

On 13 October 2000, Robin Dunn said:
> Looks like I've got wxPython building with Distutils now on win32, Linux and
> Solaris.  I thought I'd pass along the things I had to hack around so maybe
> some official solutions could be put into Distutils 1.1...

Cool!  I think we can address most of these in due course.

> Win32 (MSCVCompiler)
> --------------------
> 1. As mentioned previously I need to pass -I flags to the resource compiler
> for my .rc file.  I just hacked it to use the same flags passed to the C
> compiler but it might make sense to do it with a new attribute of Extension.

And whatever is done, it should be done for the other Windows CCompiler
implementations.  This will be helped by factoring a new class,
WindowsCCompiler, out of the {MSVC,BCPP}Compiler.  Rene Liebscher was
working on this, so hopefully it will be done and checked in soon.  I
want to hold off on changes to the Windows compiler classes until this
refactoring is done.

> 2. My secondary extensions need to link with the import lib produced from
> building the first extension.  The default location of this file is dynamic
> based on the python version, debug status, and also the relative path to the
> first source file.  Since there was no reliable way to determine before
> setup() was called where that directory would be I hacked it to just use a
> fixed location for the implib (build\ilib\).

Still pondering this one.  Wherever we put it, the directory name will
be unpredictable and not known until run-time, because all directories
in "build/" with platform-specifc files are named after the platform.

One idea: allow expansion of "configuration variables" in more values in 
the setup script.  Eg.

   setup(...
         ext_modules = [Extension('secondary',
                                  libraries=['$build_temp/primary'])]
        )

Assuming that your import library is called "primary", lives right in
build/temp.<plat>-<pyversion>, and can be accessed by adding
"-Lbuild/temp.<plat>-<pyversion> -lprimary" to the linker command line,
then all we need to do is add the ability to expand configuration
variables in certain inputs to the extension-building process.  This is
powerful and dangerous, but the precedent has been set by the "install"
command.

> 3. I implemented these two hacks by deriving from MSCVCompiler and replacing
> the whole compile and link methods, each with only a one or two line change.
> It would be nice if they were more modular so derived classes could just
> override little bits here and there instead of huge methods.

Agreed.  This should wait for the WindowsCCompiler refactoring, though.

> 4. I noticed this one on win32 but it may be a general problem.  If relative
> path names containing '..' are used for extension source files then the
> objects can be placed in odd locations, possibly not even under ./build.
> For example, '../../wx/contrib/src/stc/stc.cpp' will have its object file
> put in build\temp.win32-1.5\Release\../../wx/contrib/src/stc/stc.obj which
> defeats the purpose of having the protection of temp.win32-1.5\Release!

Oooh, tricky!  I don't have a good answer for this, except the old
standby, "Well don't do that then!".

One general weakness of the Distutils is that they really don't like
files to be outside of your source tree (ie. above the distribution
root).  

> Related to this, if a full path is given to a source file, then the object
> will be put in the same directory as the source.  I think object files
> should always be put under the build dir.

By "full path" do you mean "absolute path", or just "anything with
{slashes|backslashes|colons}"?  Let's see... yup, I can reproduce this
by putting an absolute path in the setup script.  Well, again, "don't do
that!".  I can't think of any good reason to put absolute paths in your
setup script; it destroys portability even to another machine (much less
another OS), so it's not something I ever even considered supporting.
Why do you need it?

> 5. On my Linux system (RH 6.1, RPM 3.0.3) bdist_rpm failed with this
> message:
> 
>     Could not open
>     /home/rd/wx/wxPython/build/bdist.linux-i686/rpm/RPMS/i386/wxPython-
>     2.2.2-1.i386.rpm
> 
> because bdist_rpm didn't make the i386 directory.

You must be running rpm 3.0.x, for x < 4.  Should detect this and warn
about it.

> 6. Why is -g used by the compiler when --debug is not given to distutils?

Because it's what's in Python's Makefile.  Hmmm... here's how it works
right now: if 'debug' is true (ie. --debug is given on the build_ext or
build_clib command line), then UnixCCompiler inserts "-g" into the
argument list.  But it does nothing if 'debug' is false.  Perhaps it
should explicitly remove "-g".  That feels wrong to me, though: if you
built your Python with -g, then surely you mean to build extensions with 
-g?

(Note that, at least as of Red Hat 6.2, Red Hat built Python with -g,
which belies that argument.  Still...)

> 7. It would be nice to be able to override CC, LDSAHRED, etc. (by setting
> environment variables or something) instead of just blindly taking them from
> Python's Makefile.  For example I would like to be able to use "gcc -G"
> instead of the default (with Py-1.5.2) of "ld -G" on Solaris so that the
> static C++ objects will actually get initialized instead of being full of
> garbage and causing the application to segfault.

CFLAGS and LDFLAGS have been on the TODO list for ages.  Hadn't thought
of allowing CC or LDSHARED, but I guess it would be useful.  (Can also
be dangerous, but I guess it's on your head if you compile Python with
foocc and your extension with barcc.  Don't come whining to me when it
dumps core, though!)

You'll be glad to know that LDSHARED on Solaris is fixed in Python 1.6
and later -- I patched configure.in to handle that.

Thanks for the comments -- please watch this space, hopefully in the
next couple of weeks we'll start to address some of your problems.  I
hate the idea of 400-line setup scripts out there to work around bugs or
omissions in the Distutils, but I guess we're stuck with them for a
little while.  Sigh...

        Greg
-- 
Greg Ward                                      gward@python.net
http://starship.python.net/~gward/