[Distutils] Autoconf in Python

[Greg Ward]

Those of you who follow python-checkins closely probably already know
this, but for the rest of you I think this will be news:

I now have a working demonstration of the long-mythical, frequently
procrastinated, "Autoconf-in-Python"!  Marc-Andre, you were right -- it
wasn't too bad.  I think I probably spent more time combing the Autoconf
docs for inspiration, and figuring out how to sensibly translate
Autoconf's byzantine shell/M4 interface into sensible OO Python, than I
did actually coding it.

Here's the bird's-eye view of how it works:

  * module developer defines a custom command, "config_xxx", which
    inherits from the standard "config" command (distutils.command.-
    config.config).  (I suggest we adopt the convention "config_" +
    distribution_name -- which, come to think of it, is probably
    a good convention for *all* extended commands defined in setup
    scripts.)

  * this custom "config" command then calls the various methods provided
    by the standard "config" to probe the system.  Currently available:
    'try_cpp()', 'search_cpp()', 'try_compile()', 'try_link()',
    'try_run()', 'check_header()', and 'check_func()'.  (The 'try'
    methods are fairly low-level and expect you to provide a chunk of
    custom-crafted C/C++ code; the 'check' methods just take a header
    name or function name or what-have-you, and Do The Right Thing.)

  * the custom "config" command uses the result of the system probing to 
    determine preprocessor macros to define/undefine, include
    directories to search, libraries to link against, library dirs to
    search, and whatever else might be needed.  Currently the standard
    "config" command provides no support for this, although it probably
    should.

    (My thinking is something like this: users can provide manual
    overrides by editing the "[config]" section of the setup.cfg file;
    the custom config command -- if any! -- will then either respect the
    manual overrides or probe the system (or a combination, depending on
    which manual overrides are present).  At the end of the day, there
    are a bunch of settings -- macros, include dirs, libraries, and
    library dirs -- sitting in the "config" command object, which are
    then used by the various "build" commands.

    (This is not terribly general -- what if you want to build one
    extension (or library or executable one way), but others
    differently? -- but it's good enough.  If you need to get fancy, go
    right ahead: this is Python, the only barrier is your own
    creativity.  No fighting with shell quoting rules or M4 bizarritude.
    Hooray!)

A note on portability: yes, this is all heavily geared towards C and
C++.  No matter what Sun says (lying all the way to the bank, no doubt),
I expect an Autoconf-ish facility will be necessary for grokking Java
installations to build JPython extensions -- but that ability is a long
ways off, definitely post-Distutils 1.0.

Also, I had to add a 'preprocess()' method to the CCompiler interface,
which is currently only implemented in UnixCCompiler.  Thus, the
preprocessor-depending "config" methods ('try_cpp()', 'search_cpp()',
'check_header()') will die horribly on Windows and Mac OS.  Please take
a look at the 'preprocess()' docstring and let me know if it can be
implemented portably, or if the interface will have to change.

Oh, Unix wizards: currently I just do "cc -E filename.c -o filename.i"
to run the preprocessor.  Works with GCC, and ISTR that SGI's compiler
supports "-E" too.  Anyone know how widely spread this convention is?
Should I implement some heuristics to figure out whether to use "cc -E"
or "/lib/cpp"?  What other options are there?

I'm gonna check this in now.  No time to make a snapshot tonight --
please get the latest CVS code and give this a whirl.  The only test
case is mxDateTime (examples/mxdatetime_setup.py), and only
'check_func()' (and therefore 'try_link()') have been tested.  Next up
is to add autoconfiguration to PIL's setup script.

Whee! this is fun... I guess I've written my code for the week, now back
to wading through all those patches...

        Greg
-- 
Greg Ward - nerd                                        gward@python.net
http://starship.python.net/~gward/
Support bacteria -- it's the only culture some people have!