[Distutils] Proposal: C/C++ compiler identification

[Greg Ward]

Quoth John Skaller, on 16 December 1998:
> 
>         Yes, I AM taking about building Python itself.
> Sigh. It it that hard to understand: when python is built,
> it know the name of the compiler.

[...deletia...]

>         I want the build process to put that
> information into a standard module. So I can get it out,
> and call up the compiler to make a C extension FROM PYTHON.

A-ha!  Thank you -- now it comes clear.  As it happens, this was
one of the things discussed at the Developer's Day session which I
failed to mention in the online summary.  Mea culpa.  I also forgot to
mention it on the requirements list.  If I were more weasely, I could
squirm out of it by saying "Well, this just follows from the requirement 
to easily build extension modules from source" or, "That's not a
requirement, it's an implementation detail".

But I'm not a weasel, and I think at some point I meant to mention this
in the requirements document.  So I've added it.  I also changed the
wording of the point that immediately leads into it; here're the changed
and new bullet:

      <!-- the changed one -->
      <li>It must be possible (and preferably easy) to instead download
          and build from a source distribution.  (Necessary for people
          who don't happen to be using one of the major platforms, or
          who don't trust built distributions.)

      <!-- the new one -->
      <li>To aid in installing all modules, and in building extension
          modules from C/C++ source, there must be a standardized way to
          get Python's configuration data (such as compiler, compiler
          flags, platform, default library directory, etc.).

I'm managing to keep implementation details out of the requirements
list, though.  In particular, my proposal is for a new 'sys.config'
module which would contain everything you might need to know about how
Python was configured.  The most important items are the compiler,
compiler flags, platform, and library installation directories.

For an example of the degree of flaming overkill possible in this sort
of module, try this (assuming you have a fairly recent [>= 5.003 I
think] version of Perl installed):

   perl -MConfig -e 'map { print "$_: $Config{$_}\n" } sort keys %Config;'

You'll note that Perl's Config module stores *every* piece of
information, gleaned, gathered, or queried from the user while running
Perl's 'Configure' script.  I don't think Python's sys.config need be
*quite* so exhaustive, but I can personally vouch that the Config module
has more utility than simply knowing where to install modules.  (F'r
instance, if you're wondering "Gee, is Perl's 'syscall' available on
this platform?", or, "Was this Perl built with stdio or sfio?", then you
can answer your questions with Config.  That's pretty cool.)

So, perhaps this is a good place to start a new flamewar^H^H^H^H^H^H^H^H
discussion.  Does anyone violently disagree with the concept of adding a
sys.config module?  And what should go in it?  Here's a very rough start
(loosely based on my long experience with building Perl and my very
brief experience with building Python, and a bit of poking through
Python's makefiles):

  cc              C compiler used to build Python
  cppflags        C preprocessor flags used to build Python (-I, -D, -U)
  optimize        optimization/debugging flags used to build Python
  ccsharedflags   C compiler flags required to build shared objects
  extra_ccflags   any other C compiler flags used to build Python
  ccflags         all compiler flags used to build Python:
                  cppflags + optimize + extra_ccflags
  ccextflags      all compiler flags used to build Python extensions:
                  cppflags + optimize + ccsharedflags + extra_ccflags

  ldlibdirs       -L flags used to build Python
  ldlibs          -l flags used to build Python
  extra_ldflags   any other needed linker flags
  ldsharedflags   linker flags required to build shared objects
  ldflags         all linker flags used to build Python:
                  ldlibdirs + ldlibs + extra_ldflags
  ldextflags      linker flags used to build Python extensions:
                  ldlibdirs + ldlibs + ldsharedflags + extra_ldflags

  platform        hardware/os/vendor (or something along those lines)

  install_bindir       directory to install binary executables
  install_scriptdir    directory to install non-binary executables
  install_includedir   directory to install C header files
  install_confincludedir  directory to install platform- and
                          build-specific C header files
  install_libdir       main Python library directory
  install_sitelibdir   directory for site-specific modules

Obviously this is very sketchy.  A proper proposal would have examples
and/or defaults (eg. "What do I get if I just do "configure ; make ;
make install" on a standard POSIX-compliant system?).  Any replies
should be implicitly prefixed, "Greg, you are a newbie and don't know
what you're talking about" because it's true.  *But* I have configured
and built Perl more times than I care to admit, and I know what works
and doesn't work there.  All you long-time Python hackers surely know
what works and doesn't work *here*, so together we should be able to
cobble something together.  Go to it...

        Greg

PS. oh, I guess I should briefly address John's remaining points:

>         This is the key problem for distutils BECAUSE it
> REQUIRES Guido to modify the build process to create 
> the information needed to call up the compiler
> from python script. At least, this is by far the most
> reliable way to get the information: all that autoconfig
> and makefile and setup stuff has the information,
> and it is needed to be able to build C code later as well.
> 
>         That is the connection. Distutils cannot work
> if Guido doesn't give the information. 

I agree, but I hadn't considered it the *key* problem.  After all, the
first version of distutils will have to work with 1.5.2; one possibility 
is (hack, blech, gaack) to chew up config.status and/or Makefile from
the Python source directory.  Obviously, having sys.config available in
the standard library is a far nicer solution in every respect.

-- 
Greg Ward - software developer                    gward@cnri.reston.va.us
Corporation for National Research Initiatives    
1895 Preston White Drive                      voice: +1-703-620-8990 x287
Reston, Virginia, USA  20191-5434               fax: +1-703-620-0913