[Distutils] High level options interface

[Martijn Faassen]

Hello,

I've been following the discussion on how to build C (or C++, or other
languages) extensions with interest. I'm thus talking about the
'build_extensions' command of the proposed interface, if I understand it
well.

Some talk has gone into reading a lot of low level Autoconf results into
Python, and exposing them through a sysconfig module. Some objections
were raised that this was rather low level.

It's also not very portable, right? Options that go into gcc won't make
much sense for Visual C++, I imagine.

Now, we could try to abstract a compiler framework in Python. I
understand this is what John Skaller is working on, but I may be wrong
as I haven't looked yet.

Abstracting a compiler with its myriads of options could be a pain. We
might end up having to abstract makefiles as well in Python. And
building C libraries. This could be a rather big project. We'd be
duplicating a lot of stuff, as well.

I'm not too keen on simply calling 'make' directly from our builder
script either, though. That's rather tied to the platform.

So let's try abstracting some more. See if it becomes any simpler. I
think this is what other people might've proposed as well, and if so,
I'm just saying 'me too', here.

I'll sketch out a very simplistic and anarchic high level interface
here.

The base class:

class extension_compiler:
    def compile_C(filename):
        pass    
    def compile_C++(filename):
        pass

That's about it, unless we want to choose to support more extension
languages by default.

For each platform we want to support, we supply a subclass of this. 
For example:

class gcc_extension_compiler(extension_compiler):
    def compile_C(filename):
       os.system("gcc -c " + filename)
    def compile_C++(filename):
       os.system("gcc -c " + filename)

That's just about all. We don't have any fancy options (though we might
want to enable optimizations by default).

But what about special optimizations, debugging, profiling, other fancy
stuff? Our abstraction doesn't support this! We need more methods. We
need set_optimization_level(10)!

Do we? Who does?

Generally, only the extension developer knows which special compiler
options could work best with the extension code. The extension writer
only knows this for his platform(s) of choice, too. Extensions should
still build cleanly for anyone on any platform. Thus we do best with as
abstract an interface as possible.

The extension builder should be given the option to subclass, and
provide special compilation options for optimization purposes. Profiling
and debugging builds are generally only necessary for the extension
developer, and could be handled by special throw-away subclasses. If the
developer has figured out how to make extensions build best and fastest
on Linux, he can always include a special purpose
linux_extension_compiler subclass which calls the compiler with special
optimization options.

If later it turns out we do need special option settings (for all
compilers on all platforms!), we can always add it later.

Anyway, I'm leaving lots of stuff out here. We probably need to do the
same simplistic abstraction to the linker.

The emphasis is to keep the interface as small as possible. A large
interface means supporting it on all kinds of different platforms, and
that is a lot of work. Once we got a small basic interface going, we can
always extend it, platform by platform, with more options, if necessary.

A simple 'RISC' interface does limit developers. They can't do all kinds
of fancy makefile stuff they usually do. But we don't want to have to
support all the fancy makefile stuff anyway.

....remembers ./configure ./make and why the complicated ./configure
step is there in the first place; to enhance portability. Hm. *ponder* 

Okay. Perhaps that invalidates my entire argument above. It might be
that I'm limiting developers from using any external libraries in their
extensions.

Still, philosophically, I think my arguments are sound. In practice,
however...

Leaving-confused-ly yours,

Martijn