[Distutils] conditionally compiling extensions

[Robert Kern]

Christopher Armstrong wrote:
> Twisted has a few extension modules that it only wants to build if
> certain things are available on the platform - like epoll support if
> the epoll.h header file is available, for example.
> 
> We can call a function to run some code to decide which extensions to
> build, but in certain cases having access to the compiler object in
> distutils would be preferable -- to test certain things about the
> platform. Currently we have some fairly heinous hacks to get our code
> to run *after* that compiler object is available (i.e., after it's
> constructed in build_ext.run), but these hacks are becoming
> problematic especially for easy_install support.
> 
> Before showing what these hacks are (if you really care, you can check
> the various setup.py files in the Twisted repository), I'd like to ask
> if there is any other best-practice knowledge in the community for
> conditionally building extensions based on knowledge only available by
> using a compiler.

We've been doing something along these lines in the numpy project. We've
extended the distutils command "config" to allow building and executing small C
programs to test for platform support of certain C functions.

Looking at the code right now ... it's possibly at the same level of heinous
hackishness that yours are, but it works for us. We use it to build a config.h
file using the build_src command which we added. build_src allows us to pass a
callable as an item in an Extension's sources list, and the callable will
generate a file. Inside the function that generates config.h, we grab the config
command, call .ensure_finalized() on it and then call the methods that we added
to try compiling, linking, and/or running the test files. By the time the
functions are called in build_src, the config command exists and is ready to be
used.

You might be able to compile the extensions unconditionally, but through the use
of the #defines in config.h and #ifdefs in the extension module only compile an
empty module if the feature were unsupported. That would turn the

  try:
    import _c_accelerator
    func = _c_accelerator.func
  except ImportError:
    pass

into

  import _c_accelerator
  if hasattr(_c_accelerator, 'func'):
    func = _c_accelerator.func

I leave it to you to decide which is better, but I find try: except ImportError:
to be too fragile. I spent a good amount of time at a customer's site trying to
figure out why Cheetah was unbearably slow behind Apache, but worked like blazes
when we ran the script manually on the same box. It turns out that the
permissions on the accelerator extension module were incorrect for mod_python to
import them.

The setup scripts for Marc-André Lemburg's mx extensions seem to define an
mx_autoconf distutils command that might be useful to look at, but I don't know
the details about how he uses it.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco