[Distutils] Is build an inherently arbitrary-code process?

[Paul Moore]

On 27 March 2014 20:30, Donald Stufft <donald at stufft.io> wrote:
>> When I interface with an external library that may not be in the same
>> place on every system, I'd expect to say a bit more, but mainly just
>> "look in location X, but the user can override this with a command
>> line argument".
>
> That's the job of the linker, it has built in locations to look for libraries
> and env vars/command line flags to override that. Someone packaging
> a C-ext that needs a library to compile shouldn't be specifying where
> to find it anyways.

Ideally, yes. But not in practice (at least not on Windows). This is
getting off-topic, so I'll just give a quick example and leave it at
that.

I recently wanted to build PyYAML on 64-bit Windows, which uses
libyaml if that's available. First thing I had to do therefore was to
build libyaml. That's not too hard (considering). But the build leaves
include files in the project directory and a lib file somewhere in the
bowels of the project ...\x64\Release\Output\Lib\libyaml.lib, IIRC).
There is no install step, largely because there is nowhere standard to
install include and lib files *to*.

OK, so now I build PyYAML. I don't have the compiler, linker etc on my
PATH - the distutils code automatically detects that I have the right
version of the compiler installed and sets up the environment for me.
This is a *feature*, not a problem, as otherwise I would have to
remember which compiler to use for which Python version, and activate
it as an extra step. As a result of this fact, though, I can't set LIB
or INCLUDE environment variables to point to the libyaml project
directories (nor would I particularly want to, as this is purely a
one-off build). So I need to tell the build system (distutils) where
to find libyaml and yaml.h. There's no way setup.py can guess where
they might be in the general case, so I use the build_ext -L and -I
options to specify.

Consider "the linker can just find stuff" as the "look in location X"
scenario. And the above as the reason we might need build-system
support for adding compiler/linker flags or env vars at build time.

Paul