[Python-Dev] [Distutils] FW: [issue2513] 64bit cross compilation on windows

[Mark Hammond]

Martin quoting me:

> > Currently, the "official" (by way of being de-facto) directory
> > structure for a build tree is 'PCBuild/.' for x86 builds and 
> > 'PCBuild/amd64' for x64
> > platforms.  I believe this might cause problems for people trying to
> > port their applications to 64bit platforms.  My proposal is that we 
> > change this subtly - that both 'PCBuild/x86', and 'PCBuild/amd64' 
> > exist, while 'PCBuild/.' is always the 'native' platform - ie, 
> > that 'PCBuild/.' be a copy of one of the platform subdirectories.
> 
> Can you explain what you mean by "native platform"? I usually perform
> cross-builds for amd64, so would then this directory remain empty?

By "native platform", I mean it contains binaries that are native for the
current platform.  IOW, on x86 platforms, PCBuild would contain x86
binaries, and binaries compiled for AMD64 would live in the AMD64 directory.
 
If in a given directory tree, you *only* did a cross-compile, then yes,
PCBuild would remain empty - but I don't think that is a problem; tools that
know how to cross-compile (such as the distutils after my patch is applied)
will not look in the PCBuild directory, and tools that do not cross-compile
will be unable to use the x64 binaries from the x86 platform anyway.  In
other words, I can't think of anything that would break by having an empty
PCBuild directory in a tree that only contains binaries that can't be
executed on the current platform.

> > The rationale is fairly simple: I'm quite certain that this new
> > directory layout will break existing "native only" build processes 
> > (ie, those that aren't aware of cross-compilation).  I'm quite 
> > certain that Mozilla will break, for example, and no 
> > cross-compilation process exists that I am aware
> > of.  Existing build tools already know about the PCBuild directory,
> > and are focused almost exclusively towards native compilation - all 
> > such tools on non-x86 platforms are currently broken.
> 
> The reverse may also be true: some tools may expect PCbuild to contain
> always x86 binaries, even on AMD64 - as they don't support non-x86 at
> all. Those tools might break if PCbuild suddenly starts containing
> AMD64 binaries.

I agree.  However, it is my assertion that there are very few build tools
which expect the layout you describe, simply as the scheme has only been
around for a few months, and only in Python 2.6 builds.  Further, I assert
that there are a greater number of build tools which do not support
cross-compilation, but will build natively on x64 and expect 'PCBuild' to
have libraries they can link with to create an x64 binary.

I accept my assertions may be incorrect - in which case keeping the existing
layout is indeed the best options.  But I assert that my assertions are
correct :)

> If this is all about *just* Mozilla, then I'd be much more in favour
> than if it was for some theoretical package. So if Mozilla builds with
> that kind of setup, and works correctly - go for it. If you have to
> fix Mozilla anyway, consider fixing it so it looks in the amd64
> directory.

Nope - it's not about *just* Mozilla at all, and I don't expect that I
personally will be involved in any 64bit issues for that platform.  That
just happens to be one build process that I know about, but I except it does
some "typical" things other tools might do - like executing the python
executable to sniff sys.prefix and looking for (eg) "Libs" and "PCBuild" to
locate libraries to link with seems fairly likely to me.  Even looking for
the executable itself directly in PCBuild seems likely.  On an x64 platform,
such tools may well find the 32bit version, and it may even seem to work -
but it is unlikely to be doing what they expect.  Or they may find no binary
at all.

It seems that you previously agreed with this.  From
http://mail.python.org/pipermail/distutils-sig/2007-May/007668.html; you are
replying to (I think) Kristján:

| > I am baffled about why the build environment's layout matters,
| > but once an .msi install can place the binaries in any
| > old place it wants.  The build structure doesn't have to
| > reflect the final installed structure at all.

| No. But still, people like to be able to "run" Python out of
| a source check-out. This has been supported for a long time,
| and more and more stuff was added to support it. For examples
| within Python itself, see the support in distutils, getpathp.c,
| PCbuild/rt.bat, and Tools/buildbot/*.bat. Reportedly (by
| Mark), building Mozilla (the web browser) also "knows"
| about PCbuild.

Your comments exactly reflect my concern, and I believe them relevant for
people working with native binaries on x64.  But as implied above, I
actually have zero personal interest in this at the moment - unlike the
distutils cross-compile patch, which does scratch an itch of mine.  It just
seems it would make life easier for people down the track - but I'm happy to
let it go if your position has changed or I have mis-understood it.

Thanks,

Mark