[Patches] [ python-Patches-1597850 ] Cross compiling patches for MINGW
SourceForge.net
noreply at sourceforge.net
Mon Jun 4 01:20:00 CEST 2007
Patches item #1597850, was opened at 2006-11-16 16:57
Message generated for change (Comment added) made by hanwen
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1597850&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Build
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Han-Wen Nienhuys (hanwen)
Assigned to: Nobody/Anonymous (nobody)
Summary: Cross compiling patches for MINGW
Initial Comment:
Hello,
attached tarbal is a patch bomb of 32 patches against python 2.5, that we lilypond developers use for crosscompiling python. The patches were originally written by Jan Nieuwenhuizen, my codeveloper.
These patches have been tested with Linux/x86, linux/x64 and macos 10.3 as build host and linux-{ppc,x86,x86_64}, freebsd, mingw as target platform. All packages at lilypond.org/install/ except for darwin contain the x-compiled python.
Each patch is prefixed with a small comment, but for reference, I include a snippet from the readme.
It would be nice if at least some of the patches were included. In particular, I think that X-compiling is a common request, so it warrants inclusion.
Basically, what we do is override autoconf and Makefile settings through setting enviroment variables.
**README section**
Cross Compiling
---------------
Python can be cross compiled by supplying different --build and --host
parameters to configure. Python is compiled on the "build" system and
executed on the "host" system. Cross compiling python requires a
native Python on the build host, and a natively compiled tool `Pgen'.
Before cross compiling, Python must first be compiled and installed on
the build host. The configure script will use `cc' and `python', or
environment variables CC_FOR_BUILD or PYTHON_FOR_BUILD, eg:
CC_FOR_BUILD=gcc-3.3 \
PYTHON_FOR_BUILD=python2.4 \
.../configure --build=i686-linux --host=i586-mingw32
Cross compiling has been tested under linux, mileage may vary for
other platforms.
A few reminders on using configure to cross compile:
- Cross compile tools must be in PATH,
- Cross compile tools must be prefixed with the host type
(ie i586-mingw32-gcc, i586-mingw32-ranlib, ...),
- CC, CXX, AR, and RANLIB must be undefined when running configure,
they will be auto-detected.
If you need a cross compiler, Debian ships several several (eg: avr,
m68hc1x, mingw32), while dpkg-cross easily creates others. Otherwise,
check out Dan Kegel's crosstool: http://www.kegel.com/crosstool .
----------------------------------------------------------------------
>Comment By: Han-Wen Nienhuys (hanwen)
Date: 2007-06-04 00:20
Message:
Logged In: YES
user_id=161998
Originator: YES
"1. Specifying --build should not be necessary, as it should default to"
correct.
"2. what are the implications of AC_CHECK_TOOLS wrt. the current
AC_CHECK_PROGS invocations?"
AC_CHECK_TOOLS checks for TARGET-PROG, so it uses the proper cross
compile tools.
"There is a lot of logic to determine the C++ compiler... "
Good point. This uses AC_CHECK_TOOLS too now.
"3. should we include config.sub? Can we share it easily with the libffi
one? Where do I get the most recent version of it?"
config.sub comes from autoconf, and if libffi uses it, it should be
the same one.
It's a bad idea to put generated files in SVN, as is done now; the
same holds for the toplevel configure. Standard practice is to
include an autogen.sh script that will invoke the correct
autoconf/automake/aclocal/acheader/etc. voodoo to generate the
scripts.
"4. Where does the mapping of system names from -dumpmachine come from?
What would need to be done to eliminate this altogether? What about
ac_sys_release?"
The mapping has to be maintained by someone. It's not very elegant,
but then again, there are some arbitrary mappings in configure.in in
the native build for ac_sys_release. IMO it would be better to do away
with this completely and use the autoconf/config.sub naming
exclusively.
"5. Isn't there some autoconf way for detecting a C compiler for the
build
system? It shouldn't default to 'cc'."
I'm not aware of any; what do you think the default should be, if not
cc? (There is AC_PROG_CC, but that will look for the X-compiler).
"6. I don't think the "skipping import check" warning is needed. Just
silently don't perform this check."
OK. - note that there are several other "skipping import check"
messages, for cygwin and Carbon.
"7. What is the meaning of CROSS_TARGET? In some place, it is used like
sys.platform (so it should take one of the possible values for
sys.platform), in configure.in, it is set to ac_sys_system. I think you
should just use MACHDEP here."
Good point. I changed this.
"8. Why is /usr/local/lib excluded when cross-compiling? Please add a
comment (likewise for lib64)"
/usr/local/lib/ in general contains libraries for the build system,
not the target system. Linking them in will either generate bogus
linker errors ("wrong architecture binary") or worse create a module
which isn't loadable on the target platform, because the library is
missing there.
I've added a more generic mechanism, which filters include_dirs so
they can only start with $CROSS_ROOT. Something similar should really
be applied to the linker directories, but I'm not sure which variable
to modify.
"Otherwise, it looks fine; I haven't been able to test it yet, though."
I've added some more information. If you want to get a cross-compile
environtment, you could do
mkdir gub ; cd gub ; git init ; git pull
git://git.sv.gnu.org/lilypond.git gub:
make -f lilypond.make PLATFORMS="freebsd-x86" cross-compilers
this will leave a cross compiler in
gub/target/freebsd-x86/usr/cross/bin
File Added: cross-55748.patch
----------------------------------------------------------------------
Comment By: Martin v. Löwis (loewis)
Date: 2007-03-07 12:25
Message:
Logged In: YES
user_id=21627
Originator: NO
A few more comments:
1. Specifying --build should not be necessary, as it should default to
config.guess, right? If so, we should document that you cross-compile by
passing --host. OTOH, I see it is a bug that you cannot just specify
--host...
2. what are the implications of AC_CHECK_TOOLS wrt. the current
AC_CHECK_PROGS invocations? There is a lot of logic to determine the C++
compiler...
3. should we include config.sub? Can we share it easily with the libffi
one? Where do I get the most recent version of it?
4. Where does the mapping of system names from -dumpmachine come from?
What would need to be done to eliminate this altogether? What about
ac_sys_release?
5. Isn't there some autoconf way for detecting a C compiler for the build
system? It shouldn't default to 'cc'.
6. I don't think the "skipping import check" warning is needed. Just
silently don't perform this check.
7. What is the meaning of CROSS_TARGET? In some place, it is used like
sys.platform (so it should take one of the possible values for
sys.platform), in configure.in, it is set to ac_sys_system. I think you
should just use MACHDEP here.
8. Why is /usr/local/lib excluded when cross-compiling? Please add a
comment (likewise for lib64)
Otherwise, it looks fine; I haven't been able to test it yet, though.
----------------------------------------------------------------------
Comment By: Han-Wen Nienhuys (hanwen)
Date: 2007-01-08 07:59
Message:
Logged In: YES
user_id=161998
Originator: YES
Regarding --config-cache, yes you're correct.
Regarding extending configure.in, it does already say
"configure: error: cannot check for file
existence when cross compiling"
and exit.
What more would you like it to do? I could add a check
that the --config-cache is given, although that is not strictly
necessary (You can also set the variables in the environment.)
----------------------------------------------------------------------
Comment By: Richard Tew (rmt38)
Date: 2007-01-07 22:03
Message:
Logged In: YES
user_id=1417949
Originator: NO
config.cache is not generated or used on my Windows installation of MinGW
unless --config-cache is also given as argument to configure, and from the
autoconf documentation this seems to be the default behaviour. So you
might want to amend the instructions to take that into account.
Isn't requiring the user to manually create and edit config.cache
resulting in unnecessary work and confusion for the them when it can be
addressed in configure.in? Given that checking files is an operation which
does not work when cross_compiling is set and checking them results in
configure exiting because of this, configure.in can check cross_compiling
before trying these checks and avoid them allowing configure to complete.
----------------------------------------------------------------------
Comment By: Han-Wen Nienhuys (hanwen)
Date: 2007-01-07 02:37
Message:
Logged In: YES
user_id=161998
Originator: YES
"checking for /dev/ptmx... configure: error: cannot check for file
existence when cross compiling"
You need to set up a config.cache file that contains
the correct entry for
ac_cv_file__dev_ptmx
----------------------------------------------------------------------
Comment By: Han-Wen Nienhuys (hanwen)
Date: 2007-01-07 02:37
Message:
Logged In: YES
user_id=161998
Originator: YES
"checking for /dev/ptmx... configure: error: cannot check for file
existence when cross compiling"
You need to set up a config.cache file that contains
the correct entry for
ac_cv_file__dev_ptmx
----------------------------------------------------------------------
Comment By: Richard Tew (rmt38)
Date: 2007-01-07 01:50
Message:
Logged In: YES
user_id=1417949
Originator: NO
This:
AC_CHECK_FILE(/dev/ptmx, AC_DEFINE(HAVE_DEV_PTMX, 1, [Define if we have
/dev/ptmx.]))
Is being translated into:
echo "$as_me:$LINENO: checking for /dev/ptmx" >&5
echo $ECHO_N "checking for /dev/ptmx... $ECHO_C" >&6
if test "${ac_cv_file__dev_ptmx+set}" = set; then
echo $ECHO_N "(cached) $ECHO_C" >&6
else
test "$cross_compiling" = yes &&
{ { echo "$as_me:$LINENO: error: cannot check for file existence when
cross compiling" >&5
echo "$as_me: error: cannot check for file existence when cross compiling"
>&2;}
{ (exit 1); exit 1; }; }
if test -r "/dev/ptmx"; then
ac_cv_file__dev_ptmx=yes
else
ac_cv_file__dev_ptmx=no
fi
fi
Which exits when I do:
$ export CC_FOR_BUILD=gcc
$ sh configure --host=arm-eabi
With an error like:
checking for /dev/ptmx... configure: error: cannot check for file
existence when cross compiling
I am using the latest version of msys/mingw with devkitarm to cross
compile. Is this supposed to happen?
----------------------------------------------------------------------
Comment By: Han-Wen Nienhuys (hanwen)
Date: 2006-12-09 23:50
Message:
Logged In: YES
user_id=161998
Originator: YES
this is a patch against a SVN checkout of last week.
----------------------------------------------------------------------
Comment By: Han-Wen Nienhuys (hanwen)
Date: 2006-12-09 23:48
Message:
Logged In: YES
user_id=161998
Originator: YES
With cross.patch I've been able to build a working freebsd
python on linux.
Since you had little problems with the X-compile patches, I'm
resubmitting those first. I'd like to give our (admittedly: oddball)
mingw version another go when the X-compile patches are in python SVN.
Regarding your comments:
* what would be a better to import the SO setting?
the most reliable way to get something out of a makefile into python is
VAR=foo
export VAR
..
os.environ['VAR']
this doesn't introduce any fragility in parsing/expanding/(un)quoting, so
it's
actually pretty good.
Right now, I'm overriding sysconfig wholesale in setup.py with a
sysconfig._config_vars.update (os.environ)
but I'm not sure that this affects the settings in build_ext.py.
A freebsd -> linux compile does not touch that code, so
if you dislike it, we can leave it out.
* I've documented the .x extension
File Added: cross.patch
----------------------------------------------------------------------
Comment By: Martin v. Löwis (loewis)
Date: 2006-12-06 20:12
Message:
Logged In: YES
user_id=21627
Originator: NO
One more note: it would be best if the patches were against the subversion
trunk. They won't be included in the 2.5 maintenance branch (as they are a
new feature), so they need to be ported to the trunk, anyway.
----------------------------------------------------------------------
Comment By: Martin v. Löwis (loewis)
Date: 2006-12-06 20:06
Message:
Logged In: YES
user_id=21627
Originator: NO
I'll add my comments as I go through the patches.
cab1e7d1e54d14a8aab52f0c3b3073c93f75d4fc:
- why is there now a mingw32msvc2 platform? If the target is mingw (rather
than Cygwin), I'd expect that the target is just Win32/Windows, and that
all symbolic constants provided be usable across all Win32 Pythons.
- why is h2py run for /usr/include/netinet/in.h? Shouldn't it operate on a
target header file?
- please include any plat-* files that you generate in the patch.
- why do you need dl_nt.c in Modules? Please make it use the one from PC
(consider updating the comment about
calling initall)
b52dbbbbc3adece61496b161d8c22599caae2311
- please combine all patches adding support for __MINGW32__ into a single
one. Why is anything needed here at all? I thought Python compiles already
with mingw32 (on Windows)?
- what is the exclusion of freezing for?
059af829d362b10bb5921367c93a56dbb51ef31b
- Why are you taking timeval from winsock2.h? It should come from
sys/time.h, and does in my copy of Debian mingw32-runtime.
6a742fb15b28564f9a1bc916c76a28dc672a9b2c
- Why are these changes needed? It's Windows, and that is already
supported.
a838b4780998ef98ae4880c3916274d45b661c82
- Why doesn't that already work on Windows+cygwin+mingw32?
f452fe4b95085d8c1ba838bf302a6a48df3c1d31
- I think this should target msvcr71.dll, not msvcrt.dll
Please also combine the cross-compilation patches into a single one.
- there is no need to provide pyconfig.h.in changes; I'll regenerate that,
anyway.
9c022e407c366c9f175e9168542ccc76eae9e3f0
- please integrate those into the large AC_CHECK_FUNCS that already
exists
540684d696df6057ee2c9c4e13e33fe450605ffa
- Why are you stripping -Wl?
64f5018e975419b2d37c39f457c8732def3288df
- Try getting SO from the Makefile, not from the environment (I assume
this is also meant to support true distutils packages some day).
7a4e50fb1cf5ff3481aaf7515a784621cbbdac6c
- again: what is the "mingw" platform?
7d3a45788a0d83608d10e5c0a34f08b426d62e92
- is this really necessary? I suggest to drop it
23a2dd14933a2aee69f7cdc9f838e4b9c26c1eea
- don't include bits/time.h; it's not meant for direct inclusion
6689ca9dea07afbe8a77b7787a5c4e1642f803a1
- what's a .x file?
----------------------------------------------------------------------
Comment By: Han-Wen Nienhuys (hanwen)
Date: 2006-11-25 15:12
Message:
Logged In: YES
user_id=161998
Originator: YES
I've sent the agreement by snailmail.
----------------------------------------------------------------------
Comment By: Jan Nieuwenhuizen (janneke-sf)
Date: 2006-11-17 19:57
Message:
Logged In: YES
user_id=1368960
Originator: NO
I do not mind either. I've just signed and faxed contrib-form.html.
----------------------------------------------------------------------
Comment By: Han-Wen Nienhuys (hanwen)
Date: 2006-11-17 00:33
Message:
Logged In: YES
user_id=161998
Originator: YES
note that not all of the patch needs to go in its current form. In
particular, setup.py should be
much more clever to look into build-root for finding libs and include
files.
----------------------------------------------------------------------
Comment By: Han-Wen Nienhuys (hanwen)
Date: 2006-11-17 00:32
Message:
Logged In: YES
user_id=161998
Originator: YES
I don't mind, and I expect Jan won't have a problem either.
What's the procedure: do we send the disclaimer first, or do you do the
review, or does everything happen in parallel?
----------------------------------------------------------------------
Comment By: Martin v. Löwis (loewis)
Date: 2006-11-16 21:47
Message:
Logged In: YES
user_id=21627
Originator: NO
Would you and Jan Nieuwenhuizen be willing to sign the contributor
agreement, at
http://www.python.org/psf/contrib.html
I haven't reviewed the patch yet; if they can be integrated, that will
only happen in the trunk (i.e. not for 2.5.x).
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1597850&group_id=5470
More information about the Patches
mailing list