On Sun, Jan 6, 2013 at 3:21 AM, Ondřej Čertík <ondrej.certik at gmail.com>wrote:
> Hi,
>
> Currently the NumPy binaries are built using the pavement.py script,
> which uses the following Pythons:
>
> MPKG_PYTHON = {
> "2.5":
> ["/Library/Frameworks/Python.framework/Versions/2.5/bin/python"],
> "2.6":
> ["/Library/Frameworks/Python.framework/Versions/2.6/bin/python"],
> "2.7":
> ["/Library/Frameworks/Python.framework/Versions/2.7/bin/python"],
> "3.1":
> ["/Library/Frameworks/Python.framework/Versions/3.1/bin/python3"],
> "3.2":
> ["/Library/Frameworks/Python.framework/Versions/3.2/bin/python3"],
> "3.3":
> ["/Library/Frameworks/Python.framework/Versions/3.3/bin/python3"],
> }
>
> So for example I can easily create the 2.6 binary if that Python is
> pre-installed on the Mac box that I am using.
> On one of the Mac boxes that I am using, the 2.7 is missing, so are
> 3.1, 3.2 and 3.3. So I was thinking
> of updating my Fabric fab file to automatically install all Pythons
> from source and build against that, just like I do for Wine.
>
> Which exact Python do we need to use on Mac? Do we need to use the
> binary installer from python.org?
>
Yes, the one from python.org.
> Or can I install it from source? Finally, for which Python versions
> should we provide binary installers for Mac?
> For reference, the 1.6.2 had installers for 2.5, 2.6 and 2.7 only for
> OS X 10.3. There is only 2.7 version for OS X 10.6.
>
The provided installers and naming scheme should match what's done for
Python itself on python.org.
The 10.3 installers for 2.5, 2.6 and 2.7 should be compiled on OS X 10.5.
This is kind of hard to come by these days, but Vincent Davis maintains a
build machine for numpy and scipy. That's already set up correctly, so all
you have to do is connect to it via ssh, check out v.17.0 in ~/Code/numpy,
check in release.sh that the section for OS X 10.6 is disabled and for 10.5
enabled and run it.
OS X 10.6 broke support for previous versions in some subtle ways, so even
when using the 10.4 SDK numpy compiled on 10.6 won't run on 10.5. As long
as we're supporting 10.5 you therefore need to compile on it.
The 10.7 --> 10.6 support hasn't been checked, but I wouldn't trust it. I
have a 10.6 machine, so I can compile those binaries if needed.
> Also, what is the meaning of the following piece of code in pavement.py:
>
> def _build_mpkg(pyver):
> # account for differences between Python 2.7.1 versions from
> python.org
> if os.environ.get('MACOSX_DEPLOYMENT_TARGET', None) == "10.6":
> ldflags = "-undefined dynamic_lookup -bundle -arch i386 -arch
> x86_64 -Wl,-search_paths_first"
> else:
> ldflags = "-undefined dynamic_lookup -bundle -arch i386 -arch
> ppc -Wl,-search_paths_first"
> ldflags += " -L%s" % os.path.join(os.path.dirname(__file__), "build")
The 10.6 binaries support only Intel Macs, both 32-bit and 64-bit. The 10.3
binaries support PPC Macs and 32-bit Intel. That's what the above does.
Note that we simply follow the choice made by the Python release managers
here.
> if pyver == "2.5":
> sh("CC=gcc-4.0 LDFLAGS='%s' %s setupegg.py bdist_mpkg" %
> (ldflags, " ".join(MPKG_PYTHON[pyver])))
> else:
> sh("LDFLAGS='%s' %s setupegg.py bdist_mpkg" % (ldflags, "
> ".join(MPKG_PYTHON[pyver])))
>
This is necessary because in Python 2.5, distutils asks for "gcc" instead
of "gcc-4.0", so you may get the wrong one without CC=gcc-4.0. From Python
2.6 on this was fixed.
> In particular, the last line gets executed and it then fails with:
>
> paver dmg -p 2.6
> ---> pavement.dmg
> ---> pavement.clean
> LDFLAGS='-undefined dynamic_lookup -bundle -arch i386 -arch ppc
> -Wl,-search_paths_first -Lbuild'
> /Library/Frameworks/Python.framework/Versions/2.6/bin/python
> setupegg.py bdist_mpkg
> Traceback (most recent call last):
> File "setupegg.py", line 17, in <module>
> from setuptools import setup
> ImportError: No module named setuptools
>
>
> The reason is (I think) that if the Python binary is called explicitly
> with /Library/Frameworks/Python.framework/Versions/2.6/bin/python,
> then the paths are not setup properly in virtualenv, and thus
> setuptools (which is only installed in virtualenv, but not in system
> Python) fails to import. The solution is to simply apply this patch:
>
Avoid using system Python for anything. The first thing to do on any new OS
X system is install Python some other way, preferably from python.org.
> diff --git a/pavement.py b/pavement.py
> index e693016..0c637f8 100644
> --- a/pavement.py
> +++ b/pavement.py
> @@ -449,7 +449,7 @@ def _build_mpkg(pyver):
> if pyver == "2.5":
> sh("CC=gcc-4.0 LDFLAGS='%s' %s setupegg.py bdist_mpkg" %
> (ldflags, " ".join(MPKG_PYTHON[pyver])))
> else:
> - sh("LDFLAGS='%s' %s setupegg.py bdist_mpkg" % (ldflags, "
> ".join(MPKG_PYTHON[pyver])))
> + sh("python setupegg.py bdist_mpkg")
>
This doesn't work unless using virtualenvs, you're just throwing away the
version selection here. If you can support virtualenvs in addition to
python.org pythons, that would be useful. But being able to build binaries
when needed simply by "paver dmg -p 2.x" is quite useful.
>
> @task
> def simple_dmg():
>
>
> and then things work. So an obvious question is --- why do we need to
> fiddle with LDFLAGS and paths to the exact Python version? Here is a
> proposed simpler version of the build_mpkg() function:
>
> def _build_mpkg(pyver):
> sh("python setupegg.py bdist_mpkg")
>
> Thanks for any tips.
>
Did you see the release.sh script? Some of the answers to your questions
were already documented there, and it should do the job out of the box.
Last note: bdist_mpkg is unmaintained and doesn't support Python 3.x. Most
recent version is at: https://github.com/matthew-brett/bdist_mpkg, for
previous versions numpy releases I've used that at commit
e81a58a471<https://github.com/rgommers/bdist_mpkg/commit/e81a58a47120522fc0e9374d2d2cf5403d24cb6a>
If we want 3.x binaries, then we should fix that or (preferably) build
binaries with Bento. Bento has grown support for mpkg's; I'm not sure how
robust that is.
Cheers,
Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130106/5ce93cac/attachment.html>