[Pythonmac-SIG] Problem with pip

[Joni Orponen]

On Thu, Jan 18, 2018 at 5:05 PM, Karsten Wolf via Pythonmac-SIG <
pythonmac-sig at python.org> wrote:

>
> > On 18.01.2018, at 14:58, Joni Orponen <j.orponen at 4teamwork.ch> wrote:
> >
> > The offical CPython distributions seem to be compiled on 10.6 for
> compatibility reasons. It is easier to compile CPython forwards-compatible
> than to manually patch pyconfig.h after configure to exclude the syscalls
> not available on your older target platforms.
>
> With the following config line you compile a backwards compatible Python
> (here 64-bit only since I don't like fiddling around with hard to compile
> fat-libs).
>
> # for 2.7
> env MACOSX_DEPLOYMENT_TARGET=10.6 CFLAGS="-arch x86_64 " LDFLAGS="-arch
> x86_64 " ./configure --enable-framework --enable-ipv6 --enable-toolbox-glue
>
> and it yields a Python that can be py2apped and run on 10.6 as long as you
> don't use newer Frameworks or libs that need OSX > 1.6
>

I'm not sharing your experience.

Adding -Wl,-no_weak_imports to LDFLAGS flags things up and those then
actually fail hard at runtime. The easiest one to spot and verify yourself
is getentropy() when using TLS. I'm having to patch out the detected
support for those syscalls manually out of the generated pyconfig.h when
compiling something backwards-compatibly for distribution.

I'm also passing through -mmacosx-version-min in both LDFLAGS and CFLAGS as
not all egg specific build systems pass in the environment correctly, but
there is still some hope of them to use whatever it is that Python was
built with as Python exposes that at runtime. There are eggs one still has
to patch up manually for distribution. This is also useful to be in the
know of for some of the dependencies of any CPython modules you might have
to bundle as static libraries for distribution purposes.

For the framework incompatibilities you can get to about the same detection
ability if you use -Wunguarded-availability in CFLAGS. There is
also -Wno-unguarded-availability-new which combines to provide the ability
for not yet caring of non-fatal issues for the newest one (as in the one
you are currently building on with the intention of being
backwards-compatible only), but this is rather new [1].

I do not recommend -Werror and -Wunguarded-availability together when
compiling backwards-compatible as you'd have to fix the use of all the
deprecated, but not yet broken, bits in CPython itself.

So for targeting 10.6 I'd use something roughly like (also allowing you to
switch between Xcode versions via xcode-select, if needed):

CC="xcrun cc"
CXX="$CC"

MACOSX_DEPLOYMENT_TARGET="10.6"

XCODE_ROOT="$(xcode-select --print-path)"
SYSROOT="$XCODE_ROOT/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk"

COMMON_FLAGS="--sysroot=$SYSROOT
-mmacosx-version-min=$MACOSX_DEPLOYMENT_TARGET"

CFLAGS="-I$SYSROOT/usr/include $COMMON_FLAGS
-Wno-unguarded-availability-new -Wunguarded-availability"
CPPFLAGS="$CFLAGS"

LDFLAGS="-L$SYSROOT/usr/lib $COMMON_FLAGS -Wl,-no_weak_imports"

In practice I'd not target such an old OS version when building
backwards-compatibly as the associated workload would be rather large.
Building on the latest version and supporting a couple of versions back
down is still rather manageable and sane in my experience.

Against this backdrop, I'm understanding of people, especially open source
volunteers, choosing to build both CPython and their distributable wheels
on 10.6 as the amount of effort to validate and support more permutations
is rather large for almost no benefit.

The expectation levels on the Windows side have risen and thus since Python
3.5 that stack has moved to build with a newer Visual Studio and is also
offering a nicer userspace install. I'm taking this thread as a signal of
the expectations of the macOS crowd having risen as well.

A midway compromise point might be to coerce the community to start
building on 10.11 as 10.12 was again a great hardware support divider such
as 10.7 was and it's starting to have been long enough since 10.11.

On the actual topic: If one explicitly needs some build-time details to be
specific, one can just build the egg oneself. Otherwise whatever pip pulls
in and is working is good enough for the job.

-- 
Joni Orponen

[1] https://reviews.llvm.org/D34264
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pythonmac-sig/attachments/20180118/95644d5f/attachment.html>