[Distutils] Working toward Linux wheel support

[Nate Coraor]

On Mon, Sep 7, 2015 at 12:02 PM, Donald Stufft <donald at stufft.io> wrote:

> On September 3, 2015 at 1:23:03 PM, Nate Coraor (nate at bx.psu.edu) wrote:
> > >>>
> > >>> I'll create PRs for this against wheel and pip shortly. I can also
> work
> > >>> on a PEP for the platform tag - I don't think it's going to need to
> be a
> > >>> big one. Are there any preferences as to whether this should be a
> new PEP
> > >>> or an update to 425?
> > >>>
>
> Coming back to this, I'm wondering if we should include the libc
> implementation/version in a less generic, but still generic linux wheel.
> Right
> now if you staticly link I think the only platform ABIs you need to worry
> about
> are libc and Python itself. Python itself is handled already but libc is
> not.

The only thing I've seen so far is "build on an old enough version of glibc
> that it handles anything sane", but not all versions of Linux even use
> glibc at
> all.


This proposal makes a lot of sense to me. pip will need an update to do the
backwards compatibility, and it may be a little ugly to do this all on the
platform tag. For example, linux_x86_64_ubuntu_12_04 wheels should not be
installed on systems that identify as linux_x86_64_ubuntu_14_04, but
linux_x86_64_glibc_2_15 wheels can be installed on systems that identify as
linux_x86_64_glibc_2_19. pip would need to maintain a list of which tag
prefixes or patterns should be considered backward compatible, and which
should not. Granted, new libcs do not pop up overnight, so it's not exactly
a nightmare scenario.

Wheel should be updated to generate the "libc-generic" wheels by default
when nothing other than libc is dynamically linked. It'll need libc
vendor/version detection.

Alternatively, the platform tag could be split in two, in which case you
have a "generic" portion (which would probably be what it currently is,
distutils.util.get_platform()) and a "specific" portion (the distro or
libc), possibly prefixed with something to avoid having to maintain a list
of what's version compatible and what's not, (e.g. 'd_ubuntu_14_04' vs.
'c_glibc_2_19')?

I don't think there is a strong case to include the libc version in the
specific portion when a distro version will also be specified, because the
distro is supposed to define the ABI (at least in the case of distros with
stable ABIs), and that includes the libc compatibility. So for psycopg2
wheels you'd get a "distro" wheel (linux_x86_64-d_ubuntu_14_04) but for
SQLAlchemy, you'd get a "libc-generic" wheel (linux_x86_64-c_glibc_2_19).

It's then up to PyPI project owners to build on whatever platforms they
wish to support.

--nate


>
> -----------------
> Donald Stufft
> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372
> DCFA
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20150908/98f613cc/attachment.html>