[Distutils] 400 Client Error: Binary wheel for an unsupported platform

[Donald Stufft]

On July 7, 2015 at 10:22:55 AM, Nick Coghlan (ncoghlan at gmail.com) wrote:
> On 8 July 2015 at 00:07, Antoine Pitrou wrote:
> > On Tue, 7 Jul 2015 23:53:59 +1000
> > Nick Coghlan wrote:
> >> Unfortunately, the compatibility tagging for Linux wheels is currently
> >> so thoroughly inadequate that even in tightly controlled environments
> >> having a wheel file escape from its "intended" target platforms can
> >> cause hard to debug problems.
> >
> > I'm not sure what you're pointing to, could you elaborate a bit?
>  
> That was a reference to a case of someone building for Debian (I
> think), and then having one of their wheel files end up installed on a
> CentOS system and wondering why things weren't working.
>  
> > For the record, building against a well-known, old glibc + gcc has
> > served the Anaconda platform well.
>  
> The key problem is that there's no straightforward way for us to
> verify that folks are actually building against a suitably limited set
> of platform APIs that all Linux distros in widespread use provide.
>  
> And when it inevitably fails (which it will, Python and PyPI are too
> popular for it not to), the folks running into the problem are
> unlikely to be able to diagnose what has happened, and even once we
> figure out what has gone wrong, we'd be left having to explain how to
> blacklist wheel files, and the UX would just generally be terrible
> (and the burden of dealing with that would fall on the pip and PyPI
> maintainers, not on the individual projects publishing insufficiently
> conservative Linux wheel files).
>  
> That's why the platform override tags are such a nice idea, as it
> becomes possible to start iterating on possible solutions to the
> problem without affecting the default installation UX in the near
> term.
>  
>

pip 7+ actually has the UI for blacklisting binary packages now, primarily to
ask "no don't build a wheel for X", but it also functions to ask *not* to
accept wheels for a particular project from PyPI.

In my mind, the biggest reason to not just open up the ability to upload even
generic linux wheels right now is the lack of a safe-ish default. I think if
we added a few things:

* Default to per platform tags (e.g. ubuntu_14_04), but allow this to be
  customized and also accept "Generic" Linux wheels as well.
* Put the libc into the file name as well since it's reasonable to build a
  "generic" linux wheel that statically links all dependencies (afaik), however
  it does not really work to statically link glibc. This means that even if
  you build against an old version of glibc if you're running on a Linux that
  *doesnt* use glibc (like Alpine which uses MUSL) you'll run into problems.

I think that it is entirely possible to build a generic linux wheel that will
work on any Linux built the same libc* of the same or newer version, however I
think that you have to be careful if you do it. You have to ensure all your
dependencies are statically linked (if you have any) and you have to ensure
that you build against an old enough Linux (likely some form of CentOS).

* Side question, since I don't actually know how a computer works: Is it even
  possible to have a CPython extension link against a different libc than
  CPython itself is linked against? What if static linking is involved since
  there are non glibc libcs which actually do support static linking?

---  
Donald Stufft  
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA