[Distutils] Working toward Linux wheel support

[Leonardo Rochael Almeida]

Hi,

Going back in time to this old post, but I think it becomes more relevant
now that Nate's work is being completed:

On 13 August 2015 at 22:47, Nathaniel Smith <njs at pobox.com> wrote:

> On Thu, Aug 13, 2015 at 12:30 PM, Leonardo Rochael Almeida
> <leorochael at gmail.com> wrote:
> >
> > On 13 August 2015 at 11:07, Nate Coraor <nate at bx.psu.edu> wrote:
> >>
> >> On Wed, Aug 12, 2015 at 9:05 PM, Nathaniel Smith <njs at pobox.com> wrote:
> >>>
> [...]
> >>> (2) the special hard-coded tag "centos5". (That's what everyone
> actually
> >>> uses in practice, right?)
> >>
> >>
> >> The idea here is that we should attempt to install centos5 wheels if
> more
> >> specific wheels for the platform aren't available?
> >
> >
> > Just my opinion, but although I'm +1 on Nate's efforts, I'm -1 on both
> the
> > standard behavior for installation being the exact platform tag, and an
> > automatic fallback to cento5.
> >
> > IMO, on Linux, the default should always be to opt in to the desired
> > platform tags.
> >
> > We could make it so that the word `default` inside
> > `binary-compatibility.cfg` means an exact match on the distro version, so
> > that we could simplify the documentation.
> >
> > But I don't want to upgrade to pip and suddenly find myself installing
> > binary wheels compiled by whomever for whatever platform I have no
> control
> > with, even assuming the best of the package builders intentions.
> >
> > And I certainly don't want centos5 wheels accidentally installed on my
> > ubuntu servers unless I very specifically asked for them.
> >
> > The tiny pain inflicted by telling users to add a one-line text file in a
> > very well known location (or two lines, for the added centos5), so that
> they
> > can get the benefit of binary wheels on linux, is very small compared to
> the
> > pain of repeatable install scripts suddenly behaving differently and
> > installing binary wheels in systems that were prepared to pay the price
> of
> > source installs, including the setting of build environment variables
> that
> > correctly tweaked their build process.
>
> I think there are two issues here:
>
> 1) You don't want centos5 wheels "accidentally" installed on an ubuntu
> server: Fair enough, you're right; we should probably make the "this
> wheel should work on pretty much any linux out there" tag be something
> that distributors have to explicitly opt into (similar to how they
> have to opt into creating universal wheels), rather than having it be
> something you could get by just typing 'pip wheel foo' on the right
> (wrong) machine.
>

I agree that generating something like "this linux binary wheel is
generically installable" should be opt-in, yes. But I also feel strongly
that installing such a generic wheel should also be opt in.

I guess that if we go into the direction of being able to generate wheels
with a libc tag rather than a distro tag, like Nate and Donald are now
discussing, then we could get both kinds of opt-in by specifying the libc
tag in `binary-compatibility.cfg`.


> 2) You want it to be the case that if I type 'pip install foo' on a
> Linux machine, and pip finds both an sdist and a wheel, where the
> wheel is definitely compatible with the current system, then it should
> still always prefer the sdist unless configured otherwise: Here I
> disagree strongly. This is inconsistent with how things work on every
> other platform, it's inconsistent with how pip is being used on Linux
> right now with private wheelhouses, and the "tiny pain" of editing a
> file in /etc is a huge barrier to new users, many of whom are
> uncomfortable editing config files and may not have root access.


Not having root access shouldn't be an issue, as there should be a
user-level and virtualenv level equivalents to a `binary-compatibility.cfg`
on `/etc`, and perhaps it could even be included in `requirements.txt` for
a project, so users of a project might not even have to bother setting up
`binary-compatibility.cfg`.

However, you make an excellent point: not handling binary wheels on Linux
by default (at least with exact platform tag matching) would mean having
different behavior between Linux and Mac/Windows.

Still, I wouldn't want a random binary wheel suddenly finding its way into
my servers, and I would like a way to opt out of it, for "reasons" (ex. I
might have special build flags, or a special compiler, or maybe I'm still
waiting for TUF before trusting other peoples binaries on my servers).

So I'd like to propose that the installation tooling (eg. `pip`, `distlib`)
should allow the user to specify which index servers to trust for receiving
binary wheels and which to trust only for pure python wheels or sdists.

It could trust them all by default, to maintain the current behavior (where
`all` means only pypi unless I specified more, obviously), but I'd like a
switch to limit this trust to a subset of the specified index servers.

Regards,

Leo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20150908/36736b51/attachment.html>