From matthew.brett at gmail.com  Thu Dec  8 14:30:46 2016
From: matthew.brett at gmail.com (Matthew Brett)
Date: Thu, 8 Dec 2016 14:30:46 -0500
Subject: [Wheel-builders] Pre-release Python on docker image?
In-Reply-To: <CADiSq7ekuUMePwP0bFepaCJSUpzqwJi_A5DLPBaHnwXKUy_P5w@mail.gmail.com>
References: <CAH6Pt5pZrP6Ay7YKyaYGXQ8CQZVNgA8C+twL4347Aa3Zw9EKdg@mail.gmail.com>
 <CAPJVwBmpXDQm8tWONtE9JqJz9uuEKiO9huHeA4MrPypoCV4k5A@mail.gmail.com>
 <CAH6Pt5p+ZG=yqBqsgFoATwsA5jtgXCT_OZaJwyeVq=kQWOX34Q@mail.gmail.com>
 <CADiSq7ekuUMePwP0bFepaCJSUpzqwJi_A5DLPBaHnwXKUy_P5w@mail.gmail.com>
Message-ID: <CAH6Pt5qKLXpOkaGdwmi8ahckVFh5Ts0ctsNO-KdZaf1L9Swaxg@mail.gmail.com>

Hi,

On Wed, Nov 16, 2016 at 10:54 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On 19 July 2016 at 09:04, Matthew Brett <matthew.brett at gmail.com> wrote:
>> On Mon, Jul 18, 2016 at 11:58 PM, Nathaniel Smith <njs at pobox.com> wrote:
>>> On Mon, Jul 18, 2016 at 3:43 PM, Matthew Brett <matthew.brett at gmail.com> wrote:
>>>> What to do about manylinux?   I guess we need to have a pre-release
>>>> Python built on the docker image.
>>>
>>> The problem here is that the 3.6 ABI will not be finalized until
>>> 3.6-final is actually released -- any wheels built on 3.6-prereleases
>>> could potentially segfault or whatever with the 3.6-final release.
>>> Hopefully this won't happen very often in practice, esp. for the later
>>> prereleases like the actual release candidates, but in principle it
>>> could happen. (For 3.5, the "final rc" was rc3... until they found a
>>> nasty problem with how they were building extension modules on
>>> Windows, which forced a last minute rc4 [1].)
>>>
>>> I agree that it would be really good to improve the UX for wheels on
>>> new Python releases, but I think it will require some discussion with
>>> the Python release managers (and possibly distutils-sig, in case we
>>> need finer grained Python version tags than just "3.6").
>>>
>>> Definitely no-one should be uploading 3.6.0a3 wheels to PyPI :-)
>>
>> OK - good to know.  I think we do need a solution of some sort.  I'm
>> guessing that people are starting to rely on wheels already, and they
>> aren't going to be happy when everything breaks down for a few weeks
>> after a new Python release.
>
> Resurrecting an old thread: this just came up on python-dev in the
> context of ABI changes between alpha 3 and beta 3, and I've proposed
> declaring the public ABI locked as part of the upcoming beta 4 release
> (which is 3 weeks before the planned final release, while there's only
> a single week planned between the first RC and the final release) to
> give folks that want to pre-publish Python 3.6 wheel files more of an
> opportunity to do so.

This just came up over at https://github.com/pypa/manylinux/issues/82

Can we depend on the current Python 3.6.0rc1 having the same ABI as
the final release?  And therefore start building 3.6 wheels with it?

Thanks for thinking about this,

Matthew

From njs at pobox.com  Fri Dec 23 20:02:56 2016
From: njs at pobox.com (Nathaniel Smith)
Date: Fri, 23 Dec 2016 17:02:56 -0800
Subject: [Wheel-builders] Heads-up re: new kernel configurations breaking
 the manylinux docker image
Message-ID: <CAPJVwB=xPVBDn+3i5Wwk=YpVSdhJ+SO6R67WreFRf_EBJfboUg@mail.gmail.com>

Hi all,

Just ran into this, and it took me a bit to track down, so wanted to
let people know.

The "vsyscall" mechanism is a clever hack/trick that Linux uses to
speed up some syscalls (most notably gettimeofday), and involves the
kernel injecting some code in processes' memory maps, that glibc then
knows to call. How does glibc know how to call it? In the old days,
the answer was simple: the kernel always injected it at the same fixed
address, so glibc was hard-coded to just "know" that e.g. gettimeofday
was at 0xffffffffff600400.

In these less trusting times, these hard-coded addresses are
considered a security risk (they violate ASLR etc.), so they were
deprecated a long time ago. The kernel now has configuration options
to control whether it injects this code at the fixed address. If you
flip the options to disable the old vsyscall stuff, then it makes your
system more secure... but, of course, if you try to run an old binary
that blindly uses the hardcoded addresses then it will segfault as
soon as it tries to call gettimeofday.

Debian has recently flipped the switch to disable this on their
kernels, so if you're running a recent Debian testing or unstable
(kernel 4.8 or better), then your experience of the manylinux_x86_64
docker image is now... not so great:

~$ docker run -it --rm quay.io/pypa/manylinux1_x86_64 /bin/bash
~$

Yeah, it just... silently exits when trying to do almost anything. One
clue as to what's going on is that the exit code is 139:

~$ echo $?
139

which means "segfault". I also get this in my kernel log (dmesg),
which is a little more informative:

[1060150.319215] bash[18225] vsyscall attempted with vsyscall=none
ip:ffffffffff600400 cs:33 sp:7ffff5521ea8 ax:ffffffffff600400
si:7ffff5522f80 di:0
[1060150.319216] bash[18225]: segfault at ffffffffff600400 ip
ffffffffff600400 sp 00007ffff5521ea8 error 15

This affects any docker image that uses glibc 2.13 or earlier. This
includes both CentOS 5 and CentOS 6.

The workaround is to reboot and add the option 'vsyscall=emulate' to
the kernel command line.

So far I guess it's only Arch and Debian testing/unstable doing this,
but I'd expect we may see this popping up with other distributors in
the future.

Bug report of this happening with Arch:
  https://github.com/CentOS/sig-cloud-instance-images/issues/62
Debian bug report:
  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=847154
Docker bug report:
  https://github.com/docker/docker/issues/28705

-n

-- 
Nathaniel J. Smith -- https://vorpus.org

From florian at apolloner.eu  Sat Dec 24 06:23:14 2016
From: florian at apolloner.eu (Florian Apolloner)
Date: Sat, 24 Dec 2016 12:23:14 +0100
Subject: [Wheel-builders] Heads-up re: new kernel configurations
 breaking the manylinux docker image
In-Reply-To: <CAPJVwB=xPVBDn+3i5Wwk=YpVSdhJ+SO6R67WreFRf_EBJfboUg@mail.gmail.com>
References: <CAPJVwB=xPVBDn+3i5Wwk=YpVSdhJ+SO6R67WreFRf_EBJfboUg@mail.gmail.com>
Message-ID: <1482578594.274133.828657321.7A297ACD@webmail.messagingengine.com>

On Sat, Dec 24, 2016, at 02:02 AM, Nathaniel Smith wrote:
> So far I guess it's only Arch and Debian testing/unstable doing this,
> but I'd expect we may see this popping up with other distributors in
> the future.

Afaik Debian already reverted that change for now. In the longrun though
it will be something which will get enabled.

From njs at pobox.com  Sat Dec 24 20:18:00 2016
From: njs at pobox.com (Nathaniel Smith)
Date: Sat, 24 Dec 2016 17:18:00 -0800
Subject: [Wheel-builders] Heads-up re: new kernel configurations
 breaking the manylinux docker image
In-Reply-To: <1482578594.274133.828657321.7A297ACD@webmail.messagingengine.com>
References: <CAPJVwB=xPVBDn+3i5Wwk=YpVSdhJ+SO6R67WreFRf_EBJfboUg@mail.gmail.com>
 <1482578594.274133.828657321.7A297ACD@webmail.messagingengine.com>
Message-ID: <CAPJVwBnxAg_POovXfc-bqQ+YRdfa1hhG2Ah2xY7WVvJe+VeBtg@mail.gmail.com>

On Sat, Dec 24, 2016 at 3:23 AM, Florian Apolloner <florian at apolloner.eu> wrote:
> On Sat, Dec 24, 2016, at 02:02 AM, Nathaniel Smith wrote:
>> So far I guess it's only Arch and Debian testing/unstable doing this,
>> but I'd expect we may see this popping up with other distributors in
>> the future.
>
> Afaik Debian already reverted that change for now. In the longrun though
> it will be something which will get enabled.

Ah, okay, I see it now:
  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=847154#37
It looks like the reversion is in 4.8.15-1, which has not yet made it
into testing. Not sure when that will happen given the freeze, but I
assume they'll make sure the reversion makes it into stretch.

-n

-- 
Nathaniel J. Smith -- https://vorpus.org

From njs at pobox.com  Sat Dec 31 18:21:56 2016
From: njs at pobox.com (Nathaniel Smith)
Date: Sat, 31 Dec 2016 15:21:56 -0800
Subject: [Wheel-builders] manylinux docker image now has python 3.6
Message-ID: <CAPJVwBkEJ_AyMxewjHhH9pqtJQ1mEVSCTppuHEbOVQnRci6rBw@mail.gmail.com>

Just as a heads up for anyone who was waiting before building wheels.

There's one minor issue, which is that it turns out the sqlite package
in 3.6 is incompatible with the libsqlite on centos 5, so for now the
3.6 builds have no sqlite package. This shouldn't you unless you
somehow use sqlite as part of your build process. (Does anyone use
sqlite as part of their build process?) Tracked here:
    https://github.com/pypa/manylinux/issues/90

-n

-- 
Nathaniel J. Smith -- https://vorpus.org