From lele at metapensiero.it  Thu Aug 17 09:49:30 2017
From: lele at metapensiero.it (Lele Gaifax)
Date: Thu, 17 Aug 2017 15:49:30 +0200
Subject: [Wheel-builders] Problem with "too-recent versioned symbols"
Message-ID: <87o9remhhx.fsf@metapensiero.it>

Hi all,

since a few commits on a project I'm maintaining[1] the automatic build of
binary wheels (based on cibuildwheel[2]) on Travis CI started to fail[3],
although only for Python 3.6. It's a C++ extension module, and just a few days
ago I was able to release a first version[4] on PyPI with manylinux1&windows
wheels for Python 3.4, 3.5 and 3.6.

I tried to bisect the commits, but I could not figure out what triggered it;
furthermore, manually building that wheel within a pypa/manylinux docker
container on my local development machine and then repairing the result with
auditwheel tool, works without problem!

Do you have any hint about how I could investigate further, or an explanation
on why the job on Travis CI can give different results from the (virtually)
same context running on my GNU/Linux machine?

Thanks in advance,
ciao, lele.

[1] https://github.com/python-rapidjson/python-rapidjson
[2] https://pypi.python.org/pypi/cibuildwheel
[3] https://travis-ci.org/python-rapidjson/python-rapidjson/jobs/265453792#L1278
[4] https://pypi.python.org/pypi/python-rapidjson
-- 
nickname: Lele Gaifax | Quando vivr? di quello che ho pensato ieri
real: Emanuele Gaifas | comincer? ad aver paura di chi mi copia.
lele at metapensiero.it  |                 -- Fortunato Depero, 1929.

From ncoghlan at gmail.com  Thu Aug 17 21:17:04 2017
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 18 Aug 2017 11:17:04 +1000
Subject: [Wheel-builders] Problem with "too-recent versioned symbols"
In-Reply-To: <87o9remhhx.fsf@metapensiero.it>
References: <87o9remhhx.fsf@metapensiero.it>
Message-ID: <CADiSq7eKqxyxkR6Q5QQt9K1QQn8-b89aD8YGzDC7Jh8nMSz_1Q@mail.gmail.com>

On 17 August 2017 at 23:49, Lele Gaifax <lele at metapensiero.it> wrote:
> Do you have any hint about how I could investigate further, or an explanation
> on why the job on Travis CI can give different results from the (virtually)
> same context running on my GNU/Linux machine?

I was going to suggest that the use of Ubuntu 14.04 vs 12.04 might be
the issue, but it looks like your last successful build also used
14.04: https://travis-ci.org/python-rapidjson/python-rapidjson/builds/265206684

The fact that building with the pypa/manylinux images still works
doesn't actually tell you much - as far as I am aware, those *are*
CentOS 5 images, so their glibc (et al) won't have any newer symbol
versions to potentially pick up.

One thing you might want to try is to send yourself a PR that reverts
everything back to the way it was for your 0.1.0 release, and see if
*that* builds. If it doesn't, you can plausibly guess that something
changed unexpectedly in the Travis build environment. If it does
build, then you can start selectively removing things from the
reversion PR until the build breaks again.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From lele at metapensiero.it  Fri Aug 18 05:41:59 2017
From: lele at metapensiero.it (Lele Gaifax)
Date: Fri, 18 Aug 2017 11:41:59 +0200
Subject: [Wheel-builders] Problem with "too-recent versioned symbols"
In-Reply-To: <CADiSq7eKqxyxkR6Q5QQt9K1QQn8-b89aD8YGzDC7Jh8nMSz_1Q@mail.gmail.com>
References: <87o9remhhx.fsf@metapensiero.it>
 <CADiSq7eKqxyxkR6Q5QQt9K1QQn8-b89aD8YGzDC7Jh8nMSz_1Q@mail.gmail.com>
Message-ID: <87inhlus9k.fsf@metapensiero.it>

Nick Coghlan <ncoghlan at gmail.com> writes:

> One thing you might want to try is to send yourself a PR that reverts
> everything back to the way it was for your 0.1.0 release, and see if
> *that* builds. If it doesn't, you can plausibly guess that something
> changed unexpectedly in the Travis build environment. If it does
> build, then you can start selectively removing things from the
> reversion PR until the build breaks again.

Thank you Nick,

this morning I ran a dozen of different attempts, and even if I was able to
isolate the commit that makes the difference, I'm still lost in determining
exactly *what* it does "wrong".

Just for the history, and maybe to relax a bit, here is a brain dump of my
discovering:

* the commit that triggers the problem[1] is apparently innocuous, the
  prominent change is that instead of #including a "version.h" with some
  #defines it keeps the module's version in a toplevel "version.txt" and the
  actual version is passed to the compiler with a -D option

* the immediate parent commit[2] does not exhibit the issue, 

* I tried to wander around the problematic commit trying out several different
  approaches and trialing various assumptions, without luck: my last
  attempt[3] (for today...) seems to prove that it's not the -D option the
  cause

What a strange and boring problem :-\

Thanks for any further suggestion,
ciao, lele.
  
[1] https://github.com/python-rapidjson/python-rapidjson/commit/957be619ae24e5758bfd93711713d0296724acb9
[2] https://github.com/python-rapidjson/python-rapidjson/commit/3f350c7e3362d23713c06fa2884d92e10e0cc018
[3] https://github.com/python-rapidjson/python-rapidjson/commit/f750e43c23e3e990a4e596029c0029fab2793d55
-- 
nickname: Lele Gaifax | Quando vivr? di quello che ho pensato ieri
real: Emanuele Gaifas | comincer? ad aver paura di chi mi copia.
lele at metapensiero.it  |                 -- Fortunato Depero, 1929.

From ncoghlan at gmail.com  Fri Aug 18 09:40:42 2017
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 18 Aug 2017 23:40:42 +1000
Subject: [Wheel-builders] Problem with "too-recent versioned symbols"
In-Reply-To: <87inhlus9k.fsf@metapensiero.it>
References: <87o9remhhx.fsf@metapensiero.it>
 <CADiSq7eKqxyxkR6Q5QQt9K1QQn8-b89aD8YGzDC7Jh8nMSz_1Q@mail.gmail.com>
 <87inhlus9k.fsf@metapensiero.it>
Message-ID: <CADiSq7dXuXSX2GbxTAn2xAPKQvfhTiiR+w_hxTWA-ifX6zgqrg@mail.gmail.com>

On 18 August 2017 at 19:41, Lele Gaifax <lele at metapensiero.it> wrote:
> * I tried to wander around the problematic commit trying out several different
>   approaches and trialing various assumptions, without luck: my last
>   attempt[3] (for today...) seems to prove that it's not the -D option the
>   cause

D'oh, after reading the first problematic commit, the "define_macros"
argument to Extension was going to be my suggestion for a possible
culprit, since I wasn't sure if it might have any side effects. But
you already tried removing it again, with no improvement :(

The only outlandish idea I can offer is to wonder whether or not gcc
was seeing the "version.h", assuming it meant the kernel header, and
implicitly setting some flags that changed the default linker
behaviour (I'm aware that suggestion makes no sense, but neither does
the behaviour you're seeing).

Beyond that, all I can personally offer is the suggestion to look into
whether or not you can get ciwheelbuilder and/or auditwheel to tell
exactly *what* symbols are coming from the wrong version of *which*
library.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From lele at metapensiero.it  Fri Aug 18 11:05:19 2017
From: lele at metapensiero.it (Lele Gaifax)
Date: Fri, 18 Aug 2017 17:05:19 +0200
Subject: [Wheel-builders] Problem with "too-recent versioned symbols"
In-Reply-To: <CADiSq7dXuXSX2GbxTAn2xAPKQvfhTiiR+w_hxTWA-ifX6zgqrg@mail.gmail.com>
References: <87o9remhhx.fsf@metapensiero.it>
 <CADiSq7eKqxyxkR6Q5QQt9K1QQn8-b89aD8YGzDC7Jh8nMSz_1Q@mail.gmail.com>
 <87inhlus9k.fsf@metapensiero.it>
 <CADiSq7dXuXSX2GbxTAn2xAPKQvfhTiiR+w_hxTWA-ifX6zgqrg@mail.gmail.com>
Message-ID: <874lt4q5lc.fsf@metapensiero.it>

Nick Coghlan <ncoghlan at gmail.com> writes:

> The only outlandish idea I can offer is to wonder whether or not gcc
> was seeing the "version.h", assuming it meant the kernel header, and
> implicitly setting some flags that changed the default linker
> behaviour (I'm aware that suggestion makes no sense, but neither does
> the behaviour you're seeing).

Well, given that I already tried to reject the "makes no sense" excuse for
other attempts, I will try the above too.

> Beyond that, all I can personally offer is the suggestion to look into
> whether or not you can get ciwheelbuilder and/or auditwheel to tell
> exactly *what* symbols are coming from the wrong version of *which*
> library.

I installed a customized cibuildwheel that executes a "auditwheel show" just
before repairing the wheel, and below I'm reporting three cases, respectively
"working Python 3.5", "failing Python 3.6" and "working Python 3.6".

Just after the commit[1] that triggered the problem, it outputs the following
for Python 3.5:

  auditwheel show /tmp/built_wheel/python_rapidjson-0.1.0-cp35-cp35m-linux_x86_64.whl

  python_rapidjson-0.1.0-cp35-cp35m-linux_x86_64.whl is consistent with
  the following platform tag: "manylinux1_x86_64".

  The wheel references external versioned symbols in these system-
  provided shared libraries: libgcc_s.so.1 with versions {'GCC_3.0'},
  libpthread.so.0 with versions {'GLIBC_2.2.5'}, libstdc++.so.6 with
  versions {'GLIBCXX_3.4', 'CXXABI_1.3'}, libc.so.6 with versions
  {'GLIBC_2.2.5'}

  The following external shared libraries are required by the wheel:
  {
      "libc.so.6": "/lib64/libc-2.5.so",
      "libgcc_s.so.1": "/lib64/libgcc_s-4.1.2-20080825.so.1",
      "libm.so.6": "/lib64/libm-2.5.so",
      "libpthread.so.0": "/lib64/libpthread-2.5.so",
      "libstdc++.so.6": "/usr/lib64/libstdc++.so.6.0.8"
  }

While for Python 3.6 it tells:

  auditwheel show /tmp/built_wheel/python_rapidjson-0.1.0-cp36-cp36m-linux_x86_64.whl

  python_rapidjson-0.1.0-cp36-cp36m-linux_x86_64.whl is consistent with
  the following platform tag: "linux_x86_64".

  The wheel references external versioned symbols in these system-
  provided shared libraries: libgcc_s.so.1 with versions {'GCC_3.0'},
  libstdc++.so.6 with versions {'GLIBCXX_3.4', 'CXXABI_1.3'}, libc.so.6
  with versions {'GLIBC_2.14', 'GLIBC_2.2.5', 'GLIBC_2.4', 'GLIBC_2.3',
  'GLIBC_2.3.4'}

  This constrains the platform tag to "linux_x86_64". In order to
  achieve a more compatible tag, you would to recompile a new wheel from
  source on a system with earlier versions of these libraries, such as
  CentOS 5.

At this point I made a branch from the v0.1.0 tag and used the customized
cibuildwheel; in that case the output for Python 3.6 is:

  auditwheel show /tmp/built_wheel/python_rapidjson-0.1.0-cp36-cp36m-linux_x86_64.whl

  python_rapidjson-0.1.0-cp36-cp36m-linux_x86_64.whl is consistent with
  the following platform tag: "manylinux1_x86_64".

  The wheel references external versioned symbols in these system-
  provided shared libraries: libgcc_s.so.1 with versions {'GCC_3.0'},
  libpthread.so.0 with versions {'GLIBC_2.2.5'}, libstdc++.so.6 with
  versions {'GLIBCXX_3.4', 'CXXABI_1.3'}, libc.so.6 with versions
  {'GLIBC_2.2.5'}

  The following external shared libraries are required by the wheel:
  {
      "libc.so.6": "/lib64/libc-2.5.so",
      "libgcc_s.so.1": "/lib64/libgcc_s-4.1.2-20080825.so.1",
      "libm.so.6": "/lib64/libm-2.5.so",
      "libpthread.so.0": "/lib64/libpthread-2.5.so",
      "libstdc++.so.6": "/usr/lib64/libstdc++.so.6.0.8"
  }

So, I can spot two differences, in the failing case

a) there's no reference to libpthread.so
b) there are five GLIBC_xxx versions instead of a single GLIBC_2.2.5

I have no clue on what that could mean :-) Does anybody?

As usual, many thanks anyway!

ciao, lele.

[1] https://github.com/python-rapidjson/python-rapidjson/commit/957be619ae24e5758bfd93711713d0296724acb9
-- 
nickname: Lele Gaifax | Quando vivr? di quello che ho pensato ieri
real: Emanuele Gaifas | comincer? ad aver paura di chi mi copia.
lele at metapensiero.it  |                 -- Fortunato Depero, 1929.

From njs at pobox.com  Fri Aug 18 20:09:30 2017
From: njs at pobox.com (Nathaniel Smith)
Date: Fri, 18 Aug 2017 17:09:30 -0700
Subject: [Wheel-builders] Problem with "too-recent versioned symbols"
In-Reply-To: <874lt4q5lc.fsf@metapensiero.it>
References: <87o9remhhx.fsf@metapensiero.it>
 <CADiSq7eKqxyxkR6Q5QQt9K1QQn8-b89aD8YGzDC7Jh8nMSz_1Q@mail.gmail.com>
 <87inhlus9k.fsf@metapensiero.it>
 <CADiSq7dXuXSX2GbxTAn2xAPKQvfhTiiR+w_hxTWA-ifX6zgqrg@mail.gmail.com>
 <874lt4q5lc.fsf@metapensiero.it>
Message-ID: <CAPJVwBmJHgbO=x-5wJQ+GDEuSKUuUSzDG9xsDuzjbm1jbg1DPQ@mail.gmail.com>

On Fri, Aug 18, 2017 at 8:05 AM, Lele Gaifax <lele at metapensiero.it> wrote:
> While for Python 3.6 it tells:
>
>   auditwheel show /tmp/built_wheel/python_rapidjson-0.1.0-cp36-cp36m-linux_x86_64.whl
>
>   python_rapidjson-0.1.0-cp36-cp36m-linux_x86_64.whl is consistent with
>   the following platform tag: "linux_x86_64".
>
>   The wheel references external versioned symbols in these system-
>   provided shared libraries: libgcc_s.so.1 with versions {'GCC_3.0'},
>   libstdc++.so.6 with versions {'GLIBCXX_3.4', 'CXXABI_1.3'}, libc.so.6
>   with versions {'GLIBC_2.14', 'GLIBC_2.2.5', 'GLIBC_2.4', 'GLIBC_2.3',
>   'GLIBC_2.3.4'}

It's the GLIBC_2.14 version that's causing the problem: that means a
symbol that was added in glibc 2.14, but manylinux binaries are
supposed to work against glibc 2.5 and later.

It's very odd that you're getting that -- it shouldn't be possible to
get a GLIBC_2.14 version without building against glibc 2.14 or
later... but there is no such glibc in the manylinux docker images.

Probably the first thing to is to figure out what symbol is causing
the problem. Something like:

readelf -s your/built/extension.so | grep GLIBC_2.14

should say. Looking at the symbols that 2.14 changed though, I bet it's memcpy.

Maybe the devtools compilers are doing something weird and actually
injecting a copy of glibc 2.14's memcpy into your binary as a globally
visible symbol, or something like that? But if this was triggered
every time someone used memcpy we'd definitely have noticed it before
now...

-n

-- 
Nathaniel J. Smith -- https://vorpus.org

From lele at metapensiero.it  Sat Aug 19 03:37:31 2017
From: lele at metapensiero.it (Lele Gaifax)
Date: Sat, 19 Aug 2017 09:37:31 +0200
Subject: [Wheel-builders] Problem with "too-recent versioned symbols"
In-Reply-To: <CAPJVwBmJHgbO=x-5wJQ+GDEuSKUuUSzDG9xsDuzjbm1jbg1DPQ@mail.gmail.com>
References: <87o9remhhx.fsf@metapensiero.it>
 <CADiSq7eKqxyxkR6Q5QQt9K1QQn8-b89aD8YGzDC7Jh8nMSz_1Q@mail.gmail.com>
 <87inhlus9k.fsf@metapensiero.it>
 <CADiSq7dXuXSX2GbxTAn2xAPKQvfhTiiR+w_hxTWA-ifX6zgqrg@mail.gmail.com>
 <874lt4q5lc.fsf@metapensiero.it>
 <CAPJVwBmJHgbO=x-5wJQ+GDEuSKUuUSzDG9xsDuzjbm1jbg1DPQ@mail.gmail.com>
Message-ID: <877ey0j9dw.fsf@metapensiero.it>

Nathaniel Smith <njs at pobox.com> writes:

> Probably the first thing to is to figure out what symbol is causing
> the problem. Something like:
>
> readelf -s your/built/extension.so | grep GLIBC_2.14
>
> should say. Looking at the symbols that 2.14 changed though, I bet it's memcpy.
>
> Maybe the devtools compilers are doing something weird and actually
> injecting a copy of glibc 2.14's memcpy into your binary as a globally
> visible symbol, or something like that? But if this was triggered
> every time someone used memcpy we'd definitely have noticed it before
> now...

Thank you Nathaniel,

I added the ``readelf`` command to my hacked cibuildwheel, and this is the output:

  + /opt/python/cp36-cp36m/bin/python -m zipfile -e /tmp/built_wheel/python_rapidjson-0.1.0-cp36-cp36m-linux_x86_64.whl /tmp/extracted_wheel
  + readelf -s /tmp/extracted_wheel/rapidjson.cpython-36m-x86_64-linux-gnu.so
  + grep GLIBC_2.14
      76: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND memcpy at GLIBC_2.14 (9)
     230: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND memcpy@@GLIBC_2.14
  + ls -l /lib64/libc-2.5.so /lib64/libcap.so.1 /lib64/libcap.so.1.10 /lib64/libcidn-2.5.so /lib64/libcidn.so.1 /lib64/libcom_err.so.2 /lib64/libcom_err.so.2.1 /lib64/libcrypt-2.5.so /lib64/libcrypto.so.0.9.8e /lib64/libcrypto.so.6 /lib64/libcrypt.so.1 /lib64/libc.so.6
  -rwxr-xr-x 1 root root 1720712 Aug 17  2015 /lib64/libc-2.5.so
  lrwxrwxrwx 1 root root      14 Mar  4  2015 /lib64/libcap.so.1 -> libcap.so.1.10
  -rwxr-xr-x 1 root root   15120 Mar 14  2007 /lib64/libcap.so.1.10
  -rwxr-xr-x 1 root root  197744 Aug 17  2015 /lib64/libcidn-2.5.so
  lrwxrwxrwx 1 root root      14 May 26 22:30 /lib64/libcidn.so.1 -> libcidn-2.5.so
  lrwxrwxrwx 1 root root      17 Mar  4  2015 /lib64/libcom_err.so.2 -> libcom_err.so.2.1
  -rwxr-xr-x 1 root root    7832 Sep 18  2014 /lib64/libcom_err.so.2.1
  -rwxr-xr-x 1 root root   45728 Aug 17  2015 /lib64/libcrypt-2.5.so
  -rwxr-xr-x 1 root root 1367280 May 31  2016 /lib64/libcrypto.so.0.9.8e
  lrwxrwxrwx 1 root root      19 May 26 22:30 /lib64/libcrypto.so.6 -> libcrypto.so.0.9.8e
  lrwxrwxrwx 1 root root      15 May 26 22:30 /lib64/libcrypt.so.1 -> libcrypt-2.5.so
  lrwxrwxrwx 1 root root      11 May 26 22:30 /lib64/libc.so.6 -> libc-2.5.so

So, you were right guessing about the memcpy symbol. Also, as the ``ls``
command shows, effectively the glibc available within the manylinux image is
2.5...

I'm still puzzled about *how* that can happen, given that I do not seem to
have altered the source code in a way that would justify that, but even more,
as you say, how can that glibc version be referenced, if it does not even
exist in the context.

I remain willing to try any suggestion,
ciao, lele.
-- 
nickname: Lele Gaifax | Quando vivr? di quello che ho pensato ieri
real: Emanuele Gaifas | comincer? ad aver paura di chi mi copia.
lele at metapensiero.it  |                 -- Fortunato Depero, 1929.

From ncoghlan at gmail.com  Sat Aug 19 06:00:21 2017
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 19 Aug 2017 20:00:21 +1000
Subject: [Wheel-builders] Problem with "too-recent versioned symbols"
In-Reply-To: <877ey0j9dw.fsf@metapensiero.it>
References: <87o9remhhx.fsf@metapensiero.it>
 <CADiSq7eKqxyxkR6Q5QQt9K1QQn8-b89aD8YGzDC7Jh8nMSz_1Q@mail.gmail.com>
 <87inhlus9k.fsf@metapensiero.it>
 <CADiSq7dXuXSX2GbxTAn2xAPKQvfhTiiR+w_hxTWA-ifX6zgqrg@mail.gmail.com>
 <874lt4q5lc.fsf@metapensiero.it>
 <CAPJVwBmJHgbO=x-5wJQ+GDEuSKUuUSzDG9xsDuzjbm1jbg1DPQ@mail.gmail.com>
 <877ey0j9dw.fsf@metapensiero.it>
Message-ID: <CADiSq7eRU4W1Fpoz021v0mnnWWmZtW8eyag+d350nHwY+OpLzg@mail.gmail.com>

On 19 August 2017 at 17:37, Lele Gaifax <lele at metapensiero.it> wrote:
> Nathaniel Smith <njs at pobox.com> writes:
> I'm still puzzled about *how* that can happen, given that I do not seem to
> have altered the source code in a way that would justify that, but even more,
> as you say, how can that glibc version be referenced, if it does not even
> exist in the context.
>
> I remain willing to try any suggestion,

While it won't answer the "how", it may be worth trying adding this to
see if you can keep it from happening:

    #ifdef __GLIBC__
    __asm__(".symver memcpy,memcpy at GLIBC_2.2.5");
    #endif

Beyond that, reading
https://github.com/joerick/cibuildwheel/blob/master/cibuildwheel/linux.py
means I share the confusion as to *how* you could be picking up a
newer glibc version (unless there's a bug in the manylinux1 docker
images themselves)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From lele at metapensiero.it  Sat Aug 19 06:15:34 2017
From: lele at metapensiero.it (Lele Gaifax)
Date: Sat, 19 Aug 2017 12:15:34 +0200
Subject: [Wheel-builders] Problem with "too-recent versioned symbols"
In-Reply-To: <CADiSq7eRU4W1Fpoz021v0mnnWWmZtW8eyag+d350nHwY+OpLzg@mail.gmail.com>
References: <87o9remhhx.fsf@metapensiero.it>
 <CADiSq7eKqxyxkR6Q5QQt9K1QQn8-b89aD8YGzDC7Jh8nMSz_1Q@mail.gmail.com>
 <87inhlus9k.fsf@metapensiero.it>
 <CADiSq7dXuXSX2GbxTAn2xAPKQvfhTiiR+w_hxTWA-ifX6zgqrg@mail.gmail.com>
 <874lt4q5lc.fsf@metapensiero.it>
 <CAPJVwBmJHgbO=x-5wJQ+GDEuSKUuUSzDG9xsDuzjbm1jbg1DPQ@mail.gmail.com>
 <877ey0j9dw.fsf@metapensiero.it>
 <CADiSq7eRU4W1Fpoz021v0mnnWWmZtW8eyag+d350nHwY+OpLzg@mail.gmail.com>
Message-ID: <87378nkgmx.fsf@metapensiero.it>

Nick Coghlan <ncoghlan at gmail.com> writes:

> While it won't answer the "how", it may be worth trying adding this to
> see if you can keep it from happening:
>
>     #ifdef __GLIBC__
>     __asm__(".symver memcpy,memcpy at GLIBC_2.2.5");
>     #endif

Yes, I thought about that, after reading the recent "Building manylinux1
wheels with newer toolchains" thread in this ML: I will try and report back.

> Beyond that, reading
> https://github.com/joerick/cibuildwheel/blob/master/cibuildwheel/linux.py
> means I share the confusion as to *how* you could be picking up a
> newer glibc version (unless there's a bug in the manylinux1 docker
> images themselves)

Yeah, but there's a double strangeness: I also tried to do the "build&repair"
steps on my local machine, within a quay.io/pypa/manylinux1_x86_64 container
(that AFAICT is the same image used by Travis CI itself)[1], where everything
went smooth, producing a working Py36 manylinux1 wheel, ... weird!

Thank you,
ciao, lele.

[1] thanks to Nathaniel for the thread "Heads-up re: new kernel
    configurations...", that finally explained why I wasn't able to get even a
    prompt... :)
-- 
nickname: Lele Gaifax | Quando vivr? di quello che ho pensato ieri
real: Emanuele Gaifas | comincer? ad aver paura di chi mi copia.
lele at metapensiero.it  |                 -- Fortunato Depero, 1929.

From lele at metapensiero.it  Sat Aug 19 06:56:46 2017
From: lele at metapensiero.it (Lele Gaifax)
Date: Sat, 19 Aug 2017 12:56:46 +0200
Subject: [Wheel-builders] Problem with "too-recent versioned symbols"
In-Reply-To: <87378nkgmx.fsf@metapensiero.it>
References: <87o9remhhx.fsf@metapensiero.it>
 <CADiSq7eKqxyxkR6Q5QQt9K1QQn8-b89aD8YGzDC7Jh8nMSz_1Q@mail.gmail.com>
 <87inhlus9k.fsf@metapensiero.it>
 <CADiSq7dXuXSX2GbxTAn2xAPKQvfhTiiR+w_hxTWA-ifX6zgqrg@mail.gmail.com>
 <874lt4q5lc.fsf@metapensiero.it>
 <CAPJVwBmJHgbO=x-5wJQ+GDEuSKUuUSzDG9xsDuzjbm1jbg1DPQ@mail.gmail.com>
 <877ey0j9dw.fsf@metapensiero.it>
 <CADiSq7eRU4W1Fpoz021v0mnnWWmZtW8eyag+d350nHwY+OpLzg@mail.gmail.com>
 <87378nkgmx.fsf@metapensiero.it>
Message-ID: <87y3qfj05t.fsf@metapensiero.it>

Lele Gaifax <lele at metapensiero.it> writes:

> Nick Coghlan <ncoghlan at gmail.com> writes:
>
>> While it won't answer the "how", it may be worth trying adding this to
>> see if you can keep it from happening:
>>
>>     #ifdef __GLIBC__
>>     __asm__(".symver memcpy,memcpy at GLIBC_2.2.5");
>>     #endif
>
> Yes, I thought about that, after reading the recent "Building manylinux1
> wheels with newer toolchains" thread in this ML: I will try and report back.
>

I added that trick[1], that seems to be effectively fixing the problem: on my
local machine, inspecting the result .so with ``readelf`` shows the expected
version of memcpy, and the whole test suite pass. What happens on Travis CI is
different though, the wheel gets correctly repaired:

  auditwheel repair /tmp/built_wheel/python_rapidjson-0.1.0-cp36-cp36m-linux_x86_64.whl -w /tmp/delocated_wheel

  Repairing python_rapidjson-0.1.0-cp36-cp36m-linux_x86_64.whl
  Previous filename tags: linux_x86_64
  New filename tags: manylinux1_x86_64
  Previous WHEEL info tags: cp36-cp36m-linux_x86_64
  New WHEEL info tags: cp36-cp36m-manylinux1_x86_64

but as soon cibuildwheel runs the test suite it immediately fails with:

  Traceback (most recent call last):
    File "/opt/_internal/cpython-3.6.0/lib/python3.6/site-packages/_pytest/config.py", line 379, in _importconftest
      mod = conftestpath.pyimport()
    File "/opt/_internal/cpython-3.6.0/lib/python3.6/site-packages/py/_path/local.py", line 662, in pyimport
      __import__(modname)
    File "/opt/_internal/cpython-3.6.0/lib/python3.6/site-packages/_pytest/assertion/rewrite.py", line 212, in load_module
      py.builtin.exec_(co, mod.__dict__)
    File "/project/tests/conftest.py", line 26, in <module>
      import rapidjson as rj
  ImportError: /opt/_internal/cpython-3.6.0/lib/python3.6/site-packages/rapidjson.cpython-36m-x86_64-linux-gnu.so: ELF file OS ABI invalid

Do you think I should try to open an issue on GH pypa/manylinux1, referencing
this thread, and see if anybody has a clue on what's going on?

Thanks again for your time,
ciao, lele.

[1] https://github.com/python-rapidjson/python-rapidjson/commit/7fc196333cd33242b715c6092253b1036473898b
-- 
nickname: Lele Gaifax | Quando vivr? di quello che ho pensato ieri
real: Emanuele Gaifas | comincer? ad aver paura di chi mi copia.
lele at metapensiero.it  |                 -- Fortunato Depero, 1929.

From lele at metapensiero.it  Mon Aug 21 06:54:32 2017
From: lele at metapensiero.it (Lele Gaifax)
Date: Mon, 21 Aug 2017 12:54:32 +0200
Subject: [Wheel-builders] Problem with "too-recent versioned symbols"
In-Reply-To: <87y3qfj05t.fsf@metapensiero.it>
References: <87o9remhhx.fsf@metapensiero.it>
 <CADiSq7eKqxyxkR6Q5QQt9K1QQn8-b89aD8YGzDC7Jh8nMSz_1Q@mail.gmail.com>
 <87inhlus9k.fsf@metapensiero.it>
 <CADiSq7dXuXSX2GbxTAn2xAPKQvfhTiiR+w_hxTWA-ifX6zgqrg@mail.gmail.com>
 <874lt4q5lc.fsf@metapensiero.it>
 <CAPJVwBmJHgbO=x-5wJQ+GDEuSKUuUSzDG9xsDuzjbm1jbg1DPQ@mail.gmail.com>
 <877ey0j9dw.fsf@metapensiero.it>
 <CADiSq7eRU4W1Fpoz021v0mnnWWmZtW8eyag+d350nHwY+OpLzg@mail.gmail.com>
 <87378nkgmx.fsf@metapensiero.it> <87y3qfj05t.fsf@metapensiero.it>
Message-ID: <87a82tjimv.fsf@metapensiero.it>

Just for the record, I tried also the variant[1] suggested by Robert
T. McGibbon in the thread "Building manylinux1 wheels with newer toolchains",
but got the very same error when loading the resulting wheel[2].

I'm dejected and will sadly consider to either attempt to revert the offending
commit, or disable the production of Python 3.6 manylinux1 wheel.

Thanks for any further encouragement,
ciao, lele.

[1] https://github.com/python-rapidjson/python-rapidjson/commit/9dbad39e625577beeb1a2c890c08de9f6df6875f
[2] https://travis-ci.org/python-rapidjson/python-rapidjson/builds/266748999#L1420
-- 
nickname: Lele Gaifax | Quando vivr? di quello che ho pensato ieri
real: Emanuele Gaifas | comincer? ad aver paura di chi mi copia.
lele at metapensiero.it  |                 -- Fortunato Depero, 1929.

From lele at metapensiero.it  Wed Aug 23 08:59:13 2017
From: lele at metapensiero.it (Lele Gaifax)
Date: Wed, 23 Aug 2017 14:59:13 +0200
Subject: [Wheel-builders] Problem with "too-recent versioned symbols"
In-Reply-To: <87a82tjimv.fsf@metapensiero.it>
References: <87o9remhhx.fsf@metapensiero.it>
 <CADiSq7eKqxyxkR6Q5QQt9K1QQn8-b89aD8YGzDC7Jh8nMSz_1Q@mail.gmail.com>
 <87inhlus9k.fsf@metapensiero.it>
 <CADiSq7dXuXSX2GbxTAn2xAPKQvfhTiiR+w_hxTWA-ifX6zgqrg@mail.gmail.com>
 <874lt4q5lc.fsf@metapensiero.it>
 <CAPJVwBmJHgbO=x-5wJQ+GDEuSKUuUSzDG9xsDuzjbm1jbg1DPQ@mail.gmail.com>
 <877ey0j9dw.fsf@metapensiero.it>
 <CADiSq7eRU4W1Fpoz021v0mnnWWmZtW8eyag+d350nHwY+OpLzg@mail.gmail.com>
 <87378nkgmx.fsf@metapensiero.it> <87y3qfj05t.fsf@metapensiero.it>
 <87a82tjimv.fsf@metapensiero.it>
Message-ID: <87a82qh23i.fsf@metapensiero.it>

Hi all,

the (interim?) solution I found is using a different CI service to build the
Python 3.6 wheels. This[1] seems to prove that there something strange going
on in the Travis CI service, as the very same source, compiled within the same
manylinux1 Docker image, does not exhibit the problem I explained in this
thread.

Thanks&bye, lele.

[1] https://gitlab.com/metapensiero/python-rapidjson/-/jobs/30065688
-- 
nickname: Lele Gaifax | Quando vivr? di quello che ho pensato ieri
real: Emanuele Gaifas | comincer? ad aver paura di chi mi copia.
lele at metapensiero.it  |                 -- Fortunato Depero, 1929.

From njs at pobox.com  Wed Aug 23 20:31:46 2017
From: njs at pobox.com (Nathaniel Smith)
Date: Wed, 23 Aug 2017 17:31:46 -0700
Subject: [Wheel-builders] Problem with "too-recent versioned symbols"
In-Reply-To: <87a82qh23i.fsf@metapensiero.it>
References: <87o9remhhx.fsf@metapensiero.it>
 <CADiSq7eKqxyxkR6Q5QQt9K1QQn8-b89aD8YGzDC7Jh8nMSz_1Q@mail.gmail.com>
 <87inhlus9k.fsf@metapensiero.it>
 <CADiSq7dXuXSX2GbxTAn2xAPKQvfhTiiR+w_hxTWA-ifX6zgqrg@mail.gmail.com>
 <874lt4q5lc.fsf@metapensiero.it>
 <CAPJVwBmJHgbO=x-5wJQ+GDEuSKUuUSzDG9xsDuzjbm1jbg1DPQ@mail.gmail.com>
 <877ey0j9dw.fsf@metapensiero.it>
 <CADiSq7eRU4W1Fpoz021v0mnnWWmZtW8eyag+d350nHwY+OpLzg@mail.gmail.com>
 <87378nkgmx.fsf@metapensiero.it> <87y3qfj05t.fsf@metapensiero.it>
 <87a82tjimv.fsf@metapensiero.it> <87a82qh23i.fsf@metapensiero.it>
Message-ID: <CAPJVwBmJoEk1NDjx4GB=Ab0sRW40LxPgpD6HOo_WidTKWzCH0A@mail.gmail.com>

All I can think of is that there must be some strange bug in the
devtools toolchain we're using, possibly triggered by some kernel
versions and not others (since that's the main thing that docker
containers fail to isolate processes from).

If that's right though then I'm not sure it's worth trying to track
down the exact details -- we're using the devtools-2 release, while
upstream is on devtools-6. And the only reason we're still using this
toolchain is that it's the last version to support CentOS 5, which is
EOLed, and we'll be abandoning CentOS 5 and devtools-2 as soon as
someone has a chance to put together manylinux2.

On Wed, Aug 23, 2017 at 5:59 AM, Lele Gaifax <lele at metapensiero.it> wrote:
> Hi all,
>
> the (interim?) solution I found is using a different CI service to build the
> Python 3.6 wheels. This[1] seems to prove that there something strange going
> on in the Travis CI service, as the very same source, compiled within the same
> manylinux1 Docker image, does not exhibit the problem I explained in this
> thread.
>
> Thanks&bye, lele.
>
> [1] https://gitlab.com/metapensiero/python-rapidjson/-/jobs/30065688
> --
> nickname: Lele Gaifax | Quando vivr? di quello che ho pensato ieri
> real: Emanuele Gaifas | comincer? ad aver paura di chi mi copia.
> lele at metapensiero.it  |                 -- Fortunato Depero, 1929.



-- 
Nathaniel J. Smith -- https://vorpus.org