[Distutils] draft PEP: manylinux1

[M.-A. Lemburg]

On 21.01.2016 11:11, Nick Coghlan wrote:
> On 21 January 2016 at 20:05, M.-A. Lemburg <mal at egenix.com> wrote:
>> On 21.01.2016 10:31, Nick Coghlan wrote:
>>> On 21 January 2016 at 19:03, M.-A. Lemburg <mal at egenix.com> wrote:
>>>> By using the version based approach, we'd not run into this
>>>> problem and gain a lot more.
>>>
>>> I think it's better to start with a small core that we *know* works,
>>> then expand later, rather than trying to make the first iteration too
>>> wide. The "manylinux1" tag itself is versioned (hence the "1" at the
>>> end), so "manylinux2" may simply have *more* libraries defined, rather
>>> than newer ones.
>>
>> My argument is that the file based approach taken by the PEP
>> is too limiting to actually make things work for a large
>> set of Python packages.
>>
>> It will basically only work for packages that do not interface
>> to other external libraries (except for the few cases listed in
>> the PEP, e.g. X11, GL, which aren't always installed or
>> available either).
>>
>> IMO, testing the versions of a set of libraries is a safer
>> approach.
> 
> I still don't really understand what you mean by "testing the versions
> of a set of libraries", but if you have the time available to propose
> a competing PEP, that always leads to a stronger result than when we
> only have only proposed approach to consider.

I think the PEP is fine as it is, just the restriction to test
for the library file names is something that would need to be
changed to implement the different approach:

"""
For these reasons, we define a set of library versions that
are supported by a wide range of Linux distributions. We therefore
pick library versions which have been around for at least 5 years.

When using these external libraries, Python wheels should
only depend on library versions listed in the section below.

Python wheels are free to depend on additional libraries
not included in this set, however, care should be taken
that these additional libraries do not depend on later
versions of the listed libraries, e.g. OpenSSL libraries
compiled against the C library versions listed below.

The ``manylinux1`` policy thus encompasses a standard for which
versions of these external shared libraries a wheel may depend on,
and the maximum depended-upon symbol versions therein.

Future versions of the manylinux policy may include more
libraries, or move on to later versions.

The permitted external shared libraries and versions for
``manylinux1``are: ::

    libpanelw.so.5
    libncursesw.so.5
    libgcc_s.so.1
    libstdc++.so.6
    ...
"""

This will still lead to cases where a package doesn't work
because of missing system packages, but at least they won't
fail due to mismatch in basic C library versions, which is the
most problematic case for users.

The PEP will also have to address to problems introduced
by versioned symbols in more recent Linux shared libs:
even though the library file names have not changed,
they may well include different support levels for the
various APIs, e.g. glibc 2.1, 2.2, 2.3, etc.

For our binaries, we have chosen to use a system where
this versioning has not yet been enabled for system libs.
We did this because we found using a library compiled
against a versioned lib on a system which comes with
an unversioned lib causes warnings to be issued.

For openSUSE the change was applied between the 11.3 and
11.4 releases.

Some references which show case the problem:
 - ​
http://stackoverflow.com/questions/137773/what-does-the-no-version-information-available-error-from-linux-dynamic-linker/156387#156387
 - ​
http://superuser.com/questions/305055/how-to-diagnosis-and-resolve-usr-lib64-libz-so-1-no-version-information-avail
 - ​
http://forums.opensuse.org/english/get-technical-help-here/applications/466547-google-chrome-issues.html


-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Jan 21 2016)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...           http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...           http://zope.egenix.com/
________________________________________________________________________

::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/
                      http://www.malemburg.com/