[Distutils] Does anyone understand what's going on with libpython on Linux?

[Robert T. McGibbon]

> One option we have then is to remove all DT_NEEDED references to
libpython in manylinux wheels. We get instant compatibility for bare
Debian / Ubuntu Python installs, at the cost of causing some puzzling
crash for the case of: dlopened library with embedded Python
interpreter where the embedded Python interpreter imports a manylinux
wheel.

I don't think this is acceptable, since it's going to break some packages
that depend
on dlopen.

> On the other hand, presumably this same crash will occur for nearly
all Debian-packaged Python extension modules (if it is true that they
do not specify a libpython dependency) - so it seems unlikely that
this is a common problem.

I don't think so. Debian-packaged extensions that require libpython to exist
(a minority of them to be sure, but ones that use complex shared library
layouts)
just declare a dependency on libpython. For example, python-pyside has a
Depends on libpython2.7:

```
$ apt-cache depends python-pyside.qtcore
python-pyside.qtcore
  Depends: libc6
  Depends: libgcc1
  Depends: libpyside1.2
  Depends: libpython2.7
  Depends: libqtcore4
  Depends: libshiboken1.2v5
  Depends: libstdc++6
  Depends: python
  Depends: python
  Conflicts: python-pyside.qtcore:i386
```

-Robert




On Sun, Feb 7, 2016 at 2:06 PM, Matthew Brett <matthew.brett at gmail.com>
wrote:

> On Sun, Feb 7, 2016 at 4:38 AM, Antoine Pitrou <solipsis at pitrou.net>
> wrote:
> > On Sun, 7 Feb 2016 00:25:57 -0800
> > "Robert T. McGibbon" <rmcgibbo at gmail.com> wrote:
> >> > What are Debian/Ubuntu doing in distutils so that extensions don't
> link
> >> to libpython by default?
> >>
> >> I don't know exactly, but one way to reproduce this is simply to build
> the
> >> interpreter without `--enable-shared`.
> >
> > See https://bugs.python.org/issue21536. It would be nice if you could
> > lobby for this issue to be resolved... (though that would only be for
> > 3.6, presumably)
>
> Just to unpack from that issue - and quoting a nice summary by you
> (Antoine):
>
> "... the -l flag was added in #832799, for a rather complicated case
> where the interpreter is linked with a library dlopened by an
> embedding application (I suppose for some kind of plugin system)."
>
> Following the link to https://bugs.python.org/issue832799 - the `-l`
> flag (and therefore the dependency on libpython was added at Python
> 2.3 for the case where an executable A dlopens a library B.so . B.so
> has an embedded Python interpreter and is linked to libpython.
> However, when the embedded Python interpreter in B.so loads an
> extension module mymodule.so , mymodule.so does not inherit a
> namespace with the libpython symbols already loaded. See
> https://bugs.python.org/msg18810 .
>
> One option we have then is to remove all DT_NEEDED references to
> libpython in manylinux wheels. We get instant compatibility for bare
> Debian / Ubuntu Python installs, at the cost of causing some puzzling
> crash for the case of: dlopened library with embedded Python
> interpreter where the embedded Python interpreter imports a manylinux
> wheel.
>
> On the other hand, presumably this same crash will occur for nearly
> all Debian-packaged Python extension modules (if it is true that they
> do not specify a libpython dependency) - so it seems unlikely that
> this is a common problem.
>
> Cheers,
>
> Matthew
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
-Robert
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160207/390eb173/attachment.html>