[New-bugs-announce] [issue36753] Python modules not linking to libpython causes issues for RTLD_LOCAL system-wide

[reimar]

New submission from reimar <pybug at reimardoeffinger.de>:

Most affected platforms: RedHat and Debian, but with the changes from issue21536 probably all Linux distributions will be affected.

issue34814 and issue21536 and https://bugzilla.redhat.com/show_bug.cgi?id=1585201 make statements along the lines of "In short, RTLD_LOCAL is not supported."
This might have been considered a reasonable stance because of the specific example opening libpython directly.
However Python modules not linking to libpython also breaks things when libpython is loaded in the most indirect ways via dlopen.
E.g. dlopen("libA.so", RTLD_LOCAL | RTLD_NOW)
libA might have linked against libB, libB against libC and libC might optionally link against libpython.

As a developer generally cannot really know if some library might ever pull in a most indirect reference to libpython, not supporting RTLD_LOCAL in Python essentially means RTLD_LOCAL can NEVER EVER be used safely.

A test-case that will fail the import command when modules have not been linked against libpython is attached (demonstrating only one layer of indirection, but much more complex cases are of course possible).
You will need to adjust the (include, lib) paths in test.sh for your Python version, it was written to demonstrate the issue against RedHat's modifications of Python 2.7 (to my knowledge, RedHat and Debian has been affected by this issue much longer than mainline Python).

While dlmopen is an alternative with similar behaviour to RTLD_LOCAL on recent Linux versions for this case, it is not portable.

----------
components: Library (Lib)
files: pytest.tar.gz
messages: 341094
nosy: reimar
priority: normal
severity: normal
status: open
title: Python modules not linking to libpython causes issues for RTLD_LOCAL system-wide
type: behavior
Added file: https://bugs.python.org/file48291/pytest.tar.gz

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36753>
_______________________________________