[New-bugs-announce] [issue46653] sys.path entries normalization in site.py doesn't follow POSIX symlink behaviour

[Josselin Poiret]

New submission from Josselin Poiret <dev at jpoiret.xyz>:

Whenever sys.prefix contains a symlink followed by a '..', the corresponding part of sys.path entries will not refer to the parent directory of the directory pointed to by the symlink, but rather to the directory in which the symlink is. Thus, it will be impossible to import standard Python modules installed at sys.prefix, among other things.

Here is an example:
Suppose you have installed Python with prefix /usr. Create a symlink /tmp/symlink pointing to /usr/lib, and launch `PYTHONHOME=/tmp/symlink/.. python3`. In that REPL, `import warnings` will fail to find the correct module, as evidenced by the value of `sys.path` containing entries such as `/tmp/lib/python3.X/` instead of the expected `/usr/lib/python3.X/`.

This issue is caused by the makepath function (among others) in Lib/site.py using os.path.abspath instead of os.path.realpath, which does not follow POSIX as the documentation of os.path.normpath (used internally by abspath) suggests.

I propose replacing all four instances of abspath in Lib/site.py to realpath instead. This is a breaking change for users that relied on non-conforming symlink semantics (although that user-base might be quite small), but in my opinion Python should be expected to follow the behaviour of the platform it is running on.

This issue was raised while investigating a bug [1] in the relocatable packs feature of GNU Guix [2], which makes use of symlinks to achieve relocatability.

[1] https://issues.guix.gnu.org/53258
[2] https://guix.gnu.org/en/blog/2018/tarballs-the-ultimate-container-image-format/

----------
components: Library (Lib)
messages: 412600
nosy: jpoiret
priority: normal
severity: normal
status: open
title: sys.path entries normalization in site.py doesn't follow POSIX symlink behaviour
type: behavior
versions: Python 3.10, Python 3.11, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue46653>
_______________________________________