Is sys.executable not supposed to follow symlinks?

[Eryk Sun]

On 7/26/20, Bolun Thompson <abolunthompson at gmail.com> wrote:
> In the sys.executable documentation (
> https://docs.python.org/3/library/sys.html#sys.executable), they don't
> specify if it follows symlinks. From my limited testing, it does not. Is
> this behavior guaranteed?

In Unix, the interpreter resolves the target of sys.executable if it's
a symlink in order to set the prefix directory, which is the base path
for the standard library (i.e. "prefix/lib/pythonX.Y"). For a
contrived example, compare using a symlink vs. a hardlink when the
landmark "lib/python3.8/os.py" exists relative to the directory of the
link. With a hardlink, it will accept the link directory as the prefix
path, without falling back on the configured default prefix path (e.g.
"/usr"), and the interpreter will fail to start. But with a symlink,
in an strace you can see the executable getting resolved at the start
via readlink(), and it will try setting the prefix to "/usr" and
eventually find the landmark "/usr/lib/python3.8/os.py".

In Windows, for Python 3.8 finding python38.dll and vcruntime140.dll
reliably depends on their being beside the executable in the
application directory. The loader does not resolve symlinks, and
depending on PATH to find python38.dll is unreliable (e.g. it could
find an x86 32-bit DLL when it needs an x64 DLL). For argument's sake,
it's possible to link python.exe, python38.dll, python3.dll, and
vcruntime140.dll all together in the target directory (as would be
done in a venv that uses symlinks). In this case, if you repeat the
above contrived example by creating the landmark r"Lib\os.py" relative
to the directory of the symlinks, then the interpreter will fail to
start.

There's no reason stopping Python in Windows from resolving the target
of a symlink'd executable when searching for the prefix directory, but
it's not a pressing concern since using symlinks is relatively
uncommon in Windows. A virtual environment may use symlinks, but
pyvenv.cfg is used to find the standard library in that case. Also,
other than the above contrived example, an installed Python has a
default "PythonPath" set in its registry key, which it uses when the
prefix directory isn't found otherwise.