[New-bugs-announce] [issue17639] symlinking .py files creates unexpected sys.path[0]

[Kristján Valur Jónsson]

New submission from Kristján Valur Jónsson:

When .py files are assembled into a directory structure using direct symbolic links to the files, something odd happens to sys.path[0].

Consider this file structure:
/pystuff/
  foo.py -> /scripts/foo.py
  bar.py -> /libs/bar.py

foo.py contains the line: "import bar"
"python /pystuff/foo.py" will now fail, because when foo.py is run, sys.path[0] will contain "/scripts", rather than the expected "/pystuff".

It would appear that the algorithm for finding sys.path[0] is:
sys.path[0] = os.dirname(os.realpath(filename)).
IMO, it should be:
sys.path[0] = os.realpath(os.dirname(filename)).

I say that this behaviour is unexpected, because symlinking to individual files normally has the semantics of "pulling that file in" rather than "hopping to that file's real dir".

As an example, the following works C, and other languages too, I should imagine:
/code/
  myfile.c -> /sources/myfile.c
  mylib.h  -> /libs/mylib.h
  libmylib.so -> /libs/libmylib.so

an "#include "mylib.h" in myfile.c would look for the file in /code and find it.
a "cc myfile.c -lmylib" would find the libmylib.so in /code

This problem was observed on linux, when running hadoop script jobs.  The hadoop code (cloudera CDH4) creates a symlink copy of your file structure, where each file is individually symlinked to an place in a file cache, where each file may sit in a different physical dir, like this:

tmp1/
 a.py -> /secret/filecache/0001/a.py
 b.py -> /secret/filecache/0002/b.py
 c.py -> /secret/filecache/0003/c.py

Suddenly, importing b and c from a.py won't work.
if a, b, and c were .h files, then "#include "b.h"" from a.h would work.

----------
components: Interpreter Core, Library (Lib)
messages: 186069
nosy: kristjan.jonsson, ncoghlan, neologix
priority: normal
severity: normal
status: open
title: symlinking .py files creates unexpected sys.path[0]
type: behavior
versions: Python 2.7, Python 3.3, Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue17639>
_______________________________________