[New-bugs-announce] [issue15230] runpy.run_path broken: Breaks scoping; pollutes sys.modules; doesn't set __package__

[Benjamin S Wolf]

New submission from Benjamin S Wolf <jokeserver at gmail.com>:

(Python 3.2.3)

1. After discarding the module run_path used to run the code in, all references to variables from local scopes (even if they are references to global variables) are bound to None, preventing any code in functions from running properly.

/tmp/a.py --------------------------------------
FOO = 'bar'

def f():
    print(FOO)

f()
------------------------------------------------
/tmp/b.py --------------------------------------
# Hack needed for:
#  python3 /tmp/b.py,
#  python3 -m /tmp/b
#  runpy.run_path('/tmp/b.py')
from os.path import dirname
__path__ = [dirname(__file__)]
del dirname

# Hack needed for:
#  python3 -m /tmp/b
if __name__ == '__main__' and not __package__:
    __package__ = '__main__'

from . import a

def g():
    print(a.FOO)

g()
------------------------------------------------

~$ python3
>>> import runpy
>>> d = runpy.run_module('/tmp/a')
bar
>>> d2 = runpy.run_path('/tmp/a.py')
bar
>>> d['f']
<function f at 0xb7451b6c>
>>> d['FOO']
'bar'
>>> d['f']()
bar
>>> d2['f']
<function f at 0xb7451bac>
>>> d2['FOO']
'bar'
>>> d2['f']()
None
>>> d3 = runpy.run_path('/tmp/b.py')
bar
bar
>>> d3['g']
<function g at 0xb746e26c>
>>> d3['a']
<module '<run_path>.a' from '/tmp/a.py'>
>>> d3['g']()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/tmp/b.py", line 15, in g
    print(a.FOO)
AttributeError: 'NoneType' object has no attribute 'FOO'

Notice that run_module gets this right, as d['f']() prints 'bar' but d2['f']() and d3['g']() do not.


2. run_path pollutes the module namespace when running a module that uses relative imports. This prevents any code that imports the same module in the same manner from running.

Continuing from #1 without having closed the interpreter:
>>> d4 = runpy.run_path('/tmp/b.py')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.2/runpy.py", line 250, in run_path
    return _run_module_code(code, init_globals, run_name, path_name)
  File "/usr/lib/python3.2/runpy.py", line 83, in _run_module_code
    mod_name, mod_fname, mod_loader, pkg_name)
  File "/usr/lib/python3.2/runpy.py", line 73, in _run_code
    exec(code, run_globals)
  File "/tmp/b.py", line 12, in <module>
    from . import a
ImportError: cannot import name a
>>> '<run_path>' in sys.modules
False
>>> '<run_path>.a' in sys.modules
True
>>> d3['a'].f()
bar
>>> del sys.modules['<run_path>.a']
>>> d4 = runpy.run_path('/tmp/b.py')
bar
bar
>>>

run_module, on the other hand, also alters sys.modules, but this does not prevent the module from being run, only from the secondary module from being re-imported:
[Create an empty file /tmp/__init__.py]
>>> sys.path = ['/'] + sys.path
>>> d5 = runpy.run_module('tmp.b')
bar
bar
>>> d6 = runpy.run_module('tmp.b')
bar
>>> d7 = runpy.run_module('tmp.b')
bar
>>> 'tmp' in sys.modules
True
>>> 'tmp.b' in sys.modules
False
>>> 'tmp.a' in sys.modules
True

[This was the only way I could get run_module to run /tmp/b.py, regardless of the presence or lack of the path and __package__ hacks at the top of the file, or any other changes I've experimented with. runpy.run_module('/tmp/b'), runpy.run_module('b') [with '/tmp' in sys.path] would generally result in:
  ValueError: Attempted relative import in non-package
and setting run_name='__main__' alongside any of the other changes would result in:
  ImportError: cannot import name a

python3 /tmp/b.py and python3 -m /tmp/b run fine.]


3. And finally, an examination of the run_path code shows that it doesn't, as the docs indicate, set __package__ to be run_name.rpartition('.')[0], but either the empty string or None: http://hg.python.org/cpython/file/3.2/Lib/runpy.py#l269

----------
components: Library (Lib)
messages: 164437
nosy: Benjamin.S.Wolf
priority: normal
severity: normal
status: open
title: runpy.run_path broken: Breaks scoping; pollutes sys.modules; doesn't set __package__
type: behavior
versions: Python 3.2

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue15230>
_______________________________________