[New-bugs-announce] [issue23615] Reloading tokenize breaks tokenize.open()

[Thomas Kluyver]

New submission from Thomas Kluyver:

Issue #22599 changed tokenize.open() from using builtins.open() to having a module-level reference to _builtin_open, stored by doing _builtin_open = open. However, on reloading the module, _builtin_open is pointed to tokenize.open from the last execution of the code, breaking tokenize.open():

>>> import tokenize
>>> tokenize.open
<function open at 0x7f15b660fc80>
>>> tokenize._builtin_open
<built-in function open>
>>> import imp; imp.reload(tokenize)
<module 'tokenize' from '/home/takluyver/miniconda3/envs/py34/lib/python3.4/tokenize.py'>
>>> tokenize.open
<function open at 0x7f15b660fbf8>
>>> tokenize._builtin_open
<function open at 0x7f15b660fc80>
>>> tokenize.open('foo.py')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/takluyver/miniconda3/envs/py34/lib/python3.4/tokenize.py", line 438, in open
    buffer = _builtin_open(filename, 'rb')
TypeError: open() takes 1 positional argument but 2 were given

The actual case where I'm seeing this error is nose logging a failure in a test suite, so it's not clear what is reloading tokenize, but it appears that something is. This just started failing when our Windows test system updated to Python 3.4.3.

----------
components: Library (Lib)
messages: 237588
nosy: haypo, takluyver
priority: normal
severity: normal
status: open
title: Reloading tokenize breaks tokenize.open()
versions: Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23615>
_______________________________________