[ python-Bugs-1589480 ] inspect.py imports local "tokenize.py" file

[SourceForge.net]

Bugs item #1589480, was opened at 2006-11-02 12:10
Message generated for change (Comment added) made by drfarina
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1589480&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.4
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Daniel Farina (drfarina)
Assigned to: Nobody/Anonymous (nobody)
Summary: inspect.py imports local "tokenize.py" file

Initial Comment:
urllib2 may do a relative import of tokenize.py, which
can cause it to function abnormally when the user has a
file named "tokenizer.py" in the directory as a script
that utilizes urllib2.

The attached tarball has a shell script called
"showme.sh" that will give standard input to afile.py,
which contains two import statements and nothing else.
Code in the neighboring tokenize.py will be executed,
resulting in printing those lines to standard output.

Expected behavior:
no code in tokenize.py should be executed.

Reproducible on Ubuntu 6.10


----------------------------------------------------------------------

>Comment By: Daniel Farina (drfarina)
Date: 2006-11-04 02:09

Message:
Logged In: YES 
user_id=425987

I have done something slightly less lazy and had the
tokenize.py file throw an exception. The result was not in
fact stemming from urllib2, but the inspect.py file. I have
duplicated this issue on a fresh install of Edgy on a VM at
work (from scratch, no home directory preservation or
anything). I'm perfectly willing to accept that I should
file this in Ubuntu's launchpad instead. I pray I am not a
complete crackpot.

Here is a new transcription of my interactions (slightly
edited for brevity):

[fdr at Tenacity ~/inspect_bug]$ more *
::::::::::::::
afile.py
::::::::::::::
import inspect

#some random text.
#more text.
#maybe even some more.
::::::::::::::
show.sh
::::::::::::::
#!/bin/sh
cat afile.py | python afile.py
::::::::::::::
tokenize.py
::::::::::::::
import sys

for line in sys.stdin:
    print line

raise Exception, 'Shouldn\'t be here'
[fdr at Tenacity ~/inspect_bug]$ ./show.sh 
import inspect



#some random text.

#more text.

#maybe even some more.

Traceback (most recent call last):
  File "afile.py", line 1, in ?
    import inspect
  File "/usr/lib/python2.4/inspect.py", line 31, in ?
    import sys, os, types, string, re, dis, imp, tokenize,
linecache
  File "/home/fdr/inspect_bug/tokenize.py", line 6, in ?
    raise Exception, 'Shouldn\'t be here'
Exception: Shouldn't be here
[fdr at Tenacity ~/inspect_bug]$ 



----------------------------------------------------------------------

Comment By: Daniel Farina (drfarina)
Date: 2006-11-03 10:26

Message:
Logged In: YES 
user_id=425987

I have a fresh Ubuntu Edgy install (although my home
directory is quite old and persistent). I have installed the
Python 2.5 package, but am not using it in this case.

You are correct, urllib2 doesn't contain the import. I still
get the behavior on my machine. 

On my machine, the following interaction takes place:
fdr at Tenacity:~/urllib2_bug$ ls
afile.py  show.sh  tokenize.py  tokenize.pyc
fdr at Tenacity:~/urllib2_bug$ ./show.sh
import sys

import urllib2

fdr at Tenacity:~/urllib2_bug$ more show.sh
#!/bin/sh
cat afile.py | python afile.py
fdr at Tenacity:~/urllib2_bug$ more afile.py
import sys
import urllib2
fdr at Tenacity:~/urllib2_bug$

As we can see from the contents of afile.py, it shouldn't be
printing anything.

----------------------------------------------------------------------

Comment By: Georg Brandl (gbrandl)
Date: 2006-11-03 08:20

Message:
Logged In: YES 
user_id=849994

I can't reproduce that here with your example code, and
there's also no mention of "tokenize" in urllib2.py.

In any case, "import tokenize" is not a relative import, and
it's the only way a standard library module can import
another standard library module. That this can interfere
with user-defined modules is known and must be worked around
by not naming them like builtin modules.

----------------------------------------------------------------------

Comment By: Daniel Farina (drfarina)
Date: 2006-11-02 12:16

Message:
Logged In: YES 
user_id=425987

Yet another typo: the script is called "show.sh"

It's the only shell script in there, so no fear.

----------------------------------------------------------------------

Comment By: Daniel Farina (drfarina)
Date: 2006-11-02 12:11

Message:
Logged In: YES 
user_id=425987

Typo in the above:

"tokenizer.py" should just be "tokenize.py"

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1589480&group_id=5470