import bug

[Gabriel Genellina]

En Sat, 31 Oct 2009 12:12:21 -0300, kj <no.email at please.post> escribió:

> I'm running into an ugly bug, which, IMHO, is really a bug in the
> design of Python's module import scheme.

The basic problem is that the "import scheme" was not designed in advance.  
It was a very simple thing at first. Then came packages. And then the   
__import__ builtin. And later some import hooks. And later support for zip  
files. And more import hooks and meta hooks. And namespace packages. And  
relative imports, absolute imports, and mixed imports. And now it's a mess.

> Consider the following
> directory structure:
> [containing a re.py file in the same directory as the main script]
>
> If I now run the innocent-looking ham/spam.py, I get the following
> error:
>
> % python26 ham/spam.py
> Traceback (most recent call last):
>   [...]
>   File "/usr/local/python-2.6.1/lib/python2.6/string.py", line 116, in  
> __init__
>     'delim' : _re.escape(cls.delimiter),
> AttributeError: 'module' object has no attribute 'escape'

> My sin appears to be having the (empty) file ham/re.py.  So Python
> is confusing it with the re module of the standard library, and
> using it when the inspect module tries to import re.

Exactly; that's the root of your problem, and has been a problem ever  
since import existed.

En Sat, 31 Oct 2009 13:27:20 -0300, kj <no.email at please.post> escribió:

>> 2) this has been fixed in Py3
>
> In my post I illustrated that the failure occurs both with Python
> 2.6 *and* Python 3.0.  Did you have a particular version of Python
> 3 in mind?

If the `re` module had been previously loaded (the true one, from the  
standard library) then this bug is not apparent. This may happen if re is  
imported from site.py, sitecustomize.py, any .pth file, the PYTHONSTARTUP  
script, perhaps other sources...

The same error happens if ham\spam.py contains the single line: import  
smtpd, and instead of re.py there is an empty asyncore.py file; that fails  
on 3.1 too.

En Sat, 31 Oct 2009 22:27:09 -0300, Steven D'Aprano  
<steve at remove-this-cybersource.com.au> escribió:
> On Sat, 31 Oct 2009 16:27:20 +0000, kj wrote:
>
>>> 1) it's a bad idea to name your own modules after modules in the stdlib
>>
>> Obviously, since it leads to the headaches this thread illustrates. But
>> there is nothing intrisically wrong with it.  The fact that it is
>> problematic in Python is a design bug, plain and simple.  There's no
>> rational basis for it,
>
> Incorrect. Simplicity of implementation and API is a virtue, in and of
> itself. The existing module machinery is quite simple to understand, use
> and maintain.

Uhm... module objects might be quite simple to understand, but module  
handling is everything but simple! (simplicity of implem...? quite simple  
to WHAT? ROTFLOL!!! :) )

> Dealing with name clashes doesn't come for free. If you
> think it does, I encourage you to write a patch implementing the
> behaviour you would prefer.

I'd say it is really a bug, and has existed for a long time.
One way to avoid name clashes would be to put the entire standard library  
under a package; a program that wants the standard re module would write  
"import std.re" instead of "import re", or something similar.
Every time the std package is suggested, the main argument against it is  
backwards compatibility.

> In addition, there are use-cases where the current behaviour is the
> correct behaviour. Here's one way to backport (say) functools to older
> versions of Python (untested):

You still would be able to backport or patch modules, even if the standard  
ones live in the "std" package.

En Sat, 31 Oct 2009 12:12:21 -0300, kj <no.email at please.post> escribió:

> I've tried a lot of things to appease Python on this one, including
> a liberal sprinkling of "from __future__ import absolute_import"
> all over the place (except, of course, in inspect.py, which I don't
> control), but to no avail.

I think the only way is to make sure *your* modules always come *after*  
the standard ones in sys.path; try using this code right at the top of  
your main script:

import sys, os.path
if sys.argv[0]:
     script_path = os.path.dirname(os.path.abspath(sys.argv[0]))
else:
     script_path = ''
if script_path in sys.path:
     sys.path.remove(script_path)
     sys.path.append(script_path)

(I'd want to put such code in sitecustomize.py, but sys.argv doesnt't  
exist yet at the time sitecustomize.py is executed)

-- 
Gabriel Genellina