Relative import from script with same name as package

[Oktay Şafak]

14.08.2011 00:39, OKB (not okblacke) yazmış:
>      	I'm using Python 2.6.5.  I have a directory structure like this:
>
> thetest/
>      	__init__.py
>      	thetest.py
>      	theother.py
>
> __init__.py is an empty file.  theother.py contains a function foo().
> The package is accessible from sys.path, so that if I open the
> interpreter and do "import thetest" or "from thetest import thetest" or
> "import thetest.thetest", it works fine.

It should.

>
> Inside thetest.py I have code like this:
>
> ###
> from __future__ import absolute_import
>
> if __name__ == "__main__" and __package__ is None:
> 	import thetest
> 	__package__ = "thetest"
>
> from .theother import foo
> ###
>
...
>
>      	If I run foo.py directly, I receive a traceback like this:
>

Wait! What's foo.py? I guess you mean thetest.py as you say "(the same 
file that is currently being run)" below.

> Traceback (most recent call last):
>    File "C:\...\thetest\thetest.py", line 4, in<module>
>      import thetest
>    File "C:\...\thetest\thetest.py", line 11, in<module>
>      from .theother import foo
> ValueError: Attempted relative import in non-package
>
>      	It appears that Python is reading "import thetest" as importing
> thetest.py (the same file that is currently being run).  When it tries
> to run that file a second time, the relative import fails.
>

No, there is no such thing happening. Read the error message more 
carefully: the error happens when your code reaches the line "from 
.theother import foo", and it fails because you are trying to execute an 
"explicit" relative import statement (with leading dot notation) as 
introduced by PEP 328. What you see is perfectly expected behaviour as 
explained in detail in the PEP because the python interpreter can only 
make sense of that statement if that code is *imported* for use by code 
that resides *outside* the package. That error message is what you see 
when you try to *run* a package member module which uses explicit 
relative imports. Let me try to explain a bit further:

You say that your package is accessible from sys.path. So let's say you 
fire up your interpreter and type

	import thetest.thetest

Here the interpreter first imports the *package* thetest and then the 
module *thetest.py*, therefore it knows that the module is a member of 
the package. And when executing the code in the module thetest.py, it 
can make sense of the leading dot in "import .theother", it can see that 
the leading dot means the packege thetest and find the right module 
inside that package.

But when you run thetest.py directly, the interpreter has no idea what 
the leading dot means because it has not imported any package 
beforehand. So it complains about that.
  >      	But why?  That __future__ import is supposed to make absolute
> imports the default, so why is "import thetest" importing thetest.py
> instead of the package called thetest?

This is not what is happening.

> The absolute import should make
> it look in sys.path first and not try to import from the script
> directory, right?

As I explained above, that's not the error but I think this sentence 
shows your misunderstanding of this relative imports issue. Never mind, 
I found the documentation very confusing at first and took me a great 
deal of effort to finally "get it". Let me share my "distilled wisdom" 
about this:

- First of all, never perform your experiments about relative/absolute 
imports using IDLE! I guess IDLE has its own path manipulation that 
doesn't do *the right thing*.

- "Absolute import" means importing from sys.path only.

- "Relative import" means a "package member" module importing another 
"package member", using a not fully qualified name (i.e X.Y.a importing 
X.Y.b as import b, instead of using import X.Y.b). When Python saw 
(before PEP 328) "import b" or "from b import ..." *in a package 
member*, it first imported the local b if available. (This business of a 
package member importing another package member is called relative 
import, only) Since this had the same notation (syntax) with other (non 
package member) imports, intra-package imports were considered 
"ambiguous" when the module to be imported had the same name with a 
module reachable from sys.path: it was not immediately clear which one 
was meant for untrained eyes. Also, it had the potential of shadowing a 
stdlib module: in that case no clean method of getting at the stdlib 
module was available.

- PEP 328 is about giving "relative imports" a new syntax so that this 
ambiguity no longer exists and the possibility of shadowing a stdlib 
module is removed altogether. The new syntax is called "*explicit* 
relative imports". The former method can thus be called "implicit 
relative imports" or "ambiguous relative imports"

- BIG SOURCE OF CONFUSION: Relative imports (both implicit and explicit) 
is all about imports *inside* packages. When you have 2 scripts app.py 
and string.py sitting in a *non-package* folder (they are called 
top-level modules), that folder is always the first entry in sys.path 
when you run either of those modules directly, so "import string" in 
app.py will always import the string.py in that folder, not the stdlib 
one. (But not when you try with IDLE!) In this case, if you really want 
the standard module, you must manipulate sys.path before your import 
statement (or change the name of your local string.py). THE PEP (328) 
HAS NO EFFECT IN THAT CASE!

- So, "relative imports" is not a concept introduced by PEP 328, it 
means "importing a package member from another member of that package" 
and it existed well before PEP 328. But its syntax was ambiguous so it 
was clarified by PEP 328 by giving it a special syntax (the leading dot 
notation). But that change introduced many small gotchas, including the 
one you are seeing: you can't directly run a package member module if 
you have used this new syntax (leading dot notation) in it. It works 
fine only when you import the package members for use by a module 
outside of that package. This of course brings difficulties with testing 
but it is another story.

I think you will now find it much easier to get what PEP 328 is all about.

Oktay