[Python-Dev] Broken import?

[Terry Reedy]

Guido van Rossum wrote:
> On Mon, Mar 30, 2009 at 6:17 PM, Guido van Rossum <guido at python.org> wrote:
>> [Adding python-dev. I'm quoting the entire original message.]
>>
>>> On Thu, Mar 19, 2009 at 6:40 PM, Fredrik Lundh <fredriklundh at google.com> wrote:
>>>> PS. Is it just me, or is import broken in 3.0?  Consider this:
[snip]

>>>> Sure, it's a recursive import, but there's no recursive dependency
>>>> here - nobody will access the module contents until the main program
>>>> calls the library.  What am I missing?

Problems with recursive imports are a perennial topic on Python list.  A 
common suggestion is to refactor to avoid them.

>> On Mon, Mar 30, 2009 at 5:44 PM, Guido van Rossum <guido at python.org> wrote:
>>> I reproduced this, but it seems to have more to do with "from . import
>>> ..." than with the Python version. If I add the "from ." before each
>>> of the imports, "python -c 'import p.a' " fails with roughly the above
>>> traceback for any version of Python that supports this syntax, while
>>> without that it passes for any 2.x.
>>>
>>> If I use the "from ." syntax in a.py but not in b.py, "import p.a"
>>> passes but "import p.b" fails.
>>>
>>> I'll see if anyone present at the sprints has a clue.
>> Made some progress. Anything using "from <whatever> import b" (where
>> <whatever> is either '.' or 'p') will fail when b's import is not
>> completed. OTOH using "import p.b" works. I reduced it to:
>>
>> p/a.py == "from p import b"
>> p/b.py == "import a"
>> python -c "import p.b"
>>
>> The reason seems to be that until the outermost import (in this case
>> p.b) is completed, while sys.modules has the (incomplete) modules 'p',
>> 'p.a' and 'p.b', the attributes p.a and p.b aren't added until after
>> their import is completed. Which it isn't during recursive import.
>> Apparently 'from <anything> import <something>' looks for the
>> <something> attribute in the <parent> object. This is because
>> "from...import" can also be used to import objects other than modules
>> (e.g. "from M import C"). I'm guessing that setting the attribute is
>> delayed until the import is totally complete, because upon a failed
>> import we remove the half-imported module object from sys.modules, but
>> apparently we didn 't want to be in the business of removing the
>> attribute from the parent package, so that's only set after the import
>> is deemed successful.
>>
>> At least, this is my hypothesis, thinking about it -- I might look at
>> the code later. :-)
>>
>> The most portable solution is to avoid "from...import"

When doing recursive imports (it seems to work fine otherwise).

> and instead write something like
>>
>> import p.b as b
> 
> So it turns out that "from X import Y" compiles into this bytecode:
> 
>               0 LOAD_CONST               0 (-1)
>               3 LOAD_CONST               1 (('Y',))
>               6 IMPORT_NAME              0 (X)
>               9 IMPORT_FROM              1 (Y)
>              12 STORE_NAME               1 (Y)
>              15 POP_TOP
> 
> The first three opcodes (through IMPORT_NAME) call __import__('X',
> None, None, ('Y',)) and push the result on top of the stack; this
> result is the toplevel package X. The IMPORT_FROM opcode is
> essentially a getattr call that turns an AttributeError into an
> ImportError exception. I changed p/a.py into
> 
> p = __import__('p', None, None, ['b'])
> print(p.b)
> 
> and confirmed that it fails on the print() line in p.b.

If I understand, you are saying that

from x import y

is equivalent in effect to

import x
y = x.y
del x

except that the binding of 'x' never happens.

This is pretty much what the (3.0.1) doc says: "The from form does not 
bind the module name: it goes through the list of identifiers, looks 
each one of them up in the module found in step (1), and binds the name 
in the local namespace to the object thus found. " where step 1 is the 
(completed) initialization of the module.

So it seems to me that the behavior Fredrik noticed is implied by the 
doc.  It could be main plainer though.  I have not read Brett's proposed 
import doc yet.

> Does anyone feel that this ought to be fixed?

What would be the new doc?

Terry Jan Reedy