[Import-SIG] PEP 402: specification questions

[Nick Coghlan]

On Thu, Nov 10, 2011 at 3:32 PM, "Martin v. Löwis" <martin at v.loewis.de> wrote:
>> Indeed, I don't see PEP 382 reducing the number of "Why doesn't my
>> 'foo' package work?" questions from beginners either
>
> Do beginners really have that question (i.e. can you kindly point
> me to archived examples of that question)? I'd expect beginners to
> do whatever tutorials and examples tell them to do. If these refer
> to P.pyp folders, beginners will just take that as given, and copy
> it.

Yep, beginners do ask that question, particularly if they have
experience with other languages that don't require explicit markers
for package directories (hence the tone of PEP 402). I mostly saw this
when I was following the Stack Overflow python RSS feed - here's one
such example: http://stackoverflow.com/questions/456481/cant-get-python-to-import-from-a-different-folder

>> since it just replaces "add an __init__.py" with "change your
> directory name to
>> 'foo.pyp'".
>
> Why should users have to *change* the directory name? When they
> create a package, they would create the .pyp directory to begin
> with, so no need to change it.

Because they start by doing the wrong thing, and then go to places
like SO to ask why it doesn't work and are told how to fix it. With
PEP 402, they wouldn't have to be told how to fix it because it would
just work the way they expected.

>> PEP 402, by contrast, should *just work* in the most
>> natural way possible. Similarly, "fixing" packaging conflicts just
>> becomes a matter of making sure that *none* of the distro packages
>> involved install an __init__.py file. By contrast, PEP 382 requires
>> that *all* of the distro packages be updated to install to "foo.pyp"
>> directories instead of "foo" directories.
>
> See my message to Barry: setuptools/distribute could do that, with
> no change to the source tree.

It still rings alarm bells for me - there's a non-trivial transform
going on between what's in the source tree and what's expected on
deployment with that approach, and that's going to break a lot of
things. (e.g. symlinking source checkouts into place in order to
pretend that plugins are installed)

> However, with PEP 382, they don't *have* to do that: some portions
> of a namespace package may keep the __init__.py, others may drop it,
> and it would still form a single namespace. OTOH, with PEP 402,
> *all* portions of the namespace would have simultaneously to agree
> to use the PEP 402 mechanism, since that mechanism will be ineffective
> if there is an __init__.py.

That's a pretty good argument in favour of dropping the
"self-contained package" concept from PEP 402, but aside from that
aspect, it doesn't help decide between the two.

>> While there's no reference implementation for PEP 402 that updates the
>> standard import machinery as yet, it's worth taking a look at Greg
>> Slodkowic's importlib-based implementation that came out of GSoC this
>> year: https://bitbucket.org/jergosh/pep-402
>
> Not sure whether that's the right way to use it, but I tried
> setting builtins.__import__ = importlib.__import__. Then,
> with "foo/bar.py" on disk, "import foo.bar" works fine. Interestingly,
> "import foo" fails afterwards, even though "foo" is in sys.path.
> Consequently, "from foo import bar" also fails, just as Phillip
> predicted. I presume that will have to be fixed in the PEP and
> the implementation.

Yeah, I haven't actually had a chance to try it out yet. It sounds
like it's just an implementation bug in the "import foo" part, though,
since PJE is correct in his recollection that the "from x import y"
algorithm is along the lines of:

    import x
    if hasattr(x, 'y'):
        return x.y
    else:
        import x.y
        return x.y

(It doesn't *quite* work that way, but that's the gist of it:
http://hg.python.org/cpython/file/default/Python/import.c#l3171)

Cheers,
Nick.


-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia