[Python-Dev] Dropping __init__.py requirement for subpackages

[Phillip J. Eby]

At 11:50 AM 4/26/2006 -0700, Guido van Rossum wrote:
>I'm not sure what you mean by "one directory read". You'd have to list
>the entire directory, which may require reading more than one block if
>the directory is large.

You have to do this to find an __init__.py too, don't you?  Technically, 
there's going to be a search for a .pyc or .pyo first, anyway.  I'm just 
saying you can stop as soon as you hit an extension that's in 
imp.get_suffixes().


> > Are you sure you wouldn't rather just write a GoogleImporter class to fix
> > this problem?
>
>No, because that would require more setup code with a requirement to
>properly enable it, etc., etc., more failure modes, etc., etc.

I don't understand.  I thought you said you had only *one* top-level 
package.  Fix *that* package, by putting the code in its __init__.py.  Job 
done.


> >  Append it to sys.path_hooks, clear sys.path_importer_cache,
> > and you're all set.  For that matter, if you have only one top-level
> > package, put the class and the installation code in that top-level
> > __init__, and you're set to go.
>
>I wish it were that easy.

Well, if there's more than one top-level package, put the code in a module 
called google_imports and "import google_import" in each top-level 
package's __init__.py.

I'm not sure I understand why a solution that works with released versions 
of Python that allows you to do exactly what you want, is inferior to a 
hypothetical solution for an unreleased version of Python that forces 
everybody else to update their tools.

Unless of course the problem you're trying to solve is a political one 
rather than a technical one, that is.  Or perhaps it wasn't clear from my 
explanation that my proposal will work the way you need it to, or I 
misunderstand what you're trying to do.

Anyway, I'm not opposed to the idea of supporting this in future Pythons, 
but I definitely think it falls under the "but sometimes never is better 
than RIGHT now" rule where 2.5 is concerned.  :)  In particular, I'm 
worried that you're shrugging off the extent of the collateral damage here, 
and I'd be happiest if we waited until 3.0 before changing this particular 
rule -- and if we changed it in favor of namespace packages, which will 
more closely match naive user expectations.

However, the "fix the tools" argument is weak, IMO.  Zipfile imports have 
been a fairly half-assed feature for 2.3 and 2.4 because nobody took the 
time to make the *rest* of the stdlib work with zip imports.  It's not a 
good idea to change machinery like this without considering at least what's 
going to have to be fixed in the stdlib.  At a minimum, pydoc and distutils 
have embedded assumptions regarding __init__ modules, and I wouldn't be 
surprised if ihooks, imputil, and others do as well.  If we can't keep the 
stdlib up to date with changes in the language, how can we expect anybody 
else to keep their code up to date?

Finally, as others have pointed out, requiring __init__ at the top level 
just means that this isn't going to help anybody but Google.  ISTM that in 
most cases outside Google, Python newbies are more likely to be creating 
top-level packages *first*, so the implicit __init__ doesn't help them.

So, to summarize:

1. It only really helps Google
2. It inconveniences others who have to update their tools in order to 
support people who end up using it (even if by accident)
3. It's not a small change, unless you leave the rest of the stdlib 
unreviewed for impacts
4. It could be fixed at Google by adding a very small amount of code to the 
top of your __init__.py files (although apparently this is prevented for 
mysterious reasons that can't be shared)

What's not to like?  ;)