[Python-ideas] My objections to implicit package directories

[Nick Coghlan]

On Tue, Mar 13, 2012 at 4:07 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> It still seems to me that moving to a marker *suffix* (rather than a
> marker file) as PEP 382 proposes brings all the real practical
> benefits of implicit package directories (i.e. no empty __init__.py
> files wasting space) and absolutely *none* of the pain (i.e. no
> backwards compatibility concerns, no ambiguity in the filesystem to
> module hierarchy mapping, still able to fix direct execution of
> modules inside packages rather than having to explain forevermore why
> it doesn't work), but Guido clearly feels otherwise.

I think this paragraph really gets to the heart of what I'm objecting to.

I agree wholeheartedly with the objective of eliminating __init__.py
files, there's no need to convince me of that.

However, *two* proposals were made to that end:
PEP 382 kept the explicit marker, simply changing it to a directory
suffix rather than a separate file. Simple, clean, straightforward,
minimalist, effective.
PEP 402 threw away the marker entirely, and then had to patch the
package finding algorithm with a whole series of complications to
avoid breaking backwards compatibility with Python 3.2. It also has
the side effect of eliminating the 1:1 mapping between the filesystem
and the module hierarchy. Once we lose that, there's no going back.

What I really want out of the new PEP is a clear rationale for why the
horrible package finding algorithm hacks needed to make the PEP 402
approach work in a backwards compatible way are to be preferred to the
explicitly marked PEP 382 approach which *doesn't pose a backwards
compatibility problem in the first place*.

The other thing to keep in mind is that, if, for whatever reason, we
decided further down the road that the explicit directory suffix
solution wasn't good enough, then *we could change our minds* and
allow implicit package directories after all (just as the formats for
valid C extension module names have changed over time).

There's no such freedom with implicit package directories - once
they're in, they're in and we can never introduce a requirement for an
explicit marker again without breaking working packages.

Is it so bad that I want us to take baby steps here, rather than
jumping straight to the implicit solution?

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia