[Distutils] PEP 517: Build system API

[Thomas Kluyver]

On Thu, Nov 24, 2016, at 11:10 PM, Paul Moore wrote:
> Honestly, I don't see what's so bad about pth files. They are a
> standard supported Python approach. Maybe setuptools' use of them is
> messy? I recall it was possible to end up with a lot of clutter, but
> that was going back to the egg days, I'm not sure if it's like that
> any more. Why not just have a single pth file, maintained by the build
> tool, for all editable installs? Most users would then only have one
> or two pth files.

They would definitely be a lot more tolerable if it wasn't for
setuptools' use of the execution loophole, and indeed that misfeature as
a whole.

But even without that, they're not ideal. I routinely have development
installs of quite a lot of different packages. If each one of those is
an entry on sys.path from a .pth file, there's a performance penalty, as
every 'import qux' has to stat() in dozens of directories which don't
contain qux at all:

stat('.../foo/qux.py')
stat('.../foo/qux/__init__.py')
stat('.../foo/qux.so')
stat('.../bar/qux.py')
stat('.../bar/qux/__init__.py')
...
stat('.../qux/qux.py')   # At last!

If you put tooling scripts/modules in the root of your project
directory, it also increases the chance of name clashes.

Maybe we should look into something more like symlinks for doing
editable installs, so a file called '.../site-packages/qux.pylink' would
contain the path to the qux module/package. This would only be read on
'import qux', so it wouldn't affect other imports at all.

Brett has merged my PR for option 1c, so PEP 517 no longer specifies a
hook for editable installs. If we need extra hooks to support editable
installs, we can specify them separately in a later PEP.

Thomas