[Distutils] Q about best practices now (or near future)

[PJ Eby]

On Sat, Jul 20, 2013 at 10:54 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Extras should just be a way to ask "are these optional dependencies present
> on this system?", without needing to worry about how they got there.

Technically, they are a way to ask "can you get this for me?", since
pkg_resources' API allows you to specify an installer callback when
you ask to load an entry point.  This means that an installer tool can
dynamically obtain any extras it needs, not just check for their
installation.

To put it another way, it's not "exported only if extra is available",
it's "exported, but make sure you have this first."  A subtle
difference, but important to the original use cases (see below).


> For now, I'll switch export specifiers back to the concise
> "modulename:qualname" entry point format and add "Do we need to
> support the exported-only-if-extra-is-available feature?" as an open
> question. My current thinking is that the point you made about script
> wrappers (putting the wrapper in separate distribution and depending
> on that from an extra) applies to other plugins as well.

Now that I'm thinking about it some more, one of the motivating use
cases for extras in entry points was startup performance in
plugin-heavy GUI applications like Chandler.  The use of extras allows
for late-loading of additions to sys.path.  IOW, it's intended more
for a situation where not only are the entry points imported late, but
you also want as few plugins as possible on sys.path to start with, in
order to have fast startup.

The other use case is similar, in that a plugin-heavy environment with
self-upgrading abilities can defer *installation* of parts of a
plug-in until it is actually used.  (Which is why EntryPoint objects
have a .require() method separate from .load() - you can loop over a
relevant set of entry points to pre-test or pre-ensure that they're
all available and dependencies are installed before importing any of
them, even though .load() will also do that for a single entry point.)

For the specific case of the meta build system itself, these use cases
may be moot.  For the overall use of exports, however, the use cases
are still valuable for plugin-heavy apps.  (Specifically, applications
that use lots of plugins written by different people, and don't want
to have to import everything at startup.)

Indeed, this is the original use case for exports in the first place:
it's a plugin system that doesn't require importing any plugins until
you actually need a particular plugin's functionality.  Extras just
expand that slightly to "don't require installing things or putting
them on sys.path until you need their functionality".

Heck, if pip itself were split into two distributions, one of which
were a command line script declared with an extra, pointing into the
second distribution, it'd have dynamic bootstrapping.   (Were it not
for the part where it would need pip available to *do* the
bootstrapping, of course.  ;-) )