[Distutils] Q about best practices now (or near future)

[PJ Eby]

On Thu, Jul 18, 2013 at 7:09 PM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> PJ Eby <pje <at> telecommunity.com> writes:
>> While other strategies are definitely possible, distlib's approach is
>> not backward-compatible, as it means installing new versions of a
>
> Correct, because distlib does not support multiple installed versions of the
> same distribution, nor does it do the sys.path manipulations on the fly which
> have caused many people to have a problem with setuptools.
>
> Do people see this as a problem? I would have thought that venvs would allow
> people to deal with multiple versions in a less magical way.

So does buildout, which doesn't need venvs; it just (if you configure
it that way) puts all your eggs in a giant cache directory and writes
scripts with hardcoded sys.path to include the right ones.  This is
actually more explicit than venvs, since it doesn't depend on
environment variables or on installation state.

IOW, there are other choices available besides "implicit
environment-based path" and "dynamically generated path".  Even
setuptools doesn't require that you have a dynamic path.

> If that is a real requirement which should be supported, shouldn't there be a PEP for it, if it's coming into Python? It's not supported by distutils, and it has been a point of contention.

Distutils lets you install things wherever you want; in the naive case
you could use install --root to install every package to a
version-specific directory and then use something like Gnu Stow to
create symlink farms.  Python supports explicit sys.path construction
and modification, and of course people certainly  "vendor" (i.e.
bundle) their dependencies directly in order to have a specific
version of them.  So, I don't think it's accurate to consider
multi-version installation a totally new feature.  (And AFAIK, the
point of contention isn't that setuptools *supports* multi-version
installation, it's that it's the *default* implementation.)

In any event, wheels are designed to be able to be used in the same
way as eggs for multi-version installs.  The question of *how* has
been brought up by Nick before, and I've thrown out some
counter-proposals.  It's still an open issue as to how much *active*
support will be provided, but my impression of the discussion is that
even if the stdlib isn't exactly *encouraging* multi-version installs,
we don't want to *break* them.

Hence my suggestion that if you want to drop pkg_resources use from
generated scripts, you should use buildout's approach (explicit
sys.path baked into the script) rather than distlib's current
laissez-faire approach.

Or you can just check versions, I'm not that picky.  All I want is
that if you install a new version of a package and still have an old
copy of the script, the old script should still run the old version,
or at least give you an error telling you the script wasn't updated,
rather than silently running a different version.  Buildout's approach
accomplishes this by hardcoding egg paths, so as long as you don't
delete the eggs, everything is fine, and if you do delete any of them,
you can see what's wrong by looking at the script source.