[Distutils] Request for comment: Proposal to change behaviour of pip install

[Nathaniel Smith]

On Sat, Jun 25, 2016 at 10:29 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
[...]
> By contrast, for folks that *aren't* using something like VersionEye
> or requires.io to stay on top of security updates, "always run the
> latest version of everything, and try to keep up with that upgrade
> treadmill" really is the safest way to go, and that's what the current
> eager upgrade behaviour provides.

It's really not, though :-(.

I am *incredibly* sympathetic to the idea that we should be doing
whatever we can to nudge users into keeping up to date. If there was a
button I could push that would enable Android-style updates (= "hey
the elves upgraded everything while you were sleeping, hope you like
it") by default, then I would push that button (as long as there was
an option to opt-out). In numpy-land we have really damaging feedback
loop where users don't upgrade numpy, so downstream packages insist on
supporting old numpy's b/c users have them, so downstream packages
insist on working around numpy limitations instead of fixing them
because fixes will only be in new versions, and then the teetering
pile of workarounds further rusts-over numpy's brokenness, which makes
it more likely that changes break things, so users don't want to
upgrades, ... it's bad.

But, given pip and its context, the right way to do this is:

- make explicit upgrades like 'pip install -U foo' non-recursive
- provide a 'pip upgrade-all' command (under whatever spelling)
- provide messaging and hints to encourage people to use it ("pip
install foo" -> "okay done, and fyi you have 12 out-of-date packages,
you should run pip upgrade-all")

The advantage of this is that it puts the user in control. When I want
to install or upgrade a specific package, then I can do that. When I
want to upgrade everything, I can do that. Everything is predictable,
and does what it says on the tin. Each command addresses one specific
problem that users understand. Pip is my friend who works with me to
help me accomplish my goals.

The current 'pip install -U' is none of these things. I say "I want to
upgrade foo", and then pip looks at that like "ah-HAH I really want to
upgrade all the things, it's for your own good, and you just gave me
permission to do that, or at least you gave me permission to do
something *like* that, close enough that I can pretend, so I'm just
going to go ahead and do the most that I think I can get away with,
don't worry, you'll totally appreciate this someday, and anyway, I'm
just doing what you told me to do (kinda)".

This is, like... just rude and disrespectful. It takes away my agency
as a user, with a bit of gaslighting on top. Obviously the context is
totally different, I'm not going to take this next analogy any
further, but notice that this is literally the same basic
interactional pattern as men who are like "oh that woman nodded at me
while passing in the subway aisle, I'm going to assume that that means
she wants to have a long conversation with me for the rest of the ride
and nothing will convince me otherwise, I'm a really awesome guy,
she'll see that eventually, and anyway, she totally asked me for it".

Yes, sure, pip upgrading packages is for my own good, but users hate
being condescended to by computers. And as a user I can't predict
what's going to happen ("I asked for a new version of Pyramid and it's
upgrading setuptools?"), it's not what I asked for, and compared to a
real 'upgrade-all' command the end result is *still* a haphazard mix
of up-to-date and non-up-to-date packages, so even in the best case
it's a lousy piece of social engineering that doesn't accomplish the
stated goal. And then it pisses users off so much that they implement
elaborate workarounds to take control back from pip:

  http://article.gmane.org/gmane.comp.python.distutils.devel/24218

and the basic relationship between users and pip becomes adversarial
rather than cooperative.

tl;dr: +100 on finding ways to keep users up to date on package
versions, but having recursive upgrades by default is an ineffective
mechanism that causes lots of collateral damage, we should find a
different mechanism that works better and doesn't make users hate us.

-n

-- 
Nathaniel J. Smith -- https://vorpus.org