[Distutils] PEP 438, pip and --allow-external (was: "pip: cdecimal an externally hosted file and may be unreliable" from python-dev)

[Paul Moore]

On 9 May 2014 16:56, Donald Stufft <donald at stufft.io> wrote:
> Right, but I think a similar win can be had just by folding —allow-external
> into —allow-unverifiable and make it —allow-off-pypi (needs a better name,
> maybe just keep it as --allow-external?). This would effectively mean that
> an end user cannot say "allow safe downloading X externally but disallow
> downloading it unsafely externally".

I still find this hard to understand. If I get what you're saying, you
would rather have a single flag that claims to be to allow externally
hosted files to be downloaded, regardless of whether they are safe or
not than have a clean security model that says you need to opt into
downloading unverifiable files simply to avoid allowing users to
download argparse (or any of the other 0.x% of files that are safe but
external) by default?

Once again, I'm struggling to see why *safe* externally hosted files
are such a bad thing.

> I'm normally someone who advocates towards better decisions on the security
> side of things, however if most people are going to need to use the
> --allow-unverifiable flag anyways then I think the benefits of having the
> two separated isn't very large. There is still a benefit to not installing
> externally hosted things by default which is why I think that just rolling
> the two options together is better.

This is what bothers me about your position. I would expect you to be
insisting that unverifiable downloads *have* to be opt-in, and that's
why I've never advocated removing or changing the meaning of the
--allow-unverifiable flag. I agree with that position, and want things
to stay as they are for unverifiable links. And yet you seem to be in
favour of diluting that straightforward, strong security message just
to make users opt into a tiny minority of files that are completely
safe to download, but which are not hosted on PyPI.

I'm genuinely concerned here that I'm missing a glaringly obvious
reason why off-PyPI safe files are such a bad thing. You (and Nick,
and the authors of PEP 438) seem to be willing to accept a lot of
negative feeling and user unhappiness to defend making pip a
PyPI-only-by-default tool. I'd much rather that PyPI stand on its own
merits (which are many and compelling) rather than need a "use us or
pip will make your life inconvenient" crutch, which is what the
current behaviour feels like.

Paul