[Distutils] PEP 470, round 4 - Using Multi Repository Support for External to PyPI Package File Hosting

[Nick Coghlan]

On 3 October 2014 22:02, Donald Stufft <donald at stufft.io> wrote:
>
> As far as simplication goes, I don't believe it simplifies the implementation
> of PyPI at all, it just shuffles things around and creates work on my part
> in order to get PyPI supporting the new stuff. It does however let installers
> become simpler and it enables installers to present accurate error information
> that actually helps determine the root cause of a failure instead of the
> current silent failure with a confusing error message model.
>
> I look forward to your suggestions, but I'm not hopeful. I've been thus far
> unable to determine a way to improve the current solution in a way that isn't
> just papering over one problem without solving the fundamental issue.

Donald's perspective here matches my own. I'll be interested to hear
alternative proposals, but they should aim to address at least the
following user experience expectations:

1. Easily allow external hosting to "just work" when appropriately
configured at the system, user or virtual environment level (pip
already supports this at the user level, and will support it at the
system and environment level in the next version).

2. Easily allow package authors to tell PyPI "my releases are hosted
<here>" and have that advertised in such a way that tools can clearly
communicate it to users, without silently introducing unexpected
dependencies on third party services.

3. Eliminate any and all references to the confusing "verifiable
external" and "unverifiable external" distinction from the user
experience (both when installing and when releasing packages).

4. The repository aspects of PyPI should become *just* the default
package hosting location (i.e. the only one that is treated as opt-out
rather than opt-in by most client tools in their default
configuration). Aside from that aspect, hosting on PyPI should not
otherwise provide an enhanced user experience over hosting your own
package repository.

5. Do all of the above while providing default behaviour that is
secure against most attackers below the nation state adversary level.

In my view, the most debatable part of Donald's latest proposal would
be the handling of projects that don't get updated to properly
register an external URL before the link spidering support is removed
from the client applications. That aspect should arguably include a
step where the decision on whether or not to disable that support is
based on *looking at the numbers again* before turning the feature off
on the server, and perhaps also monitoring for user complaints for a
period after it is first turned off, before the feature is removed
from the clients.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia