[Distutils] Reviving PEP 470 - Removing External Hosting support on PyPI

[Paul Moore]

On 27 August 2015 at 02:24, Donald Stufft <donald at stufft.io> wrote:
> I do need a BDFL Delegate for this PEP, Richard does not have the time to do it and the other logical candidate for a PyPI centric PEP is myself, but I don’t feel it’s appropriate to BDFL Delegate my own PEP.

I've agreed to be BDFL-Delegate for this PEP.

Overall, I believe the PEP is looking good. I've reviewed the latest
version of the PEP, and as much as I can of the discussion - but the
history of the whole issue is pretty messy and I may well have missed
something. If so, please speak up!

I do have some specific points I'd like to see addressed by the PEP:

1. While the migration process talks about how we will assist existing
users of the off-PyPI hosting option to migrate, the PEP doesn't
include any provision for *new* projects who prefer not to host on
PyPI. In the spirit of ensuring that off-PyPI hosting is seen as a
fully supported option, I'd like to see the PEP include a statement
that the process for setting up an external index will be added to the
packaging user guide (that documentation is already being included in
the emails being sent out, let's just also make sure it's published
somewhere permanent as well).

2. The PEP says very little on how users should find out when an
external index is required, and how to configure it. I'd suggest a
couple of explicit points get included here - in "Multiple
Repository/Index Support" a note should be added saying that projects
that need an external index SHOULD document the index location
prominently in their PyPI index page (i.e. near the top of their
long_description), and installers SHOULD provide an example of
configuring an external index in their documentation.

3. In the section "Allow easier discovery of externally hosted
indexes", maybe replace the paragraph "This idea is rejected because
it provides a similar painful end user experience where people will
first attempt to install something, get an error, then have to re-run
the installation with the correct options." with "This feature has
been removed from the scope of the PEP because it proved too difficult
to develop a solution that avoided UX issues similar to those that
caused so many problems with the PEP 438 solution. If needed, a future
PEP could revisit this idea."

4. The "Opposition" section still feels unsatisfying to me. It seems
to me that the point of this section is to try to address specific
points that came up in the discussion, so why not make that explicit
and turn it into a "Frequently Asked Questions" section? Something
along the lines of the following:
    * I can't host my project on PyPI because of <X>, what should I do?
      (Data sovereignty would be one question in this category). The
answer would be to host externally and instruct users to add your
index to their config.
    * But this provides a worse user experience for my users than the
current situation - how do I explain that to my users?
      There are two aspects here. On the one hand, you have to explain
to your users why you don't host on PyPI. That has to be a
project-specific issue, and the PEP can't offer any help with that. On
the other hand, the existing transparent use of external links has
been removed for security, reliability and user friendliness reasons
that have been covered elsewhere in the PEP.
    * Switching my current hosting to an index-style structure breaks
my workflow/doesn't fit my hosting provider's rules/...
      I believe the answer here was to host an index on
pythonhosted.org pointing to the existing files. But it's a fair
question, and one the PEP should probably cover in a bit more detail.
    * Why don't you provide <X>?
      Generally, the answer here is that the PEP authors don't have
sufficient experience with the subject matter behind X. This PEP is
intended to be a straightforward, easily understood baseline, similar
to existing models such as Linux distribution repositories. Additional
PEPs covering extra functionality to address further specialised
requirements are welcomed, but would require someone with a good
understanding of the underlying issue to develop.

If anyone has any further points to raise, now is the time to do so! I
don't see any need for another extended debate, hopefully most of the
issues have already been discussed, and I'm going to assume unless
told otherwise that people are happy they are covered properly in the
PEP. In particular, if anyone wants to vote an explicit -1 on the
current proposal, then please do so now.

Paul