[Distutils] on integrated docs in Warehouse and PyPI

[Donald Stufft]

> On Jun 4, 2016, at 9:33 AM, Nathaniel Smith <njs at pobox.com> wrote:
> 
> I think everyone would agree that having some nice doc hosting service available as an option would be, well, nice. Everyone likes options. But the current doc hosting is unpopular and feature poor, falls outside of the PyPI core mission, and is redundant with other more popular services, at a time when the PyPI developers are struggling to maintain core services.


To add to what Nathaniel said here, there are a few problems with the current situation:

Documentation hosting largely worked “OK” when it was just writing files out to disk, however we’ve since removed all use of the local disk (so that we can scale past 1 machine) and we’re now storing things in S3. This makes documentation hosting particularly expensive in terms of API calls because we need to do expensive list key operations to discover which files exist (versus package files where we have a database full of files).

On top of that, S3 is an eventually consistent store, while we have worked around this for package files (by changing the URL to include a hash of the file contents) this is not something that we can do for documentation. This means that while it will often be updated in a short time period, it could be hours and hours (or even days) for someone to see their uploaded documentation reflected on PyPI.

I feel that the current documentation hosting is a bit of an attractive nuisance for people. It’s easy to get started with, but it lacks even basic features— features that I don’t have time to implement. This causes people to request these features be added and for me to have to tell them no (or to just ignore them) because I don’t have the time for it. I feel that people are going to be much better served using some mechanism whose core competency is this, then they are going to be by a barely supported feature on PyPI. If they’re using Sphinx they can use RTD and get powerful features like automatic building of the documentation on push, or if they like the “just upload a tarball” approach, they can get something similar with S3 on their own, or using Github pages, or Gitlab pages. 

So all in all, while I think there are some nice benefits of having this attached to PyPI, I think that we don’t currently have the resources to actually make it a good service, and I think it’s better to drop a badly supported service than it is to leave it there to take up time and effort of both PyPI maintainers, and for people using it not knowing that it’s barely supported.

For what it’s worth, here’s the previous discussion on this topic https://mail.python.org/pipermail/distutils-sig/2015-May/026327.html <https://mail.python.org/pipermail/distutils-sig/2015-May/026327.html>

—
Donald Stufft



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160604/0013466e/attachment-0001.html>