[Catalog-sig] pre-PEP: transition to release-file hosting at pypi site

[PJ Eby]

On Tue, Mar 12, 2013 at 4:14 PM, Carl Meyer <carl at oddbird.net> wrote:
> You say below that "nobody has proposed a 'trust everything' flag." If
> there is no "trust everything" flag, then it seems to me that with
> either option A or option B the user needs to specify what they intend
> to trust. I.e. if you make the default value of allow-hosts the index
> url host, as you said you plan to do at some point, users would need to
> override it with the hosts they want to allow.
>
> It seems like maybe what you are wanting is automatically-discoverable
> installation from externally-hosted files? I.e. that I could say
> "easy_install Foo --allow-external", without needing to know any
> specific external url for Foo?
>
> This is what I was characterizing as a "trust everything" flag, but on
> reflection I don't think I have any problem with that.

Here's a story to illustrate what I mean:

Joe wants to install foo.  He runs "easy_install Foo".  Foo is hosted
externally to PyPI, so easy_install says:

URL foo.com/downloads/foo-1.2.tgz BLOCKED by allow-hosts option --
install failed.

(Or words to that effect; I'd have to check the source to get you the
exact phrasing).

The point is, Joe now *knows where to get foo from*, because PyPI
still had the information.  Joe can now decide whether he wants to
download it manually and inspect it first, expand his allow-hosts
option, or give Foo a pass.

The proposals that call for banning all links from the /simple index,
prevent Joe from being able to do this at all.


> This is partly true. An explicit flag grants package owners more control
> in that right now they don't have a choice about whether external links
> to tarballs in their long_description automatically get sucked into the
> simple index. This is not hypothetical; even if there were no rel-link
> scraping, I've had cases where package owners have complained to me
> about pip installing an RC tarball they had linked directly from their
> long-description, not intending it to be auto-installable.

Fair enough.  Thank you for actually providing an illustration of a
problem.  There's been far too much handwaving of problems without any
explicit description of what the problem *is*.

I would support making references to external links explicit rather
than implicit.


> I think it would be preferable if in the future package owners wouldn't
> need to be careful what release-file links they might place in their
> long_description, and release files would be only explicitly nominated.

Ok.


> I think the current "automatically suck in links to simple/" behavior is
> only useful as a backwards-compatibility hack, which is why I think an
> explicit switch to disable it (on by default for newly-registered
> projects, slowly, gently, carefully migrated to on for existing
> projects) is better than keeping this link-scraping behavior
> indefinitely for all projects and asking package owners to clean up
> their long-descriptions.

I would agree with dropping link parsing from the description field,
provided that an alternative way is provided for projects to
explicitly add external links to /simple, concurrent with the other
changes.

Thank you for taking the time to engage and re-engage on this issue,
and to "Explain It Like I'm Five" for me, with an illustration of an
actual problematic use case.  ;-)