[Distutils] PEP470, backward compat is a ...

[Carl Meyer]

On 05/16/2014 04:00 PM, Paul Moore wrote:
> On 16 May 2014 20:27, Carl Meyer <carl at oddbird.net> wrote:
>>>> Or, thirdly, Paul's proposal could solve this, if PyPI automatically
>>>> generated an "external legacy index" for any packages that haven't
>>>> generated their own external index URL by a certain date. Really in a
>>>> way this is similar to Holger's proposal, except it uses
>>>> external-indexes instead of verified-external-URLs, and is again a bit
>>>> more explicit about what's going on (at the cost of requiring more
>>>> adjustment from users).
>>>
>>> It’s an interesting idea. I’d have to think about it. There is of course nothing
>>> stopping anyone from doing this and shoving it on pythonhosted.org.
>>
>> The part that not anyone could do would be auto-populating the
>> discoverable external-index-url metadata with this auto-generated index
>> url, for inactive projects. That would require PyPI admin intervention.
>> That part is key, because it's the only way the user of such a package
>> ever finds out about this new external index for it.
> 
> I'm not sure I understand this. What I was proposing is entirely
> doable by anyone. Simply scrape every
> https://pypi.python.org/simple/XXX page looking for external links.
> (You'd need to do the same link chasing and scraping as pip does, to
> discover the actual downloadable file URLs). Bung them all on a simple
> index page. Do that once and publish the result. That's it. It's a
> one-off exercise, I explicitly *don't* propose refreshing the page
> after it's created.

Right, I agree that part can be done by anyone. And nope, I wasn't
proposing ever refreshing it either.

> Oh, wait - you mean putting a link to that static index page on the
> project simple index page for any project we index here? Yes, you
> can't do that, but I never intended that we should. My assumption was
> that if people wanted a legacy package, they would currently be using
> some combination of --allow-external and --allow-unverifiable. We just
> tell them "If you're using those flags, and the project you depend on
> isn't showing a proper external index, you can use the legacy index to
> make things work again - but it's not any more secure or trustworthy
> than the --allow-XXX flags. You should do your own security and
> supportability review if you care."

The question is _who_ tells them about this external index (or multiple
external indices, one per project), how, and when. It's not like we can
just post about it on distutils-sig and assume that every user of a
legacy project will find out about it :-)

I was proposing that that mechanism would be to auto-populate the new
PEP 470 external-index-url metadata for any unresponsive project after
some period of time with this auto-generated "external index" - that way
pip would tell them about the index URLs they need automatically, under
the existing wording of PEP 470. That approach would need to be done by
a PyPI admin.

I don't really see any viable approach that wouldn't either need
official buy-in from PyPI or pip in some form.

Carl

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20140516/bfbb77cf/attachment.sig>