[Catalog-sig] Deprecate External Links

[Noah Kantrowitz]

On Feb 27, 2013, at 12:16 PM, holger krekel wrote:

> On Wed, Feb 27, 2013 at 14:49 -0500, Monty Taylor wrote:
>> On 02/27/2013 02:47 PM, Aaron Meurer wrote:
>>> On Wed, Feb 27, 2013 at 11:37 AM, holger krekel <holger at merlinux.eu> wrote:
>>>> On Wed, Feb 27, 2013 at 19:34 +0100, Lennart Regebro wrote:
>>>>> On Wed, Feb 27, 2013 at 5:34 PM, M.-A. Lemburg <mal at egenix.com> wrote:
>>>>>> I'm not saying that it's not a good idea to host packages on PyPI,
>>>>>> but forcing the community into doing this is not a good idea.
>>>>> 
>>>>> I still don't understand why not. The only reasons I've seen are
>>>>> "Because they don't want to" or "because they don't trust PyPI". And
>>>>> in the latter case I'm assuming they wouldn't use PyPI at all.
>>>>> 
>>>>> And of course, nobody is forcing anyone, just like nobody is forcing
>>>>> you to use PyPI. :-)
>>>> 
>>>> I understood there is the idea to disable external links within a couple
>>>> of months.  That does break backward compatibility in a considerable way.
>>>> 
>>>> holger
>>> 
>>> But wouldn't this only be a change in pip/easy_install, not PyPI
>>> itself? I suppose you could explicitly break the external links by
>>> having them point to nothing if you are worried about the security or
>>> if it's some performance issue (that would indeed be a bad
>>> compatibility break, in case people are using those for other
>>> purposes).  Otherwise, if it's a problem, then just use the old
>>> version of pip.
>> 
>> If we don't remove the feature from pypi itself, then it won't help the
>> folks for whom its a problem, because there will be no incentive for the
>> folks hosting their software that way to actually upload their stuff to
>> PyPI - which means that client-side disabling of external_links is
>> fairly likely to never be usable.
> 
> I can see it's tempting to just try to "force" everyone to upload
> their stuff to pypi.python.org.  I am very skeptical about this approach.
> 
> There already is a high frustration with the packaging ecology
> in Python.  When we remove external links on the server side, installs
> for many people and companies are going to break, no matter what.  And
> they would have no client-side switch anymore to make things working.
> Requiring to use older setuptools/pip versions would not help because
> the server information is gone.  That'd just increase frustration.
> 
> So at the very least using external links needs to be a client-side
> installer choice for a long while and the server needs to offer
> the according information.
> 
> I'd generally prefer to think hard about ways to improve the situation
> without breaking things.  Putting simple/ and packaging serving on a CDN
> is one such step and a good idea i think.  Establishing a
> signing/verification mechanism is another.  Refining py2/py3 dependency
> discovery yet another good one.

None of these things have anything to do with improving _this issue_, though they would make PyPI better and will be tackled at some point. This is a feature that must be removed if we are going to operate a trustable packaging network. Today, tomorrow, or six months from now, but this feature will be going away, the only question is how do we get there? Yes things will break. We also broke old users of pypissh a few weeks ago as part of the SSL lockdown, this is an acceptable loss as deprecation schedules were made and followed. We will not randomly disable these links today, as you said the right first move will be to show a warning (and then an error) in pip/buildout when using these links. Donald has already begun that conversation with each of the tool developers. We will need a global plan though, an overarching schedule to work with pip and buildout (and easy_install if someone does the legwork there) about how to announce this removal and how to ensure we break as few people as possible over the course of the plan. That is what this discussion is about.

--Noah

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 203 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20130227/5c41afb8/attachment.pgp>