[Catalog-sig] pre-PEP: transition to release-file hosting at pypi site

[PJ Eby]

On Tue, Mar 12, 2013 at 3:36 PM, Jacob Kaplan-Moss <jacob at jacobian.org> wrote:
> On Tue, Mar 12, 2013 at 2:21 PM, PJ Eby <pje at telecommunity.com> wrote:
>> The *only* thing I object to is the part where some people want to ban
>> external links from /simple, always and forever, regardless of the
>> package authors' choice in the matter.
>
> Here's the thing though, there are already a bunch of other ways users
> can install packages from external repositories. I can think of at
> least two:
>
> * I can pip/easy_install a given URL (e.g. easy_install
> https://www.djangoproject.com/download/1.5/tarball/)
> * I can use a custom index server (pip install -i http://localserver/ django)
>
> The important part is that in each of those cases I can see clearly
> where I'm getting things from.
>

>
> From where I stand the absolutely non-negotiable part is that
> `pip/easy_install/whatever package` should NEVER access an external
> host (after some suitable transition period). This needs to include
> older installer software, and it needs to make it hard for new tools
> to do the wrong thing. How this is achieved really doesn't matter to
> me -- if there's a "pip install --insecure Django" that's fine too --
> but to me it's non-negotiable that the out-of-the-box configuration
> not allow external hosts.

I'm confused by this statement.  "never access an external host" is
not consistent with "have the option to specify what hosts you trust",
while still keeping PyPI as a universal index of Python software.


> Yes, this means taking some options away from the package creator. It
> means that when I'm wearing my author-of-Django hat I can't choose to
> list Django on PyPI but provide the download elsewhere. That's not
> perfect, but given a "creator choice" vs "out of the box security"
> choice the latter has to win. [And as a package creator I still have
> options: I can run my own package server, fairly easy to do these
> days.]
>
> Again, the *how* isn't a big deal to me, but the result is really
> important: the tooling has to be secure-by-default, and that means
> (among other things) `pip install package` can never hit something
> that's not PyPI without me explicitly asking for it.

That part's fine.  As I've said repeatedly, though, it's the removing
other links from the /simple index entirely that's the problem.

Under what I've proposed, as soon as the tools are updated to
secure-default (and the situation *now* if you set your --allow-hosts
to PyPI-only), is that easy_install will announce what URLs it is
skipping because they're not on PyPI.  (pip too, IIUC.)

I can't tell you how to configure pip for this, but if you want to
configure easy_install to be secure right *now*, add:

[easy_install]
allow_hosts=pypi.python.org

to your user-level or site-wide distutils .cfg file.

Better yet, encourage other people to add it now, find out what they
can no longer install, and talk to their upstream providers about
moving to PyPI.

This is all good.

I'm just saying, we don't need to change PyPI to do anything but drop
the rel="" links, and change the tools to default allow-hosts to equal
index-url.  (pip has the same parameters, not sure what config files
it uses, though.  I don't think it inherits [easy_install] settings,
though.)