[Catalog-sig] [DRAFT] Proposal for fixing PyPI/pip security

[Justin Cappos]

> Thanks. I've replied to your comments.
>

Sure, thanks a lot for clearing up some of my confusion.

>
> After reading the doc, I'm not clear on how mirrors / CDNs / separate file
> servers will be used in the system and what sort of trust you are placing
> in them.  In particular, much of the text about PyPI may or may not apply
> to mirrors. These are a major headache from a security standpoint and
> something we've really tried to minimize in TUF.
>
>
> I think the current PyPI mirror can be highly simplified once we introduce
> end-to-end authenticity with GPG. My suggestion would be to make them
> simple file servers, or even drop them and switch to commercial CDN, that
> would simplify lots of management. What we should drop is the concept of a
> full mirror, as it creates lots of security headaches as you say. I think
> the PSF board is open to a proposal to set up a budget for this.
>

I definitely agree that simple file servers are by far the easiest thing to
secure.   I think you can have mirrors, even ones run by untrusted parties,
and still have strong security.   It's really the metadata that you need to
be most worried about the correctness of (except for a few oddball
attacks).

I'll talk more about this when providing more details.


>
> I've also thought more about how TUF would address the issues you've
> mentioned. I believe TUF addressed the concerns mentioned in the doc
> (except of course things like password storage which are PyPI website
> changes). We also all of the proposed future enhancements mentioned at the
> end of the document.
>
>
> I think TUF is a large superset of what I proposed, that means that it is
> also a large superset of what it is (likely) needed. I'm still worried of
> how we can simplify TUF from an UX and IT perspective. I think that I need
> some inputs from you. Can you please write down something that describes:
>

Sure, we will get a document together and follow up with you.   However, in
the meantime, I'll give you some basic answers.


>
> 1) What is exactly expected from a package maintainer to do to:
>  1a) register themselves as package maintainers on PyPI
>

So the normal process would still apply.   We may encourage them to
generate and upload a key, but otherwise it is the same.   The PyPI
maintainer needs to delegate trust to their target package.   This could be
an online action (for ease of operation) or an offline one (for security).


>  1b) sign/publish a new package
>

Run a single command to do the signature in the normal case.   If they
choose to use a threshold of keys, it may require multiple devels to do so.
  They need to upload the signature and the new package.

 1c) hide/show a package version
>

I need to look into this more.   There are several ways this can be set up
and I need to understand more to know how to respond.  Offhand, I would say
that having the developer sign and upload metadata indicating hidden vs.
visible is the most secure.  From a usability perspective, PyPI could sign
something stating this instead, but this requires trusting PyPI more than
may be wise.  Were it my system, I'd prefer the former (and can talk more
about risks with the latter), but either choice seems reasonable.



> 2) What modifications are required on the PyPI server?
>

As for PyPI mods, we'll look into this and get back to you.  TUF was
designed to slot into a repo that is on a static webserver (or similar).
I don't know if PyPI will cause any problems.


> How many GPG keys the server would need to handle? Would they be online or
> offline? What processes do we need to setup?
>

Parts of this are also up for debate.   TUF isn't meant to be a rigid set
of rules w/ keys one must follow.   It's meant to be flexible enough that
it can be used for a variety of environments.

For example, you likely want to have a threshold of keys for the root
signing keys.   These would be rarely used (only for key revocation) and
should be kept offline.   So you may collectively decide to have one for
every board member of the PSF (for example).   Alternatively, you may
decide to give them to the 5 people who are most involved with PyPI.  I
don't know enough to know what makes sense politically or for your workload.

What I will do is come up with something based upon my understanding of
what might work.   Feel free to push back if something seems to onerous and
let us know if you don't understand why we said you should have a role for
X, etc.

I also might need some PyPI stats to make sane recommendations.   I'll let
you know...



> I would expect such document to describe also required changes to
> distutils and PyPI protocols, if required.
>
>
Sure.   This is an important concern and one we've been looking into.

>  I must confess I'm still digging out after my deadline, so my responses
> may be delayed.
>
>
> There is no specific hurry, though I would like these issues to be sorted
> out. I'm happy to integrate TUF if it's the best solution, but we need to
> discuss how.
>

Okay, this sounds good.   We'll get you a thorough answer fairly soon.

Thanks,
Justin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20130222/5c68a2c4/attachment-0001.html>