[Catalog-sig] [Draft] Package signing and verification process

[M.-A. Lemburg]

On 07.02.2013 16:04, Giovanni Bajo wrote:
> Il giorno 07/feb/2013, alle ore 15:35, "M.-A. Lemburg" <mal at egenix.com> ha scritto:
> 
>> On 07.02.2013 15:13, Giovanni Bajo wrote:
>>> Il giorno 07/feb/2013, alle ore 12:55, "M.-A. Lemburg" <mal at egenix.com> ha scritto:
>>>>> Can you please describe an attack that can be mounted against PyPI/pip that is prevented by having this additional signature?
>>>>
>>>> This is not about preventing some kind of attack. It's to simplify
>>>> the setup for the user of PyPI (via the package manager).
>>>>
>>>> The user will no longer have to install several tens or even
>>>> hundreds of different uploader GPG keys locally just to be able
>>>> to verify the downloads. Instead, just the PyPI key is needed.
>>>>
>>>> I think that's important to not disrupt the PyPI user experience.
>>>>
>>>> Additionally, as already mentioned by Lennart, all the GPG interaction
>>>> could be handled by the package managers.
>>>
>>>
>>> Yes, but *all* of the above requirements can be obtained by simply having PyPI tell pip "key ABCD1234 is authoritative for package django". pip can then tell GPG to go getting the key automatically from a first-party or third-party keyserver (eg: launchpad).
>>>
>>> I'm absolutely *not* suggesting the user to go downloading tons of GPG keys manually. 
>>
>> I don't think anyone would want to have pip installing hundreds
>> of PyPI uploader GPG keys locally, even less so, if just one is
>> enough :-)
> 
> OK so we need to both make happy Jesse that doesn't even want pip to run GPG under the hood without him even realizing that gpg exists and is being used as a crypto primitive, and you that want to keep a clean keychain that might become too cluttered by too many keys :)
> 
> I'm sure Jesse doesn't care if the GPG keychain (which he doesn't even want to have) becomes too cluttered, because he doesn't even want to learn how to dump the keychain contents, or to install a GUI tool to inspect it. I think this will be the case for the large majority of users that simpy run "apt-get install gpg" once and then forget about it and go on with their normal pip work (with a fully transparent level of additional security).
>
>> I, for one, certainly wouldn't want to have my keyring cluttered up
>> with all those GPG keys, or managing the trust state of all those
>> keys to prevent GPG warnings such as:
>>
>> gpg: WARNING: This key is not certified with a trusted signature!
>> gpg:          There is no indication that the signature belongs to the owner.
> 
> You wouldn't need to manage the trust of any key. The trust is on PyPI. Once PyPI tells you that "key ABCD1234 is trusted for package django", you just check whether there is a valid signature from ABCD1234 for all downloads related to package Django. That is irrespective on your default trust level for key ABCD1234.
> 
> In other words, I don't think it's correct to use the trust level in the keychain; if ABCD1234 is Denis Bilenko's keys, I don't want to say "this is trusted, so please install *any* packaged signed by ABCD1234"; I want to say "this is trusted FOR GEVENT, so please install gevent only if signed by Denis". This cannot be expressed by the GPG keychain trust levels. My idea is that PyPI will supply this list of trusts to users by default eg: as a text configuration file, downloadable over HTTPS, that can be automatically updated by pip every once in a while:
> 
> gevent = abcd1234
> django = 45678ad,bd14578,ce1244ab
> [...]
> 
> Very advanced users might want to hand-edit it in some way (eg: trimming the list of packages, so that some packages cannot be installed on that system, so to block possible attack vectors), and even block automatic updates of such list from PyPI (so not to trust PyPI for it).
> 
> (PS: I'm using short fingerprints in all my examples, but I'm aware of the security implications, and I think we should use the full key ID everywhere).

I'm not sure I follow you. The GPG output is generated when running
the verify command on a signature where you do have the key in the
keyring, but have not set the trust setting of that key in the
ring.

The trust flag in GPG is normally used to indicate that you have
checked that the key does indeed belong to the person it is
assigned to. Such checks can be done at code signing parties, over
the telephone, etc.

You'd normally not set the trust flag without having gone through
such a procedure, so the above warning won't go away.

If you only have to manage one key, the PyPI key, you can verify
the key by looking on the PyPI site and comparing fingerprints.
It will most likely also get signatures from core developers,
so that information can also be used to check the key.

As a result, trusting this one key is well possible and easily
manageable. You could even ship pip with a keyring that already
has that key setup with the trust flag and use this keyring
for verification, leaving the user's own keyring completely
untouched.

>> Having PyPI sign the file would also provide a possibility to keep files,
>> for which the uploader key was later revoked or which expired,
>> in a verifiable state.
> 
> When a key is revoked, all signatures made by that key *even it the past* should be ignored and should be treated as providing no information. A revocation is an explicit process in which the key owner declares that the key has been compromised, at which point you cannot trust *anything* that was signed by that key at *any* time. So if a key is revoked, we should simply delete all signature files made by that key from all of PyPI mirrors. 

That's a bit drastic, IMO. Yes, you can use that approach, but you don't
have to. After all, PyPI would only sign the uploaded files, in case
uploader's key verifies, which implies that it is not revoked at the
time of signing the file.

If an uploader finds the key compromised, s/he can remove the affected
uploaded files from PyPI together with the key, but still leave the
older release files around.

> On the other hand, the case of expired keys is already handled by GPG, since a signature embeds a timestamp so you can check whether the key was valid at the time of the signature (irrespective of whether the key is expired or not at the time of the check).
> 
> Anyway, I'm open to having PyPI sign packages; it's not wrong per-se. I just don't think it's required from a security perspective, and I think it will involve more work.

It's not required from a security perspective, but I think it is
needed from a usability perspective.

Ideally, the users of pip should not notice any change in behavior
or setup compared to what they have now. rpm, apt-get, zypper, etc.
all do this transparently as well, so it's certainly within range :-)

Sometimes I wish we'd just use one of those existing tools
to do all this package management stuff. Would safe us a lot
of grey hair...

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Feb 07 2013)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/