[Distutils] PEP draft on PyPI/pip package signing

[Giovanni Bajo]

Il giorno 28/lug/2014, alle ore 21:13, Paul Moore <p.f.moore at gmail.com> ha scritto:

> On 28 July 2014 16:01, Giovanni Bajo <rasky at develer.com> wrote:
>> I thus solicit a second round of review of my proposal; if you want me to upload to Google Docs for easier of commenting, I can do that as well.
>> I would love to get the PEP to its final form and then ask for a pronouncement.
> 
> I have only scanned the initial part of the proposal thus far, but I
> have some comments.

Thanks for reviewing my proposal.

> First of all, the proposal is well-written - I'm not a security expert
> and my eyes very rapidly glaze over when reading security documents.
> (I know what you mean about the TUF docs!) Your PEP was pretty easy
> for me to follow, so many thanks for that :-)

Thanks. I personally think that there is a value in simpleness of security architecture, but many would disagree. 

> * I assume that installation of unsigned packages would not need GPG
> or any form of key download, and would work as now. That's crucial,
> and without that, the proposal is a non-starter (consider pip running
> in an environment not connected to the internet, installing local
> builds that don't need signing and haven't been).

Yes. If you ran pip with option “—allow-unsigned-packages”, it would not require GPG at all.

> * I didn't look at how the signature metatata was supplied, but I
> assume it would only be served from full indexes and not from
> --find-links locations (relevant in the above scenario).

Yes, that was my idea, but I’m a little bit fuzzy on the real-world —find-links use case; in fact, this is one of the cases where I need some guidance to complete my PEP.

> * I am strongly against pip depending on an external GPG. Even though
> it may be a simple install, it may not be allowed in locked-down
> environments, and on virtual machines and testing services (like
> Travis or Appveyor) installing may be non-trivial or simply an
> annoying extra step.
> * Given that this leaves a pure-python GPG implementation, does one
> exist? Is it robust? I wouldn't want to rely on a low-quality
> implementation.

My opinion is that depending on an external GPG is the best option, but I think that ultimately this is a decision for pip maintainers and is pretty orthogonal to my PEP.

Notice that most (all?) Linux machines come with gpg preinstalled as it is a dependency of both rpm and apt, so I don’t think that a Linux-based CI system would be in trouble because of this.

> * Also, would it be fast enough? Speed of building virtualenvs has
> always been something users care about (it was behind the development
> of wheels for example) and a key-checking step could slow down builds
> noticeably.

Assuming an empty environment:

1) I/O time required to download the trust file. See my discussion with Donald for some size estimates. Notice that you can ship it (or a subset of it, for increased security) to your build environment together with the project you’re building. Being a global append-only file, it might eventually become to come pre-shipped in Python build environments.
2) I/O time to download the keys. Keys are quite small and need to be downloaded by a keyserver; assuming we can’t find a fast enough keyserver to use, we might consider implementing a keyserver in PyPI itself, with Fastly as frontend (not sure about the details of the hkp protocol though, but the ‘h’ stands for http so I hope that we can easy the CDN with it as well). 	

> * Other tools would need changing as well. There's distlib, and PyPI
> mirroring tools like devpi.

Yes.

> * There will always need to be an option to install unsigned packages,
> even if it's only for local packages served up by a private index.

Yes.

> * I didn't see a discussion of what happens if a maintainer loses his
> GPG key (not compromised, just lost - say he accidentally deleted his
> only copy). Would he have to generate a new one and re-sign
> everything? How would that affect users of his packages?

I haven’t fully-fledged the details yet. Technically, when you revoke a key, you declare the date from which the key was compromised; everything signed before that date is still considered valid, so there is no need to resign releases that predates the compromisation.

My idea would be that PyPI would have a background job which routinely checks for revoked keys; when it finds one, it automatically deassociates it from the maintainers account, and removes (hides?) any package whose signature is not valid anymore. The maintainer would then have to login to PyPI, register a new GPG fingerprint, and resign releases which were disabled.

> * Also what about a maintainer working on a different PC where he
> doesn't have his key available? I guess the answer there may be
> "tough, you can't maintain your package without your key available".
> (How secure would services like SkyDrive and Dropbox be considered in
> that regard? I normally work between 2 PCs where the only reliable
> shared resource I have is SkyDrive).

This enters the realm of GPG best practices, which would eventually end up on documentation for package users such as the pypa docs). Historically, people tend to have one GPG private key that they synchronize among all their computers. Since you just need to copy the key once, you don’t really need cloud-syncing, as there’s nothing to continually sync. 

-- 
Giovanni Bajo   ::  rasky at develer.com
Develer S.r.l.  ::  http://www.develer.com

My Blog: http://giovanni.bajo.it





-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4207 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20140729/8b0a79c3/attachment-0001.bin>