[Distutils] Status report on PyPI+pip+TUF

[holger krekel]

Hi Trishank,

thanks for the high level overview.  Do you have a current web page with
more detailed technical info with respect to PyPI/TUF?

best,
holger

On Wed, Jul 31, 2013 at 07:27 -0400, Trishank Karthik Kuppusamy wrote:
> Hello Nick and the PyPI community,
> 
> This is a brief status report on the integration of PyPI and pip with TUF.
> 
> (A quick reminder: TUF is a general "plug-n-play" update framework
> designed to introduce usable security to community software
> repositories such as PyPI. If you think of PyPI as HTTP, then TUF is
> like adding SSL, and more, to HTTP. More information may be found at
> [https://www.updateframework.com/].)
> 
> Firstly, thanks to the generous funding of the National Science
> Foundation, we are pleased to introduce the addition of a full-time
> developer, Vladimir Diaz, to our team. Vladimir has been
> instrumental to the development of TUF, and we are excited to have
> him join us full-time. (Now we do not just have one PhD student who
> works on TUF when he is not busy working on other projects!) We are
> also happy to have a few interns --- Zane Fisher, Tian Tian, John
> Ward, and Yuyu Zheng --- on board for the summer.
> 
> Since the security attacks on the Python wiki infrastructure earlier
> this year, we have been closely following Distutils-SIG to see what
> we could do to help secure PyPI. We use Python heavily in all of our
> projects, and would love to help in any way we can.
> 
> Here is what we have done:
> ==========================
> 
> 1. At PyCon 2013, we showed that pip needs very little modification
> to work with a TUF-enabled PyPI mirror.
> 
> 2. Soon after (during the spring break), we wrote automation to
> build a TUF-secured PyPI mirror (which is indistinguishable from any
> other PyPI mirror except that it has signed metadata about all of
> the files on PyPI).
> 
> 3. At the same time, thanks to efforts of Konstantin Andrianov, we
> also wrote a lot of unit and integration tests to show the attacks
> that are possible without TUF and impossible with TUF.
> 
> 4. After that, we started investigating the most efficient way to
> build TUF metadata for PyPI. We found that requiring a separate key
> for every package on PyPI may sound like a good idea, but besides
> generating too much metadata, this scheme also makes key management
> difficult.
> 
> Here is what we are doing now:
> ==============================
> 
> We are designing a usable key management scheme, coupled with
> efficient generation and download of metadata, which we think should
> make for a smooth integration of PyPI with TUF. We are actively
> working on this and think that we are almost there. As a
> conservative estimate, we do not believe that this should take
> longer than two weeks.
> 
> Here is what we are going to do next:
> =====================================
> 
> In about a month, we will present to you a demonstration of a PyPI
> mirror and a pip client which are robust against entire classes of
> security attacks. We welcome you then to try our demo, be really
> critical of it and tell us what you think about what we could do
> better. Our goal with TUF is to provide a framework that works with
> as many software community repositories as possible and that secures
> as many users as possible.
> 
> More details on our development are available at our mailing list:
> https://groups.google.com/forum/#!forum/theupdateframework
> 
> We hope this gives you a good idea of the current status of
> integrating TUF with PyPI and pip. Let us know if you have
> questions.
> 
> Thanks,
> The TUF team
> 
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
> 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 490 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20130731/6eceb868/attachment-0001.pgp>