[Catalog-sig] ANN: pythonpackages.com beta

[Eric P. Mangold]

On Tue, Jul 31, 2012 at 11:52:22AM -0400, Alex Clark wrote:
> Hi Eric,
> 
> 
> (Continuing this discussion from twisted list)
> 
> 
> On 7/31/12 10:24 AM, Eric P. Mangold wrote:
> >On Mon, Jul 30, 2012 at 05:09:07PM -0400, Alex Clark wrote:
> >>Hi Eric,
> >>
> >>On 7/30/12 4:49 PM, Eric P. Mangold wrote:
> >>>On Mon, Jul 30, 2012 at 12:49:56PM -0400, Alex Clark wrote:
> >>>>Hi,
> >>>>
> >>>>
> >>>>On 7/30/12 12:31 PM, Eric P. Mangold wrote:
> >>>>>Alex,
> >>>>>
> >>>>>I'm not sure if this is borderline off-topic, or not... but anyway..
> >>>>>
> >>>>>I'm sure starting a discussion here IS offtopic.
> >>>>>
> >>>>>But I have one question:
> >>>>>
> >>>>>How do package authors verify the integrity of their packages built "through the web"?
> >>>>
> >>>>
> >>>>Good question, I just created:
> >>>>
> >>>>-
> >>>>http://docs.pythonpackages.com/en/latest/faq.html#how-do-package-authors-verify-the-integrity-of-packages-built-through-the-web
> >>>
> >>>Let me be clear:
> >>>
> >>>Is it possible to have any assurance that your system has faithfully built the package, and/or that your servers have not been compromised?
> >>>
> >>>Why would anyone trust your web service to build packages, when it is *their* pgp, reputation and users that are at stake?
> >>>(Yes, I would ask Launchpad/Canonical, et. all the same question...)
> >>>
> >>>(Also, if you're suggesting MD5 (following your link..) for anything related to security or data authenticity, then I *know* you're way off base.......)
> >>
> >>
> >>The point about md5 is not to suggest using it for security or data
> >>authenticity,
> >
> >Sorry, I think I have a problem with taking the exact text of my question,
> >on your wiki, and casting it to be a different question entirely. (through
> >no fault of your own, I'm sure)
> 
> 
> Sorry, removed! Let me know if there is something better I can put
> in its place.
> 
> 
> >
> >I think I've clarified what my orignal question was meant to ask, namely how do
> >we trust YOU and YOUR build infrastructure, not "how do we verify that the data
> >you're give us hasn't been damaged in transit".
> >
> >If you wouldn't mind editing your wiki to reflect my clarifications, I would
> >very much appreciate it :)
> 
> 
> OK Let me work on it.
> 
> 
> >
> >>it's to clarify that whatever security is currently place
> >>with PyPI (not a lot, admittedly) still applies, for whatever that is
> >>worth (not much, apparently).
> >>
> >>
> >>>
> >>>Sorry if this is harsh - but it's intended. Without any kind of verifiable guarantee (get to work on that! :)) I don't think I could ever possibly use such a thing, and would advise against it.
> >>>
> >>>Getting software to end-users is a tough challenge, and I applaude your efforts to try and make it easier. A system with a single point of failure and a single point of trust just isn't feasible or desirable, imho.Administrators need to know who has final responsibility and *authority*
> >>over the software that they are consuming. If "the cloud" is the last
> >>link in that chain, then you have a big problem, I think.
> >>
> >>
> >>The last link in the chain is PyPI (or a private index). The node before
> >>that is typically your laptop. I'm suggesting you make it
> >>pythonpackages.com instead.
> >
> >Clearly PyPI is inadequate. Jumping on solutions, for HARD problems that always
> >require some form of cryptography to solve, is no more palettable.
> >
> >And PyPI is also just a publishing platform - the packages have already been
> >minted by that point.
> >
> >So as you suggest, the last point is the developer/release-manager, as it should
> >be.
> >
> >I think my point is that ideally you don't want to trust anyone except the
> >developer/package-maintainer/release-manager.
> >
> >Debian et. all solve this with signed packages. I would be happy to download
> >Debian packages from http://pythonpackages.com all day long :)
> 
> 
> That's good to know, and probably I direction I'd like to head in.
> To be clear: I want to do any-useful-thing-I-can (within the
> ballpark) in order to start alleviating pain points for folks today.
 
Cool,

Well one thing would be to make all of your source code open-source, if that is not already the case(?)

I can imagine wanting to run some pythonpackaging.com infrastructure outside of pythonpackages.com
 
> >Debian also rely upon trusted build machines. But they are a more-or-less open
> >organization with open review of what goes on.
> >
> >That said, I don't have a problem with people placing their trust in you. I don't
> >know you, and don't have any opinion on it to be honest. You're probably a good guy ;)
> >
> >I would suggest working toward BEING a better PyPI mirror. Build
> >the infrastructure necessary for people to publish python SOURCE packages,
> >as they are, to PyPI, to pythonpackages.com, etc. etc. There is a lot of value
> >to be added there.
> 
> 
> Actually I'm mostly relying on the crate.io project (Donald Stufft)
> for this. I don't want pythonpackages.com to be a PyPI mirror,
> because other people are already doing this. The only related
> feature I'm considering (because folks have asked for it) is private
> PyPIs (something like index.pythonpackages.com only persistent).
> 
> 
> >
> >Build tools to make python packaging easy. On your laptop. On the cloud. Wherever.
> >Open SOURCE is good like that.
> 
> Indeed! Currently working on a Windows version of pythonpackages.com
> to build Windows binaries (currently it only builds on Ubuntu).
> 

The key point I was making was that SOURCE is good, because then it's not just "some cloud service"
that could be here today and gone tomorrow - It's actually something people can rely on moving
forward. (in addition to being a service you run).

> 
> Alex
> 

--
Regards,
Eric Mangold