[Catalog-sig] ANN: pythonpackages.com beta

[Alex Clark]

Hi Eric,


(Continuing this discussion from twisted list)


On 7/31/12 10:24 AM, Eric P. Mangold wrote:
> On Mon, Jul 30, 2012 at 05:09:07PM -0400, Alex Clark wrote:
>> Hi Eric,
>>
>> On 7/30/12 4:49 PM, Eric P. Mangold wrote:
>>> On Mon, Jul 30, 2012 at 12:49:56PM -0400, Alex Clark wrote:
>>>> Hi,
>>>>
>>>>
>>>> On 7/30/12 12:31 PM, Eric P. Mangold wrote:
>>>>> Alex,
>>>>>
>>>>> I'm not sure if this is borderline off-topic, or not... but anyway..
>>>>>
>>>>> I'm sure starting a discussion here IS offtopic.
>>>>>
>>>>> But I have one question:
>>>>>
>>>>> How do package authors verify the integrity of their packages built "through the web"?
>>>>
>>>>
>>>> Good question, I just created:
>>>>
>>>> -
>>>> http://docs.pythonpackages.com/en/latest/faq.html#how-do-package-authors-verify-the-integrity-of-packages-built-through-the-web
>>>
>>> Let me be clear:
>>>
>>> Is it possible to have any assurance that your system has faithfully built the package, and/or that your servers have not been compromised?
>>>
>>> Why would anyone trust your web service to build packages, when it is *their* pgp, reputation and users that are at stake?
>>> (Yes, I would ask Launchpad/Canonical, et. all the same question...)
>>>
>>> (Also, if you're suggesting MD5 (following your link..) for anything related to security or data authenticity, then I *know* you're way off base.......)
>>
>>
>> The point about md5 is not to suggest using it for security or data
>> authenticity,
>
> Sorry, I think I have a problem with taking the exact text of my question,
> on your wiki, and casting it to be a different question entirely. (through
> no fault of your own, I'm sure)


Sorry, removed! Let me know if there is something better I can put in 
its place.


>
> I think I've clarified what my orignal question was meant to ask, namely how do
> we trust YOU and YOUR build infrastructure, not "how do we verify that the data
> you're give us hasn't been damaged in transit".
>
> If you wouldn't mind editing your wiki to reflect my clarifications, I would
> very much appreciate it :)


OK Let me work on it.


>
>> it's to clarify that whatever security is currently place
>> with PyPI (not a lot, admittedly) still applies, for whatever that is
>> worth (not much, apparently).
>>
>>
>>>
>>> Sorry if this is harsh - but it's intended. Without any kind of verifiable guarantee (get to work on that! :)) I don't think I could ever possibly use such a thing, and would advise against it.
>>>
>>> Getting software to end-users is a tough challenge, and I applaude your efforts to try and make it easier. A system with a single point of failure and a single point of trust just isn't feasible or desirable, imho.Administrators need to know who has final responsibility and *authority*
>> over the software that they are consuming. If "the cloud" is the last
>> link in that chain, then you have a big problem, I think.
>>
>>
>> The last link in the chain is PyPI (or a private index). The node before
>> that is typically your laptop. I'm suggesting you make it
>> pythonpackages.com instead.
>
> Clearly PyPI is inadequate. Jumping on solutions, for HARD problems that always
> require some form of cryptography to solve, is no more palettable.
>
> And PyPI is also just a publishing platform - the packages have already been
> minted by that point.
>
> So as you suggest, the last point is the developer/release-manager, as it should
> be.
>
> I think my point is that ideally you don't want to trust anyone except the
> developer/package-maintainer/release-manager.
>
> Debian et. all solve this with signed packages. I would be happy to download
> Debian packages from http://pythonpackages.com all day long :)


That's good to know, and probably I direction I'd like to head in. To be 
clear: I want to do any-useful-thing-I-can (within the ballpark) in 
order to start alleviating pain points for folks today.


>
> Debian also rely upon trusted build machines. But they are a more-or-less open
> organization with open review of what goes on.
>
> That said, I don't have a problem with people placing their trust in you. I don't
> know you, and don't have any opinion on it to be honest. You're probably a good guy ;)
>
> I would suggest working toward BEING a better PyPI mirror. Build
> the infrastructure necessary for people to publish python SOURCE packages,
> as they are, to PyPI, to pythonpackages.com, etc. etc. There is a lot of value
> to be added there.


Actually I'm mostly relying on the crate.io project (Donald Stufft) for 
this. I don't want pythonpackages.com to be a PyPI mirror, because other 
people are already doing this. The only related feature I'm considering 
(because folks have asked for it) is private PyPIs (something like 
index.pythonpackages.com only persistent).


>
> Build tools to make python packaging easy. On your laptop. On the cloud. Wherever.
> Open SOURCE is good like that.



Indeed! Currently working on a Windows version of pythonpackages.com to 
build Windows binaries (currently it only builds on Ubuntu).



Alex





>
> Regards,
> Eric Mangold
>


-- 
Alex Clark · http://pythonpackages.com/ONE_CLICK