From eric at teratorn.org Wed Aug 1 20:09:42 2012 From: eric at teratorn.org (Eric P. Mangold) Date: Wed, 1 Aug 2012 14:09:42 -0400 Subject: [Catalog-sig] ANN: pythonpackages.com beta In-Reply-To: References: <20120730163118.GH10379@ragnarok.teratorn.org> <20120730204904.GJ10379@ragnarok.teratorn.org> <20120731142402.GL10379@ragnarok.teratorn.org> Message-ID: <20120801180942.GQ10379@ragnarok.teratorn.org> On Tue, Jul 31, 2012 at 11:52:22AM -0400, Alex Clark wrote: > Hi Eric, > > > (Continuing this discussion from twisted list) > > > On 7/31/12 10:24 AM, Eric P. Mangold wrote: > >On Mon, Jul 30, 2012 at 05:09:07PM -0400, Alex Clark wrote: > >>Hi Eric, > >> > >>On 7/30/12 4:49 PM, Eric P. Mangold wrote: > >>>On Mon, Jul 30, 2012 at 12:49:56PM -0400, Alex Clark wrote: > >>>>Hi, > >>>> > >>>> > >>>>On 7/30/12 12:31 PM, Eric P. Mangold wrote: > >>>>>Alex, > >>>>> > >>>>>I'm not sure if this is borderline off-topic, or not... but anyway.. > >>>>> > >>>>>I'm sure starting a discussion here IS offtopic. > >>>>> > >>>>>But I have one question: > >>>>> > >>>>>How do package authors verify the integrity of their packages built "through the web"? > >>>> > >>>> > >>>>Good question, I just created: > >>>> > >>>>- > >>>>http://docs.pythonpackages.com/en/latest/faq.html#how-do-package-authors-verify-the-integrity-of-packages-built-through-the-web > >>> > >>>Let me be clear: > >>> > >>>Is it possible to have any assurance that your system has faithfully built the package, and/or that your servers have not been compromised? > >>> > >>>Why would anyone trust your web service to build packages, when it is *their* pgp, reputation and users that are at stake? > >>>(Yes, I would ask Launchpad/Canonical, et. all the same question...) > >>> > >>>(Also, if you're suggesting MD5 (following your link..) for anything related to security or data authenticity, then I *know* you're way off base.......) > >> > >> > >>The point about md5 is not to suggest using it for security or data > >>authenticity, > > > >Sorry, I think I have a problem with taking the exact text of my question, > >on your wiki, and casting it to be a different question entirely. (through > >no fault of your own, I'm sure) > > > Sorry, removed! Let me know if there is something better I can put > in its place. > > > > > >I think I've clarified what my orignal question was meant to ask, namely how do > >we trust YOU and YOUR build infrastructure, not "how do we verify that the data > >you're give us hasn't been damaged in transit". > > > >If you wouldn't mind editing your wiki to reflect my clarifications, I would > >very much appreciate it :) > > > OK Let me work on it. > > > > > >>it's to clarify that whatever security is currently place > >>with PyPI (not a lot, admittedly) still applies, for whatever that is > >>worth (not much, apparently). > >> > >> > >>> > >>>Sorry if this is harsh - but it's intended. Without any kind of verifiable guarantee (get to work on that! :)) I don't think I could ever possibly use such a thing, and would advise against it. > >>> > >>>Getting software to end-users is a tough challenge, and I applaude your efforts to try and make it easier. A system with a single point of failure and a single point of trust just isn't feasible or desirable, imho.Administrators need to know who has final responsibility and *authority* > >>over the software that they are consuming. If "the cloud" is the last > >>link in that chain, then you have a big problem, I think. > >> > >> > >>The last link in the chain is PyPI (or a private index). The node before > >>that is typically your laptop. I'm suggesting you make it > >>pythonpackages.com instead. > > > >Clearly PyPI is inadequate. Jumping on solutions, for HARD problems that always > >require some form of cryptography to solve, is no more palettable. > > > >And PyPI is also just a publishing platform - the packages have already been > >minted by that point. > > > >So as you suggest, the last point is the developer/release-manager, as it should > >be. > > > >I think my point is that ideally you don't want to trust anyone except the > >developer/package-maintainer/release-manager. > > > >Debian et. all solve this with signed packages. I would be happy to download > >Debian packages from http://pythonpackages.com all day long :) > > > That's good to know, and probably I direction I'd like to head in. > To be clear: I want to do any-useful-thing-I-can (within the > ballpark) in order to start alleviating pain points for folks today. Cool, Well one thing would be to make all of your source code open-source, if that is not already the case(?) I can imagine wanting to run some pythonpackaging.com infrastructure outside of pythonpackages.com > >Debian also rely upon trusted build machines. But they are a more-or-less open > >organization with open review of what goes on. > > > >That said, I don't have a problem with people placing their trust in you. I don't > >know you, and don't have any opinion on it to be honest. You're probably a good guy ;) > > > >I would suggest working toward BEING a better PyPI mirror. Build > >the infrastructure necessary for people to publish python SOURCE packages, > >as they are, to PyPI, to pythonpackages.com, etc. etc. There is a lot of value > >to be added there. > > > Actually I'm mostly relying on the crate.io project (Donald Stufft) > for this. I don't want pythonpackages.com to be a PyPI mirror, > because other people are already doing this. The only related > feature I'm considering (because folks have asked for it) is private > PyPIs (something like index.pythonpackages.com only persistent). > > > > > >Build tools to make python packaging easy. On your laptop. On the cloud. Wherever. > >Open SOURCE is good like that. > > Indeed! Currently working on a Windows version of pythonpackages.com > to build Windows binaries (currently it only builds on Ubuntu). > The key point I was making was that SOURCE is good, because then it's not just "some cloud service" that could be here today and gone tomorrow - It's actually something people can rely on moving forward. (in addition to being a service you run). > > Alex > -- Regards, Eric Mangold From eric at teratorn.org Wed Aug 1 20:11:53 2012 From: eric at teratorn.org (Eric P. Mangold) Date: Wed, 1 Aug 2012 14:11:53 -0400 Subject: [Catalog-sig] ANN: pythonpackages.com beta In-Reply-To: References: <20120730163118.GH10379@ragnarok.teratorn.org> <20120730204904.GJ10379@ragnarok.teratorn.org> <20120731142402.GL10379@ragnarok.teratorn.org> Message-ID: <20120801181153.GR10379@ragnarok.teratorn.org> On Tue, Jul 31, 2012 at 01:43:42PM -0400, Daniel Holth wrote: > Perhaps you would be interested in the Wheel package format's upcoming > public key signature system (wheel.rtfd.orgl#signed-wheel-files). The > (undocumented) plan will include per-buildserver, per-package and > possibly per-package-version signing keys via a pluggable trust model, > instead of the PGP model where a signing key is an e-mail address. Sounds really interesting. > About wheel > "A wheel is a ZIP-format archive with a specially formatted filename > and the .whl extension. It is designed to contain all the files for a > PEP 376 compatible install in a way that is very close to the on-disk > format." > > A simple wheel for Package-1.0 would contain > > package.py > > Package-1.0.dist-info/METADATA (PEP-376, PEP-345 'Metadata 1.2') > > Package-1.0.dist-info/WHEEL (metadata for this build of the dist) > > Package-1.0.dist-info/RECORD (extended PEP-376) > > The bdist_wheel setuptools plugin and egg2wheel and wininst2wheel > allow you to create wheel archives without having to modify source > dists. Neat. I look forward to using this stuff... -- -E From aclark at aclark.net Wed Aug 1 20:19:52 2012 From: aclark at aclark.net (Alex Clark) Date: Wed, 01 Aug 2012 14:19:52 -0400 Subject: [Catalog-sig] ANN: pythonpackages.com beta In-Reply-To: <20120801180942.GQ10379@ragnarok.teratorn.org> References: <20120730163118.GH10379@ragnarok.teratorn.org> <20120730204904.GJ10379@ragnarok.teratorn.org> <20120731142402.GL10379@ragnarok.teratorn.org> <20120801180942.GQ10379@ragnarok.teratorn.org> Message-ID: Hi On 8/1/12 2:09 PM, Eric P. Mangold wrote: [snip] >>> >>> Debian et. all solve this with signed packages. I would be happy to download >>> Debian packages from http://pythonpackages.com all day long :) >> >> >> That's good to know, and probably I direction I'd like to head in. >> To be clear: I want to do any-useful-thing-I-can (within the >> ballpark) in order to start alleviating pain points for folks today. > > Cool, > > Well one thing would be to make all of your source code open-source, if that is not already the case(?) > > I can imagine wanting to run some pythonpackaging.com infrastructure outside of pythonpackages.com I <3 open source and it could happen, but it hasn't yet (for various reasons). I have a FAQ about it here: - http://docs.pythonpackages.com/en/latest/faq.html#is-pythonpackages-com-open-source >>> Debian also rely upon trusted build machines. But they are a more-or-less open >>> organization with open review of what goes on. >>> >>> That said, I don't have a problem with people placing their trust in you. I don't >>> know you, and don't have any opinion on it to be honest. You're probably a good guy ;) >>> >>> I would suggest working toward BEING a better PyPI mirror. Build >>> the infrastructure necessary for people to publish python SOURCE packages, >>> as they are, to PyPI, to pythonpackages.com, etc. etc. There is a lot of value >>> to be added there. >> >> >> Actually I'm mostly relying on the crate.io project (Donald Stufft) >> for this. I don't want pythonpackages.com to be a PyPI mirror, >> because other people are already doing this. The only related >> feature I'm considering (because folks have asked for it) is private >> PyPIs (something like index.pythonpackages.com only persistent). >> >> >>> >>> Build tools to make python packaging easy. On your laptop. On the cloud. Wherever. >>> Open SOURCE is good like that. >> >> Indeed! Currently working on a Windows version of pythonpackages.com >> to build Windows binaries (currently it only builds on Ubuntu). >> > > The key point I was making was that SOURCE is good, because then it's not just "some cloud service" > that could be here today and gone tomorrow - It's actually something people can rely on moving > forward. (in addition to being a service you run). I don't disagree, but I'm also not convinced that it has to be that way to be successful. Alex > >> >> Alex >> > > -- > Regards, > Eric Mangold > -- Alex Clark ? http://pythonpackages.com/ONE_CLICK From dholth at gmail.com Wed Aug 1 21:12:14 2012 From: dholth at gmail.com (Daniel Holth) Date: Wed, 1 Aug 2012 15:12:14 -0400 Subject: [Catalog-sig] ANN: pythonpackages.com beta In-Reply-To: <20120801181153.GR10379@ragnarok.teratorn.org> References: <20120730163118.GH10379@ragnarok.teratorn.org> <20120730204904.GJ10379@ragnarok.teratorn.org> <20120731142402.GL10379@ragnarok.teratorn.org> <20120801181153.GR10379@ragnarok.teratorn.org> Message-ID: On Wed, Aug 1, 2012 at 2:11 PM, Eric P. Mangold wrote: > Neat. I look forward to using this stuff... Try the demo. The format works well for the initial use case "lxml takes too long to compile" and can be used to build virtualenvs in record time. The digital signatures piece is not finished, and the installer does not currently check the compatibility tags (whether the wheel is expected to run on the installing Python), but "pip install --find-links directory-of-cached-wheels somepackage" works great (with the patched pip). I mention wheel in this forum because pypi doesn't accept them for upload yet, a feature which we will want after getting enough feedback on the basics of the format. For test.pypi? Daniel Holth From ubershmekel at gmail.com Wed Aug 1 23:12:34 2012 From: ubershmekel at gmail.com (Yuval Greenfield) Date: Thu, 2 Aug 2012 00:12:34 +0300 Subject: [Catalog-sig] ANN: pythonpackages.com beta In-Reply-To: References: <20120730163118.GH10379@ragnarok.teratorn.org> <20120730204904.GJ10379@ragnarok.teratorn.org> <20120731142402.GL10379@ragnarok.teratorn.org> <20120801180942.GQ10379@ragnarok.teratorn.org> Message-ID: On Wed, Aug 1, 2012 at 9:19 PM, Alex Clark wrote: > Hi > > On 8/1/12 2:09 PM, Eric P. Mangold wrote: > [snip] > > >>>> Debian et. all solve this with signed packages. I would be happy to >>>> download >>>> Debian packages from http://pythonpackages.com all day long :) >>>> >>> >>> >>> That's good to know, and probably I direction I'd like to head in. >>> To be clear: I want to do any-useful-thing-I-can (within the >>> ballpark) in order to start alleviating pain points for folks today. >>> >> >> Cool, >> >> Well one thing would be to make all of your source code open-source, if >> that is not already the case(?) >> >> I can imagine wanting to run some pythonpackaging.com infrastructure >> outside of pythonpackages.com >> > > > I <3 open source and it could happen, but it hasn't yet (for various > reasons). I have a FAQ about it here: > > - http://docs.pythonpackages.**com/en/latest/faq.html#is-** > pythonpackages-com-open-source > > > Pasted what the FAQ says for reference: Is pythonpackages.com open source? The web application that powers pythonpackages.com is not open source, > however it uses open source software where and when applicable, and > permissible by license, in order to facilitate its operation. Furthermore, > pythonpackages.com has a large committment to the open source software > community in general, and strives to contribute as much as possible. All of > pythonpackages.com?s open source offerings are made available here: > https://github.com/pythonpackages. > It doesn't really explain why pythonpackages.com isn't open source. Keeping the project closed is your right. Though I can definitely understand why Eric or any package maintainer would worry of introducing a dependency on a black box that does binary magic. Yuval -------------- next part -------------- An HTML attachment was scrubbed... URL: From aclark at aclark.net Thu Aug 2 02:16:32 2012 From: aclark at aclark.net (Alex Clark) Date: Wed, 01 Aug 2012 20:16:32 -0400 Subject: [Catalog-sig] ANN: pythonpackages.com beta In-Reply-To: References: <20120730163118.GH10379@ragnarok.teratorn.org> <20120730204904.GJ10379@ragnarok.teratorn.org> <20120731142402.GL10379@ragnarok.teratorn.org> <20120801180942.GQ10379@ragnarok.teratorn.org> Message-ID: Hi On 8/1/12 5:12 PM, Yuval Greenfield wrote: [snip] > > > > Pasted what the FAQ says for reference: > > Is pythonpackages.com open source? > > The web application that powers pythonpackages.com > is not open source, however it uses open > source software where and when applicable, and permissible by > license, in order to facilitate its operation. Furthermore, > pythonpackages.com has a large > committment to the open source software community in general, and > strives to contribute as much as possible. All of pythonpackages.com > ?s open source offerings are made > available here: https://github.com/pythonpackages. > > > It doesn't really explain why pythonpackages.com > isn't open source. True, I've added this: - http://docs.pythonpackages.com/en/latest/faq.html#why-isn-t-pythonpackages-com-open-source > Keeping the project > closed is your right. Though I can definitely understand why Eric or any > package maintainer would worry of introducing a dependency on a black > box that does binary magic. It's a valid point, but I've purposely chosen to automate tasks that are relatively trivial to perform locally, in hopes that the automation will create value and help folks' do their jobs better. The "binary magic" FWIW is currently: - Redis-backed pyramid application with deform forms to facilitate user sign up and package management. - requests library to do GitHub API calls. - Hacks around GitHub and PyPI API limitations (the latter of which is currently being addressed, and is probably the most relevant topic to discuss on this list). - pbs library to make `python setup.py` calls. Alex > Yuval > > > _______________________________________________ > Catalog-SIG mailing list > Catalog-SIG at python.org > http://mail.python.org/mailman/listinfo/catalog-sig > -- Alex Clark ? http://pythonpackages.com/ONE_CLICK From eric at teratorn.org Thu Aug 2 02:37:27 2012 From: eric at teratorn.org (Eric P. Mangold) Date: Wed, 1 Aug 2012 20:37:27 -0400 Subject: [Catalog-sig] ANN: pythonpackages.com beta In-Reply-To: References: <20120730163118.GH10379@ragnarok.teratorn.org> <20120730204904.GJ10379@ragnarok.teratorn.org> <20120731142402.GL10379@ragnarok.teratorn.org> <20120801181153.GR10379@ragnarok.teratorn.org> Message-ID: <20120802003727.GS10379@ragnarok.teratorn.org> On Wed, Aug 01, 2012 at 03:12:14PM -0400, Daniel Holth wrote: > On Wed, Aug 1, 2012 at 2:11 PM, Eric P. Mangold wrote: > > > Neat. I look forward to using this stuff... > > Try the demo. The format works well for the initial use case "lxml > takes too long to compile" and can be used to build virtualenvs in > record time. The digital signatures piece is not finished, and the > installer does not currently check the compatibility tags (whether the > wheel is expected to run on the installing Python), but "pip install > --find-links directory-of-cached-wheels somepackage" works great (with > the patched pip). sounds cool > I mention wheel in this forum because pypi doesn't accept them for > upload yet, a feature which we will want after getting enough feedback > on the basics of the format. For test.pypi? > > Daniel Holth Would be nice to start using post-egg infrastructure. -- Cheers, -E From eric at teratorn.org Thu Aug 2 02:48:18 2012 From: eric at teratorn.org (Eric P. Mangold) Date: Wed, 1 Aug 2012 20:48:18 -0400 Subject: [Catalog-sig] ANN: pythonpackages.com beta In-Reply-To: References: <20120730163118.GH10379@ragnarok.teratorn.org> <20120730204904.GJ10379@ragnarok.teratorn.org> <20120731142402.GL10379@ragnarok.teratorn.org> <20120801180942.GQ10379@ragnarok.teratorn.org> Message-ID: <20120802004818.GT10379@ragnarok.teratorn.org> On Wed, Aug 01, 2012 at 02:19:52PM -0400, Alex Clark wrote: > Hi > > On 8/1/12 2:09 PM, Eric P. Mangold wrote: > [snip] > >>> > >>>Debian et. all solve this with signed packages. I would be happy to download > >>>Debian packages from http://pythonpackages.com all day long :) > >> > >> > >>That's good to know, and probably I direction I'd like to head in. > >>To be clear: I want to do any-useful-thing-I-can (within the > >>ballpark) in order to start alleviating pain points for folks today. > > > >Cool, > > > >Well one thing would be to make all of your source code open-source, if that is not already the case(?) > > > >I can imagine wanting to run some pythonpackaging.com infrastructure outside of pythonpackages.com > > > I <3 open source and it could happen, but it hasn't yet (for various > reasons). I have a FAQ about it here: > > - http://docs.pythonpackages.com/en/latest/faq.html#is-pythonpackages-com-open-source Well since you're a commercial service I can understand your reluctance. :) > >>>Debian also rely upon trusted build machines. But they are a more-or-less open > >>>organization with open review of what goes on. > >>> > >>>That said, I don't have a problem with people placing their trust in you. I don't > >>>know you, and don't have any opinion on it to be honest. You're probably a good guy ;) > >>> > >>>I would suggest working toward BEING a better PyPI mirror. Build > >>>the infrastructure necessary for people to publish python SOURCE packages, > >>>as they are, to PyPI, to pythonpackages.com, etc. etc. There is a lot of value > >>>to be added there. > >> > >> > >>Actually I'm mostly relying on the crate.io project (Donald Stufft) > >>for this. I don't want pythonpackages.com to be a PyPI mirror, > >>because other people are already doing this. The only related > >>feature I'm considering (because folks have asked for it) is private > >>PyPIs (something like index.pythonpackages.com only persistent). > >> > >> > >>> > >>>Build tools to make python packaging easy. On your laptop. On the cloud. Wherever. > >>>Open SOURCE is good like that. > >> > >>Indeed! Currently working on a Windows version of pythonpackages.com > >>to build Windows binaries (currently it only builds on Ubuntu). > >> > > > >The key point I was making was that SOURCE is good, because then it's not just "some cloud service" > >that could be here today and gone tomorrow - It's actually something people can rely on moving > >forward. (in addition to being a service you run). > > > I don't disagree, but I'm also not convinced that it has to be that > way to be successful. > Well good luck with that. I think you should consider open-sourcing, and consider the on-prem market. Given those two things, I could even imagine myself as a customer. Cheers, -E From richard at python.org Thu Aug 2 02:52:44 2012 From: richard at python.org (Richard Jones) Date: Thu, 2 Aug 2012 10:52:44 +1000 Subject: [Catalog-sig] ANN: pythonpackages.com beta In-Reply-To: References: <20120730163118.GH10379@ragnarok.teratorn.org> <20120730204904.GJ10379@ragnarok.teratorn.org> <20120731142402.GL10379@ragnarok.teratorn.org> <20120801181153.GR10379@ragnarok.teratorn.org> Message-ID: On 2 August 2012 05:12, Daniel Holth wrote: > I mention wheel in this forum because pypi doesn't accept them for > upload yet, a feature which we will want after getting enough feedback > on the basics of the format. For test.pypi? Once you're confident of the format and the community thinks they're worthwhile I'll happily add support for wheel files - I just need some basic validation rules to ensure we aren't getting garbage submitted. Richard From martin at v.loewis.de Thu Aug 2 11:32:24 2012 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 02 Aug 2012 11:32:24 +0200 Subject: [Catalog-sig] e.pypi.python.org now in China Message-ID: <501A4928.7060609@v.loewis.de> Thanks to Aron Xu, e.pypi.python.org is now located in China; I had to close my own mirror right at the moment when he offered to provide one. Regards, Martin From michael at voidspace.org.uk Sun Aug 5 13:35:18 2012 From: michael at voidspace.org.uk (Michael Foord) Date: Sun, 5 Aug 2012 12:35:18 +0100 Subject: [Catalog-sig] Fwd: No registration email References: Message-ID: <38B71006-2F1A-4691-ACF7-31C85B476EEB@voidspace.org.uk> Begin forwarded message: > From: Paul Backhouse > Subject: No registration email > Date: 5 August 2012 10:26:30 BST > To: > > Hi, > > I'm having issues registring with bugs.python.com. I enter correct details, username "paulb" email "pcbackhouse at hotmail.co.uk" etc and get taken to the "Registration in Progress" page. However I never receive a confirmation email. I've tried to register a few time now. > > Thanks, > > Paul -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From pmartin at yaco.es Thu Aug 9 17:32:37 2012 From: pmartin at yaco.es (Pablo Martin) Date: Thu, 9 Aug 2012 17:32:37 +0200 Subject: [Catalog-sig] I removed a egg in pypi Message-ID: Hi, I am Pablo Mart?n, developer in python by 5 years ago. I have a little problem.... I removed a egg [1], by mistake. I only wanted to remove a release.... I have got to recover every release from a mirror [2], and I uploaded these. But of course I have lost the meta information (stats, date to upload etc) is it possible to recover this? REF's 1. http://pypi.python.org/pypi/django-inplaceedit 2. http://f.pypi.python.org/simple/django-inplaceedit/ Thanks you! -- Pablo Mart?n pmartin at yaco.es Yaco Sistemas S.L. http://www.yaco.es/ C/ Rioja 5, 41001 Sevilla Tel?fono +34 954 50 00 57 Fax +34 954 50 09 29 -------------- next part -------------- An HTML attachment was scrubbed... URL: From julien at tayon.net Thu Aug 9 17:46:35 2012 From: julien at tayon.net (julien tayon) Date: Thu, 9 Aug 2012 17:46:35 +0200 Subject: [Catalog-sig] I removed a egg in pypi In-Reply-To: References: Message-ID: 2012/8/9 Pablo Martin : > Hi, Hello, For the future, you can use http://pypi.python.org/pypi/pypi-stat/1.2.2 : it stores a time serie of a package stats, upload, revisions ... locally in an easily accessible json. btw, I intend for research purpose to upload a malvelant package on pypi to test the security. Would calling it dont_install a good idea? (it would modify a dotfile (.bashrc), delete or create a file in the PATH, call an outer webservice to simulate an information leak). The doc would ofc tell DONT INSTALL. I also want to test the openBSD pkg_add (systrace jails/stuff) to propose an automated installation checking for malvolent stuff this way. Cheers, -- Julien From pmartin at yaco.es Thu Aug 9 17:51:49 2012 From: pmartin at yaco.es (Pablo Martin) Date: Thu, 9 Aug 2012 17:51:49 +0200 Subject: [Catalog-sig] I removed a egg in pypi In-Reply-To: References: Message-ID: 2012/8/9 julien tayon > 2012/8/9 Pablo Martin : > > Hi, > Hello, > > For the future, you can use > http://pypi.python.org/pypi/pypi-stat/1.2.2 : it stores a time serie > of a package stats, upload, revisions ... locally in an easily > accessible json. > > Ok, thanks. But is there some possibility to get the info removed? is there some way to restore the old data? Sorry by my clumsiness > btw, I intend for research purpose to upload a malvelant package on > pypi to test the security. Would calling it dont_install a good idea? > (it would modify a dotfile (.bashrc), delete or create a file in the > PATH, call an outer webservice to simulate an information leak). The > doc would ofc tell DONT INSTALL. > > I also want to test the openBSD pkg_add (systrace jails/stuff) to > propose an automated installation checking for malvolent stuff this > way. > > Cheers, > -- > Julien > -------------- next part -------------- An HTML attachment was scrubbed... URL: From martin at v.loewis.de Thu Aug 9 19:19:16 2012 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 09 Aug 2012 19:19:16 +0200 Subject: [Catalog-sig] I removed a egg in pypi In-Reply-To: References: Message-ID: <5023F114.2050004@v.loewis.de> > is it possible to recover this? In principle, there are backups, but they are intended for catastrophic failures of PyPI itself, such as the server-side removal of all files. Since accessing the backups is fairly expensive (in terms of man-hours), we do not offer that for recovery from end-user mistakes; you would have to run your own backups for that. Regards, Martin From pmartin at yaco.es Sun Aug 12 22:22:10 2012 From: pmartin at yaco.es (Pablo Martin) Date: Sun, 12 Aug 2012 22:22:10 +0200 Subject: [Catalog-sig] I removed a egg in pypi In-Reply-To: <5023F114.2050004@v.loewis.de> References: <5023F114.2050004@v.loewis.de> Message-ID: 2012/8/9 "Martin v. L?wis" > is it possible to recover this? >> > > In principle, there are backups, but they are intended for catastrophic > failures of PyPI itself, such as the server-side removal of all files. > Since accessing the backups is fairly expensive (in terms of man-hours), > we do not offer that for recovery from end-user mistakes; you would > have to run your own backups for that. > > Ok, no problem, I got every version of the edge. Only I lost the meta-information (stats, dates, etc). I know that this is my problem, I was wrong the other day... but I wanted to know if a admin of pypi could remove my action. I think that I am not the first :-) > Regards, > Martin > Thanks anyway, -- Pablo Martin -------------- next part -------------- An HTML attachment was scrubbed... URL: From martin at v.loewis.de Mon Aug 13 18:01:01 2012 From: martin at v.loewis.de (martin at v.loewis.de) Date: Mon, 13 Aug 2012 18:01:01 +0200 Subject: [Catalog-sig] PyPI migration Message-ID: <20120813180101.Horde.EXUuALuWis5QKSS95boj2qA@webmail.df.eu> I'll be moving PyPI to new hardware tomorrow in the UTC morning; expect an outage of no more than one hour. Regards, Martin From martin at v.loewis.de Tue Aug 14 12:18:33 2012 From: martin at v.loewis.de (martin at v.loewis.de) Date: Tue, 14 Aug 2012 12:18:33 +0200 Subject: [Catalog-sig] PyPI migration done Message-ID: <20120814121833.Horde.KX0sSdjz9kRQKiX50KMwVcA@webmail.df.eu> PyPI is now hosted at OSU/OSL; thanks to Noah Kantrowitz for preparing the infrastructure. We still may need tune this installation over the coming weeks; further outages will be announced again. Regards, Martin From donald.stufft at gmail.com Tue Aug 14 13:03:17 2012 From: donald.stufft at gmail.com (Donald Stufft) Date: Tue, 14 Aug 2012 07:03:17 -0400 Subject: [Catalog-sig] PyPI migration done In-Reply-To: <20120814121833.Horde.KX0sSdjz9kRQKiX50KMwVcA@webmail.df.eu> References: <20120814121833.Horde.KX0sSdjz9kRQKiX50KMwVcA@webmail.df.eu> Message-ID: <6C4806A0AC12421AAC17E7C6F18AB497@gmail.com> The serverkey appears to be missing, or it was moved? curl -I http://pypi.python.org/serverkey HTTP/1.1 404 Not Found Server: nginx/1.1.19 Date: Tue, 14 Aug 2012 11:02:08 GMT Content-Type: text/html Content-Length: 169 Connection: keep-alive On Tuesday, August 14, 2012 at 6:18 AM, martin at v.loewis.de wrote: > PyPI is now hosted at OSU/OSL; thanks to Noah Kantrowitz for preparing > the infrastructure. > > We still may need tune this installation over the coming weeks; further > outages will be announced again. > > Regards, > Martin > > > > _______________________________________________ > Catalog-SIG mailing list > Catalog-SIG at python.org (mailto:Catalog-SIG at python.org) > http://mail.python.org/mailman/listinfo/catalog-sig > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From martin at v.loewis.de Tue Aug 14 13:27:11 2012 From: martin at v.loewis.de (martin at v.loewis.de) Date: Tue, 14 Aug 2012 13:27:11 +0200 Subject: [Catalog-sig] PyPI migration done In-Reply-To: <6C4806A0AC12421AAC17E7C6F18AB497@gmail.com> References: <20120814121833.Horde.KX0sSdjz9kRQKiX50KMwVcA@webmail.df.eu> <6C4806A0AC12421AAC17E7C6F18AB497@gmail.com> Message-ID: <20120814132711.Horde.b_viINjz9kRQKjYPlnqRRcA@webmail.df.eu> Zitat von Donald Stufft : > The serverkey appears to be missing, or it was moved? No, that was a mistake - which is now fixed. Thanks, Martin From jwilk at jwilk.net Wed Aug 15 20:22:50 2012 From: jwilk at jwilk.net (Jakub Wilk) Date: Wed, 15 Aug 2012 20:22:50 +0200 Subject: [Catalog-sig] PyPI migration done In-Reply-To: <20120814121833.Horde.KX0sSdjz9kRQKiX50KMwVcA@webmail.df.eu> References: <20120814121833.Horde.KX0sSdjz9kRQKiX50KMwVcA@webmail.df.eu> Message-ID: <20120815182250.GA563@jwilk.net> It looks like the migration broke SSH access: $ ssh -T submit at ssh.pypi.python.org submit at ssh.pypi.python.org's password: Of course, I don't don't know submit's password. ;) -- Jakub Wilk From martin at v.loewis.de Fri Aug 17 14:45:27 2012 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 17 Aug 2012 14:45:27 +0200 Subject: [Catalog-sig] Wheel format now supported Message-ID: <502E3CE7.7090205@v.loewis.de> I just added support for the wheel format to the PyPI file upload interface. If this causes problems to existing tools, please submit a bug report. Regards, Martin From dholth at gmail.com Fri Aug 17 15:13:00 2012 From: dholth at gmail.com (Daniel Holth) Date: Fri, 17 Aug 2012 09:13:00 -0400 Subject: [Catalog-sig] Wheel format now supported In-Reply-To: <502E3CE7.7090205@v.loewis.de> References: <502E3CE7.7090205@v.loewis.de> Message-ID: On Fri, Aug 17, 2012 at 8:45 AM, "Martin v. L?wis" wrote: > I just added support for the wheel format to the PyPI > file upload interface. If this causes problems to existing > tools, please submit a bug report. +1 From dholth at gmail.com Fri Aug 17 19:57:51 2012 From: dholth at gmail.com (Daniel Holth) Date: Fri, 17 Aug 2012 13:57:51 -0400 Subject: [Catalog-sig] Wheel format now supported In-Reply-To: References: <502E3CE7.7090205@v.loewis.de> Message-ID: Uploading a wheel to the cheese shop: Install wheel (from pypi) Use setuptools in your setup.py 'python setup.py bdist_wheel upload' ? Profit! Thanks again for implementing this in pypi. Daniel From pydanny at gmail.com Fri Aug 17 20:17:42 2012 From: pydanny at gmail.com (Daniel Greenfeld) Date: Fri, 17 Aug 2012 11:17:42 -0700 Subject: [Catalog-sig] Wheel format now supported In-Reply-To: References: <502E3CE7.7090205@v.loewis.de> Message-ID: Martin, Daniel, Will Wheel work with non-patched pip? Will we need to handle multiple pip versions to get projects running that have the existing format and this new Wheel format? Will we have to teach incoming developers to use standard pip and patched pip? How does Wheel work with virtualenv and buildout? I don't mind moving forward, this just isn't very clear to me. Danny On Fri, Aug 17, 2012 at 10:57 AM, Daniel Holth wrote: > Uploading a wheel to the cheese shop: > > Install wheel (from pypi) > > Use setuptools in your setup.py > > 'python setup.py bdist_wheel upload' > > ? > > Profit! > > Thanks again for implementing this in pypi. > > Daniel > _______________________________________________ > Catalog-SIG mailing list > Catalog-SIG at python.org > http://mail.python.org/mailman/listinfo/catalog-sig -- 'Knowledge is Power' Daniel Greenfeld http://pydanny.com From pydanny at gmail.com Fri Aug 17 20:19:10 2012 From: pydanny at gmail.com (Daniel Greenfeld) Date: Fri, 17 Aug 2012 11:19:10 -0700 Subject: [Catalog-sig] Wheel format now supported In-Reply-To: References: <502E3CE7.7090205@v.loewis.de> Message-ID: Also, I've been told that Windows users have to use easy_install unless they are on cygwin. Is there a patched version of easy_install that supports Wheel? All of this might be moot. This just seems rather sudden and unclear. Daniel Greenfeld On Fri, Aug 17, 2012 at 11:17 AM, Daniel Greenfeld wrote: > Martin, Daniel, > > Will Wheel work with non-patched pip? Will we need to handle multiple > pip versions to get projects running that have the existing format and > this new Wheel format? Will we have to teach incoming developers to > use standard pip and patched pip? How does Wheel work with virtualenv > and buildout? > > I don't mind moving forward, this just isn't very clear to me. > > Danny > > On Fri, Aug 17, 2012 at 10:57 AM, Daniel Holth wrote: >> Uploading a wheel to the cheese shop: >> >> Install wheel (from pypi) >> >> Use setuptools in your setup.py >> >> 'python setup.py bdist_wheel upload' >> >> ? >> >> Profit! >> >> Thanks again for implementing this in pypi. >> >> Daniel >> _______________________________________________ >> Catalog-SIG mailing list >> Catalog-SIG at python.org >> http://mail.python.org/mailman/listinfo/catalog-sig > > > > -- > 'Knowledge is Power' > Daniel Greenfeld > http://pydanny.com -- 'Knowledge is Power' Daniel Greenfeld http://pydanny.com From donald.stufft at gmail.com Fri Aug 17 20:21:19 2012 From: donald.stufft at gmail.com (Donald Stufft) Date: Fri, 17 Aug 2012 14:21:19 -0400 Subject: [Catalog-sig] Wheel format now supported In-Reply-To: References: <502E3CE7.7090205@v.loewis.de> Message-ID: <05E52348A06147F1B9C4B03923CC3113@gmail.com> Pretty sure it will require patching pip, but I believe it'll be added into pip proper in the future. As for windows users I believe they need to use easy_install not because pip doesn't work and easy_install does, but because pip doesn't support any binary formats and getting a compiler setup on Windows is a pain. Wheels should make pip more viable on windows. On Friday, August 17, 2012 at 2:17 PM, Daniel Greenfeld wrote: > Martin, Daniel, > > Will Wheel work with non-patched pip? Will we need to handle multiple > pip versions to get projects running that have the existing format and > this new Wheel format? Will we have to teach incoming developers to > use standard pip and patched pip? How does Wheel work with virtualenv > and buildout? > > I don't mind moving forward, this just isn't very clear to me. > > Danny > > On Fri, Aug 17, 2012 at 10:57 AM, Daniel Holth wrote: > > Uploading a wheel to the cheese shop: > > > > Install wheel (from pypi) > > > > Use setuptools in your setup.py > > > > 'python setup.py bdist_wheel upload' > > > > ? > > > > Profit! > > > > Thanks again for implementing this in pypi. > > > > Daniel > > _______________________________________________ > > Catalog-SIG mailing list > > Catalog-SIG at python.org (mailto:Catalog-SIG at python.org) > > http://mail.python.org/mailman/listinfo/catalog-sig > > > > > > > -- > 'Knowledge is Power' > Daniel Greenfeld > http://pydanny.com > _______________________________________________ > Catalog-SIG mailing list > Catalog-SIG at python.org (mailto:Catalog-SIG at python.org) > http://mail.python.org/mailman/listinfo/catalog-sig > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dholth at gmail.com Fri Aug 17 20:30:36 2012 From: dholth at gmail.com (Daniel Holth) Date: Fri, 17 Aug 2012 14:30:36 -0400 Subject: [Catalog-sig] Wheel format now supported In-Reply-To: References: <502E3CE7.7090205@v.loewis.de> Message-ID: The pip developers are interesting in supporting wheel after their upcoming release. Wheel installations work as long as you are using distribute >= 0.6.28. It works with virtualenv, but no one has tried to write what would surely be called zc.recipe.wheel for buildout. After you install a wheel, it is just a PEP-376 compatible distribution, so it stores its metadata in a .dist-info directory instead of in .egg-info. From martin at v.loewis.de Fri Aug 17 20:37:46 2012 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 17 Aug 2012 20:37:46 +0200 Subject: [Catalog-sig] Wheel format now supported In-Reply-To: References: <502E3CE7.7090205@v.loewis.de> Message-ID: <502E8F7A.9050500@v.loewis.de> On 17.08.2012 20:17, Daniel Greenfeld wrote: > Martin, Daniel, > > Will Wheel work with non-patched pip? I can't say - I just added it because a) Michele Lacchia requested that I do, and b) I cannot see anything wrong with it. Personally, I'm much in dislike of any non-platform distribution format - be they called eggs, gems, wheels, or whatnot. As a PyPI maintainer, I get requests to add new formats every year or two years. At this rate, it seems harmless - only time will decide whether it's useful. I guess in a few years, we can evaluate how often people upload these, and decide whether it would be better to remove the support again. Perhaps they have replaced eggs, perhaps not. Regards, Martin From kencochrane at gmail.com Tue Aug 21 15:41:05 2012 From: kencochrane at gmail.com (ken cochrane) Date: Tue, 21 Aug 2012 09:41:05 -0400 Subject: [Catalog-sig] e.pypi.python.org now in China In-Reply-To: <501A4928.7060609@v.loewis.de> References: <501A4928.7060609@v.loewis.de> Message-ID: It looks like this mirror is a little out of date. Do you have an email address for Aron Xu, so I could let them know? http://www.pypi-mirrors.org Thanks, Ken On Thu, Aug 2, 2012 at 5:32 AM, "Martin v. L?wis" wrote: > Thanks to Aron Xu, e.pypi.python.org is now located in China; > I had to close my own mirror right at the moment when he offered > to provide one. > > Regards, > Martin > _______________________________________________ > Catalog-SIG mailing list > Catalog-SIG at python.org > http://mail.python.org/mailman/listinfo/catalog-sig > -------------- next part -------------- An HTML attachment was scrubbed... URL: From martin at v.loewis.de Tue Aug 21 18:03:18 2012 From: martin at v.loewis.de (martin at v.loewis.de) Date: Tue, 21 Aug 2012 18:03:18 +0200 Subject: [Catalog-sig] Pypi outage Message-ID: <20120821180318.Horde.IODFNruWis5QM7FGcnpTzbA@webmail.df.eu> Tomorrow morning (UTC), there will be another brief PyPI outage. I'll send an announcement when the maintenance is over. Regards, Martin From happyaron.xu at gmail.com Wed Aug 22 05:28:33 2012 From: happyaron.xu at gmail.com (Aron Xu) Date: Wed, 22 Aug 2012 11:28:33 +0800 Subject: [Catalog-sig] e.pypi.python.org now in China In-Reply-To: References: <501A4928.7060609@v.loewis.de> Message-ID: Hi, Our disk was full since the night of the day before yesterday, and we have worked around it. The mirror is fresh now. We are working on another 4TB NAS to fix the disk starvation eventually, sorry for the inconvenience. On Tue, Aug 21, 2012 at 9:41 PM, ken cochrane wrote: > It looks like this mirror is a little out of date. Do you have an email > address for Aron Xu, so I could let them know? > > http://www.pypi-mirrors.org > > Thanks, > Ken > > > On Thu, Aug 2, 2012 at 5:32 AM, "Martin v. L?wis" > wrote: >> >> Thanks to Aron Xu, e.pypi.python.org is now located in China; >> I had to close my own mirror right at the moment when he offered >> to provide one. >> >> Regards, >> Martin >> _______________________________________________ >> Catalog-SIG mailing list >> Catalog-SIG at python.org >> http://mail.python.org/mailman/listinfo/catalog-sig > > > > _______________________________________________ > Catalog-SIG mailing list > Catalog-SIG at python.org > http://mail.python.org/mailman/listinfo/catalog-sig > -- Regards, Aron Xu From martin at v.loewis.de Wed Aug 22 20:52:34 2012 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 22 Aug 2012 20:52:34 +0200 Subject: [Catalog-sig] [Infrastructure] Pypi outage In-Reply-To: <20120821180318.Horde.IODFNruWis5QM7FGcnpTzbA@webmail.df.eu> References: <20120821180318.Horde.IODFNruWis5QM7FGcnpTzbA@webmail.df.eu> Message-ID: <50352A72.2010906@v.loewis.de> On 21.08.2012 18:03, martin at v.loewis.de wrote: > Tomorrow morning (UTC), there will be another brief PyPI outage. > I'll send an announcement when the maintenance is over. I didn't manage to get to it today; will retry tomorrow. Regards, Martin From martin at v.loewis.de Thu Aug 23 09:33:55 2012 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 23 Aug 2012 09:33:55 +0200 Subject: [Catalog-sig] PyPI maintenance over Message-ID: <5035DCE3.3040104@v.loewis.de> I just moved the Postgres database to the new server; everything should be working again. Regards, Martin From martin at v.loewis.de Thu Aug 23 09:49:55 2012 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 23 Aug 2012 09:49:55 +0200 Subject: [Catalog-sig] PyPI migration done In-Reply-To: <20120815182250.GA563@jwilk.net> References: <20120814121833.Horde.KX0sSdjz9kRQKiX50KMwVcA@webmail.df.eu> <20120815182250.GA563@jwilk.net> Message-ID: <5035E0A3.1080506@v.loewis.de> On 15.08.2012 20:22, Jakub Wilk wrote: > It looks like the migration broke SSH access: Indeed it did; this should be fixed now. Regards, Martin