From Gude.Roland at dlr.de Wed Sep 3 11:26:17 2008 From: Gude.Roland at dlr.de (Gude.Roland at dlr.de) Date: Wed, 3 Sep 2008 11:26:17 +0200 Subject: [Catalog-sig] High-availability for PyPI, mirroring infrastructures? Message-ID: <97A3CC5C4C8069449795A84ED88D631BD7318E@exbe01.intra.dlr.de> Hello Everybody, I am new to this list and found the discussion about PyPI Mirrors in the archieves. I am working at the German Aerospace Center where I have just started development of a PyPI mirroring solution which addresses a few issues in QA and Continous Integration we encountered here. For Java/maven based projects we use Sonatype-Nexus (http://nexus.sonatype.org/) Which solves a lot of problems there but does not help with python. It should be able to proxy/mirror any number of package indexes (storing all the files that have been downloaded once.) It should be able to host multiple indexes which are available under different urls and which may or may not restrict access to certain users. It should be possible to combine several mirrors and/or hosted repositories into groups which appear to be a single repository. It should integrate with LDAP/Activedirectory (at some time) Anyway... the project is in early planning and will be available under Apache License 2.0 The project has been registered at Sourceforge just yesterday (https://sourceforge.net/projects/hatchery/) There is nothing there yet, but hopefully that will change soon. If you have specific features you would wish for in a mirroring/hosting solution Please tell me. Discussion and participation is always welcome. Kind regards, Roland Gude From lists at zopyx.com Wed Sep 3 11:51:49 2008 From: lists at zopyx.com (Andreas Jung) Date: Wed, 03 Sep 2008 11:51:49 +0200 Subject: [Catalog-sig] High-availability for PyPI, mirroring infrastructures? In-Reply-To: <97A3CC5C4C8069449795A84ED88D631BD7318E@exbe01.intra.dlr.de> References: <97A3CC5C4C8069449795A84ED88D631BD7318E@exbe01.intra.dlr.de> Message-ID: --On 3. September 2008 11:26:17 +0200 Gude.Roland at dlr.de wrote: > > Anyway... the project is in early planning and will be available under > Apache License 2.0 > The project has been registered at Sourceforge just yesterday > (https://sourceforge.net/projects/hatchery/) > How is it different from out project ? Especially: where is the code? -aj -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 194 bytes Desc: not available URL: From ben at groovie.org Sun Sep 7 08:39:55 2008 From: ben at groovie.org (Ben Bangert) Date: Sat, 6 Sep 2008 23:39:55 -0700 Subject: [Catalog-sig] Hosting documentation on PyPI In-Reply-To: <489A017B.6050708@colorstudy.com> References: <48995F4B.6060409@v.loewis.de> <4899CE06.3060206@colorstudy.com> <4899D508.7030001@v.loewis.de> <4899DB3B.5000100@colorstudy.com> <4899DC03.3010203@colorstudy.com> <4899E3FF.2000600@v.loewis.de> <489A017B.6050708@colorstudy.com> Message-ID: <8DFA3184-B138-453E-9DC3-BCCDDCC42252@groovie.org> On Aug 6, 2008, at 12:54 PM, Ian Bicking wrote: > Also, I'd like the "current" version to be bookmarkable. If every > URL is bound to a version then you can't do that. Actually, the only way you can bookmark is if it includes the version. What's the point of bookmarking URL's that may likely disappear in version 2.0, 3.0, etc? Unless your bookmark is tied to the version, there's no saying if its forever... as the feature/module it documents may be removed, moved, merged elsewhere, etc. - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2507 bytes Desc: not available URL: From skip at pobox.com Sun Sep 14 05:42:14 2008 From: skip at pobox.com (skip at pobox.com) Date: Sat, 13 Sep 2008 22:42:14 -0500 Subject: [Catalog-sig] Access to uploaded documentation? Message-ID: <18636.34838.200049.343500@montanaro-dyndns-org.local> I noticed that you can upload documentation for packages in PyPI, so I took advantage of that for the lockfile module: http://pypi.python.org/pypi/lockfile http://packages.python.org/lockfile/ I don't see any links to the latter URL at the former though. How are users supposed to know documentation is available online? I don't see a link on the lockfile page. Thx, Skip From lists at zopyx.com Sun Sep 14 08:20:31 2008 From: lists at zopyx.com (Andreas Jung) Date: Sun, 14 Sep 2008 08:20:31 +0200 Subject: [Catalog-sig] Access to uploaded documentation? In-Reply-To: <18636.34838.200049.343500@montanaro-dyndns-org.local> References: <18636.34838.200049.343500@montanaro-dyndns-org.local> Message-ID: <2D134BEF6316B04D0E9654B3@[192.168.0.24]> --On 13. September 2008 22:42:14 -0500 skip at pobox.com wrote: > I noticed that you can upload documentation for packages in PyPI, so I > took advantage of that for the lockfile module: > > http://pypi.python.org/pypi/lockfile > http://packages.python.org/lockfile/ > > I don't see any links to the latter URL at the former though. How are > users supposed to know documentation is available online? I don't see a > link on the lockfile page. We have the 'long_description' field of the metadata for providing documentation. See the various zope.* packages as an example. -aj -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 194 bytes Desc: not available URL: From gary at modernsongs.com Sun Sep 14 19:17:56 2008 From: gary at modernsongs.com (Gary Poster) Date: Sun, 14 Sep 2008 13:17:56 -0400 Subject: [Catalog-sig] Trouble changing my email address Message-ID: <5A2A7077-281B-4727-9DA1-8F51879521F7@modernsongs.com> Hi. I need to change my email address on PyPI but I'm having problems. I log in, click on "Your Details" on the top right, and try to change my email address. If I try to just change my email address, I get this error message: ----- Error processing form password is required ----- If I then enter my password (in addition to my email change), I get this error message: ----- User registration user "gary" already exists ------ Could someone help? Thank you Gary From flo at chaoflow.net Mon Sep 15 14:38:07 2008 From: flo at chaoflow.net (Florian Friesdorf) Date: Mon, 15 Sep 2008 14:38:07 +0200 Subject: [Catalog-sig] APGL classifier is missing Message-ID: <20080915123807.GA30791@marvin> Hi, the classifier: License :: OSI Approved :: GNU Affero General Public License (AGPL) is missing. florian -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: not available URL: From chris at simplistix.co.uk Tue Sep 30 17:01:01 2008 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 30 Sep 2008 16:01:01 +0100 Subject: [Catalog-sig] [Distutils] PEP for distutils In-Reply-To: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com> References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com> Message-ID: <48E23F2D.3000802@simplistix.co.uk> Tarek Ziad? wrote: > I started to write a new PEP (well a wiki page in fact...) that > describes a new package called "pypi" that would be dedicated to package > registering and uploading mechanisms. > It would also provide enhancements like a proper password hash, or > deepers metadata controls > > http://wiki.python.org/moin/A_new_pypi_module > > Any opinions about this PEP ? I tried to include all the problems people > are having with register and upload. I think that catalog-sig would be interested in this. That said, I didn't see any indication of what I consider to be a critical failure in PyPI: No dependency metadata prior to downloading the package. cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk From ianb at colorstudy.com Tue Sep 30 17:41:11 2008 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 30 Sep 2008 10:41:11 -0500 Subject: [Catalog-sig] [Distutils] PEP for distutils In-Reply-To: <20080930153822.GJ26878@phare.normalesup.org> References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com> <48E23F2D.3000802@simplistix.co.uk> <20080930153822.GJ26878@phare.normalesup.org> Message-ID: <48E24897.8010800@colorstudy.com> Gael Varoquaux wrote: > On Tue, Sep 30, 2008 at 04:01:01PM +0100, Chris Withers wrote: >> That said, I didn't see any indication of what I consider to be a critical >> failure in PyPI: No dependency metadata prior to downloading the package. > > +1. I want to be able do list all the packages an easy_install run will > download without running it. Something like the "-s" option of apt-get. > In addition, I want this information to be available programmatically (ie > with a good api, not something that expects to be called from the command > line) to be able to use it to build dependency graphs, generate conflicts > list, or simply tell me that I have requested something that is > impossible. > > There is nothing that I hate more than easy_install failing after having > half-installed a package because of a missing dependency. This is one of > the reasons I am never too happy when I have to run easy_install. FWIW, pyinstall can collect all the packages before installing any of them. You do have to download all packages, though, as that's the only way to get the metadata. -- Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org From ziade.tarek at gmail.com Tue Sep 30 17:43:51 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 30 Sep 2008 17:43:51 +0200 Subject: [Catalog-sig] [Distutils] PEP for distutils In-Reply-To: <48E24897.8010800@colorstudy.com> References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com> <48E23F2D.3000802@simplistix.co.uk> <20080930153822.GJ26878@phare.normalesup.org> <48E24897.8010800@colorstudy.com> Message-ID: <94bdd2610809300843r516851aasdca03f7ce60878f9@mail.gmail.com> On Tue, Sep 30, 2008 at 5:41 PM, Ian Bicking wrote: > Gael Varoquaux wrote: >> >> On Tue, Sep 30, 2008 at 04:01:01PM +0100, Chris Withers wrote: >>> >>> That said, I didn't see any indication of what I consider to be a >>> critical failure in PyPI: No dependency metadata prior to downloading the >>> package. >> >> +1. I want to be able do list all the packages an easy_install run will >> download without running it. Something like the "-s" option of apt-get. >> In addition, I want this information to be available programmatically (ie >> with a good api, not something that expects to be called from the command >> line) to be able to use it to build dependency graphs, generate conflicts >> list, or simply tell me that I have requested something that is >> impossible. >> >> There is nothing that I hate more than easy_install failing after having >> half-installed a package because of a missing dependency. This is one of >> the reasons I am never too happy when I have to run easy_install. > > FWIW, pyinstall can collect all the packages before installing any of them. > You do have to download all packages, though, as that's the only way to get > the metadata. Yes, so having them at PyPI would be a good idea indeed, I am adding that to that small PEP > > > -- > Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ From jp.camguilhem at gmail.com Tue Sep 30 17:53:22 2008 From: jp.camguilhem at gmail.com (Jean-Philippe CAMGUILHEM) Date: Tue, 30 Sep 2008 17:53:22 +0200 Subject: [Catalog-sig] [Distutils] PEP for distutils In-Reply-To: <48E24897.8010800@colorstudy.com> References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com> <48E23F2D.3000802@simplistix.co.uk> <20080930153822.GJ26878@phare.normalesup.org> <48E24897.8010800@colorstudy.com> Message-ID: On Tue, Sep 30, 2008 at 5:41 PM, Ian Bicking wrote: > Gael Varoquaux wrote: > >> On Tue, Sep 30, 2008 at 04:01:01PM +0100, Chris Withers wrote: >> >>> That said, I didn't see any indication of what I consider to be a >>> critical failure in PyPI: No dependency metadata prior to downloading the >>> package. >>> >> >> +1. I want to be able do list all the packages an easy_install run will >> download without running it. Something like the "-s" option of apt-get. >> In addition, I want this information to be available programmatically (ie >> with a good api, not something that expects to be called from the command >> line) to be able to use it to build dependency graphs, generate conflicts >> list, or simply tell me that I have requested something that is >> impossible. >> >> There is nothing that I hate more than easy_install failing after having >> half-installed a package because of a missing dependency. This is one of >> the reasons I am never too happy when I have to run easy_install. >> > > FWIW, pyinstall can collect all the packages before installing any of them. > You do have to download all packages, though, as that's the only way to get > the metadata. is a "simple catalog "db storage for metadata like /usr/ports/ on freebsd a bad idea ? the idea is to not download all packages to get the metadata, but just query the catalog/db Cheers, Jean-Philippe > > > > -- > Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris at simplistix.co.uk Tue Sep 30 18:13:18 2008 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 30 Sep 2008 17:13:18 +0100 Subject: [Catalog-sig] [Distutils] PEP for distutils In-Reply-To: <48E24897.8010800@colorstudy.com> References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com> <48E23F2D.3000802@simplistix.co.uk> <20080930153822.GJ26878@phare.normalesup.org> <48E24897.8010800@colorstudy.com> Message-ID: <48E2501E.90202@simplistix.co.uk> Ian Bicking wrote: > FWIW, pyinstall can collect all the packages before installing any of > them. You do have to download all packages, though, as that's the only > way to get the metadata. ...yes, and this is why PyPI should change! Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk From amk at amk.ca Tue Sep 30 18:25:59 2008 From: amk at amk.ca (A.M. Kuchling) Date: Tue, 30 Sep 2008 12:25:59 -0400 Subject: [Catalog-sig] [Distutils] PEP for distutils In-Reply-To: <48E24897.8010800@colorstudy.com> References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com> <48E23F2D.3000802@simplistix.co.uk> <20080930153822.GJ26878@phare.normalesup.org> <48E24897.8010800@colorstudy.com> Message-ID: <20080930162559.GA11804@amk-desktop.matrixgroup.net> On Tue, Sep 30, 2008 at 10:41:11AM -0500, Ian Bicking wrote: > FWIW, pyinstall can collect all the packages before installing any of > them. You do have to download all packages, though, as that's the only > way to get the metadata. Does the DOAP output for a package not contain enough metadata? --amk From gael.varoquaux at normalesup.org Tue Sep 30 17:38:22 2008 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Tue, 30 Sep 2008 17:38:22 +0200 Subject: [Catalog-sig] [Distutils] PEP for distutils In-Reply-To: <48E23F2D.3000802@simplistix.co.uk> References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com> <48E23F2D.3000802@simplistix.co.uk> Message-ID: <20080930153822.GJ26878@phare.normalesup.org> On Tue, Sep 30, 2008 at 04:01:01PM +0100, Chris Withers wrote: > That said, I didn't see any indication of what I consider to be a critical > failure in PyPI: No dependency metadata prior to downloading the package. +1. I want to be able do list all the packages an easy_install run will download without running it. Something like the "-s" option of apt-get. In addition, I want this information to be available programmatically (ie with a good api, not something that expects to be called from the command line) to be able to use it to build dependency graphs, generate conflicts list, or simply tell me that I have requested something that is impossible. There is nothing that I hate more than easy_install failing after having half-installed a package because of a missing dependency. This is one of the reasons I am never too happy when I have to run easy_install. Ga?l From ianb at colorstudy.com Tue Sep 30 18:37:01 2008 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 30 Sep 2008 11:37:01 -0500 Subject: [Catalog-sig] [Distutils] PEP for distutils In-Reply-To: <20080930162559.GA11804@amk-desktop.matrixgroup.net> References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com> <48E23F2D.3000802@simplistix.co.uk> <20080930153822.GJ26878@phare.normalesup.org> <48E24897.8010800@colorstudy.com> <20080930162559.GA11804@amk-desktop.matrixgroup.net> Message-ID: <48E255AD.7020408@colorstudy.com> A.M. Kuchling wrote: > On Tue, Sep 30, 2008 at 10:41:11AM -0500, Ian Bicking wrote: >> FWIW, pyinstall can collect all the packages before installing any of >> them. You do have to download all packages, though, as that's the only >> way to get the metadata. > > Does the DOAP output for a package not contain enough metadata? No. It probably could hold that information, but currently PyPI doesn't keep any record of requirements, and so the DOAP file it generates doesn't indicate requirements either. -- Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org From ziade.tarek at gmail.com Tue Sep 30 19:13:03 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 30 Sep 2008 19:13:03 +0200 Subject: [Catalog-sig] [Distutils] "just use debian" In-Reply-To: <48E255B9.50806@colorstudy.com> References: <20080917171851.GA27261@logilab.fr> <94bdd2610809300649q6bc9cdf3t84d5221d956bfc9d@mail.gmail.com> <1222784868.4166.88.camel@shizuru> <94bdd2610809300820g36a3f1bbpc2a04e03b43ee9c9@mail.gmail.com> <48E24765.1020908@simplistix.co.uk> <48E24BDA.2050104@simplistix.co.uk> <94bdd2610809300901o3fa62601q985784f01a8ad774@mail.gmail.com> <48E24FF7.9000300@simplistix.co.uk> <48E255B9.50806@colorstudy.com> Message-ID: <94bdd2610809301013m5d07865eu7eabc4cb3b62d354@mail.gmail.com> On Tue, Sep 30, 2008 at 6:37 PM, Ian Bicking wrote: > Chris Withers wrote: >> >> Tarek Ziad? wrote: >>>> >>>> Tarek Ziade wrote: >>>>> >>>>> For KGS I agree that this is a big work, but there's the need to work >>>>> at a >>>>> higher level that in your package >>>> >>>> Why? You really need to explain to me why the dependency information in >>>> each >>>> of the packages isn't enough? >>> >>> Because you can keep up with the dependencies changes, removed, or >>> introduced >>> by a package you depend on. >> >> Why can this not be expressed in the dependency information in the >> package? > > I tried this briefly for a while when Setuptools first came out, and I found > it completely unmaintainable. > > Say I have a package that represents an application. We'll call it FooBlog. > I release version 1.0. It uses the Turplango web framework (1.5 at the > time of release) and the Storchalmy ORM (0.4), and Turplango uses HardJSON > (1.2.1). > > I want my version 1.0 to keep working. So, I figure I'll add the > dependencies: > > Turplango==1.5 > Storchalmy==0.4 > > Then HardJSON 2.0 is released, and Turplango only required HardJSON>=1.2, so > new installations start installing HardJSON 2.0. But my app happens not to > be compatible with that library, and so it's broken. OK... so, I could add > HardJSON==1.2.1 in my requirements. > > But then a small bug fix, HardJSON 1.2.2 comes out, that fixes a security > bug. Turplango releases version 1.5.1 that requires HardJSON>=1.2.2. I now > have have to update FooBlog to require both Turplango==1.5.1 and > HardJSON==1.2.2. > > Later on, I decide that Turplango 1.6 fixes some important bugs, and I want > to try it with my app. I can install Turplango 1.6, but I can't start my > app because I'll get a version conflict. So to even experiment with a new > version of the app, I have to check out FooBlog, update setup.py, reinstall > (setup.py develop) the package, and then I can start using it. But if I've > made other hard requirements of packages like HardJSON, I'll have to update > all those too. > > So... that's the kind of thing I encountered with just a couple > dependencies, but in practice it was much worse because there were a lot > more than 3 libraries involved. I now think it is best to only use version > requirements to express known conflicts. For future versions of packages > you can't really know if they will cause conflicts until they are released. Exactly, you can't control everything from your package unless you work in an isolated environement like virtualenv or zc.buildout provides, so I can't see any solution unless someone is taking care of it at a higher level :( maybe PyPI though, can automate this, when a package is uploaded, by browsing all dependency and finding relevant conflict ? PyPI "knows" all the packages out there. At least display those conflicts somehow ? or warn about them. (I am pushing this to catalog-sig as well, sorry for the cross-post. I do thing though, these mailing lists should merge) > > > -- > Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org > -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ From ianb at colorstudy.com Tue Sep 30 19:25:17 2008 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 30 Sep 2008 12:25:17 -0500 Subject: [Catalog-sig] [Distutils] "just use debian" In-Reply-To: <94bdd2610809301013m5d07865eu7eabc4cb3b62d354@mail.gmail.com> References: <20080917171851.GA27261@logilab.fr> <94bdd2610809300649q6bc9cdf3t84d5221d956bfc9d@mail.gmail.com> <1222784868.4166.88.camel@shizuru> <94bdd2610809300820g36a3f1bbpc2a04e03b43ee9c9@mail.gmail.com> <48E24765.1020908@simplistix.co.uk> <48E24BDA.2050104@simplistix.co.uk> <94bdd2610809300901o3fa62601q985784f01a8ad774@mail.gmail.com> <48E24FF7.9000300@simplistix.co.uk> <48E255B9.50806@colorstudy.com> <94bdd2610809301013m5d07865eu7eabc4cb3b62d354@mail.gmail.com> Message-ID: <48E260FD.9010309@colorstudy.com> Tarek Ziad? wrote: >> So... that's the kind of thing I encountered with just a couple >> dependencies, but in practice it was much worse because there were a lot >> more than 3 libraries involved. I now think it is best to only use version >> requirements to express known conflicts. For future versions of packages >> you can't really know if they will cause conflicts until they are released. > > Exactly, you can't control everything from your package unless you > work in an isolated environement like virtualenv or zc.buildout > provides, so I can't see any solution unless someone is taking care of > it at a higher level :( > > maybe PyPI though, can automate this, when a package is uploaded, by > browsing all dependency and > finding relevant conflict ? PyPI "knows" all the packages out there. > > At least display those conflicts somehow ? or warn about them. Yes, keeping this version information separate from packages would help, I think. If you find out more information about a conflict it shouldn't require a new release -- new releases take a while to do, and have cascading effects. This kind of metadata isn't so much about the package, as about how the package relates to other packages. If we could somewhat safely have collaborative conflict information that would be nice, though there's different kinds of conflicts so it might be infeasible. It's all too common for a person to just poke around with version stuff until something works, but in a way that is only accurate for the context of their application, and if they submit that information upstream they could easily break other people's setups unnecessarily. -- Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org From gael.varoquaux at normalesup.org Tue Sep 30 19:38:26 2008 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Tue, 30 Sep 2008 19:38:26 +0200 Subject: [Catalog-sig] [Distutils] PEP for distutils In-Reply-To: <48E24897.8010800@colorstudy.com> References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com> <48E23F2D.3000802@simplistix.co.uk> <20080930153822.GJ26878@phare.normalesup.org> <48E24897.8010800@colorstudy.com> Message-ID: <20080930173826.GL26878@phare.normalesup.org> On Tue, Sep 30, 2008 at 10:41:11AM -0500, Ian Bicking wrote: >> There is nothing that I hate more than easy_install failing after having >> half-installed a package because of a missing dependency. This is one of >> the reasons I am never too happy when I have to run easy_install. > > FWIW, pyinstall can collect all the packages before installing any of them. > You do have to download all packages, though, as that's the only way to > get the metadata. Yes, I have seen that. I was very happy to witness the release of this tool. Thank you. Ga?l From pje at telecommunity.com Tue Sep 30 19:51:30 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 30 Sep 2008 13:51:30 -0400 Subject: [Catalog-sig] [Distutils] PEP for distutils In-Reply-To: <20080930162559.GA11804@amk-desktop.matrixgroup.net> References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com> <48E23F2D.3000802@simplistix.co.uk> <20080930153822.GJ26878@phare.normalesup.org> <48E24897.8010800@colorstudy.com> <20080930162559.GA11804@amk-desktop.matrixgroup.net> Message-ID: <20080930175201.106863A409C@sparrow.telecommunity.com> At 12:25 PM 9/30/2008 -0400, A.M. Kuchling wrote: >On Tue, Sep 30, 2008 at 10:41:11AM -0500, Ian Bicking wrote: > > FWIW, pyinstall can collect all the packages before installing any of > > them. You do have to download all packages, though, as that's the only > > way to get the metadata. > >Does the DOAP output for a package not contain enough metadata? Nope. And it can't possibly do so, unless it contains dependency data for every possible variation of the package. For example, a package might dynamically declare dependency on ctypes, depending on whether you're installing it for Python 2.4 or Python 2.5. (Dependencies can also be platform-specific and build-option-specific, as well as Python-version-specific.) From cakebread at gmail.com Tue Sep 30 20:21:54 2008 From: cakebread at gmail.com (Rob Cakebread) Date: Tue, 30 Sep 2008 11:21:54 -0700 Subject: [Catalog-sig] [Distutils] PEP for distutils In-Reply-To: <20080930175201.106863A409C@sparrow.telecommunity.com> References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com> <48E23F2D.3000802@simplistix.co.uk> <20080930153822.GJ26878@phare.normalesup.org> <48E24897.8010800@colorstudy.com> <20080930162559.GA11804@amk-desktop.matrixgroup.net> <20080930175201.106863A409C@sparrow.telecommunity.com> Message-ID: <9b06ffb10809301121r589e1784je1300fb2632ab46@mail.gmail.com> On Tue, Sep 30, 2008 at 10:51 AM, Phillip J. Eby wrote: > At 12:25 PM 9/30/2008 -0400, A.M. Kuchling wrote: >> >> On Tue, Sep 30, 2008 at 10:41:11AM -0500, Ian Bicking wrote: >> > FWIW, pyinstall can collect all the packages before installing any of >> > them. You do have to download all packages, though, as that's the only >> > way to get the metadata. >> >> Does the DOAP output for a package not contain enough metadata? > > Nope. And it can't possibly do so, unless it contains dependency data for > every possible variation of the package. For example, a package might > dynamically declare dependency on ctypes, depending on whether you're > installing it for Python 2.4 or Python 2.5. (Dependencies can also be > platform-specific and build-option-specific, as well as > Python-version-specific.) > Not to mention the DOAP vocabulary lacks a way to describe dependency information. This is planned but it has to be well thought out because of all the variations Philip mentions. The good news is much of this dependency info is already in existence in Linux distributions. Take a Gentoo ebuild, for example. It has separate run-time, build-time and test dependency info, dependencies based on enabled features, and dependencies based on the version of Python used. Ebuilds also have metadata mapping the PyPI name to the Gentoo package name, so it'll be easy enough to create a database with all this info. I'm working on this now at http://doapspace.org/ where you can find DOAP for Python packages with a bit more metadata than the DOAP supplied on PyPI ( http://doapspace.org/doap/py/PYPI_PKG_NAME ) From martin at v.loewis.de Tue Sep 30 22:08:25 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 30 Sep 2008 22:08:25 +0200 Subject: [Catalog-sig] [Distutils] PEP for distutils In-Reply-To: <48E23F2D.3000802@simplistix.co.uk> References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com> <48E23F2D.3000802@simplistix.co.uk> Message-ID: <48E28739.7080901@v.loewis.de> > That said, I didn't see any indication of what I consider to be a > critical failure in PyPI: No dependency metadata prior to downloading > the package. Contributions are welcome. The source code of PyPI is available publically, and I'm willing to accept patches. I won't have time to work on this in the next 12 months myself. Regards, Martin From chris at simplistix.co.uk Tue Sep 30 22:26:35 2008 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 30 Sep 2008 21:26:35 +0100 Subject: [Catalog-sig] [Distutils] PEP for distutils In-Reply-To: <48E28739.7080901@v.loewis.de> References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com> <48E23F2D.3000802@simplistix.co.uk> <48E28739.7080901@v.loewis.de> Message-ID: <48E28B7B.3060602@simplistix.co.uk> Martin v. L?wis wrote: >> That said, I didn't see any indication of what I consider to be a >> critical failure in PyPI: No dependency metadata prior to downloading >> the package. > > Contributions are welcome. The source code of PyPI is available > publically, Where? > and I'm willing to accept patches. I won't have time > to work on this in the next 12 months myself. These two don't seem to go hand in hand and don't really seem to fit my experiene of the catalog-sig :-( Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk From ziade.tarek at gmail.com Tue Sep 30 22:41:12 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 30 Sep 2008 22:41:12 +0200 Subject: [Catalog-sig] [Distutils] PEP for distutils In-Reply-To: <9b06ffb10809301121r589e1784je1300fb2632ab46@mail.gmail.com> References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com> <48E23F2D.3000802@simplistix.co.uk> <20080930153822.GJ26878@phare.normalesup.org> <48E24897.8010800@colorstudy.com> <20080930162559.GA11804@amk-desktop.matrixgroup.net> <20080930175201.106863A409C@sparrow.telecommunity.com> <9b06ffb10809301121r589e1784je1300fb2632ab46@mail.gmail.com> Message-ID: <94bdd2610809301341j60830399s37cf310939783b86@mail.gmail.com> On Tue, Sep 30, 2008 at 8:21 PM, Rob Cakebread wrote: > On Tue, Sep 30, 2008 at 10:51 AM, Phillip J. Eby wrote: >> At 12:25 PM 9/30/2008 -0400, A.M. Kuchling wrote: >>> >>> On Tue, Sep 30, 2008 at 10:41:11AM -0500, Ian Bicking wrote: >>> > FWIW, pyinstall can collect all the packages before installing any of >>> > them. You do have to download all packages, though, as that's the only >>> > way to get the metadata. >>> >>> Does the DOAP output for a package not contain enough metadata? >> >> Nope. And it can't possibly do so, unless it contains dependency data for >> every possible variation of the package. For example, a package might >> dynamically declare dependency on ctypes, depending on whether you're >> installing it for Python 2.4 or Python 2.5. (Dependencies can also be >> platform-specific and build-option-specific, as well as >> Python-version-specific.) >> > > Not to mention the DOAP vocabulary lacks a way to describe dependency > information. This is planned but it has to be well thought out because of > all the variations Philip mentions. out of curiosity, : - can a RDF-based database can possibly handle such a graph ? - would it make sense for PyPI to query the doap server to get those dependency infos ? - > > The good news is much of this dependency info is already in existence in > Linux distributions. Take a Gentoo ebuild, for example. It has separate > run-time, build-time and test dependency info, dependencies based on > enabled features, and dependencies based on the version of Python used. > > Ebuilds also have metadata mapping the PyPI name to the Gentoo > package name, so it'll be easy enough to create a database with all this info. > > I'm working on this now at http://doapspace.org/ where you can find DOAP > for Python packages with a bit more metadata than the DOAP supplied on > PyPI ( http://doapspace.org/doap/py/PYPI_PKG_NAME ) > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ From martin at v.loewis.de Tue Sep 30 22:49:12 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 30 Sep 2008 22:49:12 +0200 Subject: [Catalog-sig] [Distutils] PEP for distutils In-Reply-To: <48E28B7B.3060602@simplistix.co.uk> References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com> <48E23F2D.3000802@simplistix.co.uk> <48E28739.7080901@v.loewis.de> <48E28B7B.3060602@simplistix.co.uk> Message-ID: <48E290C8.4030008@v.loewis.de> Chris Withers wrote: >> Contributions are welcome. The source code of PyPI is available >> publically, > > Where? https://svn.python.org/packages/trunk/ >> and I'm willing to accept patches. I won't have time >> to work on this in the next 12 months myself. > > These two don't seem to go hand in hand and don't really seem to fit my > experiene of the catalog-sig :-( Saying what? that I do have time in the next twelve months? That's nice to hear. Regards, Martin