From richardjones at optushome.com.au Sun Jan 2 23:47:35 2005 From: richardjones at optushome.com.au (Richard Jones) Date: Sun Jan 2 23:47:44 2005 Subject: [Catalog-sig] PyCon sprint for PEPs 314 and 243? In-Reply-To: <14E239E4-5B11-11D9-95EA-000A9567635C@redivi.com> References: <200412301535.34522.richardjones@optushome.com.au> <14E239E4-5B11-11D9-95EA-000A9567635C@redivi.com> Message-ID: <200501030947.39857.richardjones@optushome.com.au> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Fri, 31 Dec 2004 08:48 pm, Bob Ippolito wrote: > I'm interested in helping this along. Similar to your revisions with > PEP-243, here's what I think should happen to PEP 314: > [python module names only as provides/requires] This seems reasonable to me. In particular: > Requires should not be arbitrary. I really can't see a reasonable situation where anything *other* than this could be the case (racking my brain to try to think of *any* python module / package which could be replaced with another, retaining API compatibility). The current list of changes to PEP 314 version 1.15 are: * Platform and License need to be altered to indicate they take text describing the platform/full license text in the case where there is no appropriate Classifier. * Requires/Provides need to specify *only* python package or module names. Abstract names are not considered useful. * Conflicts, on the other hand, probably *does* need to specify actual distutils package names, rather than python package names or module names, as the latter can be determined from the Provides field. I've had another glance at DOAP, but I think I'd need to dedicate at least half a day to it, which I don't have at the moment. Is anyone up to speed enough with it to give a summary of what we'd need to change to become compatible? Or does the patch against PyPI: http://sourceforge.net/tracker/index.php?func=detail&aid=1079962&group_id=66150&atid=513503 and its comments capture it all? Richard -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) iD8DBQFB2HoLrGisBEHG6TARAugSAJ9QCbr2nJCeNwEm8sBd2fNS3QmvWgCdEyd7 uZ/67mfzzfWnx3kYCcr4HO4= =09AP -----END PGP SIGNATURE----- From bob at redivi.com Mon Jan 3 00:13:56 2005 From: bob at redivi.com (Bob Ippolito) Date: Mon Jan 3 01:11:48 2005 Subject: [Catalog-sig] PyCon sprint for PEPs 314 and 243? In-Reply-To: <200501030947.39857.richardjones@optushome.com.au> References: <200412301535.34522.richardjones@optushome.com.au> <14E239E4-5B11-11D9-95EA-000A9567635C@redivi.com> <200501030947.39857.richardjones@optushome.com.au> Message-ID: On Jan 2, 2005, at 5:47 PM, Richard Jones wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Fri, 31 Dec 2004 08:48 pm, Bob Ippolito wrote: >> I'm interested in helping this along. Similar to your revisions with >> PEP-243, here's what I think should happen to PEP 314: >> [python module names only as provides/requires] > > This seems reasonable to me. In particular: > > >> Requires should not be arbitrary. > > I really can't see a reasonable situation where anything *other* than > this > could be the case (racking my brain to try to think of *any* python > module / > package which could be replaced with another, retaining API > compatibility). > > The current list of changes to PEP 314 version 1.15 are: > > * Platform and License need to be altered to indicate they take text > describing the platform/full license text in the case where there is no > appropriate Classifier. > * Requires/Provides need to specify *only* python package or module > names. > Abstract names are not considered useful. > * Conflicts, on the other hand, probably *does* need to specify actual > distutils package names, rather than python package names or module > names, as > the latter can be determined from the Provides field. I'm unconvinced that Conflicts does anything useful at all. It's obvious that installing two packages with the same module name is a conflict (probably an upgrade). Anything else couldn't possibly conflict in any concrete way in the same way that Provides and Requires can't mean anything useful without specifying actual module names. I say get rid of Conflicts altogether. -bob From jafo at tummy.com Sun Jan 16 01:23:22 2005 From: jafo at tummy.com (Sean Reifschneider) Date: Sun Jan 16 01:23:39 2005 Subject: [Catalog-sig] PyCon sprint for PEPs 314 and 243? In-Reply-To: <200412312028.35688.richardjones@optushome.com.au> References: <200412301556.28923.richardjones@optushome.com.au> <20041230171827.GA11064@tummy.com> <200412312028.35688.richardjones@optushome.com.au> Message-ID: <20050116002322.GB2701@tummy.com> On Fri, Dec 31, 2004 at 08:28:35PM +1100, Richard Jones wrote: >Without looking into it too closely, it seems like yum is closely tied to RPMs >and their associated package databases. I would suspect that the code for handling mirror lists and downloading of files is not at all tied to the RPM file format. That code has had something like a year of development on it, and probably can be used with few changes. There are 3 different downloaded file types: RPM packages, RPM headers, and the repository information. The repository information would probably require some tweeks, and the RPM packages and headers would need to be replaced with something else. Unless we were to decide to use RPM as the package format. Which is probably also not a bad idea. >Also, I'm not sure I want to try to solve dependencies. I'm not sure you have a choice not to. I'm also not sure it's as hard a problem as some people seem to think. Red Hat and Debian have fairly good solutions to this problem with years and years of thought behind them. Sean -- Like its politicians and its wars, society has the teenagers it deserves. -- J. B. Priestley Sean Reifschneider, Member of Technical Staff tummy.com, ltd. - Linux Consulting since 1995. Qmail, Python, SysAdmin From bob at redivi.com Sun Jan 16 01:53:51 2005 From: bob at redivi.com (Bob Ippolito) Date: Sun Jan 16 04:27:43 2005 Subject: [Catalog-sig] PyCon sprint for PEPs 314 and 243? In-Reply-To: <20050116002322.GB2701@tummy.com> References: <200412301556.28923.richardjones@optushome.com.au> <20041230171827.GA11064@tummy.com> <200412312028.35688.richardjones@optushome.com.au> <20050116002322.GB2701@tummy.com> Message-ID: <16C0F668-6759-11D9-BFCD-000A95BA5446@redivi.com> On Jan 15, 2005, at 19:23, Sean Reifschneider wrote: > On Fri, Dec 31, 2004 at 08:28:35PM +1100, Richard Jones wrote: >> Without looking into it too closely, it seems like yum is closely >> tied to RPMs >> and their associated package databases. > > I would suspect that the code for handling mirror lists and > downloading of > files is not at all tied to the RPM file format. That code has had > something like a year of development on it, and probably can be used > with > few changes. Well, yum is GPL and therefore not suitable for inclusion into Python. It doesn't really matter if the code is already written in Python and implements some useful things, unless you're certain that you can get those components relicensed to suit Python. > There are 3 different downloaded file types: RPM packages, RPM > headers, and > the repository information. The repository information would probably > require some tweeks, and the RPM packages and headers would need to be > replaced with something else. Unless we were to decide to use RPM as > the > package format. Which is probably also not a bad idea. Is the RPM format itself well specified? Is there already a suitably licensed pure Python implementation for creating and using RPM files? Platforms that don't already have RPM are going to need it, and I believe that librpm is GPL (and probably also not portable everywhere Python is), so we couldn't add that dependency for an extension to wrap this functionality. >> Also, I'm not sure I want to try to solve dependencies. > > I'm not sure you have a choice not to. I'm also not sure it's as hard > a > problem as some people seem to think. Red Hat and Debian have fairly > good > solutions to this problem with years and years of thought behind them. Dependency tracking and resolution requires package maintainers. I think we'd be better off deferring the implementation of this, because it would take more effort to get this solution out the door and it would require an additional burden to make these packages. If PyPI had tried to solve packaging, distribution, dependency tracking, etc. from the start, it would've never finished. I don't think Python is really ready for this kind of system anyway. I'd rather try and fix the fundamental problems of versioning and module namespaces in Python itself (which are probably 90% policy and 10% implementation). I also find the Red Hat / Debian solution to *this* issue quite obnoxious in that you often have to install or upgrade more packages than should be necessary. -bob From jafo at tummy.com Mon Jan 17 01:17:18 2005 From: jafo at tummy.com (Sean Reifschneider) Date: Mon Jan 17 01:17:44 2005 Subject: [Catalog-sig] PyCon sprint for PEPs 314 and 243? In-Reply-To: <16C0F668-6759-11D9-BFCD-000A95BA5446@redivi.com> References: <200412301556.28923.richardjones@optushome.com.au> <20041230171827.GA11064@tummy.com> <200412312028.35688.richardjones@optushome.com.au> <20050116002322.GB2701@tummy.com> <16C0F668-6759-11D9-BFCD-000A95BA5446@redivi.com> Message-ID: <20050117001718.GW4411@tummy.com> On Sat, Jan 15, 2005 at 07:53:51PM -0500, Bob Ippolito wrote: >Is the RPM format itself well specified? Yes. At it's heart it's just a cpio archive with some extra information. >Is there already a suitably >licensed pure Python implementation for creating and using RPM files? Doubtful. However, it's still valuable to use a mature package format, despite lack of existing code, as it prevents you from having to design and build a new format. .deb and .rpm have years of experience under their belts, including ability to be relocated, tools to build and manipulate them, signatures in the case of .rpm, dependencies, etc... I'd recommend .rpms primarily because they have the ability to be signed. >Platforms that don't already have RPM are going to need it, and I >believe that librpm is GPL (and probably also not portable everywhere >Python is), so we couldn't add that dependency for an extension to wrap >this functionality. Personally, I wouldn't use a system that didn't integrate with my system package format. Instead I'd prefer "python setup.py bdist_rpm" or bdist_deb. That's why I was always more interested in solving the problem of distributing these packages as opposed to building a new package format. The process of building the catalot system seems to get mired down in issues like handling all platforms and allowing non-root users to install packages outside of the system package location. >Dependency tracking and resolution requires package maintainers. I don't see how having dependencies requires that much more than the maintenance for a setup.py. In the worst case, it doesn't hurt anything to have dependencies that nobody uses. If we already have a package database, then we already know if "optik" is installed, for example, so having a package say that it requires "optik" isn't a huge deal. >From a user perspective, lacking dependencies is a huge problem. When you want to install one package which requires another, it's far better if the system can determine this and pull in the required other packages. If you'd like an example of just how big a deal users consider this, look at pretty much any discussion related to RPMs and "dependency hell". These are almost always discussions from people using "rpm" directly to install packages (which at least alerts on dependencies, but doesn't automatically install dependencies). Any discussion mentioning that Debian packages are better than RPMs is also likely about this, because "apt-get" (a layer on top of "dpkg") follows and downloads dependencies where "dpkg" and "rpm" do not. >I think we'd be better off deferring the implementation of this, because >it would take more effort to get this solution out the door and it >would require an additional burden to make these packages. It wouldn't *REQUIRE* the burden, because you can always release packages that simply do not list their dependencies. >If PyPI had >tried to solve packaging, distribution, dependency tracking, etc. from >the start, it would've never finished. That's certainly not true, as proof of it I submit "yum" which does solve all these problems, using Python, and *HAS* made progress to a usable state. I know something of what I speak about here, since I took the system which I demonstrated at Python 9 in 2001, tweeked it a bit over the next month, and turned it into a system for Red Hat package dependency tracking, installation, and downloading, which worked extremely well for our users. The difficulty with applying this solution to Python was entirely political, not at all technical. It boiled down to this: in order to reach critical mass, I was certain that we needed an easy way for module packagers to submit their packages up to a central repository. After the Catalog-SIG discussion at Python 9, I built a patch to Distutils which would do exactly this. Getting that code incorporated into the upcoming Python release was the problem. >I also find the Red Hat / Debian solution to *this* issue quite >obnoxious in that you often have to install or upgrade more packages >than should be necessary. Admittedly, I've only been using the above packaging systems for a decade, but in my experience that is absolutely not true. If an update to a package says that it needs Python >= 2.3.4 instead of Python >= 2.3, then it probably *DOES* need those other packages upgraded. If it is making unnecessary demands on other packages, then it should be reported to the packager as a bug, because that's what it is. Sean -- If you don't believe it's correct before you start testing, what could possibly convince you? -- Don Grimes, 1994 Sean Reifschneider, Member of Technical Staff tummy.com, ltd. - Linux Consulting since 1995. Qmail, Python, SysAdmin From richardjones at optushome.com.au Mon Jan 17 02:33:24 2005 From: richardjones at optushome.com.au (Richard Jones) Date: Mon Jan 17 02:33:31 2005 Subject: [Catalog-sig] PyCon sprint for PEPs 314 and 243? In-Reply-To: <20050116002322.GB2701@tummy.com> References: <200412312028.35688.richardjones@optushome.com.au> <20050116002322.GB2701@tummy.com> Message-ID: <200501171233.24529.richardjones@optushome.com.au> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sun, 16 Jan 2005 11:23 am, Sean Reifschneider wrote: > On Fri, Dec 31, 2004 at 08:28:35PM +1100, Richard Jones wrote: > >Without looking into it too closely, it seems like yum is closely tied to > > RPMs and their associated package databases. > > I would suspect that the code for handling mirror lists and downloading of > files is not at all tied to the RPM file format. That code has had > something like a year of development on it, and probably can be used with > few changes. It's been pointed out that yum is GPL, which precludes any of its code going into Python. We might be able to learn from its design though when we address downloading from mirrors. > There are 3 different downloaded file types: RPM packages, RPM headers, and > the repository information. The repository information would probably > require some tweeks, and the RPM packages and headers would need to be > replaced with something else. Unless we were to decide to use RPM as the > package format. Which is probably also not a bad idea. I don't see what RPM gains us over the sdist format. > >Also, I'm not sure I want to try to solve dependencies. > > I'm not sure you have a choice not to. I'm also not sure it's as hard a > problem as some people seem to think. Red Hat and Debian have fairly good > solutions to this problem with years and years of thought behind them. No, I do have a choice not to, and I choose not to :) I simply don't have time or energy to try to address the issue *correctly* at the moment. I *believe* I have time and energy available to try to address the other issue of source package upload. That's it. Things for some other day are: - - fetching of source packages by anything other than a web browser or ftp program - - keeping a database of installed packages - - using dependencies to figure out what else needs to be installed As with PyPI, I'm attempting to just bite off a little bit so I actually have a chance of it getting *done* :) Of course, this is just me talking. If someone else does want to bite off the whole lot, and has the time and energy to stick it through to completion, then I'm perfectly happy to let them, and I'll even make supportive noises every now and then :) Richard -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) iD8DBQFB6xXkrGisBEHG6TARAqtBAJ9i2MWPhh8hTE1bcFMk2nDeLxTx8gCfaCjd 7F62SiExLIXJzHQgkgw+wOw= =MXy1 -----END PGP SIGNATURE----- From chris.mcavoy at gmail.com Thu Jan 20 19:02:46 2005 From: chris.mcavoy at gmail.com (Chris McAvoy) Date: Thu Jan 20 19:02:49 2005 Subject: [Catalog-sig] We'd really like to Sprint. Message-ID: <3096c19d05012010023c665ce4@mail.gmail.com> Hi catalog-sig, Forgive me for being a bull in a china shop, but can someone suggest a 5 hour Sprint task for the Chicago Python Users Group to Sprint on? Is there a paragraph in a PEP that we could tackle with the catalog-sig's blessing? Thanks, Chris From richardjones at optushome.com.au Fri Jan 28 09:14:39 2005 From: richardjones at optushome.com.au (Richard Jones) Date: Fri Jan 28 09:14:27 2005 Subject: [Catalog-sig] We'd really like to Sprint. In-Reply-To: <3096c19d05012010023c665ce4@mail.gmail.com> References: <3096c19d05012010023c665ce4@mail.gmail.com> Message-ID: <200501281914.39342.richardjones@optushome.com.au> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Fri, 21 Jan 2005 05:02 am, Chris McAvoy wrote: > Forgive me for being a bull in a china shop, but can someone suggest a > 5 hour Sprint task for the Chicago Python Users Group to Sprint on? > Is there a paragraph in a PEP that we could tackle with the > catalog-sig's blessing? Sorry for the delay. 5 hours probably isn't enough for any of the things I had planned. I guess if you were able to get together a patch implementing PEP 314 then that would be great. I've still not revised the PEP, which I will try to do this evening around the most recent discussion: http://article.gmane.org/gmane.comp.python.catalog/240 I'm still keen to actually Sprint at PyCon - I'm going to put up the proposal in the wiki tonight too. If nothing else, I'll sit around before PyCon and bloody well write the thing by myself :) Richard -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) iD4DBQFB+fRvrGisBEHG6TARAgFGAJi7wOBSa3z5exW7UatWYHlWxa40AJ4iFQmH dUeRquuol7s14DsdsUvoeA== =Kna0 -----END PGP SIGNATURE----- From chris.mcavoy at gmail.com Fri Jan 28 15:49:28 2005 From: chris.mcavoy at gmail.com (Chris McAvoy) Date: Fri Jan 28 15:49:31 2005 Subject: [Catalog-sig] We'd really like to Sprint. In-Reply-To: <200501281914.39342.richardjones@optushome.com.au> References: <3096c19d05012010023c665ce4@mail.gmail.com> <200501281914.39342.richardjones@optushome.com.au> Message-ID: <3096c19d050128064920bd344c@mail.gmail.com> Is there anything that we could do in 5 hours? We had talked about an XML-RPC interface for searching, and a client. Something that could grow into the perl -MCPAN -e shell sort of interface. Is that something you'd be interested it? 5 hours could build an XML-RPC client and server with time left over for us to figure out how to Sprint. Chris On Fri, 28 Jan 2005 19:14:39 +1100, Richard Jones wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Fri, 21 Jan 2005 05:02 am, Chris McAvoy wrote: > > Forgive me for being a bull in a china shop, but can someone suggest a > > 5 hour Sprint task for the Chicago Python Users Group to Sprint on? > > Is there a paragraph in a PEP that we could tackle with the > > catalog-sig's blessing? > > Sorry for the delay. 5 hours probably isn't enough for any of the things I had > planned. I guess if you were able to get together a patch implementing PEP > 314 then that would be great. I've still not revised the PEP, which I will > try to do this evening around the most recent discussion: > > http://article.gmane.org/gmane.comp.python.catalog/240 > > I'm still keen to actually Sprint at PyCon - I'm going to put up the proposal > in the wiki tonight too. If nothing else, I'll sit around before PyCon and > bloody well write the thing by myself :) > > Richard > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.2.4 (GNU/Linux) > > iD4DBQFB+fRvrGisBEHG6TARAgFGAJi7wOBSa3z5exW7UatWYHlWxa40AJ4iFQmH > dUeRquuol7s14DsdsUvoeA== > =Kna0 > -----END PGP SIGNATURE----- > _______________________________________________ > Catalog-sig mailing list > Catalog-sig@python.org > http://mail.python.org/mailman/listinfo/catalog-sig > From ianb at colorstudy.com Fri Jan 28 18:07:17 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Fri Jan 28 18:08:34 2005 Subject: [Catalog-sig] We'd really like to Sprint. In-Reply-To: <200501281914.39342.richardjones@optushome.com.au> References: <3096c19d05012010023c665ce4@mail.gmail.com> <200501281914.39342.richardjones@optushome.com.au> Message-ID: <41FA7145.6030205@colorstudy.com> Richard Jones wrote: > On Fri, 21 Jan 2005 05:02 am, Chris McAvoy wrote: > >>Forgive me for being a bull in a china shop, but can someone suggest a >>5 hour Sprint task for the Chicago Python Users Group to Sprint on? >>Is there a paragraph in a PEP that we could tackle with the >>catalog-sig's blessing? > > > Sorry for the delay. 5 hours probably isn't enough for any of the things I had > planned. I guess if you were able to get together a patch implementing PEP > 314 then that would be great. I've still not revised the PEP, which I will > try to do this evening around the most recent discussion: There's also a lot of more boring (wait, no, straight-forward but fun!) tasks that we could do as well, like port it to WSGI so it could be run under mod_python, make it run with Postgres, and separating out the HTML into templates (ZPT I'd guess). Whatever it is, it should be something we can get cleaned up and integrated before the PyCon sprint. > http://article.gmane.org/gmane.comp.python.catalog/240 > > I'm still keen to actually Sprint at PyCon - I'm going to put up the proposal > in the wiki tonight too. If nothing else, I'll sit around before PyCon and > bloody well write the thing by myself :) Well, there might be parts we can work on in advance, based on the presumption you (and your multitude of sprinting minions!) will be filling in other parts later. E.g., we could work on a mirroring script, which might later use some appropriate metadata that's added to PyPI. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org From richardjones at optushome.com.au Fri Jan 28 23:04:34 2005 From: richardjones at optushome.com.au (Richard Jones) Date: Fri Jan 28 23:04:32 2005 Subject: [Catalog-sig] We'd really like to Sprint. In-Reply-To: <41FA7145.6030205@colorstudy.com> References: <3096c19d05012010023c665ce4@mail.gmail.com> <200501281914.39342.richardjones@optushome.com.au> <41FA7145.6030205@colorstudy.com> Message-ID: <200501290904.34314.richardjones@optushome.com.au> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sat, 29 Jan 2005 04:07 am, Ian Bicking wrote: > There's also a lot of more boring (wait, no, straight-forward but fun!) > tasks that we could do as well, like port it to WSGI so it could be run > under mod_python, make it run with Postgres, and separating out the HTML > into templates (ZPT I'd guess). Whatever it is, it should be something > we can get cleaned up and integrated before the PyCon sprint. These are great ideas. Either mod_python'ing the web interface or moving from sqlite to postgres would be highest of the three in my books. I can supply the current sqlite database for sprinters to work from. Also, investigating / fixing any of: http://sourceforge.net/tracker/?group_id=66150&atid=513503 would also be good. Richard -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) iD8DBQFB+rbyrGisBEHG6TARAuKuAJ4msYhuUfAvmw9lbQLMFuERR/t6SwCeNLSm Fh+QcYPANAv/itymLi4XHwQ= =oDTB -----END PGP SIGNATURE----- From ehs at pobox.com Fri Jan 28 23:13:29 2005 From: ehs at pobox.com (Ed Summers) Date: Fri Jan 28 23:13:38 2005 Subject: [Catalog-sig] We'd really like to Sprint. In-Reply-To: <200501290904.34314.richardjones@optushome.com.au> References: <3096c19d05012010023c665ce4@mail.gmail.com> <200501281914.39342.richardjones@optushome.com.au> <41FA7145.6030205@colorstudy.com> <200501290904.34314.richardjones@optushome.com.au> Message-ID: <20050128221329.GE16407@chloe.inkdroid.org> On Sat, Jan 29, 2005 at 09:04:34AM +1100, Richard Jones wrote: > I can supply the current sqlite database for sprinters to work from. Nice thing about sqlite is this sort of thing :) Is there a URL for the current db? //Ed From richard at mechanicalcat.net Mon Jan 31 21:40:20 2005 From: richard at mechanicalcat.net (Richard Jones) Date: Mon Jan 31 21:40:31 2005 Subject: [Catalog-sig] Re: Distutils sprinting at PyCon In-Reply-To: References: Message-ID: <200502010740.23971.richard@mechanicalcat.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 [this is a response to a comment PJE left in my weblog] On Tue, 1 Feb 2005 03:56 am, Phillip J. Eby wrote: > we're working on ... http://peak.telecommunity.com/DevCenter/PythonEggs > ... > This format would obsolete PEP 262, since removing a package will be a > matter of just deleting its .egg file. It affects PEP 314 too, because > "download URL" doesn't allow for platform-specific binary distribution like > .egg files, and because better platform information is needed. I'm not sure that having download URLs for *every* download format is *sane* for having in the PKG-INFO. I think that the only purpose for the download-url field in PKG-INFO is for the download of *the package that the PKG-INFO resides in*. The current bdist commands don't modify PKG-INFO though, which shoots this idea down pretty quickly (the platform information would need to be changed also). There may be some crossover with EGG-INFO, I don't know. How to then handle a collection of download URLs is another problem. It could partly be solved by an online repository of the sort I'm planning on implementing. This problem has had a lot of hand-waving up until now ;) Richard -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) iD8DBQFB/pe3rGisBEHG6TARAmT5AKCAwJGzc5voTZAK/p4UZ5HxX8fcsQCeJPey k1XckmJ4hrJHxPIxBX/f/mM= =DVeO -----END PGP SIGNATURE----- From pje at telecommunity.com Mon Jan 31 22:04:39 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon Jan 31 22:02:54 2005 Subject: [Catalog-sig] Re: Distutils sprinting at PyCon In-Reply-To: <200502010740.23971.richard@mechanicalcat.net> References: Message-ID: <5.1.1.6.0.20050131155704.0270f7b0@mail.telecommunity.com> At 07:40 AM 2/1/05 +1100, Richard Jones wrote: >[this is a response to a comment PJE left in my weblog] > >On Tue, 1 Feb 2005 03:56 am, Phillip J. Eby wrote: > > we're working on ... http://peak.telecommunity.com/DevCenter/PythonEggs > > ... > > This format would obsolete PEP 262, since removing a package will be a > > matter of just deleting its .egg file. It affects PEP 314 too, because > > "download URL" doesn't allow for platform-specific binary distribution like > > .egg files, and because better platform information is needed. > >I'm not sure that having download URLs for *every* download format is *sane* >for having in the PKG-INFO. I agree. The idea Bob and I are working on is that the download URL is in fact the URL of a manifest file that lists platforms and download locations. >The current bdist commands don't modify PKG-INFO >though, which shoots this idea down pretty quickly (the platform information >would need to be changed also). Exactly; platform-specific info shouldn't be in PKG-INFO. > There may be some crossover with EGG-INFO, I >don't know. The EGG-INFO directory is wide open for extension with additional metadata files; currently we only have two files in EGG-INFO whose contents are defined: native_libs.txt and eager_resources.txt. These control automatic extraction of files when other files are requested for extraction. I would suggest that a (non-platform-specific) PKG-INFO might be useful to include in the EGG-INFO directory. >How to then handle a collection of download URLs is another problem. It could >partly be solved by an online repository of the sort I'm planning on >implementing. > >This problem has had a lot of hand-waving up until now ;) We're trying to do something along the lines of Eclipse's "feature" concept, where a file describes a feature (package in this case) and lists the various downloads that are required for that feature including platform-specific URLs for those items that are platform-specific. The idea is that the download URL in PyPI and elsewhere would then be the download URL for a file that describes currently available versions of the package. Obviously there is still some handwaving here, but hopefully the waves are much smaller and better defined now, because it's more about defining a format and having tools to parse and download. (We may want that file to be XML, simply because it would allow the contents to be rendered with an XSL stylesheet for viewing by humans.) From ianb at colorstudy.com Mon Jan 31 22:11:59 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Mon Jan 31 22:13:36 2005 Subject: [Catalog-sig] Re: Distutils sprinting at PyCon In-Reply-To: <5.1.1.6.0.20050131155704.0270f7b0@mail.telecommunity.com> References: <5.1.1.6.0.20050131155704.0270f7b0@mail.telecommunity.com> Message-ID: <41FE9F1F.2030405@colorstudy.com> Phillip J. Eby wrote: > The idea is that the download URL in PyPI and elsewhere would then be > the download URL for a file that describes currently available versions > of the package. One thing I was hoping would be possible is mirroring, which means replacing download URLs with other URLs. If all the URLs are contained in a single file managed outside of PyPI, then it's hard to build a service on top of it (unless PyPI fetched the file and used that information to recreate a file that included mirrors, but that seems to defeat the purpose of an extra file). Also, it's one more file to manage; it's reasonable to manage files in your package -- either setup.py or an auxilliary file, but managing those files online is harder. Some people already seem overwhelmed with the system as it is -- and I don't know if there's anything to do about that, since it really doesn't seem that hard and maybe some people are just being a little whiney, but the sentiment does exist. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org From bob at redivi.com Mon Jan 31 21:55:05 2005 From: bob at redivi.com (Bob Ippolito) Date: Wed Feb 2 13:35:48 2005 Subject: [Catalog-sig] Re: Distutils sprinting at PyCon In-Reply-To: <200502010740.23971.richard@mechanicalcat.net> References: <200502010740.23971.richard@mechanicalcat.net> Message-ID: On Jan 31, 2005, at 15:40, Richard Jones wrote: > [this is a response to a comment PJE left in my weblog] > > On Tue, 1 Feb 2005 03:56 am, Phillip J. Eby wrote: >> we're working on ... >> http://peak.telecommunity.com/DevCenter/PythonEggs >> ... >> This format would obsolete PEP 262, since removing a package will be a >> matter of just deleting its .egg file. It affects PEP 314 too, >> because >> "download URL" doesn't allow for platform-specific binary >> distribution like >> .egg files, and because better platform information is needed. > > I'm not sure that having download URLs for *every* download format is > *sane* > for having in the PKG-INFO. I think that the only purpose for the > download-url field in PKG-INFO is for the download of *the package > that the > PKG-INFO resides in*. The current bdist commands don't modify PKG-INFO > though, which shoots this idea down pretty quickly (the platform > information > would need to be changed also). There may be some crossover with > EGG-INFO, I > don't know. > > How to then handle a collection of download URLs is another problem. > It could > partly be solved by an online repository of the sort I'm planning on > implementing. > > This problem has had a lot of hand-waving up until now ;) Beyond what's on that wiki page, we did have a discussion about how the downloading and metadata might work on the #peak IRC channel: http://peak.telecommunity.com/irc-logs/2005-01-27.txt (there wasn't a whole lot else discussed that day). There have also been discussions on the 28th and 31st (today) about eggs, but the only discussion relevant to online meta-data, dependencies, and package downloading happened on the 27th. -bob