From richardjones at optushome.com.au Thu Apr 20 11:35:42 2006 From: richardjones at optushome.com.au (Richard Jones) Date: Thu, 20 Apr 2006 19:35:42 +1000 Subject: [Catalog-sig] PyPI enhancement wish items? Message-ID: <200604201935.42426.richardjones@optushome.com.au> Please fire away. I may have some time to work on this in the near future. Current stuff I know about: - PEP for metadata 1.2 http://www.python.org/peps/pep-0345.html (though this is not finished nor has it been commented on) - incorporate Ian Bicking's XML-RPC API http://mail.python.org/pipermail/catalog-sig/2005-May/000634.html - auto-generate download_urls for package uploads - command-line tool to query pypi and fetch entries (is this necessary given easy_install?) - DOAP support - better searching - tooltips for field labels - change notification emails - admin interface for user details editing Richard From ianb at colorstudy.com Thu Apr 20 18:05:14 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Thu, 20 Apr 2006 11:05:14 -0500 Subject: [Catalog-sig] PyPI enhancement wish items? In-Reply-To: <200604201935.42426.richardjones@optushome.com.au> References: <200604201935.42426.richardjones@optushome.com.au> Message-ID: <4447B13A.1020500@colorstudy.com> Richard Jones wrote: > Please fire away. I may have some time to work on this in the near future. > Current stuff I know about: > > - PEP for metadata 1.2 > http://www.python.org/peps/pep-0345.html > (though this is not finished nor has it been commented on) The most notable addition seems to be Requires. Would this be read from install_requires? It seems like this has to go in Setuptools (to generate the PKG-INFO) as well as PyPI (to consume it). Setuptools has the extra features, which could also be shown, probably more as Recommended than as Required. Setuptools doesn't allow for requirements like C libraries, or Python versions. It would be nice if it did (at least the Python versions). I don't know if Requires in the metadata will include those kinds of requirements? > - incorporate Ian Bicking's XML-RPC API > http://mail.python.org/pipermail/catalog-sig/2005-May/000634.html > - auto-generate download_urls for package uploads > - command-line tool to query pypi and fetch entries (is this necessary > given easy_install?) I think it's still useful; easy_install only works if you know what you are looking for. Maybe it could be integrated into easy_install... but it might be best if it was a stand-alone script for now, and maybe integrate into nest later. Better keyword searching support would be good too, especially through XML-RPC or some RESTish interface. E.g.: /pypi/packages/wsgi+application Gives you a list of everything with the wsgi and application keywords. I think Kevin worked around this, but he was having problems searching for multiple keywords as well. > - DOAP support > - better searching +2! A single search box on the front page would be most appreciated. > - tooltips for field labels > - change notification emails > - admin interface for user details editing Comments might be nice, but sadly require maintenance, for spam and content. A more constrained thing might be links to other packages, so you could see what else is Some other things: Download tracking would be nice. I know at least one person has been reluctant to move all his packages to the Cheese Shop because it didn't count downloads and SF did. And I hate SF downloading so so so much. Changing the template to match the new look would be nice, of course. I look forward to the improvements! -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From pje at telecommunity.com Thu Apr 20 18:49:14 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 20 Apr 2006 12:49:14 -0400 Subject: [Catalog-sig] PyPI enhancement wish items? In-Reply-To: <200604201935.42426.richardjones@optushome.com.au> Message-ID: <5.1.1.6.0.20060420123202.01e21bc0@mail.telecommunity.com> At 07:35 PM 4/20/2006 +1000, Richard Jones wrote: >Please fire away. I may have some time to work on this in the near future. >Current stuff I know about: > >- PEP for metadata 1.2 > http://www.python.org/peps/pep-0345.html > (though this is not finished nor has it been commented on) I think that PEP needs to go back to the drawing board; it emphasizes too much the syntax of fields, while leaving their semantics entirely void. For example, it dictates what format projects may specify version numbers in, using a scheme that's too strict for even Python itself to comply with. But then it leaves the meaning of requirements to be community-defined. There's little point in having machine-readable data with no machine-comprehensible semantics. That is, why make people force fit stuff into a form that machines will never use anyway? >- auto-generate download_urls for package uploads This seems unnecessary to me, since easy_install reads the upload links just fine where they are, but of course I'm biased. :) >- command-line tool to query pypi and fetch entries (is this necessary > given easy_install?) A stable, documented interface to perform the operations that easy_install does now via screen-scraping and URL interpretation would be useful. The other things I'd add to the list are (decreasing priority): 1) the ability to treat project names and versions as case-insensitive, while removing extraneous characters (as in pkg_resources.safe_name()) for purposes both of searching and determining name uniqueness when registering. 2) Compute "cheesecake" scores for modified entries (using only the metrics that don't actually run any of the package's code, of course) and display them prominently. :) 3) Provide better explanation as to what to put in the fields: encourage people distributing via Sourceforge to put their showfiles.php URL in as the download URL, or any other page that has actual clickable links to download files. 4) More strongly encourage people to use "setup.py register", by having the web interface generate a setup.py containing the information they filled in, and suggesting that they use it, while making it hard to find the button that will go ahead and put the data in from the web. This can be billed as a new convenience feature to automatically generate a setup.py and help people improve their cheesecake scores. ;-) From grig at gheorghiu.net Thu Apr 20 19:00:34 2006 From: grig at gheorghiu.net (Grig Gheorghiu) Date: Thu, 20 Apr 2006 10:00:34 -0700 (PDT) Subject: [Catalog-sig] PyPI enhancement wish items? In-Reply-To: <5.1.1.6.0.20060420123202.01e21bc0@mail.telecommunity.com> Message-ID: <20060420170034.4957.qmail@web54512.mail.yahoo.com> --- "Phillip J. Eby" wrote: > 2) Compute "cheesecake" scores for modified entries (using only the > metrics > that don't actually run any of the package's code, of course) and > display > them prominently. :) Cheesecake! Cheesecake! I'm all for that! :-) I've kind of neglected the cheesecake project in the last couple of months, but if you guys express interest in it, I'm way more motivated to dust it off and improve it... I was actually thinking that cheesecake will never take off and gain mind share unless I actually start displaying its scores for the PyPI packages in a very prominent way :-) Grig From martin at v.loewis.de Thu Apr 20 21:54:18 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 20 Apr 2006 21:54:18 +0200 Subject: [Catalog-sig] PyPI enhancement wish items? In-Reply-To: <200604201935.42426.richardjones@optushome.com.au> References: <200604201935.42426.richardjones@optushome.com.au> Message-ID: <4447E6EA.3050105@v.loewis.de> Richard Jones wrote: > Please fire away. I may have some time to work on this in the near future. > Current stuff I know about: - Rating system for packages - procedures for "taking over" entries should the original owner of the entry go away. Regards, Martin From ianb at colorstudy.com Thu Apr 20 22:07:21 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Thu, 20 Apr 2006 15:07:21 -0500 Subject: [Catalog-sig] PyPI enhancement wish items? In-Reply-To: <5.1.1.6.0.20060420123202.01e21bc0@mail.telecommunity.com> References: <5.1.1.6.0.20060420123202.01e21bc0@mail.telecommunity.com> Message-ID: <4447E9F9.7040306@colorstudy.com> Phillip J. Eby wrote: >>- auto-generate download_urls for package uploads > > > This seems unnecessary to me, since easy_install reads the upload links > just fine where they are, but of course I'm biased. :) It might be nice if PyPI can figure out SF download links. Then if SF changes its format again PyPI can be updated and easy_install won't have to be. It's a lot easier to update the one and only PyPI than every easy_install. >>- command-line tool to query pypi and fetch entries (is this necessary >> given easy_install?) > > > A stable, documented interface to perform the operations that easy_install > does now via screen-scraping and URL interpretation would be useful. Would that be the XML-RPC interface? > The other things I'd add to the list are (decreasing priority): > > 1) the ability to treat project names and versions as case-insensitive, > while removing extraneous characters (as in pkg_resources.safe_name()) for > purposes both of searching and determining name uniqueness when registering. I'm guessing that means: if someone goes to /pypi/foo and the real name is Foo, then redirect them to /pypi/Foo. > 2) Compute "cheesecake" scores for modified entries (using only the metrics > that don't actually run any of the package's code, of course) and display > them prominently. :) Or alternately, have a way of uploading scores (from a trusted source), and then the scores can be calculated offline on a less important server. This will make it easier to develop Cheesecake too, since it can be updated without updating PyPI. I'm guessing the Cheesecake results can be summarized by a single score and then some human-readable HTML describing the score. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From richardjones at optushome.com.au Fri Apr 21 00:26:27 2006 From: richardjones at optushome.com.au (Richard Jones) Date: Fri, 21 Apr 2006 08:26:27 +1000 Subject: [Catalog-sig] PyPI enhancement wish items? In-Reply-To: <4447B13A.1020500@colorstudy.com> References: <200604201935.42426.richardjones@optushome.com.au> <4447B13A.1020500@colorstudy.com> Message-ID: <200604210826.27224.richardjones@optushome.com.au> On Friday 21 April 2006 02:05, Ian Bicking wrote: > Download tracking would be nice. I know at least one person has been > reluctant to move all his packages to the Cheese Shop because it didn't > count downloads and SF did. And I hate SF downloading so so so much. This is in place now. For example, see http://cheeseshop.python.org/pypi/roundup Richard From richardjones at optushome.com.au Fri Apr 21 00:51:59 2006 From: richardjones at optushome.com.au (Richard Jones) Date: Fri, 21 Apr 2006 08:51:59 +1000 Subject: [Catalog-sig] PyPI enhancement wish items? In-Reply-To: <200604201935.42426.richardjones@optushome.com.au> References: <200604201935.42426.richardjones@optushome.com.au> Message-ID: <200604210851.59678.richardjones@optushome.com.au> On Thursday 20 April 2006 19:35, Richard Jones wrote: > - incorporate Ian Bicking's XML-RPC API > http://mail.python.org/pipermail/catalog-sig/2005-May/000634.html *cough* of course, the TODO list I was looking at is out of date, as this is already done :) >>> import xmlrpclib >>> server = xmlrpclib.ServerProxy('http://cheeseshop.python.org/pypi') >>> server.package_releases('roundup') ['1.1.1'] >>> server.package_urls('roundup', '1.1.1') [{'url': 'http://cheeseshop.python.org/packages/any/r/roundup/roundup-1.1.1.win32.exe', 'packagetype': 'bdist_wininst'}, {'url': 'http://cheeseshop.python.org/packages/source/r/roundup/roundup-1.1.1.tar.gz', 'packagetype': 'sdist'}] Richard From pje at telecommunity.com Fri Apr 21 01:53:16 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 20 Apr 2006 19:53:16 -0400 Subject: [Catalog-sig] PyPI enhancement wish items? In-Reply-To: <200604210851.59678.richardjones@optushome.com.au> References: <200604201935.42426.richardjones@optushome.com.au> <200604201935.42426.richardjones@optushome.com.au> Message-ID: <5.1.1.6.0.20060420194753.041682b0@mail.telecommunity.com> At 08:51 AM 4/21/2006 +1000, Richard Jones wrote: >On Thursday 20 April 2006 19:35, Richard Jones wrote: > > - incorporate Ian Bicking's XML-RPC API > > http://mail.python.org/pipermail/catalog-sig/2005-May/000634.html > >*cough* of course, the TODO list I was looking at is out of date, as this is >already done :) > > >>> import xmlrpclib > >>> server = xmlrpclib.ServerProxy('http://cheeseshop.python.org/pypi') > >>> server.package_releases('roundup') >['1.1.1'] > >>> server.package_urls('roundup', '1.1.1') >[{'url': >'http://cheeseshop.python.org/packages/any/r/roundup/roundup-1.1.1.win32.exe', > >'packagetype': 'bdist_wininst'}, {'url': >'http://cheeseshop.python.org/packages/source/r/roundup/roundup-1.1.1.tar.gz', > >'packagetype': 'sdist'}] Sweet! It's fast, too, especially on search. I'll definitely upgrade easy_install to start using this, although alas, not today. From pje at telecommunity.com Sat Apr 22 00:22:15 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 21 Apr 2006 18:22:15 -0400 Subject: [Catalog-sig] Reject or update PEP 243? Message-ID: <5.1.1.6.0.20060421181756.01e3bc00@mail.telecommunity.com> I just noticed that the 2.5 What's New references PEP 243 in relation to the distutils' "upload" command, but in fact the upload command implements a different mechanism entirely. About the only thing the PEP and implementation have in common is that they do a POST to python.org/pypi and have a protocol_version field of "1". Nothing else is the same. Perhaps someone who knows how the actual implementation works could rewrite the PEP, or write another one? From richardjones at optushome.com.au Sat Apr 22 03:09:27 2006 From: richardjones at optushome.com.au (Richard Jones) Date: Sat, 22 Apr 2006 11:09:27 +1000 Subject: [Catalog-sig] Reject or update PEP 243? In-Reply-To: <5.1.1.6.0.20060421181756.01e3bc00@mail.telecommunity.com> References: <5.1.1.6.0.20060421181756.01e3bc00@mail.telecommunity.com> Message-ID: <200604221109.27540.richardjones@optushome.com.au> On Saturday 22 April 2006 08:22, Phillip J. Eby wrote: > I just noticed that the 2.5 What's New references PEP 243 in relation to > the distutils' "upload" command, but in fact the upload command implements > a different mechanism entirely. About the only thing the PEP and > implementation have in common is that they do a POST to python.org/pypi and > have a protocol_version field of "1". Nothing else is the same. It'll probably be easier to just rewrite the PEP. I'm not entirely sure I know how the inner workings of the upload command work (how distutils does command chaining, for example) but I could update the protocol if I found the time for it. Richard From pje at telecommunity.com Sat Apr 22 18:04:43 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 22 Apr 2006 12:04:43 -0400 Subject: [Catalog-sig] [Python-Dev] setuptools: past, present, future In-Reply-To: References: <5.1.1.6.0.20060421134259.0419f318@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20060422115700.0448f138@mail.telecommunity.com> At 12:34 PM 4/22/2006 +0200, Fredrik Lundh wrote: >Guido van Rossum wrote: > > > Leaving aside the Perl vs. Py thing, opinions on CPAN seem to be > > diverse -- yes, I've heard people say that this is something that > > Python sorely lacks; but I've also heard from more than one person > > that CPAN sucks from a quality perspective. So I think we shouldn't > > focus on emulating CPAN; rather, we should solve the problems we > > actually have. > >the first problem seems to be to define what those problems really >are ;-) > >(as for the CPAN quality, any public repository will end up being full >of crap; I don't see any way to work around that. automatic scoring >based on superficial aspects The purpose of automated scoring on superficial aspects isn't so much to ensure quality as it is to ensure *accessibility*, both in the sense of being able to install the thing, and meet some basic levels of having documentation. If something is accessible and trivial to install, then the market can decide which packages are better to actually use. >or ratings by small numbers of anonymous >visitors are probably among the worst ways to distinguish crap from >good stuff; for quality, you need initiatives like > > http://code.enthought.com/enthon/ > >and other "fat python" projects. Actually, every project that uses other projects' code can now be a "chubby python" by expressing dependencies. Really, one of the best ratings of a package's quality (or at least popularity) is going to be how many other projects depend on it. If everybody uploaded eggs to the Cheeseshop, it'd be possible to show links to "projects that use this project's code" by reading the dependency metadata with pkg_resources. (Not to mention "projects that this project uses"). From grig at gheorghiu.net Sat Apr 22 18:49:15 2006 From: grig at gheorghiu.net (Grig Gheorghiu) Date: Sat, 22 Apr 2006 09:49:15 -0700 (PDT) Subject: [Catalog-sig] PyPI enhancement wish items? In-Reply-To: <20060420170034.4957.qmail@web54512.mail.yahoo.com> Message-ID: <20060422164915.63521.qmail@web54512.mail.yahoo.com> Richard, Do you think it would be possible to start some integration of my Cheesecake project into PyPI? Here's the homepage of the project: http://pycheesecake.org/ For a given package, I'm computing scores for installability, documentation and code 'kwalitee'. Details are on the homepage. I'd love to see what other things people think would be worth measuring. Grig --- Grig Gheorghiu wrote: > --- "Phillip J. Eby" wrote: > > > 2) Compute "cheesecake" scores for modified entries (using only the > > metrics > > that don't actually run any of the package's code, of course) and > > display > > them prominently. :) > > Cheesecake! Cheesecake! I'm all for that! :-) > > I've kind of neglected the cheesecake project in the last couple of > months, but if you guys express interest in it, I'm way more > motivated > to dust it off and improve it... > > I was actually thinking that cheesecake will never take off and gain > mind share unless I actually start displaying its scores for the PyPI > packages in a very prominent way :-) > > Grig > > _______________________________________________ > Catalog-sig mailing list > Catalog-sig at python.org > http://mail.python.org/mailman/listinfo/catalog-sig >