[Catalog-sig] Back to RPC

[Ian Bicking]

OK, I'm trying to make package installation easier for Paste, which 
comes back to PyPI pretty quickly, of course.

I think all the information I really need is how to get an sdist 
package, or a checkout -- right now it's important to be to be able to 
install out of repositories, not just released versions.

So, I think all the data is there now.  I might search package_urls for 
the right release, packagetype 'sdist'.  Though there isn't any place 
for a repository URL, and that gets a little vague -- in theory you 
really could have a per-version repository in the form of a branch.  But 
that's not very useful information.  But maybe that can be disambiguated 
with a URL "svn-trunk" or "cvs-trunk", which doesn't imply that the 
repository is version-specific, even if the database schema requires it 
be put there.

 From there it's all client-side work to install the package, so it 
doesn't involve PyPI.  The only issue is giving URLs to packages that 
don't have URLs.  It would be a lot easier from my perspective if I 
didn't have to go through the package author when URLs were missing, 
maybe with notification going back to the owner so they can adjust their 
setup.py file.  But lets ignore that for a little while, though it's 
also very important to me, as I'd like to use this stuff sooner rather 
than later.

Anyway, summarizing:

* Agree and document release_urls.packagetype values.  I'd like:
   * sdist
   * bdist
   * bdist_dumb
   * bdist_rpm
   * bdist_wininst
   * bdist_egg
   * svn-tag
   * svn-trunk
   * cvs-trunk (is there any agreed-upon format for CVS URLs?  What about
     branches and tags?)

Also, these really need to be documented somewhere on the pypi site, 
that's the only way agreeing on it means anything ;)

* Add some functions to rpc.py:
   * package_releases(package_name): list of release versions, as strings
   * package_stable_version(package_name): packages.stable_version
   * package_urls(package_name, version): A list of {'url': url,
     'packagetype': packagetype}
   * package_data(package_name, version): A dictionary that basically
     summarizes the releases table, plus release_classifiers.
   * providing_packages(specifier): A list of (name, version) from
     release_provides.
   * requiring_packages(specifier): A list of (name, version) from
     release_requires.

I don't entirely understand "specifier" and "release_obsoletes".  Are 
specifiers package names, or a slightly more abstract version of package 
names?  Or something more structured?

While it isn't important to me now, I think it would be good to change 
rpc.search to take a single dictionary argument, which it would pass in 
as the query spec.  Probably the return value should also be cleaned up, 
as store.query_packages returns a kind of odd data structure.

The adding URL thing is a bit more of a policy decision.  Maybe we can 
just add a form to request a role, which will email the owner and give 
them a quick form (maybe just one link) to add the user.

If I can get CVS commit access I can start working on some of these; a 
recent database dump would also be useful for testing.

Incidentally, I notice the PyPI page is quite slow.  Thoughts?  I think 
I'm noticing that templates are loaded off disk for each request, which 
be really slow.  Is PyPI running under mod_python now?

-- 
Ian Bicking  /  ianb at colorstudy.com  / http://blog.ianbicking.org