[Catalog-sig] PyPI enhancement doc

[Ian Bicking]

Incidentally, in case I forget, I should remind the group I have this 
file from that last sprint that gives PyPI a WSGI frontend:
   http://lonelylion.com/chipy/pypi/pypi_wsgi.py

It's pretty straight-forward.  It might be useful in porting PyPI to 
mod_python, or some other WSGI server.  Incidentally with Paste you 
could use the configuration:

   publish_app = 'pypi.pypi_wsgi:pypi_app("/path/to/base")'

Martin v. Löwis wrote:
>>The binary distributions are all potentially platform specific, but
>>(except for Windows) the platform isn't encoded into the packagetype.
>>Should packagetype be extended to include information like Python
>>version?
> 
> 
> It was excluded because it can be implied from the package file
> (distutils makes sure the filenames don't conflict). upload.py
> puts the platform in the comment.
> 
> Standardizing on platform names is a tricky issue: Is "SuSE" the
> same platform as "Debian"? Is it the same platform as "SuSE Linux"?

Yes, I agree, it's tricky.  Lets say that the URL must include as its 
basename the filename that distutils produces, so that this can be used 
to recreate that filename:

base = url.split('/')[-1]
base = base.split('?')[0]

>>Another change would be to refactor the database, so that
>>release_files didn't have python_version, packagetype, or md5_digest
>>fields, but rather a third table (referenced by both release_files and
>>package_urls) reference this package-metadata field.
> 
> 
> What would be the advantage of doing so? If it is normalization:
> I don't see any advantage in normalization at all.

Because those fields can apply to other packages, even if they aren't 
stored in PyPI's file repository.  Just for normalization's sake I 
wouldn't really care.  And really, I don't actually care if the fields 
are copied either.

But anyway, if we are expecting the filename to be self-describing then 
this mostly doesn't matter.  Except perhaps an optional md5_digest.

>>XML-RPC methods
>>---------------
>>
>>This is an initial list of proposed methods for PyPI to support:
> 
> 
> I think this whole thing is highly debatable. Why XML-RPC? Are HTML
> forms not good enough for you? Why not CORBA? etc.

XML-RPC is already in PyPI (and live!), in rpc.py.  It just doesn't have 
any useful functions right now.

> It appears to be a design principle of PyPI to not use XML-RPC if
> it can be avoided. I don't know where that principle comes from,
> but I think it needs to be followed if indeed it is a principle.
> 
> If there is no such principle, I think all the current forms should
> be duplicated to simplify the code in the distutils commands.

You mean the (somewhat awkward, if I remember) construction of POST 
requests in distutils?  Yes, I think it would be nice to move those to 
XML-RPC.  That so many old distutils exist means the forms would have to 
stay around for some time though :(  Or does it; can we ask people to 
update at least for submission?  Eh, probably not for some time.

>>Search should be available with a single field, that searches all of
>>package name, summary, description/description_html, keywords.  This
>>should be on the front page.
> 
> 
> See
> 
> http://sourceforge.net/tracker/index.php?func=detail&aid=989926&group_id=66150&atid=513503
> 
> In general, details of the UI and the database can be easily
> integrated as long as they don't affect current users and
> as long as they don't affect distutils or other programmatic
> interfaces.
> 
> So contributions are welcome, although I understand this is
> meant to be solved by Postgres full text search.

OK.  I expect there's some sysadmin issues to enabling full text search 
in Postgres at this point as well.

>>There should be a contact form attached to each package that will be
>>emailed to any people on record as owners or maintainers, for
>>reporting bad or missing links or other incorrect data.
> 
> 
> Again, contributions are welcome. We should be careful not to implement
> another issue tracker here, though, so just sending an email might be
> enough for the moment (or else a read-only roundup installation,
> writable only to PyPI and the maintainer might be the right solution).
> As soon as tracking is available, people will start using it to
> report bugs on the actual software itself, which might defeat
> the bug reporting channels the maintainers try to establish.
> 
> So perhaps a "report problems with this record to" field might
> be sufficient, with the feedback form only available if the field
> is not filled.

Is there a missing negation there?  You mean, only allow reporting 
problems if that field is filled in?  I figure it's good enough to send 
mail to everyone who has sufficient permission to resolve the problem, 
another field shouldn't be necessary.

I'd like to display the message as well, so that readers can see message 
until it is corrected; of course, leaving it forever just builds cruft. 
  Hence the resolved/unresolved thing.

> In any case, if you have something that works, please submit
> a patch to sf.net/projects/pypi.

Well, for now I'll try to do something with rpc.py, since there's not 
enough code their to require refactoring (at which point patches start 
to become more challenging).

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org