[Catalog-sig] Prototype setuptools-specific PyPI index.

[Jim Fulton]

On Jul 21, 2007, at 3:08 PM, Martin v. Löwis wrote:

>>> - it does include a top-level index of all packages (but neither
>>>   releases nor descriptions)
>>
>> Why?  This is a relatively expensive page, due to it's size I assume,
>> that really provides no value.  This will slow down setuptools.
>
> IIUC, it won't slow down setuptools, as setuptools looks at it only
> if it cannot find the real package page due to a misspelling. So
> as long as everything is spelled correctly, it should not provide
> any slowdown.
>
> If people do misspell a package name when invoking easy_install,
> they get the feature that you consider of no value.

That is not correct. Not all packages are in PyPI.  Using a package  
that isn't in PyPI will trigger a fetch of that page.  It isn't  
misspelled, it's just not there.  People should *not* misspell pages  
when using setuptools.  They should certainly not use misspelled  
package names in requirements.  In my strongly help opinion, allowing  
imprecise names in requirements and setuptools command if of negative  
value.

> As for performance - 30 downloads take 3.9s currently from nearby.

That's nice.  For me, that page takes 3 or 4 times as long as other  
pages.

>>> - it's always current, due to being dynamically computed
>>
>> And also unreliable, for the same reason. For example, it would have
>> been inaccessible yesterday afternoon.
>
> The same could happen to Apache, too, of course. svn.python.org
> sometimes fails to restart when a restart is request on log rotation.
>
> Any software is unreliable; to reduce downtime, you need an operator
> that is available when something breaks.

Apache has a far better record than the cheeseshop.  I give up.

>> And also puts more load on the server.  It would be much better imo
>> if static pages could be written on writes.
>
> Contributions are welcome. In addition to me considering it futile,
> I also don't know how to implement it correctly.

I'd be happy to contribute my polling version.  That solves my  
problems and I can't justify the additional effort to figure out the  
cheeseshop softtware.

...
>> They lack the #egg= links.
>
> How are these computed?

By parsing the description.

Apparently, I'm going this incorrectly.  I'll have to look into that.

Jim

--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org