[Catalog-sig] start on static generation, and caching - apache config.

[Phillip J. Eby]

At 02:52 PM 7/11/2007 -0400, Benji York wrote:
>Phillip J. Eby wrote:
>>This schism between the idea of neatly cataloging things, versus 
>>being able to actually *use* that cataloging for practical purposes 
>>by automated tools (as opposed to being usable only to human 
>>readers), seems to be at the heart of some of the current discussion.
>
>Wasn't there a proposal to merge the catalog-sig and distutils-sig?

Merging the lists isn't going to merge the people or change anybody's 
point of view.  The difference in SIGs reflects, for the most part, a 
difference in Special Interest -- the "I" in SIG.

Or another way of looking at the "I" is "Itch".  The people who have 
been working on cataloging already have their itch basically 
scratched; PyPI has been sufficient for their needs for some time now.

The packaging people, OTOH, have an ever-increasing itch, as 
setuptools hits its "hockey stick" growth phase both in user volume 
and package volume.  This is understandably, of little interest to 
people who don't do lots of packaging, deployment, and distribution.

I absolutely don't want to disparage the good folks who have made 
PyPI what it is today, and I totally understand their not wanting to 
take on the burden of supporting a tool they don't use or care about 
themselves, just because it happens to use PyPI.

But it seems to me that for folks whose Interest/Itch is not merely 
finding packages, but *using* them, a different infrastructure is 
needed, treating PyPI as the ultimate *source* of the information, 
without being also its sole *distribution* point, or query interface.

There are plenty of folks who have offered to spend funds, provide 
hosting, etc. for PyPI mirrors or alternatives -- perhaps we should 
create a SIG to start figuring out *how* to provide that, ideally 
while creating the least amount of additional service burden on the Cheeseshop.

Ideally, we could then support having the Cheeseshop redirect 
existing clients to a nearby distribution index, while newer clients 
could use a distribution index to start with.

Such a discussion would need to resolve certain design tradeoffs such 
as speed and availability vs. freshness of the index vs. load on the 
primary Cheeseshop vs. ability to have lots of mirrors/distribution 
indexes vs. ease of selecting one, etc.

But I believe the main reason why such discussion hasn't gone very 
far at this point is because the packaging-interest folks have been 
looking to the cataloging-interest folks to provide direction and 
focus to the discussion of the tradeoffs, even though these things 
lie mostly outside their itch/interest.  I think it is more likely to 
be productive for the packaging-interest folks to get clear about 
what they want first, and then the cataloging-interest folks can 
chime in if they see something being proposed that might be 
especially harmful to the Cheeseshop's availability or performance.