[Catalog-sig] start on static generation, and caching - apache config.

[Jim Fulton]

On Jul 11, 2007, at 4:15 PM, Martin v. Löwis wrote:

>> Let's suppose that setuptools was changed to be aware that PyPI  
>> release
>> pages correspond to a particular version.  In that case, it would  
>> have
>> to read the package page to discover the release pages and then it  
>> would
>> have to read at least one release page.  If it had requirements other
>> than the version (e.g. Python version or platform), it might have to
>> scan several releases to find an acceptable distribution.  But, in  
>> the
>> best case, it would have to scan at least two pages.
>
> Sure. However, that makes the difference between O(1) and O(N),
> where N is the number of releases recorded. Going back to your
> original concern: you would not have to change the policy of
> keeping many different releases if the number of releases
> does not impact performance.

Yup.  Absolutely.  That's why it we should change the index or  
setuptools, or both.  IMO, it makes the most sense to change the  
index to have setuptools specific pages, in addition to the ones for  
humans, that allow:

- One page per package and

- a minimal amount of data to be downloaded and scanned per page.

   (As I noted before, release pages are meant for humans.  They  
sometimes contain *lots* of data that setuptools doesn't need.)

> When it looks for individual release pages, does it know that these
> are release pages, or does it follow all links on the package
> page?

I'll have to dig to answer that question precisely.  I'll do that  
after pausing to see if Phillip explains it first.

> If the latter, what links does it follow (there are plenty
> more on the package page)?

See: http://mail.python.org/pipermail/catalog-sig/2007-July/001217.html

It seems to only scan the release pages.  So it has some heuristic to  
know which links to follow.

Jim

--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org