[Catalog-sig] start on static generation, and caching - apache config.

[Phillip J. Eby]

At 03:15 AM 7/10/2007 +1000, René Dudfield wrote:
>def the static generation:
>    - generate file in temp file
>    - move temp file to place where static file lives.
>
>def the update code:
>    - do inserts/updates/deletes.
>    - remove static files.
>    - commit change.
>    - the static generation()

Ah - I was assuming static generation was going to be a separate process.

However, there's still a race condition here, unless you open the 
temp file exclusively before the transaction commits.  If you wait 
until after the transaction is finished, another change could occur 
to the same page after you, but finish its page write *before* you, 
causing you to overwrite it with your move!  You then end up with an 
outdated page that will stick around indefinitely.  (Yes, it's 
unlikely, but it *can* happen, and therefore eventually will.)

So, as in my suggestion, you *still* need an exclusive open of a 
pre-determined tempfile name, prior to transaction commit.  Then, 
such an occurrence is impossible.

By the way, the generate-on-change approach also means you have to do 
a big batch run to pre-generate all the existing static pages; the 
approach I suggested will simply generate them in response to actual 
demand, with no batch processing necessary.  A new PyPI installation 
would just build up its cache as it gets used, getting faster as it goes.