[Catalog-sig] start on static generation, and caching - apache config.
Jim Fulton
jim at zope.com
Mon Jul 9 16:09:37 CEST 2007
On Jul 8, 2007, at 12:07 PM, Martin v. Löwis wrote:
>>> So you are saying it's not fast enough already?
>>
>> Uh, yeah.
>
> Can you please be more precise, then? What kind of operation are
> you performing,
I'm using setuptools. Sertuptools looks at package pages (e.g.
http://www.python.org/pypi/foobar), it looks at:
http://www.python.org/pypi/ and it doenloads distributions. (AFAICT,
the later is done dynamically too, which
is especially insane.)
> how long does it take,
Lately, it's has often taken minutes. This has been the major
problem. At the best of times. well, I don't know when those are. :)
ATM, requests for http://www.python.org/pypi/zc.buildout takes about
1/3 second. Requests for http://cheeseshop.python.org/packages/2.5/z/
zc.buildout/zc.buildout-1.0.0b28-py2.5.egg take about 2.5 seconds.
Requests for http://www.python.org/pypi/ take about 10 seconds.
I would say that these times are too long.
> and how long should it
> take so that you would consider it fast enough?
IMO, it needs to be much much faster. If we were serving pages
staticially, we would be able to serve thousands of requests per
second. There's nothing about this application that would make doing
that hard.
> It's difficult to implement a system if the requirements are
> unknown to those implementing it.
I'm sorry, I've been talking about setuptools all along. I thought
the use case was understood. Also, I thought it was pretty obvious
that the performance we've been seeing lately is totally
unacceptable. It's hard to pinpoint exactly what the acceptable
performance will be, in part because, we we do better, demand will
increase. Note that, as it is now, demand is possibly decreasing
because people are building their own indexes.
If this was an application that had to be served dynamically (and of
course, parts of it are), then it would be much more interesting to
discuss targets for dynamic delivery. The performance-critical parts
of this application -- the pages that setuptools uses, can readily be
served statically, so it makes no sense not to do so.
Jim
--
Jim Fulton mailto:jim at zope.com Python Powered!
CTO (540) 361-1714 http://www.python.org
Zope Corporation http://www.zope.com http://www.zope.org
More information about the Catalog-SIG
mailing list