[Web-SIG] multi-threaded or multi-process wsgi apps

[Robert Brewer]

Graham Dumpleton wrote:
> On 27/11/2007, Robert Brewer <fumanchu at aminus.org> wrote:
> > Chris Withers wrote:
> > > Right, I'm curious as to how wsgi applications end up being
> > > multi-threaded or multi-process and if they are, how they share
> > > resources such as databases and configuration.
> > >
> > > There's a couple of reasons I'm asking...
> > >
> > > The first was something Chris McDonough said about one ofthe
issues
> > > they're having with the repoze project: when using something like
> > > mod_wsgi, it's the first person to hit each thread that takes the
> hit
> > > of loading the configuration and opening up the zodb. Opening the
> > ZODB,
> > > in particular, can take a lot of time. How should repoze be
> structured
> > > such that all the threads load their config and open their
> databases
> > > when apache is restarted rather than when each thread is first
hit?
> >
> > If I were coding it, repoze would use a database connection pool
that
> is
> > populated at (sub)process startup.
> 
> The issue with running under Apache, whether it be mod_wsgi or
> mod_python, is that the server itself doesn't necessarily know
> anything about what applications may actually need to be loaded. This
> is because both support the concept of sticking the file representing
> the entry point to the application in some file system directory. The
> first that the server knows about the application is when a URL
> arrives which maps to that application file.
> 
> Thus, in the general case one cant have pre initialisation at
> (sub)process startup. To have pre initialisation means providing an
> explicit means of configuring the server to say that it is possible
> that some application may get invoked through a URL and so just in
> case it should preload the application.
> 
> Because it involves changing main server configuration, obviously can
> only be used as an option where you control the actual web server.
> There would be no way you could use such an option if you were just a
> user in a paid shared web hosting environment. In that case you can't
> avoid doing delayed initialisation at time that first request arrives.
> 
> This is the big difference between Apache and pure Python hosting
> solutions. That is that Apache has to deal with potential shared
> hosting issues. Pure Python hosting solutions would probably always be
> under direct control of the user and be only running their own code.

True, but that doesn't change my recommendation. Even if you're willing
to live with delays on the first request, you still should do as much as
possible as early as possible. Any server, application, or framework
which *requires* me to live with those delays even though I've taken
pains to deploy in a capable, controllable environment would make me
seriously question their utility.


Robert Brewer
fumanchu at aminus.org