[Web-SIG] serving (potentially large) files through wsgi?

[Graham Dumpleton]

On 22/12/2007, Brian Smith <brian at briansmith.org> wrote:
> Manlio Perillo wrote:
> > Graham Dumpleton ha scritto:
> > > On 21/12/2007, Brian Smith <brian at briansmith.org> wrote:
> > >> The specification should then also explicitly say
> > >> that WSGI applications should not redirect logging
> > >> output to wsgi.errors or anywhere else.
> > >> In fact, if that was done, there would be no reason
> > >> to have wsgi.errors in the first place.
> > >
> > > At least in the context of Apache, wsgi.errors is
> > > different to sys.stderr or a global logging module
> > > output target. This is because wsgi.errors is linked
> > > to the actual request and so any output can be
> > > correctly redirected to a per virtual host error log.
>
> (Apache) mod_wsgi uses threads, right? You could keep a map (thread ->
> output stream), possibly using thread-local variables, to keep track of
> where to redirect stderr and logging output.

I thought about that and decided not to do it. Part of the reason is
the extra complexity, the other is that it still isn't able to cater
for all cases. The case it still doesn't catch is where user code
creates Python threads unrelated to a specific request. These still
can't be tied to the request. For that matter it also doesn't catch
user code which is triggered from atexit callbacks on process
shutdown.

It therefore seemed more consistent for only wsgi.errors to be bound
to request, given that it comes through request environment. This can
then map to internal Apache ap_log_rerror() function, allowing client
IP to be listed against message in error log file.

For sys.stderr, instead use internal Apache ap_log_error() function,
which will log against server context. For embedded mode of mod_wsgi
this is main Apache error log and for daemon mode the VirtualHost if
daemon process is associated with the virtual host. Thus if
VirtualHost has its own error log different to the main Apache error
log it will go there.

This seemed to be about the best compromise one could hope for and so
what the mod_wsgi C core code does.

FWIW, there is actually nothing to stop you in mod_wsgi from
implementing what you want at the Python code level. In other words,
mod_wsgi will not complain if you replace sys.stderr and at the same
time use a WSGI wrapper which does some registration of the thread ID
such that a smarter sys.stderr can route things back to wsgi.errors
instead. It would certainly be a lot easier to do at the Python code
level than at C code level.

I even added some hooks in latest release candidate so that such WSGI
wrappers could be transparently added into the application stack and
thus allow such things to be done across a lot of application entry
points at the same time. Users couldn't see the point of why I added
it and as such I'll likely remove the feature rather than argue the
point.

Graham