[Web-SIG] serving (potentially large) files through wsgi?

[Graham Dumpleton]

On 22/12/2007, Brian Smith <brian at briansmith.org> wrote:
> Manlio Perillo wrote:
>
> > I can modify the code, so that:
> > - sys.stderr for the main interpreter goes to the main error log
> > - sys.stderr for subinterpreters goes to the error log declared in the
> >    HTTP location where the WSGI application is "mounted"
>
> I think that makes sense. To effectively handle logging levels, you need
> to configure the root log handler appropriately as well.
>
> > Instead of using sys.stderr, a better solution is to add a new log
> > object to the WSGI environment dictionary, so that each
> > application can have its error log redirected to different files.
>
> I agree, but (a) that would have to be standardized somewhere to be
> useful, and (b) you still have to deal with code that isn't aware of
> this new functionality--especially libraries that are not WSGI-specific,
> and existing WSGI 1.0 applications.

The more and more that this discussion goes on, the conclusion I am
coming to is that WSGI applications should simply not be using the web
server log files for application logging at all.

The main problem with Apache at least is that Apache has its own
concept of a log level which is independent of any Python logging
system. Even if you were to map Python log messages to equivalent log
levels in Apache, if the Python log level threshold is set to be
higher than that which Apache is filtering at, then Apache will just
throw away the log messages. There is not way around this.

Given that Apache default is 'warn', it means that any Python log
messages at level of 'info' or 'debug' would be thrown away by Apache
if Python let them threw. It is unlikely that ISPs will run with
anything higher than 'warn' because of the extra output from Apache
modules themselves. If you are lucky they may allow 'info' within a
specific VirtualHost if it has its own log files, but certainly not
'debug'. As such, seems that using Apache error log levels for Python
is a bad match anyway.

Thus I really thing that having sys.stderr go to 'error' level in
Apache is more than reasonable as least it is displayed. If Python
logging module sends to sys.stderr, all logging will be seen without
problem. The only complaint seems to be that the Python log level is
tagged on to the start of the log message after the Apache log level
is shown.

In respect of using a web server feature for mailing out error
messages of a certain level, that just seems wrong to me. This should
be done in Python code using log handlers to the logging module if you
want to use the logging module. Alternatively, use a log handler which
sends logging to a proper logging system like syslog-ng and configure
it to do the mail outs.

Anyway, this whole discussion is taking too much of the little time I
have available at the moment. As such I intend to leave Apache
mod_wsgi as is and will not now even look at a log handler for Apache
error log files, or even a hook to allow one to log to Apache error
logs with a log level. If some official sort of specification comes up
for logging in conjunction with WSGI, or someone wants to come up with
working code that demonstrates exactly how it should work and it
addresses all issues, I'll look at it then.

Graham