[Web-SIG] serving (potentially large) files through wsgi?

[Graham Dumpleton]

On 21/12/2007, Chris Withers <chris at simplistix.co.uk> wrote:
> Manlio Perillo wrote:
> >> For me, it does feel like the responsibility of the server to
> >> configure logging, and I think this is something that should be
> >> documented somewhere. Afterall, as you guys have been discussing, it's
> >> the server that holds configuration for things like listening sockets,
> >> etc and logging feels like that kind of thing to me...
> >>
> >
> > In mod_wsgi for nginx I now redirect sys.stderr to server log file (as
> > suggested by Graham).
>
> I've never really understood this desire to do *anything* with
> sys.stderr. Writing to sys.stderr seems a very odd thing for any web
> component to use, especially as python now has a fantastic logging package.

I only ever suggested that sys.stderr be directed to the web server
log file as a fallback to cope with third party Python modules which
use it directly, or where the logging module defaults to outputing it
there.

> > However there are some problems.
> > The log object has a fixed error level (NGX_LOG_ERR);
> > this means that every message logged using this object will have this
> > error level, even if I do, as example:
> > log.info('just an info message')
>
> I'm missing the relationship between this and python's logging package.
> I'll note that the point you raise about why using sys.stderr for
> logging sucks are things I agree with ;-)

As I believe I have said before, I disagree with how you are trying to
make logging module output simply go to sys.stderr. If you want to
preserve the concept of logging levels, then expose the internal web
server logging function, including it accepting its concept of log
level, and write a log handler for the logging module that uses it
instead.

I can't get to the code right now, but have done exactly this with
Apache, where one can install a custom Apache log handler that
internally uses a SWIG'd binding for apache.http_log.ap_log_error().

So, there is no reason to force fit log levels into wsgi.errors or
sys.stderr somehow.

> > A better solution could be the integration with the logging module.
> >
> > However there are some problems here, too, since log levels from nginx
> > and the logging module differs.
>
> Why?

The possibility of mismatch on log levels is common in possible output
targets for logging module handlers. If you look at the source code
for some of the logging handlers you will see how there is the ability
to remap the log levels exposed at Python code level, to what may be
available in the output target.

Thus, I don't see this an issue, you just make it transparently map as
best you can in your nginx specific log handler for the logging
module.

Graham