[Web-SIG] Fwd: Can writing to stderr be evil for web apps?

[Alan Kennedy]

[anatoly]
> Martin expressed concerns that using logging module with stderr output
> can break web applications, such as PyPI.

Please can you specify exactly what you mean by "using logging module
with stderr output"?

Dealing with stderr is a webserver specific concern.

Consider the case where you're the author of a webserver that deals
with CGI scripts.

When you get a request for the CGI script, you start a subprocess to
run the script. You must decide what to do with the stdin, stdout and
stderr of the process.

 - CGI mandates that any content that came with the request (e.g. a
POST body) should be fed into stdin(if no other mechanism is in
place[0])
 - CGI mandates that the stdout of the process is sent back to the
client (if no other mechanism is in place[1]).
 - CGI makes no mention of stderr.

Various webservers permit configurable handling of stderr.

For example, Tomcat has a setting called "swallowOutput" which
redirects both stdout and stderr to a log file. (Obviously, Tomcat's
treatment of stdout is different for CGI)

http://tomcat.apache.org/tomcat-6.0-doc/config/context.html

WSGI has a specific mechanism for diagnostic output, wsgi.errors.

"""
wsgi.errors 	

An output stream (file-like object) to which error output can be
written, for the purpose of recording program or other errors in a
standardized and possibly centralized location. This should be a "text
mode" stream; i.e., applications should use "\n" as a line ending, and
assume that it will be converted to the correct line ending by the
server/gateway.

...

For many servers, wsgi.errors will be the server's main error log.
Alternatively, this may be sys.stderr, or a log file of some sort. The
server's documentation should include an explanation of how to
configure this or where to find the recorded output. A server or
gateway may supply different error streams to different applications,
if this is desired.
"""

Lastly, note that WSGI supplies an example CGI gateway, about which it
has this to say about error handling

"""
Note that this simple example has limited error handling, because by
default an uncaught exception will be dumped to sys.stderr and logged
by the web server.
"""

http://www.python.org/dev/peps/pep-3333/#the-server-gateway-side

So I would say that

1. If you are writing a web application, and want it run under any
WSGI container, and for the user to be able to control that output in
a way with which they are familiar (i.e. which is documented and may
have specific configuration options), send the output to wsgi.errors.

2. If you are writing a web server, you should either capture or
ignore stderr. If it is captured, then it is reasonable to, e.g.,
write it to a file so that the user can find it. It should never be
mixed with stdout if stdout is the mechanism by which the application
communicates with the webserver, as with CGI.

Alan.

[0] http://ken.coar.org/cgi/draft-coar-cgi-v11-03.txt
"""
Section 6.2 Request Message-Bodies

   As there may be a data entity attached to the request, there
   MUST be a system defined method for the script to read these
   data. Unless defined otherwise, this will be via the 'standard
   input' file descriptor.
"""

[1] http://ken.coar.org/cgi/draft-coar-cgi-v11-03.txt
"""
Section 7. Data Output from the CGI Script

   There MUST be a system defined method for the script to send
   data back to the server or client; a script MUST always return
   some data. Unless defined otherwise, this will be via the
   'standard output' file descriptor
"""