[Python-Dev] Consistent logging in the standard library

[Guido van Rossum]

> The method for doing so is modelled after the socket module's new timeout
> functionality [3], where each module that performs logging defines a pair
> of module-level functions named getdefaultlogger() and setdefaultlogger()
> that return and accept (respectively) a logging.Logger instance.  The
> initial default logger for each of these modules is named after the module
> itself (i.e. logging.getLogger(__name__)) and its behavior mimics that of
> the Python 2.3 version of the module in terms of format and destination of
> the logging messages.

Hm...  Isn't the logging module's ability to set separate *handlers*
sufficient to get the desired flexibility?  Rather than the
application specifying a different *logger* object, the application
can specify a different *handler* object.  I think this was the
originally intended design.  It places less of a burden on the modules
that use logging (no need to define getdefaultlogger() and
setdefaultlogger()) and makes it possible for the different handlers
to be specified by the logging module's configuration file, without
the application having to support this.

Then what remains is a convention for the name of the logger used by
standard library modules; perhaps the module name should suffice?

> So an application that wants to override the logging behavior of a
> standard library module can do so simply by passing a customized logger
> object to the module's setdefaultlogger() function.  This works out even
> better for modules where all the logging is encapsulated in a class.
> 
> The PEP calls for each class that performs logging to define an attribute
> called self.logger, and upon instantiation initialize self.logger to the
> module's *current* default logger (again, modelled after how the timeout
> value for a socket object is initialized).  This allows each instance of a
> class to have its own unique logging behavior.

What behavior exactly are you thinking of?  A logger object has almost
no semantics apart from passing log events on to handlers; by design,
all the work is done in the handlers.

> The first item for discussion is whether this seems like a reasonable
> approach.  The second is identifying which standard library modules are
> candidates for modification using this approach.  I've identified the
> following by simply searching for the word "logging" in the source code,
> but I'm not sure whether this list is complete:
> 
>         asyncore
>         BaseHTTPServer
>         cgi
>         doctest
>         imaplib
>         unittest
>         the distutils package
> 
> I have patches ready for asyncore and BaseHTTPServer and doctest, which
> fit the model I described above quite nicely.

I think it would be useful to let these use the logging module even
without the proposed setdefaultlogger() etc. architecture.

> I determined that no changes are necessary for the cgi module, since
> it doesn't even use the logging functions it defines (initlog,
> dolog, etc.).

They're intended for use by CGI applications.  They should probably be
rewritten to direct the calls to the logging module.

> I'm currently working on a way to replicate imaplib's command
> history using LogRecords, which may call for a new type of
> logging.BufferingHandler subclass that maintains a circular queue of
> LogRecords.

That might be a useful feature on its own for the logging package.
Note though, that at a grander scale, the RotatingFileLogger class
already supports a way to keep only the last N messages (for much
larger N, though :-).

> And the distutils package seems prepped for using the logging
> package already.
> 
> The oddball is unittest.  Unittest is what motivated me to bring up the
> logging issue in the first place, but I can't (or more precisely - am not
> in a position to) decide what direction that module should take w.r.t. if
> and how it should use the logging package.  Should the TextTestRunner
> class be modified to use a logging.Logger instead of a _WritelnDecorator,
> or should a separate TestRunner subclass (a LoggingTestRunner perhaps) be
> defined instead?  I'm hoping that Steve Purcell or others may have some
> thoughts on this.

To reach Steve, you should probably post to the PyUnit list (sorry,
you'll have to Google for details, I only vaguely recall that there is
such a list).

Personally, I think that unittest is such a special case that it might
be best to leave it alone...

> I would also appreciate hearing any alternatives to the approach I've
> suggested, as well as comments on the strengths and weaknesses of each.

--Guido van Rossum (home page: http://www.python.org/~guido/)