[Python-Dev] Passing contextual information when logging

[Vinay Sajip at Red Dove]

I recently posted the following to c.l.py, and didn't get much feedback. I'm
planning to add a new class to the logging package, and before I do so I'd
like to get feedback from python-dev. I know many of the devs may not be
monitoring c.l.py, so I'm reposting here for your feedback.

Some users of the logging package have raised an issue regarding the
difficulty of passing additional contextual information when logging. For
example, the developer of a networked application may want to log, in
addition to specifics related to to the network service being provided,
information about the IP address of the remote machine and the username of
the person logged into and using the service.

Python 2.4 introduced an 'extra' keyword argument which was intended to hold
a dict-like object containing additional information to be added to a
LogRecord. The additional information would then be printed via placeholders
in a format string.

While this works, it is a little unwieldy to use in practice, because users
need to provide the 'extra' parameter in every logging call. This has led
people in some instances to create context-specific Logger instances (e.g.
one logger for every connection). This has a drawback in that a logger name
can only provide limited contextual information, and moreover, if the number
of connections is effectively unbounded over time, the number of Logger
instances will also grow in an unbounded way. (Logger instances are never
garbage collected, because references to them are always held in the logging
package. This alleviates a burden on users in that they never have to pass
loggers around, but means that creating a lot of Logger instances will lead
to a long-lived memory burden.)

One solution is to create a generic wrapper around loggers to which a logger
name and contextual information can be passed. The wrapper would delegate
logging calls to the logger with the specified name, but would manipulate
the arguments passed to the logging call to insert the contextual
information. I have created such a wrapper class, called LoggerAdapter,
which is in the example script located at

http://dpaste.com/30613/

I would welcome your views on whether the LoggerAdapter class is suitable
for adding to the logging package in Python 2.6/3.0. Does it do what might
reasonably be expected out of the box? LoggerAdapters are, of course,
garbage collected in the normal way and so impose no particular memory
burden.

I'll wait a few days for any comments/suggestions from this list, then
(assuming no adverse comments) go ahead and add the LoggerAdapter class and
update the docs.

Best regards,

Vinay Sajip