[Python-Dev] Using logging in the stdlib and its unit tests

[Nick Coghlan]

On Sat, Dec 11, 2010 at 4:25 PM, Glenn Linderman <v+python at g.nevcal.com> wrote:
> On 12/10/2010 9:24 PM, Nick Coghlan wrote:
> This could actually make a reasonably good basic for a "task oriented"
> subsection of the logging documentation. Something like:
>
> Yep, agree.  But sadly, for each point, there may be multiple options (your
> StreamHandler, but I'd want a FileHandler; your separation of messages by
> level, my wanting them combined; etc.)

No, no, no, that's the *whole point* of using logging. The library
writer doesn't really *care* about where the messages end up - that is
entirely in the hands of the application developer when they choose
which handlers to install. The only situation that the library writer
cares about is the one that tripped up concurrent.futures and that has
already been changed for 3.2b2: that warnings and errors written to a
logger were silenced by default if the application never even called
basicConfig(). That has now been fixed so they will be emitted on
sys.stderr instead (including obeying any redirections of stderr over
the lifetime of the program).

Notice that my task list is *entirely* from the point of view of the
person emitting the messages. How those messages are later displayed
is then up to the application writer (or the logging module default
settings, if the application writer doesn't do anything specific).

> Your comment about basicConfig setting the level on the root logger, but not
> on the default handler making it useless is opaque to me,

An early version of my code used basicConfig to create the stderr
StreamHandler, but that turned out to be pointless since I wanted
different levels on the logger and the handler.

> but is there
> perhaps room for another basic setup API that could get the setup code down
> to a line or two in simple cases?

For 3.2? No.

For 3.3? Maybe.

> Would that be a useful set of functionality to bundle?  And could it be
> extended, when the user wants more power, or would it have to be replaced,
> because it gets in the way of the user that wants more power?

Logging already has powerful configuration mechanisms, especially
following the addition of dictConfig
(http://docs.python.org/dev/library/logging#logging-config-dictschema),
so it really doesn't need anything else along those lines.

At the simpler end, basicConfig already covers sending all messages to
a single stream with a single format - it's only split-stream handling
(such as stderr/stdout potentially being different endpoints) that it
can't cope with.

What may make more sense than yet another global config mechanism, is
a module level "addHandler" helper function along the following lines:

from logging import Formatter, FileHandler, StreamHandler, getLogger
def addHandler(*, handler=None, stream=None, filename=None, filemode='a',
                          format=None, datefmt=None, style='{',
                          level=None, max_level=None, filters=(),
                          logger=None):
    """stream, filename, level, format, datefmt, style: as per
logging.basicConfig

        handler: use a precreated handler instead of creating a new one
        logger: logger to add the handler to (uses root logger if none
specified)
        filters: an iterable of filters to add to the handler
        max_level: can optionally limit the handler to messages of a
certain level and below
    """
    # Create the handler if one hasn't been passed in
    if handler is None:
        if filename is not None:
            handler = FileHandler(filename, filemode)
        else:
            handler = StreamHandler(stream)
    # Set up the formatting of the log messages
    # New API, so it can default to str.format instead of %-formatting
    formatter = Formatter(format, datefmt, style)
    handler.setFormatter(formatter)
    # Set up filtering of which messages to handle
    if level is not None:
        handler.setLevel(level)
    if max_level is not None:
        def level_ok(record):
            return record.levelno <= max_level
        handler.addFilter(level_ok)
    for filter in filters:
        handler.addFilter(filter)
    # Add the fully configured handler to the specified logger
    if logger is None:
        logger = getLogger()
    logger.addHandler(handler)
    return handler

# Previous set up example is now only three lines long
import sys, logging
# Let root logger handlers see all messages
logging.getLogger().setLevel(logging.NOTSET)
# Send WARNING and above to stderr
addHandler(stream=sys.stderr, level=logging.WARNING)
# Send INFO to stdout
addHandler(stream=sys.stdout, level=logging.INFO, max_level=logging.INFO)

logging.info("Hello world!")
logging.warn("Hello world!")

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia