[Python-Dev] Using logging in the stdlib and its unit tests

[Paul Moore]

On 8 December 2010 08:32, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Nick Coghlan <ncoghlan <at> gmail.com> writes:
>> I'm not proposing that the standard library be special-cased, I'm
>> proposing that the default behaviour of an unconfigured logging module
>> in general be changed to something more useful (i.e. warnings and
>> errors printed to stderr, everything else suppressed), rather than
>> unhelpfully suppressing *all* output except for an "Oh, I'm throwing
>> output away" message the first time it happens.
>
> I don't have a philosophical problem with this, but there might be differing
> opinions about this not just on python-dev but also in the wider community. I'd
> definitely want logging to do the right thing and minimize inconvenience as much
> as possible. There's also possibly a backwards compatibility dimension to this,
> but I'm not sure to what extent a change like this would really affect people in
> practice (as nothing will change if logging is configured).

To provide an application developer point of view, I'm +1 on this change.

I want library developers to be able to report errors/warnings and
have them appear by default, without me doing anything (or even
needing to be aware that the logging module is involved). The benefit
of the library developer using the logging module to me is that if I
deem it appropriate, I can direct the warnings/errors elsewhere simply
by configuring logging.

So, I'd like the library developer to feel that they can use logging
in precisely the same ways as they would write to sys.stderr, and with
the same expectations (output will be seen by default), but with less
reluctance because they know that I can easily adapt their output
according to my needs.

>> The options for handling this are:
>>
>> 1. Write the error detail directly to stderr. (Unless the default
>> behaviour of logging changes, that is what I am going to suggest Brian
>> do, as it exactly mimics the behaviour of the PyErr_WriteUnraisable
>> API).

I find library code that does this annoying. It's the right effect,
but leaves me with little or no control should I want to do something
like log to a file.

>> 2. Write it to the root logger with the convenience APIs (Possible
>> option, but I don't like the global state impact of implicitly calling
>> basicConfig() from library code)

Library code shouldn't use the root logger, and shouldn't call
basicConfig - I agree with Nick again.

>> 3. Add a StdErr handler for the logger (this is what is currently
>> implemented, and again, I don't like it because of the global state
>> impact on something that should be completely under an application's
>> control)

That seems like a best of a bad job answer. As an application user,
I'd expect to find it awkward to modify this default, purely because I
have to undo the configuration that the library had to do to work
around the logging library's defaults.

> Thanks for the detailed explanation. I agree that unraisable warnings and errors
> need to be handled somehow. There is a way in which this can be done without
> affecting a logging configuration, viz. logging can define a "handler of last
> resort" (not attached to any logger) which is invoked when there are no
> user-specified handlers. This would by default be a StreamHandler writing to
> sys.stderr with a threshold of WARNING (or perhaps ERROR). Thus sounds like a
> better option than a direct write to sys.stderr, since you can't change the
> latter behaviour easily if you want to do something else instead.

>From my POV as an application writer, I don't care how you implement
it (but I agree that your suggestion sounds more flexible). As long as
errors and warnings are printed by default, and I can set my own
configuration without having to undo the existing configuration, then
I'm happy.

> This is of course a backwards-incompatible change to logging semantics: instead
> of saying that logging will be silent unless explicitly asked to produce output,
> we're saying that logging will always produce output for warnings and errors (or
> perhaps just errors), unless explicitly silenced. This is of course in line with
> the Zen of Python; the present behaviour, which is not so aligned, is based on
> the idea that logging should not affect program behaviour if it's not wanted by
> the program developer (as opposed to library developer).

I can see that it's an incompatible change. But I'd still be in favour
of it for 3.2. In my view, it needs to be done, and it'll only get
harder to do as time goes on.

Paul.