[Python-Dev] Using logging in the stdlib and its unit tests

[Vinay Sajip]

Robert Kern <robert.kern <at> gmail.com> writes:

> I'm sorry, but it's not at all clear that you have understood my point.
> There is no way for me to parse your words as a sensible reply to what I
> said.
> 
> Let's say I write a library called Foo. I want to add logging to my
> functions. You want to write an application called Bar that uses Foo and
> you want to configure logging for your application (at the very least to
> provide a default if not production). The warning is supposed to help you
> not make mistakes when configuring logging in your application. If I,
> library author, attach NullHandlers to all of Foo's loggers, then you will
> not get that warning if you forget to add handlers the Foo loggers.
> My adding the NullHandler to Foo prevented that warning that you consider
> to be so important.
 
I understand, and what you say above is correct.

> I don't think the warning helps much, if at all.

Perhaps not. I'm not so hung up on the importance of the message now, but it
certainly *was* important when logging was first introduced, and users couldn't
expect to get the same level of help on comp.lang.python that they now can.
Today there are quite a few people who can help people with finger-trouble
logging issues.

Consider the scenarios when not having the current behaviour would bite you:

1. You're new to Python/new to logging. You write some code, perhaps across
several modules, which code makes logging calls, but you omit to configure any
handlers, whether through forgetting or not understanding what needs to be done.
Without the message, you've no idea why no logging messages appear, no matter
how much you fiddle with levels.
2. You write some code and decide you don't want to use logging, but use some
third party libraries which do. You don't care about that, so it's annoying to
have "no handlers could be found for logger XXX" messages printed to console.
You berate the library developer for their lack of consideration.

Perhaps you don't find yourself in these situations, but surely you sympathize
with people who do? How would you propose to address both those scenarios?

> In a large package (particularly a namespace package), I can't guarantee
> that any particular module will get imported. I will want to be able to
> import just foo.bar.baz without needing to worry about whether
> foo.setup_logging got imported and ran the logging configuration as a
> side-effect. I want to be able to loosen the coupling between modules
> across my package, not add more coupling.
> 

I'm not sure what coupling you're talking about - perhaps you can illustrate
with an example. If I develop a package "foo.bar" which is part of namespace
package "foo", and use loggers named __name__ in my code, and add a NullHandler
to logger "foo.bar", that's all I have to do. Likewise, if another person
develops "foo.baz" and they add a NullHandler to "foo.baz", then where's the
coupling between the two packages? They needn't even know about each other.

> But in any case, while adding a NullHandler to just the package's root logger
> helps you to avoid needing a NullHandler on every logger, the effect is the 
> same. Almost all loggers effectively terminate in a NullHandler either
> directly or through a chain of parent loggers. So why not just make it the
> default?
 

There's no "termination" when a NullHandler is encountered. Django has dozens of
modules, many of which could use logging, but only one NullHandler needs to be
added for the whole of Django. The effect you say is the same is not: when
adding new modules to Django, for example, no additional NullHandler adding
needs to be done.

I don't want to appear rude, but this seems to be another mistake (or perhaps a
variant of the same mistake as before) you are making about how logging works.
You obviously feel strongly about it, and if you have a specific use case which
is causing/has caused you pain, please spell it out for me (on comp.lang.python,
say) and I will try to help sort out the problem.

<offtopic>

As am off-topic example, Armin Ronacher kept on saying in various posts and
presentations that you couldn't use stdlib logging for web applications, that
there were fundamental problems with it. But when he actually sent me his
specific problem statement, I gave him a solution without spending too much time
on it (see
http://plumberjack.blogspot.com/2010/09/configuring-logging-for-web.html if you
care). I'm not trying to be obstructive, honestly.

</offtopic>

> Personally, I would back the proposals being made elsewhere in this thread,
> that in the absence of configuration, warnings and errors should be printed
> to stderr no matter where they come from. This gives useful behavior
> out-of-the-box

As you might have seen in my response to Nick's post, I've made specific
proposals about this myself. I agree with Nick's viee that some more time should
be given for discussion, suggestions, others to express views, and just for the
ideas to bake for a bit.

> without configuration but remains completely configurable. Library errors 
> don't pass silently, but logging allows people to silence them explicitly.
> It separates the concerns of library authors (who should never touch logging
> configuration and shouldn't be required to think about it) from those of the
> application authors and application users.

You surprise me with "library authors (who should never touch logging
configuration and shouldn't be required to think about it)". If you use logging,
you should think about it at least a little, just as with any other software you
use. For example, about default levels, propagation, etc. etc. The Django team,
for example, have expended a reasonable amount of thought on how best to add
logging support to their library; I don't believe their time has been wasted.

Regards,


Vinay Sajip