[Python-Dev] Using logging in the stdlib and its unit tests

[Glenn Linderman]

On 12/11/2010 12:00 AM, Nick Coghlan wrote:
> On Sat, Dec 11, 2010 at 4:25 PM, Glenn Linderman<v+python at g.nevcal.com>  wrote:
>> On 12/10/2010 9:24 PM, Nick Coghlan wrote:
>> This could actually make a reasonably good basic for a "task oriented"
>> subsection of the logging documentation. Something like:
>>
>> Yep, agree.  But sadly, for each point, there may be multiple options (your
>> StreamHandler, but I'd want a FileHandler; your separation of messages by
>> level, my wanting them combined; etc.)
> No, no, no, that's the *whole point* of using logging. The library
> writer doesn't really *care* about where the messages end up - that is
> entirely in the hands of the application developer when they choose
> which handlers to install. The only situation that the library writer
> cares about is the one that tripped up concurrent.futures and that has
> already been changed for 3.2b2: that warnings and errors written to a
> logger were silenced by default if the application never even called
> basicConfig(). That has now been fixed so they will be emitted on
> sys.stderr instead (including obeying any redirections of stderr over
> the lifetime of the program).
>
> Notice that my task list is *entirely* from the point of view of the
> person emitting the messages. How those messages are later displayed
> is then up to the application writer (or the logging module default
> settings, if the application writer doesn't do anything specific).

Yes, yes, yes... that's what your list was!  I see that now!  But it is 
interesting that in missing that point, and reading your logging setup 
code, I saw that the same list of tasks is also exactly the sorts of 
things that you would want to control, for simple uses of the logger.  
Does basicConfig give that control?  I don't know, the first 8% of the 
logger documentation don't tell me that, they only give me a canned 
example (actually 3) without explanation of its full function.  Someday 
I'll read more.  Until then, basicConfig is opaque: I know only three 
incantations for using it, and not even any surety that they can be 
mixed, or that it can be called more than once in the same program.


>> Your comment about basicConfig setting the level on the root logger, but not
>> on the default handler making it useless is opaque to me,
> An early version of my code used basicConfig to create the stderr
> StreamHandler, but that turned out to be pointless since I wanted
> different levels on the logger and the handler.
>
>> but is there
>> perhaps room for another basic setup API that could get the setup code down
>> to a line or two in simple cases?
> For 3.2? No.
>
> For 3.3? Maybe.
>
>> Would that be a useful set of functionality to bundle?  And could it be
>> extended, when the user wants more power, or would it have to be replaced,
>> because it gets in the way of the user that wants more power?
> Logging already has powerful configuration mechanisms, especially
> following the addition of dictConfig
> (http://docs.python.org/dev/library/logging#logging-config-dictschema),
> so it really doesn't need anything else along those lines.

Sure; that is the replacement technique.  And maybe that is OK, if there 
are no useful intermediates between basicConfig (whatever it might be 
capable of) and dictConfig (or the other various config techniques that 
preceded it and still exist, if I understand correctly from has been 
said here on python-dev).

> At the simpler end, basicConfig already covers sending all messages to
> a single stream with a single format - it's only split-stream handling
> (such as stderr/stdout potentially being different endpoints) that it
> can't cope with.

OK, so I'd probably be fine with basicConfig, even though you weren't, 
once I understand it a little better.

> What may make more sense than yet another global config mechanism, is
> a module level "addHandler" helper function along the following lines:

And once I learn enough to understand the addHandler code, I will have 
read a lot of the logger documentation than I have to date, or it will 
have been restructured enough that re-reading the first 8%, or maybe 12% 
if the easy-to-understand part grows!  But I like what you said here:

> # Previous set up example is now only three lines long
> import sys, logging
> # Let root logger handlers see all messages
> logging.getLogger().setLevel(logging.NOTSET)
> # Send WARNING and above to stderr
> addHandler(stream=sys.stderr, level=logging.WARNING)
> # Send INFO to stdout
> addHandler(stream=sys.stdout, level=logging.INFO, max_level=logging.INFO)
>
> logging.info("Hello world!")
> logging.warn("Hello world!")

And this subset usage of addHandler is reasonably understandable for 
beginners, although clearly the complexity of all the parameters must 
add power that is not needed in the simple case, that you feel could be 
useful in the complex case.  And I would change the third line from 
level=logging.INFO to level=logging.DEBUG to see lots more on stdout, 
correct?

Looking back at the logger's simple examples section, I see that 
getLogger is used before being defined, is not explained, nor is its 
parameter explained.  So it is not clear how to go from the first simple 
example to the second simple example.  I wonder if that is where I 
started getting a bit of uncertainty about the ability to quickly learn 
what was going on... the rotating log example uses several APIs that 
haven't been used before, without explaining them at all.  Can that 
example actually be added to the previous example?  What is the magic 
'MyLogger'?  Where has basicConfig() gone in the second example?

I'm not expecting answers to the above questions here, but I'd love to 
see the documentation improved to the point that I don't have them any 
more :)

Glenn
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101211/05c86a1b/attachment.html>