Fwd: Re: [py-dev] Logging in the py library

[Grig Gheorghiu]

For some reason I only sent the message below to Holger, not to the
list. Here goes:

=============================================================

Hi, Holger

--- holger krekel <hpk at trillke.net> wrote:
 
> Adding Tracing/Logging to the py lib is definitely important. 
> I'd like to hide the logging module under some nice 
> super-simple API, though.  Or maybe not use the logging module 
> at all or only optionally :-) 
> 
> I think that the severity categorization is not enough to make
> nice use of logging information.  I don't have a fully fledged
> idea but i think that having a keyword-based approach to
> tracing would be more helpful.  Basically we could associate
> with each logging message/object [*] a set of keywords.
> severity categorizations would just be a keyword in that idea. 
> 
> Let me try an example for e.g. tracing in py/execnet/gateway.py: 
> 
>     import py  # this import is actually problematic to do in
> gateway.py, 
>                # let's forget about that detail for the moment
> 
>     trace = py.trace.keywordtracer('execnet gateway') 
> 
>     class Gateway(object): 
>         def __init__(self, ...): 
>             trace("initializing gateway ...")
>             ... 
> 
> 'trace' may by default add a 'debug' keyword if called like this. 
> If we want to print some warning, then we could do something like: 
> 
>     trace.warn("remote gateway side could not be destroyed") 
> 
> which would add the keyword 'warning' to that particular message.  


So is this association static or dynamic? If I understand correctly,
you'd want it to be dynamic, right? So for example trace.foo would add
'foo' to the message. But the severity levels need to be statically
defined, at least if we want to mimic what the logging module does. In
the logging module, you can set a default severity level, so that the
logger will output only those messages that have greater or equal
severity levels. 

> 
> By default, everything would probably go to stdout but an
> application should be able to redirect by keyword to specific
> formatting/output backends (much like the logging module).  I
> think it's important to allow this redirection to happen
> _after_ the above example is already imported, so
> redirections-by-keyword should be re-configurable at runtime. 

Yes, it would be nice to have the possibility of sending the messages
to a variety of handlers (stdout, files, sockets, HTTP servers, etc.)
One reservation I have about stdout being the default is that it might
interfere with py.test's own catching of stdout/stderr. I think it
would be better to trace to a file by default (I think that's what the
twisted.log module does too, although I haven't looked at it, I only
saw it mentioned somewhere).

> What do you think of this basic keyword idea?  (which cold
> still use the logging module underneath for accessing all 
> the backends if it makes sense). 
> 
> And sorry, but the py lib really aims at exploring
> improvements over current ways of doing things :-) 
> 

I think it's a very good idea. In my opinion, it would still be worth
using the logging module underneath, since all the grunt work is
already done there. I hate reinventing wheels. But polishing and
simplifying the interface to the logging module is certainly something
to be aimed for -- and your keyword idea goes a long way towards that
goal.

> 
> [*] one concern is how much overhead tracing adds to an application. 
>     It should be possible to trace via objects (or
> lambdas/functions?, whatever) 
>     that only produce a str() representation when there is a consumer
>     for the tracing information.  In particular, if i "ground" the
> 'debug' 
>     keyword to /dev/null then the tracing code should not spend time 
>     str()ing objects.  I don't like the logging module's way of 
>     turning messages into strings too early. So e.g. 
> 
>         trace.debug("gateway", gateway, "processes", message) 
> 
>     would not compute all the str()s of the objects if i (somehow) 
>     set the debug keyword to 'ignore'. 

Agreed.

Grig