[py-dev] py.log -- annotating logs

[Grig Gheorghiu]

--- Ian Bicking <ianb at colorstudy.com> wrote:

> I'm guessing you meant to copy the list...
> 
> Grig Gheorghiu wrote:
> > Maybe the easiest way would be to just define a consumer per
> > "application domain", i.e. for your Web app you'd have a
> > web_app_consumer() function that would know how to print all the
> info
> > you need. 
> > 
> > The way it works right now is that you need to associate a consumer
> > with one or more keywords. Then you have the flexibility of
> changing
> > consumers for those keywords dynamically via configuration files
> etc.
> > 
> > So I'd do something like this in a configuration file:
> > 
> > log = py.log.Producer("mywebapp")
> > def my_consumer(message):
> >     remote_ip = get_current_remote_ip()
> >     crt_time = get_current_time()
> >     message.args = [crt_time, remote_ip] + message.args
> >     print str(message) # prints "[mywebapp] crt_time remote_ip
> > <rest_of_message>"
> 
> This particular example doesn't work well with other consumers.  I'd 
> want to change the message and then let another consumer deal with
> that 
> message.
> 
> Unfortunately this kind of composition through dispatching is very 
> configuration-unfriendly -- that's something we're hitting with WSGI 
> middleware configuration right now.  But what makes it unfriendly is 
> also what makes it very flexible.
> 
> The wrapping consumer I gave before could, potentially, be inserted
> in a 
> number of places.  It could be put in place globally, or for a single
> 
> keyword, or whatever.  This presumes, of course, that it's possible
> to 
> find consumers without running them, so a new consumer can be
> inserted 
> instead of just overwriting the old one.
> 
> This can be challenging in itself.  There might be one root consumer.
> 
> Let's say by default it's the consumer that dispatches based on
> keyword. 
>   If we try to insert another consumer in the root consumer, and the 
> dispatching consumer has been replaced, the replacement doesn't 
> necessarily have the same API.  Should that replacement consumer send
> 
> all unknown attributes to its sub-consumer?  But the keyword
> dispatching 
> consumer doesn't have a single sub-consumer to dispatch to, so that 
> doesn't work there.  And other dispatchers are also possible -- for 
> instance, this application supports multiple clients, and if a client
> is 
> associated with a request I'd like to put the log file in a location 
> specific to that client.  But that's more programmatic than keywords,
> 
> and I can create the sub-consumers dynamically.  Or maybe some
> component 
> is generating annoying log messages, and I want to filter them out
> based 
> on some really arbitrary expression.  There's lots of possibilities. 
> But the setup can become hard.
> 
> But then, it's clear that the logging module doesn't have easy setup 
> either.
> 

You're right, you get tremendous flexibility, but the onus is on you to
cleanly configure it. I think this fits well with the overall py lib
mantra, which is "No API" :-)

Seriously though, I think Holger's intention with py.log is to keep it
as simple as possible and let the application developers worry about
creating specific consumers, etc. But if you start using it on Paste
and add your own hierarchy of consumers, then I'm sure we'll be able to
incorporate tons of features back into the py.log basic functionality.

Grig