limitations of current logging package (was Re: ANN: Logging Package v0.4.9.4 released)

[Peter Hansen]

Vinay Sajip wrote:
> What exactly is hard to use? An example of the minimalist way of
> logging:
> 
> import logging
> 
> logging.warn("This is your %s warning", "first")
> #similarly you can call logging.debug(), logging.info() etc.

The above shows clearly that one can use logging in a simple
way, but as Michael points out, it's not well documented how
simple it can be.  Even having read this thread and knowing
in advance that I could do that, I wasn't able to find the
relevant information easily in the documentation with
several minutes of reading (and several minutes covers a
lot of territory normally).  Even when I found the comments
about the shortcut functions, I wouldn't have been certain
how to use them (as above) because an example is missing.

Unlike Michele, however, I find the logging package to be
fairly straightforward and easy to use (though admittedly
after a few months of disuse maybe I will not... but in
such cases I tend to cut and paste the setup code from
past projects and don't care).

What I did find, however, were several limitations, at least
one of which I think is easily resolved.

1) I wanted to log an except from a sys.excepthook handler.
You can't, near as I can tell, without doing something like
the following incantation:

def excepthook(*args, **kwargs):
     root = logging.getLogger()
     fn, lno = root.findCaller()
     root.handle(root.makeRecord(root.name, logging.ERROR, fn, lno,
         'uncaught exception', (), exc_info=args))

Yuck!  Why is it so awkward?  Well, it appears that it is
impossible to pass exception info to a Logger, because the
special "exc_info" argument is merely a boolean flag that
tells it to retrieve the info from sys.exc_info().

What if sys.exc_info() doesn't have the data, as in the above
real-world example, and others I could make up?  Well, in that
case you just create a Formatter() and use formatException()...
except that there doesn't appear to be a (documented) way of
getting the formatter out of the root Logger...  I'm using
fileConfig() to set things up, so I don't have direct access
to any of the objects and don't see an obvious way past the
above ugly code (which is hoisted out of Logger._log()).

2) I wanted my log files written to the application's own
directory, rather than to the current directory.  Using
the fileConfig capability, you can't, at least not without
using absolute paths (near as I can tell).

My hack workaround was to prepopulate the logging module's
vars() with one call _app_path, and then to reference that
in the config file in the following way.  This works only
because (a) eval is used on the args, and (b) the os module
has already been imported by logging.__init__:

(in the main code:)
logging._app_path = MY_APP_PATH
logging.config.fileConfig(MY_CONFIG_FILE)

(in the config file:)
[handler_hand01]
class=handlers.RotatingFileHandler
level=NOTSET
formatter=form01
args=(os.path.join(_app_path, 'bod.log'), 'a', 10000000, 9)


Now I can sort of see why the second issue might be awkward
to resolve (since it is in effect a problem that appears in
various other ways in the Python world, with no standard
solution, mainly because there's no simple "sys.getappdir()"
or whatever it would be called).

For the first problem, I think the exc_info "flag" should be
changed so that if it contains actual exception info, or
maybe just a tuple of three items, then *that* information
is logged instead of a call to sys.exc_info() being made.

But these are just warts, and I'm sure there are others, but
I do find the logging facility to be fairly elegant, all
things considered, and not very hard to figure out.  (I'd
never used it before this thread came up, but the timing
was right and I just had a need for it, finally.)

-Peter