[Python-ideas] Non-intrusive debug logging

[C Anthony Risinger]

On Thu, Jan 25, 2018 at 5:09 PM, Nick Timkovich <prometheus235 at gmail.com>
wrote:

> I think part of the reason that logging appears complicated is because
> logging actually is complicated. In the myriad different contexts a Python
> program runs (daemon, command line tool, interactively), the logging output
> should be going to all sorts of different places. Thus was born handlers.
> If you want "all the logs", do you really want all the logs from some
> library and the library it calls? Thus was born filters.
>
> For better or worse, the "line cost" of a logging call encourages them to
> be useful.
>

I think that last bit is the OP's primary ask. Truly great and useful logs
are genuinely hard to write. They want a cross-cutting, differentially
updated context that no single section of code a) cares about or b)
willingly wants to incur the costs of... especially when unused.

In my mind, the most significant barrier to fantastic logging -- DEBUG in
particular -- is you must 100% understand, ahead-of-time, which data and in
which situations will yield solutions to unknown future problems, and then
must limit that extra data to relevant inputs only (eg. only DEBUG logging
a specific user or condition), ideally for a defined capture window, so you
avoid writing 100MBps to /dev/null all day long. While this proposal does
limit the line noise I don't think it makes logging any more accessible, or
useful.

The desire to tap into a running program and dynamically inspect useful
data at the time it's needed is what led to this:

Dynamic logging after the fact (injects "logging call sites" into a target
function's __code__)
https://pypi.python.org/pypi/retrospect/0.1.4

It never went beyond a basic POC and it's not meant to be a plug, only
another point of interest. Instead of explicitly calling out to some
logging library every 0.1 lines of code, I'd rather attach "something" to a
function, as needed, and tell it what I am interested in (function args,
function return, symbols, etc). This makes logging more like typing, where
you could even move logging information to a stub file of sorts and bind it
with the application, using it... or not! Sort of like a per-function
sys.settrace.

I believe this approach (ie. with something like __code__.settrace) could
fulfill the OP's original ask with additional possibilities.

-- 

C Anthony
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20180125/88dd0e8d/attachment-0001.html>