[SciPy-User] Is there existing code to log-with-bells-on for runtime algorithm diagnostics?

[Cameron Simpson]

On 21Apr2015 16:46, Rob Clewley <rob.clewley at gmail.com> wrote:
>All of these ideas and links are very helpful, thank you!

Another to add to your list, a be warned that it is baroque.

I have a context manager named "Pfx" which I use liberally in my code like 
this:

  from cs.logutils import Pfx, info

  def load(filename):
    with Pfx("loading %r", filename):
      with open(filename) as fp:
        lineno = 0
        for line in fp:
          lineno += 1
          with Pfx("%d", lineno):
            ... do stuff with line ...
            info("line value is foo=%r", foo)
            ...

This causes the info() call to report like this:

  loading 'myfile.dat': 21: line value is foo=12345

It hooks into the logging system anyway, which means that you can (a) apply a 
format to the above to get leading timestamps and so forth and (b) you can use 
logging facilities like emit() or Handlers to direct the messages where you 
like.
  
Codewise, this has the following advantages:

  - the info() call just has the local message, context is added automatically

  - it nests arbitrarily through the call chain, so outer functions using Pfx add context as you'd hope:

    2015-04-22 10:09:00 main: out_func: loading 'myfile.dat': 21: lone value is foo=12345

  - the Pfx context manager catches and reraises exceptions also and embeds the context in the except message where possible, which is incredibly useful

It is _very_ useful in hierarchical processing situations.

Let me know if you want further information or the code.

Cheers,
Cameron Simpson <cs at zip.com.au>

If you give me six lines written by the most honest man, I will find
something in them to hang him.  - Cardinal Richilieu