Logging from different python scripts to different output files

[Peter Otten]

James McMahon wrote:

> I'm struggling with Python logging. Have tried to apply several examples I
> have found in the open source literature, but can't get it to work. What I
> need to do is this: I have two python scripts, a.py and b.py. Each is
> called by NiFi ExecuteScript processor repeatedly against many incoming
> flowfiles. In every case I need to output log messages to two distinct and
> different log files. My scripts run in the same Python interpreter, which
> as i understand it means they rely on the same root level logger. I have
> tried establishing non-root loggers for each, and associating distinct
> file handles for each to those loggers.
> 
> My output still appears in multiple log files, and appears to be repeating
> in increasing numbers each time I try a test run (first test, output line
> appeared in log once. second test, twice. third test, three times, etc).
> Is there an example that anyone knows of that demonstrates how two
> concurrently running python scripts within the same interpreter can direct
> output to two distinct log files? Thanks in advance for any help.

Here's a simple example:

$ cat logme.py 
import logging

a = logging.getLogger("a")
b = logging.getLogger("b")

def add_handler(logger, filename):
    if logger.hasHandlers():
        print("add_handler() called twice for logger 
{!r}".format(logger.name))
        return
    formatter = logging.Formatter(logging.BASIC_FORMAT)
    handler = logging.FileHandler(filename)
    handler.setFormatter(formatter)
    logger.addHandler(handler)

add_handler(a, "logs/a.log")
add_handler(b, "logs/b.log")

a.warn("foo")
b.warn("bar")

The 'if logger.hasHandlers(): ...' check is for debugging purposes, in your 
actual code a.py would only contain

import logging
a = logging.getLogger("a")
a.warn("whatever")

The corresponding setup code

import logging
a = logging.getLogger("a")
add_handler(a, "logs/a.log")

should only run once.