What should go to stdout/stderr and why Python logging write everything to stderr?

[Peter J. Holzer]

On 2023-01-05 12:29:33 -0800, Grant Edwards wrote:
> On 2023-01-05, Thomas Passin <list1 at tompassin.net> wrote:
> 
> > The logging system is so configurable that...
> 
> I find it almost impossible to use unless I copy a working example I
> find somewhere. ;)

I usually copy the working example from my previous project ;-).

I think the general structure is clear enough. You need a formatter to
format your log messages, a handler to actually write them and finally a
logger to determine what goes where. I can look up the details in the
docs but generally the logging is always the same (except for path names
and log levels), so I can just copy the config from last time and adjust
those.

So I might have a config.py like this:

    # ...
    logging = {
        "version": 1,
        "disable_existing_loggers": False,
        "formatters": {
            "standard": {
                "format": "%(asctime)s %(levelname)s %(name)s %(funcName)s %(lineno)d | %(message)s"
            }
        },
        "handlers": {
            "file": {
                "class": "switchinglogfilehandlers.TimeoutSwitchingFileHandler",
                "formatter": "standard",
                "filename": "/var/log/www/XXX.hjp.at/XXX.",
            },
        },
        "loggers": {
            "": {
                "handlers": ["file"],
                "level": "INFO"
            }
        }
    }
    # ...

And then my application would start like this:

    import logging
    import logging.config
    import config
    logging.config.dictConfig(config.logging)
    log = logging.getLogger(__name__)

Plus typically every other source file contains

    import logging
    log = logging.getLogger(__name__)

somewhere near the start.

Then I can just scatter my log.debug(...) or whatever whereever I want.
When I decide that I need debug output from one module, I'll just add a
logger. Or if some library is too chatty I'll add another logger to shut
it up - no programming, just a few extra lines in a dict.

(Instead of a config.py I might use a json file, especially if I expect
the config to change often.)

> I'm not at all surprised that the OP didn't understand how it
> works.

It probably helps to have worked with log4j before that. The structure
is very similar, although I find Python logging easier to use (but then
I never did much Java programming so that's probably just a matter of
familiarity.

        hp

PS: The TimeoutSwitchingFileHandler mentioned above is one I wrote
    myself. You can find it on PyPI, but be warned that the
    documentation is (still) quite lacking.

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/python-list/attachments/20230106/1087494a/attachment.sig>