Trouble propagating logging configuration

[Loris Bennett]

"Dieter Maurer" <dieter at handshake.de> writes:

> Loris Bennett wrote at 2021-9-1 13:48 +0200:
>> ...
>>Yes, but to quote from https://docs.python.org/3.6/howto/logging.html#logging-basic-tutorial:
>>
>>  A good convention to use when naming loggers is to use a module-level
>>  logger, in each module which uses logging, named as follows:
>>
>>    logger = logging.getLogger(__name__)
>>
>>  This means that logger names track the package/module hierarchy, and
>>  it’s intuitively obvious where events are logged just from the logger
>>  name.
>>
>>so in this case the source layout is relevant, isn't it?
>
> Relevant in this case is the package/module hierarchy.
>
> Often the package/module hierarchy follows the source layout
> **BUT** this is not necessarily the case.
>
> In particular, the "start" module of a script is called `__main__`
> indepently of its location.
> Furthermore, so called "namespace packages" consist of
> decentralized (i.e. located at different places in the file system)
> subpackages.
>
> Thus, in general, the connection between pachage/module hierarchy
> and the source layout is loose.
>
>
>>> Furthermore, the place of the configuration (and where in the
>>> code it is activated) is completely irrelevant for the "inheritance".
>>
>>OK, so one issue is that I was getting confused by the *order* in which
>>modules are being called.  If I have two modules, 'foo' and 'bar', in
>>the same directory, configure the logging just in 'foo' and then call
>>
>>
>>  foo.some_method()
>>  bar.some_method()
>>
>>then both methods will be logged.   If I do
>>
>>  bar.some_method()
>>  foo.some_method()
>>
>>then only the method in 'foo' will be logged.
>
> Usually, log configuration is considered a (global) application
> (not a (local) package/module) concern:
> The components (modules) decide what to log at what level
> and global configuration decides what to do with those messages.
>
> Thus, typically, you configure the complete logging in
> your main application module and then start to import modules
> and call their functions.
>
>> ...
>>If I have
>>
>>  [loggers]
>>  keys=root,main,log_test
>>
>>in my logging configuration and initialise  the logging in run.py  ...
>
> This logging configuration is obviously not complete (thus, I cannot
> check what happens in your case).
> You may have made errors at other places in your configuration
> which may explain your observations.
>
> At your place, I would look at the `logging` source code
> to find out where you can see (in an interactive Python session)
> which loggers have been created by your configuration and
> how they are configured. For the last part you use "vars(logger)".

Thanks Peter and Dieter for all the help.  I have finally figured out
what my problem was.  If in a module 'mylibs.mylib' I have

  import logging

  logger = logging.getLogger(__name__)

and in my main script have

  import logger
  import logger.config

  import mylibs.mylib

  logging.config.fileConfig("/home/loris/config")
  logger = logging.getLogger(__name__)
 
then 'logger' in 'mylibs.mylib' is defined before
'logging.config.fileConfig' is called and the logger in the module is
not configured.

If change the order and write

  import logger
  import logger.config

  logging.config.fileConfig("/home/loris/config")
  logger = logging.getLogger(__name__)

  import mylibs.mylib

then the 'logger' in 'mylibs.mylibs' does get configured properly.

I'm still thinking what implications this has if I want to load a
configuration file via a command-line option.

Cheers,

Loris

-- 
This signature is currently under construction.