How to accept argparse.log_level parameter and propagate its value to all other modules

[Peter Otten]

On 27/01/2021 20:04, Zoran wrote:
> In the same folder I have three files: main.py, app_logger.py and mymodule.py with their contents:
> 
> # main.py
> import argparse
> import app_logger
> import mymodule
> import logging
> 
> logger = app_logger.get_logger(__name__)
> 
> def log_some_messages():
>      logger.debug(f'{__name__} - debug message')
>      logger.info(f'{__name__} - info message')
>      logger.warning(f'{__name__} - warning message')
> 
> if __name__ == '__main__':
>      parser = argparse.ArgumentParser(description="Script for executing data quality checks.")
>      parser.add_argument("--log-level", default='info', choices=['notset', 'debug', 'info', 'warning', 'error', 'critical'], help="set log level")
>      args = parser.parse_args()
> 
>      logger.setLevel(eval(f"logging.{args.log_level.upper()}"))
> 
>      log_some_messages()
>      mymodule.log_some_messages()
> 
> 
> #mymodule.py
> import logging
> 
> from script_logging import app_logger
> 
> logger = app_logger.get_logger(__name__)
> 
> 
> def log_some_messages():
>      logger.debug(f'{__name__} - debug message')
>      logger.info(f'{__name__} - info message')
>      logger.warning(f'{__name__} - warning message')
> 
> 
> #app_logger.py
> import logging
> 
> _log_format = f"%(asctime)s - [%(levelname)s] - %(name)s - (%(filename)s).%(funcName)s(%(lineno)d) - %(message)s"
> _log_level = logging.DEBUG
> 
> def get_stream_handler():
>      stream_handler = logging.StreamHandler()
>      stream_handler.setLevel(_log_level)
>      stream_handler.setFormatter(logging.Formatter(_log_format))
>      return stream_handler
> 
> def get_logger(name):
>      logger = logging.getLogger(name)
>      logger.setLevel(_log_level)
>      logger.addHandler(get_stream_handler())
>      return logger
> 
> When I run it with main.py --log-level debug I get the following output:
> 
> 2021-01-25 13:21:01,151 - [DEBUG] - __main__ - (main.py).log_some_messages(10) - __main__ - debug message
> 2021-01-25 13:21:01,151 - [INFO] - __main__ - (main.py).log_some_messages(11) - __main__ - info message
> 2021-01-25 13:21:01,152 - [WARNING] - __main__ - (main.py).log_some_messages(12) - __main__ - warning message
> 2021-01-25 13:21:01,152 - [DEBUG] - mymodule - (mymodule.py).log_some_messages(10) - mymodule - debug message
> 2021-01-25 13:21:01,152 - [INFO] - mymodule - (mymodule.py).log_some_messages(11) - mymodule - info message
> 2021-01-25 13:21:01,152 - [WARNING] - mymodule - (mymodule.py).log_some_messages(12) - mymodule - warning message
> 
> which is OK. If I change --log-level parameter to 'warning', than output is the following:
> 
> 2021-01-25 13:22:12,760 - [WARNING] - __main__ - (main.py).log_some_messages(12) - __main__ - warning message
> 2021-01-25 13:22:12,760 - [DEBUG] - mymodule - (mymodule.py).log_some_messages(10) - mymodule - debug message
> 2021-01-25 13:22:12,761 - [INFO] - mymodule - (mymodule.py).log_some_messages(11) - mymodule - info message
> 2021-01-25 13:22:12,761 - [WARNING] - mymodule - (mymodule.py).log_some_messages(12) - mymodule - warning message
> 
> As you can see, in main.py log level is set correctly, but in mymodule.py it is not.
> 
> How to accept argparse.log_level parameter in if __name__ == '__main__': and propagate its value to all other modules?

You just have to set the level in the root logger. The easiest way to 
achieve that is to forget about your app_logger wrapper around logging 
and call

logging.basicConfig(level=yourlevel, format=yourformat)

in  main.py and to create the loggers in main and mymodule

with

logger = logging.getLogger(__name__)