using configobj string interpolation and logging.config.dictConfig (was Re: dictConfig: logging.StreamHandler object is not iterable.)

[Tim Williams]

On Wednesday, May 24, 2017 at 5:47:37 PM UTC-4, Peter Otten wrote:
> Tim Williams wrote:
> 
> > Just as a followup, if I use 'unrepr=True' in my ConfigObj, I don't have
> > to convert the strings.
> 
> I'd keep it simple and would use JSON...

I looked at JSON at first, but went with configobj because I didn't see where it did string interpolation, which I needed for other parts of my INI file, and I'm trying to use it to specify my log file in my handler.

Which brings me to ...

I have this stripped down INI file:

[loggng]
version = 1
level = 'INFO'
RootDir = 'TestData'
CaptureDrive = 'C:/'
LogFile = '%(CaptureDrive)s%(RootDir)s/test.log'
	[[formatters]]
		[[[fmt1]]]
			format = '%(asctime)s: (%(levelname)s)  %(message)s'
			datefmt =
	[[handlers]]
		[[[console]]]
			class = 'logging.StreamHandler'
			level = 'INFO'
			stream = 'ext://sys.stdout'
		[[[file]]]
			class = 'logging.FileHandler'
			level = 'WARN'
			filename = '%(LogFile)s'
			#filename = 'cfg://loggng.LogFile'
	[[loggers]]
		[[[root]]]
		level = 'INFO'
		handlers = ['file','console']

When I parse the INI file with configobj:
config = configobj.ConfigObj('loggingtest.ini', unrepr=True, raise_errors=True)

config['loggng']['handlers']['file']['filename'] evaluates correctly to C:/TestData/test.log

However, when I try to call logging.config.dictConfig() on it, the stream that is opened on creating the logging.FileHandler object is "%(LogFile)s", not "C:/TestData/test.log".

Stepping into the debugger, I see that the dictionary is changed in BaseConfigurator.convert()

    def convert(self, value):
        """
        Convert values to an appropriate type. dicts, lists and tuples are
        replaced by their converting alternatives. Strings are checked to
        see if they have a conversion format and are converted if they do.
        """
        if not isinstance(value, ConvertingDict) and isinstance(value, dict):
            value = ConvertingDict(value)
            value.configurator = self

BEFORE calling ConvertingDict(value), the value dict has the correct value for key filename:

>>> value
{'class': 'logging.FileHandler', 'level': 'WARN', 'filename': 'C:/TestData/test.log'}

AFTER calling ConvertingDict(value):

>>> value
{'filename': '%(LogFile)s', 'class': 'logging.FileHandler', 'level': 'WARN'}

If I try to use 
			filename = 'cfg://loggng.LogFile'
in my INI file, I get a ValueError calling logging.config.dictConfig(config):


pydev debugger: starting (pid: 70744)
Traceback (most recent call last):
  File "C:\Python34\lib\logging\config.py", line 557, in configure
    handler = self.configure_handler(handlers[name])
  File "C:\Python34\lib\logging\config.py", line 723, in configure_handler
    kwargs = dict([(k, config[k]) for k in config if valid_ident(k)])
  File "C:\Python34\lib\logging\config.py", line 723, in <listcomp>
    kwargs = dict([(k, config[k]) for k in config if valid_ident(k)])
  File "C:\Python34\lib\logging\config.py", line 318, in __getitem__
    return self.convert_with_key(key, value)
  File "C:\Python34\lib\logging\config.py", line 284, in convert_with_key
    result = self.configurator.convert(value)
  File "C:\Python34\lib\logging\config.py", line 455, in convert
    value = converter(suffix)
  File "C:\Python34\lib\logging\config.py", line 404, in cfg_convert
    d = self.config[m.groups()[0]]
  File "C:\Python34\lib\logging\config.py", line 317, in __getitem__
    value = dict.__getitem__(self, key)
KeyError: 'loggng'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "L:\timothy.j.williams1\Documents\My Programs\eclipse_neon\eclipse\plugins\org.python.pydev_5.3.0.201610121612\pysrc\pydevd.py", line 1531, in <module>
    globals = debugger.run(setup['file'], None, None, is_module)
  File "L:\timothy.j.williams1\Documents\My Programs\eclipse_neon\eclipse\plugins\org.python.pydev_5.3.0.201610121612\pysrc\pydevd.py", line 938, in run
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "L:\timothy.j.williams1\Documents\My Programs\eclipse_neon\eclipse\plugins\org.python.pydev_5.3.0.201610121612\pysrc\_pydev_imps\_pydev_execfile.py", line 25, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "L:\workspace\MyPython\src\testlog.py", line 8, in <module>
    logging.config.dictConfig(config)
  File "C:\Python34\lib\logging\config.py", line 789, in dictConfig
    dictConfigClass(config).configure()
  File "C:\Python34\lib\logging\config.py", line 565, in configure
    '%r: %s' % (name, e))
ValueError: Unable to configure handler 'file': 'loggng'

testlog.py:
import logging, logging.config, logging.handlers
import configobj

logconfig = configobj.ConfigObj('loggingtest.ini', unrepr=True, raise_errors=True)

config=dict(logconfig['loggng'])
logging.config.dictConfig(config)
logger=logging.getLogger('root')
print(config['handlers']['file'])
print(logger.handlers[0].stream)

loggingtest.ini:
[loggng]
version = 1
level = 'INFO'
RootDir = 'TestData'
CaptureDrive = 'C:/'
LogFile = '%(CaptureDrive)s%(RootDir)s/test.log'
	[[formatters]]
		[[[fmt1]]]
			format = '%(asctime)s: (%(levelname)s)  %(message)s'
			datefmt =
	[[handlers]]
		[[[console]]]
			class = 'logging.StreamHandler'
			level = 'INFO'
			stream = 'ext://sys.stdout'
		[[[file]]]
			class = 'logging.FileHandler'
			level = 'WARN'
			filename = '%(LogFile)s'
			#filename = 'cfg://loggng.LogFile'
	[[loggers]]
		[[[root]]]
		level = 'INFO'
		handlers = ['file','console']

Thanks for any help.