Parse config file and command-line arguments, to get a single collection of options

[rzed]

Ben Finney <ben at benfinney.id.au> wrote in
news:87k4deaxfc.fsf at benfinney.id.au: 

> Howdy all,
> 
> Python's standard library has modules for configuration file
> parsing (configparser) and command-line argument parsing
> (optparse, argparse). I want to write a program that does both,
> but also: 
> 
> * Has a cascade of options: default option values, overridden by
> config 
>   file options, overridden by command-line options.
> 
> * Reads a different, or even additional, configuration file if
> specified 
>   on the command-line (e.g. --config-file foo.conf) and yet still
>   obeys the above cascade.
> 
> * Allows a single definition of an option (e.g. logging level) to
> define 
>   the same option for parsing from configuration files and the
>   command line.
> 
> * Unifies the parsed options into a single collection for the
> rest of 
>   the program to access without caring where they came from.
> 
> How can I achieve this with minimum deviation from the Python
> standard library?
> 
> 
> (For anyone interested in gaining StackOverflow points, I'm also
> asking this as a question there so feel free to post answers on
> that site 
> <URL:http://stackoverflow.com/questions/6133517/parse-config-file-
> and-command-line-arguments-to-get-a-single-collection-of-optio>.) 
> 

This seems vaguely similar to a module I wrote and use all the time. 
It allows default value specification, categorization of command-line 
options, in-line parsing of further spec file[s] and overrides of 
values in the the sequence you define, pretty much. It doesn't deal 
with the "logging level" item. I'm not sure what that would mean. The  
idea of the module is to create a namespace object (though it could 
as easily be a dict) that contains whatever values are specified.

in the program it would be invoked like this:
ctx = Cmdline( outfname='test.out', size=(250,400), ... ) (or 
whatever).

The command line can contain switches, prefixed by hyphens, spec file 
names, prefixed by @, untagged names, or key=value pairs. The values 
will be parsed as (tuples), [lists], or {dicts}, ints, floats, or 
strings. I did not, I am ashamed to say, resist the temptation to 
guess. 'Ctx' will contain the result of all this. Switches, if any 
are in a list named ctx._switches, unadorned arguments are in a list 
named ctx._vars, and the other stuff is pretty much as you would 
expect. It expects configuration files to be in a sort of ini-file 
format. 

Here's an example:

---------------------------
test.spec:
log=test.log
count=10
size=(400,200)
group1={key=value,a=alpha}
group2={b=beta,name=doogie howser}
#
temp=[1,2,5,11,22]
sub=(al,becky,carl,diane,edwin,fran)

---------------------------
test.py:
from Cmdline import Cmdline
c = Cmdline( count=5, log='pink.tank')
c.show()

---------------------------
>python test.py log=friend.txt @test.spec count=32 name=waldo

... yields:
<type 'int'> count  = 32
<type 'tuple'> sub  = ('al', 'becky', 'carl', 'diane', 'edwin', 
'fran')
<type 'str'> log  = 'test.log'
<type 'list'> temp  = [1, 2, 5, 11, 22]
<type 'dict'> group1  = {'a': 'alpha', 'key': 'value'}
<type 'dict'> group2  = {'b': 'beta', 'name': 'doogie howser'}
<type 'tuple'> size  = (400, 200)
<type 'str'> name  = 'waldo'

... while

>python @test.spec count=32 name=waldo temp=Vitalis sub='' dream -k
<type 'int'> count  = 32
<type 'list'> _vars  = ['dream']
<type 'str'> log  = 'test.log'
<type 'str'> temp  = 'Vitalis'
<type 'list'> _switches  = ['k']
<type 'str'> name  = 'waldo'
<type 'dict'> group1  = {'a': 'alpha', 'key': 'value'}
<type 'dict'> group2  = {'b': 'beta', 'name': 'doogie howser'}
<type 'tuple'> size  = (400, 200)
<type 'str'> sub  = ''

What the program does with the results is up to it, of course.

-- 
rzed