[Python-ideas] Python 3.9.9 - The 'I Have A Dream' Version

[Andrew Barnert]

To respond to the one concrete proposal anyone's been able to dig out so far:

On Wednesday, April 15, 2015 8:53 AM, Simon Kennedy <sffjunkie at gmail.com> wrote:

>On Wednesday, 15 April 2015 12:44:59 UTC+1, Chris Angelico  wrote:


>>There is a fundamental difficulty with the unification of the 
>>disparate, in that you have to start coping with all sorts of 
>>differences and try to make sense of them *in code*. When, for 
>>instance, you want to unify config handling, what you're saying is 
>>either that every config handler has to devolve to the lowest common 
>>denominator, or else that your unified config handler has to become 
>>the least common multiple.
>
>
>The stdlib is full of modules which don't attempt everything but instead attempt to do a useful subset and everyone seems to be happy they're included.
>
>Why not configuration file handling?
>
>I assume we're writing Python code and that eventually all config data needs to end up in memory that Python can access.
>

>Pulling numbers out of my posterior, lets say 80% of projects only need nothing more than a text file with a set of sections with name/value pairs under them. That can be parsed into a dict.

Except that's not actually a lowest-common denominator.

Names that aren't in pure printable ASCII don't work in all of your formats. Neither do names with embedded whitespace. In some formats, on some platforms, names are case-insensitive (anyone who's ever tried to port portable apps to Windows and had to deal with their configured "path" variable having the contents of "PATH" instead can tell you how much fun that is).

While you suggested that all of these formats can handle types, the paradigmatic one you just presented, the sectioned .ini format, cannot; it can only handle string values, without newlines or nulls, that can be encoded to the system's default encoding (and remember, for a lot of systems, that's still CP1252 or similarly limited).

Most importantly, a sectioned .ini file doesn't represent a dict of key-value pairs, it represents a dict of dicts of key-value pairs. But other formats, like environment variables, _do_ represent a single flat dict. The lowest common denominator between those two things is that absolutely nothing can be stored.

Of course people have designed "lowest-common-denominator-ish" config formats—see any portability framework like Qt or JUCE, for obvious examples—but they don't actually try to store in all formats, or layer additional functionality on top of some (e.g., using a flat dict with dotted names to represent hierarchy), or partly abandon the idea of human readability (e.g., strings are HTML-encoded, or you can store non-string data but you get a dump in some binary pickle-like format). If you think you can solve the impossible problem that nobody else has, go solve it; just implying that it would be nice if someone solved it, without even indicating that you've recognized its existence, is not a useful proposal.

>And 5% want to use Python code. That when it gets exec'ed a dict comes out the other end
>And 5% want to use a database. We can produce an adapter (say sqlite) that spits out a dict or they can.
>And 5% want to use an etcd store.
>That leaves 5% that want to do something funky. They're on their own
>
>There aren't that many configuration storage systems in the world.
>
>Then you define the order which sources (command line / env var / global config etc.) are used and merge them into a complete configuration
>

>Why not 'bless' some code which simplifies the 95%

Because not everything belongs in the stdlib. You need to do the same cost-benefit analysis for this library as for any other.

>There's a distribution on PyPi called 'layeredconfig' which does much of what I think could be done. A few Python projects I've encountered seem to follow a similar pattern.


OK, so is there something fundamentally wrong with layeredconfig? If so, it would be nice to know _what_. If not, why didn't you just say "add layeredconfig or something equivalent" instead of making us drag the same information out of you in more verbose form?