Thoughts on language-level configuration support?

[Rhodri James]

On Tue, 31 Mar 2009 07:06:50 +0100, jfager <jfager at gmail.com> wrote:

> On Mar 30, 9:31 pm, "Rhodri James" <rho... at wildebst.demon.co.uk>
> wrote:
>> On Mon, 30 Mar 2009 16:59:12 +0100, jfager <jfa... at gmail.com> wrote:
>> > It's the configuration problem.  Right now you would use something
>> > like ConfigParser or optparse to populate some configuration object,
>> > which you would then pass around and extract values from.  This would
>> > provide two advantages over these approaches:  it would separate "what
>> > can be configured" from "the mechanism by which it is configured" -
>> > i.e., I, programmer, don't have to make a decision about what the end
>> > user has to do to give me those values.
>>
>> This would be a interesting idea, but ultimately no more than a veneer
>> over the current set of configuration possibilities.  Quite how such a
>> system would tell whether to get configuration data from command line
>> parameters, a config file somewhere, or just pull bytes off the Z-wave
>>  from Mars, I'm not at all clear.
>
> Not a veneer over; a foundation under.  As I acknowledged in my blog
> post, this would require a small bit of bootstrapping, to configure
> which end-user configuration system was used.  But this would simply
> point to an adapter, that would map from the desired configuration
> interface into the canonical standard api provided by the language
> system.

By magic: see later

> The problem with the current set of configuration possibilities is
> that there's nothing really constant between them, unless the
> programmer explicitly codes it, even though they're basically
> accomplishing the same thing.  There's nothing amazingly special about
> this proposal, it's just saying:  that basic thing that we do in a lot
> of different ways, let's make as much of that as possible standard.  I
> think that cutoff point is below the end-user interface, since there
> are a lot of valid ways to want to actually inject config data
> (command line, local files, from a db, etc.), but there's no reason
> that the end-user interfaces can't rest on top of a lower-level common
> format that the programmer can rely on.

There is actually.  The reason is that those different configuration
interfaces are actually accomplishing different things.  You're lumping
together both config files (wherever they may be and whatever format
they may be in) with command line parameters and GUI command interfaces
when the one is providing long-term defaults and the other short-term
overrides.

The level of detail that you can have in a config file is something that
you never want to see on a command line.  When you do, as with say "gcc",
the results are rather mind-numbing.  This distinction is something that
your reduction of configuration to a single black box fails to capture.

>> >  And it would allow the
>> > configurable surface of the program to be discoverable; I wouldn't
>> > have to poke through code or documentation that was maintained
>> > separate from the source code.
>>
>> Except that you still do.  
>
> No, you don't.  If you do, then this proposal hasn't been implemented
> (and the little code snippet I provided in my post, please don't
> consider that a full implementation, it was just a sketch I threw
> together in a really short time).  I'm not saying discoverability
> magically arises; I'm saying that it's one the main points that would
> be deliberately put into place.

Yes you do.  This spread-about inline style of configuration is in no
sense discoverable to the casual reader, who isn't going to know that
a particular config option even exists unless he or she chances upon
the line where it's registered.  I'm not commenting on your code
snippet here -- or I'd be here all night, and you really wouldn't
like me at the end of it -- but I am commenting on the usage style
it's obviously built to support.  That usage style is much less
visible than any of the common (or uncommon, for that matter) styles
of getting config input, and discourages documentation habits that
would otherwise mitigate it.

>> You've just specified a different way in
>> which you have to do this, one that's a good deal less visible in
>> the code
>
> Why would it be less visible?  If it's syntax, you would know exactly
> where it was just by looking.  I would argue that it's more clear - no
> more boilerplate code spent shuffling around config objects that you
> peel values out of, just write the code that does the work you're
> actually trying to accomplish, with a small bit of syntax to mark that
> the end user can provide an alternate value.

As I said, it's less visible because it's decentralised.  The moment
you do that, you lose all hope of centralising the documentation, and
the visibility of your options plummets.

>> and a lot less self-documenting because it's spread out all
>> over the place.  
>
> Actually, you get the best of both worlds.  You get to see clearly in
> the code how the configured values are actually used, and you get the
> gathered summary of configurable values from the "discoverability"
> implementation.

What "discoverability"?  You still haven't told me anything to make
me believe that this configuration setup is discoverable at all without
external tools.

>> It also has the major disadvantage from my point of
>> view of requiring Python to do magic in the background to figure out
>> just what is being configured.
>
> If it's built into the language, it stops being magic, and turns into
> "how it works".

It remains magic when it requires the language to invent things on the
fly that the programmer has no means of discovering until the code is
running, by which time it's generally too late.  For example:

   spam = conf(1)

How does that translate into a command-line option?  -s?  --span?
Both?  -S, even?  --first-config-option-I-came-across?  If the first,
what happens when:

   span = conf(0)

shows up to initialise a class attribute some time later?  What
restrictions can be put on the value you get back?  What can the
help system say about this, or do we have to go back to doing all
that by hand?  Now translate all those questions into the very
different environment of a config file.  Repeat with a database,
and all it's quirks.  By the time your colossus has acquired
enough parameters to at least hint at the desirable answers to
these questions, you've effectively duplicated the interfaces to
all of the config mechanisms you're trying to replace and you've
still lost a whole heap of flexibility.

-- 
Rhodri James *-* Wildebeeste Herder to the Masses