Thoughts on language-level configuration support?

[jfager]

On Mar 31, 10:44 pm, "Rhodri James" <rho... at wildebst.demon.co.uk>
wrote:
> On Tue, 31 Mar 2009 07:06:50 +0100, jfager <jfa... at gmail.com> wrote:
> > On Mar 30, 9:31 pm, "Rhodri James" <rho... at wildebst.demon.co.uk>
> > wrote:
> >> On Mon, 30 Mar 2009 16:59:12 +0100, jfager <jfa... at gmail.com> wrote:
> >> > It's the configuration problem.  Right now you would use something
> >> > like ConfigParser or optparse to populate some configuration object,
> >> > which you would then pass around and extract values from.  This would
> >> > provide two advantages over these approaches:  it would separate "what
> >> > can be configured" from "the mechanism by which it is configured" -
> >> > i.e., I, programmer, don't have to make a decision about what the end
> >> > user has to do to give me those values.
>
> >> This would be a interesting idea, but ultimately no more than a veneer
> >> over the current set of configuration possibilities.  Quite how such a
> >> system would tell whether to get configuration data from command line
> >> parameters, a config file somewhere, or just pull bytes off the Z-wave
> >>  from Mars, I'm not at all clear.
>
> > Not a veneer over; a foundation under.  As I acknowledged in my blog
> > post, this would require a small bit of bootstrapping, to configure
> > which end-user configuration system was used.  But this would simply
> > point to an adapter, that would map from the desired configuration
> > interface into the canonical standard api provided by the language
> > system.
>
> By magic: see later
>
> > The problem with the current set of configuration possibilities is
> > that there's nothing really constant between them, unless the
> > programmer explicitly codes it, even though they're basically
> > accomplishing the same thing.  There's nothing amazingly special about
> > this proposal, it's just saying:  that basic thing that we do in a lot
> > of different ways, let's make as much of that as possible standard.  I
> > think that cutoff point is below the end-user interface, since there
> > are a lot of valid ways to want to actually inject config data
> > (command line, local files, from a db, etc.), but there's no reason
> > that the end-user interfaces can't rest on top of a lower-level common
> > format that the programmer can rely on.
>
> There is actually.  The reason is that those different configuration
> interfaces are actually accomplishing different things.  You're lumping
> together both config files (wherever they may be and whatever format
> they may be in) with command line parameters and GUI command interfaces
> when the one is providing long-term defaults and the other short-term
> overrides.
>
> The level of detail that you can have in a config file is something that
> you never want to see on a command line.  When you do, as with say "gcc",
> the results are rather mind-numbing.  This distinction is something that
> your reduction of configuration to a single black box fails to capture.

The fact that you 'can' expose something through a particular
interface doesn't mean you 'have to'.  If command line args don't make
sense for a particular program, nobody is would be forced to use
them.  That's the point - the interface the end user works with is
flexible, it's what they want to deal with.



> >> >  And it would allow the
> >> > configurable surface of the program to be discoverable; I wouldn't
> >> > have to poke through code or documentation that was maintained
> >> > separate from the source code.
>
> >> Except that you still do.  
>
> > No, you don't.  If you do, then this proposal hasn't been implemented
> > (and the little code snippet I provided in my post, please don't
> > consider that a full implementation, it was just a sketch I threw
> > together in a really short time).  I'm not saying discoverability
> > magically arises; I'm saying that it's one the main points that would
> > be deliberately put into place.
>
> Yes you do.  This spread-about inline style of configuration is in no
> sense discoverable to the casual reader, who isn't going to know that
> a particular config option even exists unless he or she chances upon
> the line where it's registered.  

"Discoverable", as in built-in tools that let you have the following
conversation:  "Program, tell me all the things I can configure about
you" - "Okay, here they all are".  No digging through the source
required.


> I'm not commenting on your code
> snippet here -- or I'd be here all night, and you really wouldn't
> like me at the end of it --

:)  Dude, that was all of 15 minutes of coding (not counting the
diversions spent poking around the bytecode tools for the first
time).


> but I am commenting on the usage style
> it's obviously built to support.  That usage style is much less
> visible than any of the common (or uncommon, for that matter) styles
> of getting config input, and discourages documentation habits that
> would otherwise mitigate it.
>
> >> You've just specified a different way in
> >> which you have to do this, one that's a good deal less visible in
> >> the code
>
> > Why would it be less visible?  If it's syntax, you would know exactly
> > where it was just by looking.  I would argue that it's more clear - no
> > more boilerplate code spent shuffling around config objects that you
> > peel values out of, just write the code that does the work you're
> > actually trying to accomplish, with a small bit of syntax to mark that
> > the end user can provide an alternate value.
>
> As I said, it's less visible because it's decentralised.  The moment
> you do that, you lose all hope of centralising the documentation, and
> the visibility of your options plummets.

The tools provided by the language do the centralization for you, that
was one of the main points.


>
> >> and a lot less self-documenting because it's spread out all
> >> over the place.  
>
> > Actually, you get the best of both worlds.  You get to see clearly in
> > the code how the configured values are actually used, and you get the
> > gathered summary of configurable values from the "discoverability"
> > implementation.
>
> What "discoverability"?  You still haven't told me anything to make
> me believe that this configuration setup is discoverable at all without
> external tools.

The whole point is that the tools for this would be provided by the
language system.



> >> It also has the major disadvantage from my point of
> >> view of requiring Python to do magic in the background to figure out
> >> just what is being configured.
>
> > If it's built into the language, it stops being magic, and turns into
> > "how it works".
>
> It remains magic when it requires the language to invent things on the
> fly that the programmer has no means of discovering until the code is
> running, by which time it's generally too late.  For example:
>
>    spam = conf(1)
>
> How does that translate into a command-line option?  -s?  --span?
> Both?  -S, even?  --first-config-option-I-came-across?  If the first,
> what happens when:
>
>    span = conf(0)
>
> shows up to initialise a class attribute some time later?  What
> restrictions can be put on the value you get back?  What can the
> help system say about this, or do we have to go back to doing all
> that by hand?  Now translate all those questions into the very
> different environment of a config file.  Repeat with a database,
> and all it's quirks.  By the time your colossus has acquired
> enough parameters to at least hint at the desirable answers to
> these questions, you've effectively duplicated the interfaces to
> all of the config mechanisms you're trying to replace and you've
> still lost a whole heap of flexibility.

Yes, you're right, the code that actually injects the configuration
isn't trivial.  I never intended to imply that it was.  But it would
probably only have to be written once (people would only write their
own if they had a special need).  The win is that the underlying code
doesn't have to change just because the end-user configuration format
did.

- Jason


>
> --
> Rhodri James *-* Wildebeeste Herder to the Masses