Thoughts on language-level configuration support?

[jfager]

On Mar 31, 2:16 pm, Lorenzo Gatti <ga... at dsdata.it> wrote:
> On 31 Mar, 09:19, jfager <jfa... at gmail.com> wrote:
>
> > On Mar 31, 2:54 am, David Stanek <dsta... at dstanek.com> wrote:
>
> > > On Mon, Mar 30, 2009 at 9:40 AM, jfager <jfa... at gmail.com> wrote:
> > > >http://jasonfager.com/?p=440.
>
> > > > The basic idea is that a language could offer syntactic support for
> > > > declaring configurable points in the program.  The language system
> > > > would then offer an api to allow the end user to discover a programs
> > > > configuration service, as well as a general api for providing
> > > > configuration values.
>
> A configuration "service"? An "end user" that bothers to discover it?
> API for "providing" configuration "values"? This suggestion, and the
> companion blog post, seem very distant from the real world for a
> number of reasons.
>
> 1) Users want to supply applications with the least amount of useful
> configuration information as rarely and easily as possible, not to use
> advanced tools to satisfy an application's crudely expressed
> configuration demands.

It depends on the user, doesn't it?  In the vast majority of cases,
you're right, the end user is more than happy (or at least, willing)
to use whatever the software developer tells them to.  Command line
interfaces, .ini files, whatever, there's no reason the language
couldn't ship with a set of standard adapters (and that the
bootstrapping question could be bypassed by the developer offering a
'hint' about which they think would be best, so the user wouldn't have
to care unless they wanted to).

I spend a lot of time in the Java world (yes, I know), where XML
reigns supreme.  I would *love* to be able to seamlessly replace that
with something human readable.


> Reducing inconvenience for the user entails sophisticated and mostly
> ad hoc techniques: deciding without asking (e.g. autoconf looking into
> C compiler headers and trying shell commands or countless applications
> with "user profiles" querying the OS for the current user's home
> directory), asking when the software is installed (e.g. what 8 bit
> character encoding should be used in a new database), designing
> sensible and safe defaults.

It seems that something like autoconf would be the perfect consumer
for something like this, both for querying what could be set in and
for injecting values back into the application.

I don't understand your point about "countless applications with "user
profiles" querying the OS for the current user's home directory", so
I'm going to skip it.

Asking for configuration when the software is installed, for something
that can't be changed be later - there are still multiple ways to want
to interact with the code that's doing that setup, aren't there?  A
GUI wizard, interactively on the command line, an automated
installation script.  And, for the main program - you know that's a
one-off thing, why would you mark it as configurable every time the
program runs?

As for deciding sensible and safe defaults, I don't see how what I
proposed at all ignores that necessity, at least any more than any
other configuration technique.


> 2) Practical complex configuration files (or their equivalent in a DB,
> a LDAP directory, etc.) are more important and more permanent than the
> applications that use them; their syntax and semantics should be
> defined by external specifications (such as manuals and examples), not
> in the code of a particular implementation.

> User documentation is necessary, and having a configuration mechanism
> that isn't subject to accidents when the application is modified is
> equally important.

Developers have to take care to maintain backwards compatibility with
entrenched config formats today, why is it a strike against this idea
that that need would continue?  It's not like an implementation of
this would seek out valuable existing configurations and destroy them
whenever the code changed.

Right now, when you have a rich configuration format, and you need to
make code changes, how do you deal with the problem?  You have a layer
of indirection, one that's responsible for keeping the end-user
configuration more or less constant while mapping that configuration
to the changing code underneath.  There's no reason that would have to
change - your project gets complicated enough, just start maintaining
your own consumer module.  Indirection is valuable from both sides.



> 3) Configuration consisting of values associated with individual
> variables is an unusually simple case. The normal case is translating
> between nontrivial sequential, hierarchical or reticular data
> structures in the configuration input and quite different ones in the
> implementation.

I don't think it's an "unusually" simple case.  It's a question of
what you're writing.  Most of what I use python is fairly small
projects that started out as one-off scripts that I gradually build
up.

And, yes, how this would handle more complicated config structures is
something that needs to be given more thought.


> 4) Your actual use case seems to be providing a lot of tests with a
> replacement for the "real" configuration of the actual application.
> Branding variables as "configuration" all over the program isn't an
> useful way to help the tests and the actual application build the same
> data structures in different ways.

Sorry, I'm not understanding you here.  Are you referring to the
testing app I mentioned in my post?  It was a webservice load testing
application.  Each test was more like a full little script defining a
scenario that could be tweaked to produce different load profiles; the
configuration requirement was to expose to the end user who was
running the load test what, exactly, could be tweaked.  Doing this in
a declarative way in the scenario files was *very* useful, there
really wasn't a better way to solve the problem.



>
> > > What value does this have over simply having a configuration file.
>
> > "Simply having a configuration file" - okay.  What format?  What if
> > the end user wants to keep their configuration info in LDAP?
>
> Wait a minute. Reading the "configuration" from a live LDAP directory
> is a major feature, with involved application specific aspects (e.g.
> error handling) and a solid justification in the application's
> requirements (e.g. ensuring up to date authentication and
> authorization data), not an interchangeable configuration provider and
> certainly not something that the user can replace.

I'm not disputing this.  I never intended to suggest I thought that
the configuration providers would be trivial pieces of software.


> Deciding where the configuration comes from is an integral part of the
> design, not something that can or should be left to the user: there
> can be value in defining common object models for various sources of
> configuration data and rules to combine them, like e.g. in the Spring
> framework for Java, but it's only a starting point for the actual
> design of the application's configuration.

This is just a starting point, too; I didn't mean to suggest
otherwise, apologies if I gave that impression.



> > > In your load testing application you could have easily checked for the
> > > settings in a config object.
>
> > Not really easily, no.  It would have been repeated boilerplate across
> > many different test cases (actually, that's what we started with and
> > refactored away), instead of a simple declaration that delegated the
> > checking to the test runner.
>
> A test runner has no business configuring tests beyond calling generic
> setup and teardown methods; tests can be designed smartly and factored
> properly to take care of their own configuration without repeating
> "boilerplate".

Yes, they can, which is why we refactored them to do so as the system
grew from 'one-off script' to 'load-testing framework'.  And the test
runner acting as a middleman between the scripts and the end-user,
that was invaluable, really.  I promise.  It let us run the same tests
easily from the command line, from a web application, from a nightly
batch process, etc, to add new tests quickly and to manage the
configuration without going crazy.


>
> > > I think that the discover-ability of
> > > configuration can be handled with example configs and documentation.
>
> > Who's keeping that up to date?  Who's making sure it stays in sync
> > with the code?  Why even bother, if you could get it automatically
> > from the code?
>
> It's the code that must remain in sync with the documentation, the
> tests, and the actual usage of the application. For example, when did
> you last see incompatible changes in Apache's httpd.conf?

> You seem to think code is central and actual use and design is a
> second class citizen. You say in your blog post: "Users shouldn’t have
> to pore through the code to find all the little bits they can tweak".
> They shouldn't because a well designed application has adequate
> documentation of what should be configured in the form of manuals,
> wizards, etc. and they shouldn't because they don't want to tweak
> little bits, not even if they have to.

The amount of software with thorough and current hand-maintained
documentation and manuals, I would guess that it's vastly outnumbered
by that without, and anyways, nothing in this proposal precludes any
of that.  This isn't "the software developer suddenly has no
responsibility for how their app is maintained", this is "the software
developer has a standard set of tools for handling common
configuration tasks, and the end user doesn't necessarily have their
hands tied by developer decisions."


Thanks for your feedback, I really do appreciate the engagement on the
topic.

- Jason


>
> Regards,
>
> Lorenzo Gatti