separate software from data

[Mark McEahern]

Here's the short form of my question:

How do people generally keep their Python modules and any configuration data
for those Python modules separate?

I'm looking for both conventions (e.g., config goes in /etc/foo) and
mechanics (e.g., use distutils with options x, y, and z).

The longer form of the question is a bit more convoluted, so for those with
patience, read on.  ;-)

I'm transitioning from Windows to Linux.  So part of my question is about
the conventions for separating software from data on Linux.  I assume it's
self-explanatory why you'd want to separate software from data.  In case
it's not, here's why I think it's important to separate:  You don't really
have to backup software (although you can if you want), but you do have to
backup data (it's not like you can re-download it).  So, if push comes to
shove, and you have limited resources/time, it's nice to be able to focus on
the important stuff and only back that up.

There's always a gray line between software and data that I label
configuration data.  In the Linux world, I guess that'd constitute anything
you do to install any piece of software beyond the equivalent of configure;
make; make install.  If you specify any configuration options, create daemon
accounts, etc.--ideally you'd do that with a script and you'd store that
script somewhere along with your data, so that you not only knew what you
did, but you could easily re-do it.

I've looked at distutils and it's not immediately obvious that it was
designed to solve the distribution problem in the particular what I want to
try to solve it, although I suppose I may be able to extend distutils with
special commands to do something like this?

I want a way to say something like this:

Install my module here:

  /usr/local/lib/pythonX.Y/site-packages/

By default, place configuration data here:

  /etc/my-package/

But if the installer specifies different configuration options when running
setup.py, then use those values when generating mypackage.conf from
mypackage.conf.template.

Of course, the person installing the package should be able to override
where configuration data is stored:

  $ python setup.py --configdir=/var/foo

If such an option existed, its effect might be to generate a configuration
file in the root of the package's module distribution folder:

  /site-packages/mymodule/mymodule.conf

containing something readable by ConfigParser:

[locations]
prefix=/etc/mymodule/
connections=%prefix/connections.xml

So that at runtime, the module first looks to the configuration information
in its package distribution to determine where it needs to look for the
distribution-specific configuration information?  Shwew.

Has anybody tackled this problem or do most folks just punt on this issue
and assume they can plop configuration information alongside their
distributed modules without providing any easy way for the person who
installs it to override that?

Thanks,

// mark