distutils - distributing non-python data files

[Micah Elliott]

On Oct 25, RickMuller wrote:
> I really appreciate the ease that the distutils make distributing
> Python modules. However, I have a question about using them to
> distribute non-Python (i.e. text) data files that support Python
> modules.

It's not clear from your questions whether this is user-configuration
data for your program, or just data files that your program generates.

If the latter, you might look into pickle'ing.  Often the best place
to look for examples is in your Python distribution itself.  My
cursory scan didn't show much this time, but the Ft module might be
worthy of note.  It installs a Share directory
(.../site-packages/Ft/Share) that contains some XML files.

> Currently when I have data of this type, I parse it into
> python objects and make a python module from it.

Sometimes this is a reasonable approach, and you might get away with
just storing it as Python code with no conversion.  Mailman (and one
of my projects) uses config files that are actually treated as Python
modules.  For configuration data Python is such a readable language
that even users unfamiliar with it can often safely edit a config
file.

> In other words,
> given a data file like
> 
> % cat grocery_list.txt
> eggs
> spam
> cheese
> 
> I would make a data structure like
> grocery_list = ['eggs', 'spam', 'cheese']
> which can be easily imported from a python file, and which the
> distutils installation programs make it easy for me to find on the
> Python path.
> 
> Of course, the data I'm using is much more complex than this, but
> you get the idea.

Then you might consider going to an XML format, and let
SAX/DOM/whatever do the work for you.  Or maybe use the Configuration
Parser <http://www.python.org/doc/2.4.2/lib/module-ConfigParser.html>,
depending on your needs.

> I'm starting to feel like this is a Bad Thing, because if the data
> file is distributed as a plain text file (e.g. data values in
> columns), I'm putting a barrier to updating the data if I have to
> parse a new file into Python.

Text, binary, columnar, XML... whatever the format, if it changes you
will always be susceptible to changing your parser.

> But I don't know whether there is a better way to include a file
> like 'grocery_list.txt' in my python distutils-distributed module so
> that I can get it from my other python modules?

If it's configuration data not stored within your distribution, you
could have some places to auto-check, e.g.: $HOME/somerc.py,
/etc.  And/or you could support an environment variable.

-- 
_ _     ___
|V|icah |- lliott  http://micah.elliott.name  mde at micah.elliott.name
" "     """