Working with fixed format text db's

[Jeremy C B Nicoll]

Neil Cerutti <horpner at yahoo.com> wrote:

> On 2007-06-08, Jeremy C B Nicoll <jeremy at omba.demon.co.uk> wrote:
> > Neil Cerutti <horpner at yahoo.com> wrote:
> >> Luckily, the output format has not changed yet, so issues with
> >> maintaining the above haven't arisen.
> >
> > The problem surely is that when you want to change the format
> > you have to do so in all files (and what about the backups
> > then?) and all programs simultaneously.
> 
> I don't have control of the format, unfortunately. It's an import
> file format for a commercial database application.

You're saying your program merely has to read data files created by that
database app?  It's not that you have a whole suite of programs that create
and read these files, nor that you have years worth of old files that would
need their format converted if the programs were changed?


> It is not actually *hard* to do this with ad-hoc code, but then
> the program is indecipherable without a hardcopy of the spec in
> hand. And also, as you say, if the spec ever does change, the
> hand-written batch of ljust, rjust and slice will be somewhat of
> a pain to reconfigure.

You could presumably define a list (of some sort, might be the wrong
terminology) that defines the 'name', type, length, justification and
padding of each field, and then make the explicit code you showed loop
through that list and do what's needed field by field.

There's a risk that abstracting the definitions will make the code less
clear to anyone else; at least it's clear what the current stuff does.

> But biggest weakness, to me, is that the specification is not in
> the code, or read and used by the code, and I think it should be.

It'd be better if you could read the data layout spec from some file
produced by the database system.  No chance perhaps of having the dat files
include some sort of dummy first record that contains the necessary info in
a form that you could interpret?
 

-- 
Jeremy C B Nicoll - my opinions are my own.