Best way to parse file into db-type layout?

[Steve Holden]

John Machin wrote:
[...]
> 
> I wouldn't use fileinput for a "commercial data processing" exercise,
> because it's slow, and (if it involved using the Python csv module) it
> opens the files in text mode, and because in such exercises I don't
> often need to process multiple files as though they were one file.
> 
If the process runs once a month, and take ten minutes to process the 
required data, isn't that fast enough. It's unwise to act as though 
"slow" is an absolute term.

> When I am interested in multiple files -- more likely a script that
> scans source files -- even though I wouldn't care about the speed nor
> the binary mode, I usually do something like:
> 
> for pattern in args: # args from an optparse parser
>     for filename in glob.glob(pattern):
>         for line in open(filename):
> 
> There is also an "on principle" element to it as well -- with
> fileinput one has to use the awkish methods like filelineno() and
> nextfile(); strikes me as a tricksy and inverted way of doing things.
> 
But if it happens to be convenient for the task at hand why deny the OP 
the use of a tool that can solve a problem? We shouldn't be so purist 
that we create extra (and unnecessary) work :-), and principles should 
be tempered with pragmatism in the real world.

regards
  Steve
-- 
Steve Holden        +1 703 861 4237  +1 800 494 3119
Holden Web LLC             http://www.holdenweb.com/
Python Web Programming  http://pydish.holdenweb.com/