Can you do it faster? (parsing text file)

[Terry Reedy]

"Marcus Stojek" <stojek at part-gmbh.de> wrote in message
news:3e3650fc.2103718 at news.easynews.net...
> Hi,
>
> I have to parse large txt-files (15Mb and more, 200000 lines).
> The file consits of comma separated keywords and alphanumerical
data.
> Each command line starts with some or no blanks, a '*' and the
keyword
> followed by comma separated paramters

I would separate the problem into two parts: 1) divide the input into
blocks (each headed by a command line); 2) turn each block into an
entry for the output list.  I would write the block generator as
that -- a generator, which lets you yield at precisely the right
point:

def blocks(infile):
  block = []
  for line in file(infile)
    if iscommand(line):
       if block: yield block
       block = [line]
    elif isnotcomment(line):
       block.append(line)

Terry J. Reedy