Read a record instead of a line from a file

[Andrew Dalke]

Donn Cave:
>So the generator stuff is just for fun, right?

Yep.  Martel, my mxTextTools based parser generator has much
faster and more powerful record readers than the generator
code I presented.  (For example, you can access the lookaheadbuffer.
This is useful in cases where several different record styles
are used in the same file, as with a header followed by 1 or
more records followed by a footer.)

One thing I'm looking forward to in 2.2 is using __iter__ rather
than using __getitem__ like this

   def __getitem__(self, i):
       assert i == self._n, "forward iteration only"
       ...
       self._n = self._n + 1


>I mean, this
>can just as easily be expressed as a conventional buffer
>object, minus the for loop application but I believe possibly
>allowing a little more flexibility in other respects.

Agreed.  It is more flexible.  But I do think the generator one
is easier to understand ... once you understand generators.  And
I'm not sure if generators are more or less complicated to
explain than classes.

>mxTextTools rules.

Whole-heartedly agree.  For one of my test cases it's only 50%
slower than grep, and in more realistic cases it's faster and
better than hand written Python and Perl code.  It's about 4
times slower than hand-written Java code, but that's not a fair
comparison because mxTextTools is doing a *lot* more work than
the Java code.  Timings at
  http://www.dalkescientific.com/Martel/ebi-talk/img37.htm

                    Andrew
                    dalke at dalkescientific.com