split large file by string/regex

[Denis S. Otkidach]

On Mon, 22 Nov 2004 08:53:02 -0500
Steve Holden <steve at holdenweb.com> wrote:

> > I am trying to split a file by a fixed string.
> > The file is too large to just read it into a string and split this.
> > I could probably use a lexer but there maybe anything more simple?
> > thanks
> > m.
> 
> Depends on your definition of "simple", I suppose. The problem with 
> *not* using a lexer is that you'd have to examine the file in a sequence 
> of overlapping chunks to make sure that a regex could pick up all 

re module works fine with mmap-ed file, so no need to read it into memory.

> matches. For me that would be more complex than using a lexer, given the 
> excellent range of modules such as SPARK and PLY, to mention but two.

-- 
Denis S. Otkidach
http://www.python.ru/      [ru]