split large file by string/regex

[Martin Dieringer]

Steve Holden <steve at holdenweb.com> writes:

> Martin Dieringer wrote:
>
>> I am trying to split a file by a fixed string.
>> The file is too large to just read it into a string and split this.
>> I could probably use a lexer but there maybe anything more simple?
>> thanks
>> m.
>
> Depends on your definition of "simple", I suppose. The problem with
> *not* using a lexer is that you'd have to examine the file in a
> sequence of overlapping chunks to make sure that a regex could pick up
> all matches. For me that would be more complex than using a lexer,
> given the excellent range of modules such as SPARK and PLY, to mention
> but two.
>

yes lexing would be the simplest, but PLY also can't read from streams
and it looks to me (from the examples) as if it's the same with SPARK.
I wonder why something like this is not in any lib.
Is there any known lexer that can do this?
I don't have to parse, just write the junks to separate files.
I really hate doing that sequence thing...

m.