Splitting strings - by iterators?

[Diez B. Roggisch]

Jeremy Sanders wrote:

> I have a large string containing lines of text separated by '\n'. I'm
> currently using text.splitlines(True) to break the text into lines, and
> I'm iterating over the resulting list.
> 
> This is very slow (when using 400000 lines!). Other than dumping the
> string to a file, and reading it back using the file iterator, is there a
> way to quickly iterate over the lines?
> 
> I tried using newpos=text.find('\n', pos), and returning the chopped text
> text[pos:newpos+1], but this is much slower than splitlines.
> 
> Any ideas?

Maybe [c]StringIO can be of help. I don't know if it's iterator is lazy. But
at least it has one, so you can try and see if it improves performance :)


-- 
Regards,

Diez B. Roggisch