How to read from a file to an arbitrary delimiter efficiently?

[Serhiy Storchaka]

On 27.02.16 11:49, Steven D'Aprano wrote:
> On Thu, 25 Feb 2016 06:30 pm, Chris Angelico wrote:
>> How bad is it if you over-read?
>
> Pretty bad :-)
>
> Ideally, I'd rather not over-read at all. I'd like the user to be able to
> swap from "read N bytes" to "read to the next delimiter" (and possibly
> even "read the next line") without losing anything.
>
>
> If there's absolutely no other way to speed this up by at least a factor of
> ten, I'll consider reading into a buffer and losing the ability to mix
> different kinds of reads.

If the file is buffered, you can use Chris's receipt, but with peek(). 
Otherwise you should fall back to slow one-byte read.