How to read from a file to an arbitrary delimiter efficiently?

[Dan Sommers]

On Sat, 27 Feb 2016 21:40:17 +1100, Steven D'Aprano wrote:

> Thanks for finding the issue, but the solutions given don't suit my
> use case. I don't want an iterator that operates on pre-read blocks, I
> want something that will read a record from a file, and leave the file
> pointer one entry past the end of the record.

A file is a stream of bytes, but you want to view it as a stream of
records.  It sounds like you want an abstraction layer, and it sounds
like you also want to let the file leak through that layer when it's
convenient.  (Yes, I spun that horribly on purpose, and I understand the
use case of imposing some structure on part of a file, and possibly a
different structure on a different part of a file.  MIME messages and
literate programming files spring to mind.)

Perhaps (as I think ChrisA suggested), you could provide your own
buffering/chunking layer between your application and the file itself,
and never let the application see the file directly.