feof status (was: Re: [Python-Dev] Rehabilitating fgets)

[Guido van Rossum]

> > So, I see no reason why the logic in your program couldn't take care
> > of this, which in general is a preferred way to solve a problem than
> > to change the language.
> 
> OK, two objections, one practical and one (more important) esthetic:
> 
> Practical: I guess I oversimplified the code for expository purposes.
> What's actually going on is that I have two parser classes both based
> on shlex -- they do character-at-a-time input and don't actually
> *have* accessible line buffers.

And what's wrong with always starting the second parser?  If the
stream was at EOF it will simply process zero lines.  Or does your
parser have a problem with empty input?

> Esthetic: Yes, I can have the first parser set a flag, or return some
> EOF token.  But this seems deeply wrong to me, because EOFness is not
> a property of the parser but of the underlying stream object.  It
> seems to me that my program ought to be able to ask the stream object
> whether it's at EOF rather than carrying its own flag for that state.

Eric, before we go furhter, can you give an exact definition of
EOFness to me?

> In Python as it is, there's no clean way to do this.  I'd have to do a
> nonzero-length read to test it (I failed to check the right alternate
> case before when I tried zero-length).  That's really broken.  What if the
> neither the underlying stream nor the parser supports pushback?
> 
> Do you see now why I think this is a more general issue?

No.  What's wrong with just setting the parser loose on the input and
letting it deal with EOF?  In your example, apparently a line
containing the word "history" signals that the rest of the file must
be parsed by the second parser.  What if "history" is the last line of
the file?  The eof() test can't tell you *that*!

> Now, another and more general way to handle this would be to make an
> equivalent of the old FIONCLEX ioctl part of Python's standard set of 
> file object methods -- a way to ask "how many bytes are ready to be
> read in this stream?  

There's no portable way to do that.

> Trivial to make it work for plain files, of course.  Harder to make it   
> work usefully for pipes/fifos/sockets/terminals.  Having it pass up the
> results of the fstat.size field (corrected for the current seek address
> if you're reading a plain file) would be a good start.

This seems totally the wrong level to solve your problem.

--Guido van Rossum (home page: http://www.python.org/~guido/)