feof status (was: Re: [Python-Dev] Rehabilitating fgets)

[Eric S. Raymond]

Guido van Rossum <guido@python.org>:
> [ESR in a later message]
> > I considered trying a zero-length read() in Python, but this strikes me 
> > as inelegant even if it would work.
> 
> I doubt that a zero-length read conveys any information.  It should
> return "" whether or not there is more to read!

Duh.  Of course it would.  

You know, I've always been half-consciously dissatisfied with Python's
use of "" as an EOF marker, and now I know why.  It's precisely
because there's no way to distinguish these cases.  I think a zero-length
read ought to return "" and a read on EOF ought to return None.

> Bizarre (given what I know about zero-length read).  But in the above
> code, you can replace "if not fp.feof()" with "if line".  In other
> words, you just have to carry the state over within your program.
> 
> So, I see no reason why the logic in your program couldn't take care
> of this, which in general is a preferred way to solve a problem than
> to change the language.

OK, two objections, one practical and one (more important) esthetic:

Practical: I guess I oversimplified the code for expository purposes.
What's actually going on is that I have two parser classes both based
on shlex -- they do character-at-a-time input and don't actually
*have* accessible line buffers.

Esthetic: Yes, I can have the first parser set a flag, or return some
EOF token.  But this seems deeply wrong to me, because EOFness is not
a property of the parser but of the underlying stream object.  It
seems to me that my program ought to be able to ask the stream object
whether it's at EOF rather than carrying its own flag for that state.

In Python as it is, there's no clean way to do this.  I'd have to do a
nonzero-length read to test it (I failed to check the right alternate
case before when I tried zero-length).  That's really broken.  What if the
neither the underlying stream nor the parser supports pushback?

Do you see now why I think this is a more general issue?

Now, another and more general way to handle this would be to make an
equivalent of the old FIONCLEX ioctl part of Python's standard set of 
file object methods -- a way to ask "how many bytes are ready to be
read in this stream?  

Trivial to make it work for plain files, of course.  Harder to make it   
work usefully for pipes/fifos/sockets/terminals.  Having it pass up the
results of the fstat.size field (corrected for the current seek address
if you're reading a plain file) would be a good start.
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

Live free or die; death is not the worst of evils.
	-- General George Stark.