feof status (was: Re: [Python-Dev] Rehabilitating fgets)

[Tim Peters]

Quickie:

[Guido]
> Eric, before we go furhter, can you give an exact definition of
> EOFness to me?

[Eric]
> A file is at EOF when attempts to read more data from it will fail
> returning no data.

To be very clear about this, that's not what C's feof() means:  in general,
the end-of-file indicator in std C stream input is set only *after* you've
attempted a read that "didn't work".  For example,

#include <stdio.h>

void
main()
{
	FILE* fp = fopen("guts", "wb");
	fputs("abc", fp);
	fclose(fp);
	fp = fopen("guts", "rb");
	for (;;) {
		int c;
		c = getc(fp);
		printf("getc returned %c (%d)\n", c, c);
		printf("At EOF after getc? %d\n", feof(fp));
		if (c == EOF)
			break;
	}
}

Unless your C is broken, feof() will return 0 after getc() returns 'a', and
again after 'b', and again after 'c'.  It's not until getc() returns EOF
that feof() first returns a non-zero result.

Then add these two lines after the "for":

	fseek(fp, 0L, SEEK_END);
	printf("after seeking to the end, feof() says %d\n", feof(fp));

Unless your fseek() is non-std, that clears the end-of-file indicator, and
regardless of to where you seek.  So the std behavior throughout libc is
much like Python's behavior:  there's nothing that can tell you whether
you're at the end of the file, in general, short of trying to read and
failing to get something back.

In your case you seem to *know* that you have a "plain old file", meaning
that its size is well-defined and that ftell() makes sense for it.  You also
seem to know that you don't have to worry about anyone else, e.g., appending
to it (or in any other way changing its size, or changing your stream's file
position), while you're mucking with it.  So why not just do f.tell() and
compare that to the size yourself?  This sounds easy for you to do, but in
this particular case you enjoy the benefits of a world of assumptions that
aren't true in general.

> ...
> This is where it bites that I can't test for EOF with a read(0).

You can't in std C using an fread of 0 bytes either -- that has no effect on
the end-of-file indicator.  Add

		if (c == 'c') {
			char buf[100];
			size_t i = fread(buf, 1, 0, fp);
			printf("after fread of 0 bytes, feof() says %d\n",
			       feof(fp));
		}

before the "(c == EOF)" test above to try that on your platform.

> ...
> I would be quite surprised if the plain-file case didn't work on Mac
> and Windows.

Don't know about Mac.  On Windows everything is grossly complicated because
of line-end translations in text mode.  Like the C std says, the only
*portable* thing you can do with an ftell() result for a text file is feed
it back unaltered to fseek().  It so happens that on Windows, using MS's
libc, if f.readline() returns "abc\n" for the first line of a native text
file, f.tell() returns 5, reflecting the actual byte offset in the file
(including the \r that .readline() doesn't show you).  So you *can* get away
with comparing f.tell() to the file's size on Windows too (using the MS C
compiler; don't know about others).

the-operational-defn-of-eof-is-the-only-portable-defn-
    there-is-ly y'rs  - tim