Request for Enhancement

[Hans Nowak]

On 30 Aug 00, at 18:46, Samuel A. Falvo II wrote:

> I have need to process very large text files in Python, but I don't have any
> idea how long the files are going to be in real-world situations.  It is
> unfortunate that there is no F.eof() function, where F is a Python file.
> 
> Here's what I *want*:
> 	
> 	while not F.eof():
> 		l = F.readline()
> 		...process line...
> 
> As it is, I have to do the following:
> 
> 	l_list = F.readlines()	#note plural
> 	for line in l_list:
> 		... process line ...
> 
> While this is fine for my test cases, it could consume unacceptable amounts
> of memory when fed large text files.
> 
> Thanks.

There *is* a way to do what you want, by writing your own eof() function 
(or method, if you want to emulate a file by writing a class). However, 
this is considered un-Pythonic (although this is probably more because of 
tradition) and requires a bit of overhead. I found the following in some 
old code; note that I didn't test it:

# 1. compute the size of the file; do this once
pos = f.tell()   # store current position of file pointer
f.seek(0,2)      # go to end of file
size = f.tell()  # get size
f.seek(pos)      # go to original position

Then, at any point you want to check for EOF, compare f.tell() to size; if 
they are the same, EOF is reached.

Assuming this works, you could write your own file object which has an 
eof() method, and which computes the size upon opening.

HTH,

--Hans Nowak (zephyrfalcon at hvision.nl)
You call me a masterless man. You are wrong. I am my own master.
May an orc make you pull over with your pegleg!