About open file for Read

[Peter Otten]

Dave Angel wrote:

> On 12/10/2012 11:36 AM, moonhkt wrote:
>> Hi All
>>
>> I am new in Python. When using open and then for line in f .
>>
>> Does it read all the data into f object ? or read line by line ?
>>
>>
>>   f=open(file, 'r')
>>            for line in f:
>>               if userstring in line:
>>                  print "file: " + os.path.join(root,file)
>>                  break
>>            f.close()
>>
>>
>> moonhk
> 
> open() does not read the whole file into any object.  There is buffering
> that goes on in the C libraries that open() calls, but that should be
> transparent to you for regular files.
> 
> When you ask for a line, it'll read enough to fulfill that request, and
> maybe some extra that'll get held somewhere in the C runtime library.
> 
> You should look into the 'with' statement, to avoid that f.close().
> That way the file will be closed, regardless of whether you get an
> exception or not.
> 
> http://docs.python.org/2/reference/compound_stmts.html#index-15
> 
>     with open(file,. "r") as f:
>         for line in f:
>              etc.
> 
> BTW, since you're in version 2.x, you should avoid hiding the builtin
> file object.  Call it something like file_name, or infile_name.
> 

Python does a bit of buffering on its own (which is why you cannot mix file 
iteration and .readline() calls):

>>> with open("tmp.txt", "w") as f: f.writelines("%s\n" % i for i in 
range(10**6))
... 
>>> f = open("tmp.txt")
>>> f.readline()
'0\n'
>>> f.tell()
2
>>> f.readline()
'1\n'
>>> f.tell()
4
>>> next(f) # a for-loop does this implicitly
'2\n'
>>> f.tell()
8196 # after a next() call or the first loop iteration
     # part of the file is now in a buffer.
>>> f.readline()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Mixing iteration and read methods would lose data
>>> f.seek(0, 2)
>>> f.tell()
6888890


This is Python 2, in Python 3 f.tell() would fail after a next(f) call, but 
f.readline() continues to work.