FileInput too slow

wiso gtu2003 at alice.it
Mon Jan 4 17:35:02 EST 2010


I'm trying the fileinput module, and I like it, but I don't understand why 
it's so slow... look:

from time import time
from fileinput import FileInput

file = ['r1_200907.log', 'r1_200908.log', 'r1_200909.log', 'r1_200910.log', 
'r1_200911.log']

def f1():
  n = 0
  for f in file:
    print "new file: %s" % f
    ff = open(f)
    for line in ff:
      n += 1
    ff.close()
  return n

def f2():
  f = FileInput(file)
  for line in f:
    if f.isfirstline(): print "new file: %s" % f.filename()
  return f.lineno()

def f3(): # f2 simpler
  f = FileInput(file)
  for line in f:
    pass
  return f.lineno()


t = time(); f1(); print time()-t # 1.0
t = time(); f2(); print time()-t # 7.0 !!!
t = time(); f3(); print time()-t # 5.5


I'm using text files, there are 2563150 lines in total.



More information about the Python-list mailing list