Fast File Input

[Skip Montanaro]

    Pádraig> This actually improved a lot with python version 2
    Pádraig> but is still quite slow as you can see here:
    Pádraig> http://www.pixelbeat.org/readline/
    Pádraig> There are a few notes within the python script there.

Your page doesn't mention precisely which version of Python 2 you used.  I
suspect a rather old one (2.0? 2.1?) because of the style of loop you used
to read from sys.stdin.  Eliminating comments, your python2 script was:

    import sys

    while 1:
	line = sys.stdin.readline()
	if line == '':
	    break
        try:
	    print line,
	except:
	    pass

Running that using the CVS version of Python feeding it my machine's
dictionary as input I got this time(1) output (fastest real time of four runs):

    % time python readltst.py < /usr/share/dict/words > /dev/null

    real    0m1.384s
    user    0m1.290s
    sys     0m0.060s

Rewriting it to eliminate the try/except statement (why did you have that
there?) got it to:

    % time python readltst.py < /usr/share/dict/words > /dev/null

    real    0m1.373s
    user    0m1.270s
    sys     0m0.040s

Further rewriting it as the more modern:

    import sys

    for line in sys.stdin:
	print line,

yielded:

    % time python readltst2.py < /usr/share/dict/words > /dev/null

    real    0m0.660s
    user    0m0.600s
    sys     0m0.060s

My guess is that your python2 times are probably at least a factor of 2 too
large if you accept that people will use a recent version of Python in which
file objects are iterators.

Skip