python disk i/o speed

[Bengt Richter]

On 9 Aug 2002 14:45:01 -0400, "Karl Vogel" <vogelke at dnaco.net> wrote:
[...]
>##############bench3.py: better python implementation
>
>import sys,xreadlines
>if len(sys.argv)<3:
>    sys.exit("bench input output")
>
>input=open(sys.argv[1])
>output=open(sys.argv[2],"w")
>
>for line in input.xreadlines():
>    part=line.split("\",\"")
>    part0=part[0][1:]
>    part1=part[1]
>    part2=part[2][:-2]
Why not instead
     part1, part2, part3 = line[1:-2].split('","'))
or
     part1, part2, part3 = map(int,line[1:-2].split('","'))) # or give map & int local names first
or maybe even don't bother with part* ( and using local names for map, int, int.__add__ & reduce )
     sum = reduce(int.__add__,map(int,line[1:-2].split('","'))))
>
>    # formatting
>    out = '"%s","%s","%s","%s"\n' % (part0, part1, part2,
>    int(part0)+int(part1)+int(part2),)
and then don't bother re-creating the original line, since you are just appending ', "<sum>"'
     out = '%s, "%s"\n' % (line[:-1], sum)
and then don't bother with 'out'
     output.write('%s, "%s"\n' % (line[:-1], sum))
and then don't bother with 'sum'
     output.write('%s, "%s"\n' % (line[:-1],
       reduce(int.__add__,map(int,line[1:-2].split('","'))))))

>
>    output.write(out)
>
I wonder what this would do. Don't forget to assign local names outside the loop and use
them inside the loop. I.e., recapping (untested(!))

 import sys,xreadlines
 if len(sys.argv)<3:
     sys.exit("bench input output")

 input=open(sys.argv[1])
 output=open(sys.argv[2],"w")
     l_int = int
     l_add = int.__add__
     l_map = map
     l_reduce = reduce
     l_outwrite = output.write
     for line in input.xreadlines():
         l_outwrite('%s, "%s"\n' % (
             line[:-1], # chop \n
             l_reduce(l_add, l_map(l_int, line[1:-2].split('","')))))
         )

Hm, slicing the line twice should be unnecessary overhead:

     for line in input.xreadlines():
         line12 = line[1:-2]
         l_outwrite('"%s", "%s"\n' % (
             line12,
             l_reduce(l_add, l_map(l_int, line12.split('","')))))
         )

Regards,
Bengt Richter