Guido's regrets: filter and map

[maney at pobox.com]

Robin Becker <robin at jessikat.fsnet.co.uk> wrote:
> I repeatedly get
> 
> for loop took 5.81
> list comprehension took 8.16
> map took 4.23
> 
> with 2.2.2 win32 on a 1GHz pentium.

Two runs each on the otherwise idle P3/850, Debian stable.  Default
Python is 2.1.3, other is 2.2.1:

maney at wheel2:~$ python regrets.py
for loop took 7.40
list comprehension took 7.98
map took 4.32

for loop took 7.34
list comprehension took 7.94
map took 4.32

maney at wheel2:~$ python2.2 regrets.py
for loop took 6.91
list comprehension took 7.35
map took 4.35

for loop took 6.91
list comprehension took 7.34
map took 4.35

BTW, your tests include the 'del S' in the timed portion.  Oddly,
moving that to come after the end of the timed period doesn't have any
effect on the for loop time, but reduces the other two tests by about
10%.  I also tried adding a pre-charging construction before the for
loop (I used the map, since it was fastest, of course), but that had no
significant effect on any of the results.  Odd.

Although after seeing a 25% hit from changing the for loop to use the
more natural S.append(...), I am left wondering how much of the speedup
from using map I would ever see in actual practice.  Hmmm...

If we add a fairly trivial filter to the test things the differences
become a great deal less significant:

maney at wheel2:~$ python sorry.py
for loop took 5.22 to make a list of 500000 items
list comprehension took 5.22 to make a list of 500000 items
map took 4.33 to make a list of 500000 items

maney at wheel2:~$ python2.2 sorry.py
for loop took 5.06 to make a list of 500000 items
list comprehension took 4.90 to make a list of 500000 items
map took 4.52 to make a list of 500000 items

The test is a bit contrived, but seems a reasonable proxy for the low
end of complexity of the sort of things I seem most often to be doing
(usually with list comprehension syntax since I've been able to migrate
away from 1.5 on all my active targets):

### revised test with filtering
from time import time

n = 1000000
N = xrange(n)

t0 = time()
S=[]
a = S.append
for x in N:
    if x % 2 == 1:
        a(str(x))
print 'for loop took %.2f to make a list of %d items' % ((time()-t0), len(S))

del S
t0 = time()
S = [str(x) for x in N if x % 2 == 1]
print 'list comprehension took %.2f to make a list of %d items' % ((time()-t0), len(S))

del S
t0 = time()
S = map(str,filter(lambda x: x % 2 == 1, N))
print 'map took %.2f to make a list of %d items' % ((time()-t0), len(S))
###

Interesting chages when the filter is changed to select every seventh
item (s/x % 2/x % 7/ in three places):

maney at wheel2:~$ python sorry.py
for loop took 2.57 to make a list of 142857 items
list comprehension took 2.57 to make a list of 142857 items
map took 2.94 to make a list of 142857 items

maney at wheel2:~$ python2.2 sorry.py
for loop took 2.58 to make a list of 142857 items
list comprehension took 2.46 to make a list of 142857 items
map took 3.24 to make a list of 142857 items

Okay, that's enough for now.