Guido's regrets: filter and map

[Jp Calderone]

On Sun, Nov 24, 2002 at 03:40:09PM +0000, maney at pobox.com wrote:
> Robin Becker <robin at jessikat.fsnet.co.uk> wrote:
> > I repeatedly get
> > 
> > for loop took 5.81
> > list comprehension took 8.16
> > map took 4.23
> > 
> > with 2.2.2 win32 on a 1GHz pentium.
> 
> [snip setup, timings]
> 
> ### revised test with filtering
> from time import time
> 
> n = 1000000
> N = xrange(n)
> 
> t0 = time()
> S=[]
> a = S.append
> for x in N:
>     if x % 2 == 1:
>         a(str(x))
> print 'for loop took %.2f to make a list of %d items' % ((time()-t0), len(S))
> 
> del S
> t0 = time()
> S = [str(x) for x in N if x % 2 == 1]
> print 'list comprehension took %.2f to make a list of %d items' % ((time()-t0), len(S))
> 
> del S
> t0 = time()
> S = map(str,filter(lambda x: x % 2 == 1, N))
> print 'map took %.2f to make a list of %d items' % ((time()-t0), len(S))
> ###
> 

  On my P2 Linux machine w/ Python 2.2.2:

for loop took 17.74 to make a list of 500000 items
list comprehension took 17.24 to make a list of 500000 items
map took 12.72 to make a list of 500000 items

However, changing the last filter to:

S = map(str,filter((2).__rmod__, N))

(which is equivalent) improves the final time to 7.74 seconds.  


> Interesting chages when the filter is changed to select every seventh
> item (s/x % 2/x % 7/ in three places):
> 
> maney at wheel2:~$ python sorry.py
> for loop took 2.57 to make a list of 142857 items
> list comprehension took 2.57 to make a list of 142857 items
> map took 2.94 to make a list of 142857 items
> 
> maney at wheel2:~$ python2.2 sorry.py
> for loop took 2.58 to make a list of 142857 items
> list comprehension took 2.46 to make a list of 142857 items
> map took 3.24 to make a list of 142857 items

  Adding this 4th test:

def seventh(n):
    for e in n:
        if e % 7 == 1:
            yield e

del S
t0 = time()
S = map(str, seventh(N))
print 'mapping generator took %.2f to make a list of %d items' % (
    (time()-t0), len(S))

 (and reverting the map/filter, since the trick doesn't work for 7 ;) gave
me...

for loop took 7.48 to make a list of 142857 items
list comprehension took 7.37 to make a list of 142857 items
map took 8.80 to make a list of 142857 items
map took 6.68 to make a list of 142857 items

  showing us once again that function calls are slow, and to be avoided
where ever possible! ;)

  I did notice one other solution that is orders of magnitudes faster than
the above, it involves passing a couple extra parameters to range().  I'll
leave determining exactly what it is as an exercise for the reader...

    Jp

> 
> Okay, that's enough for now.
> -- 
> http://mail.python.org/mailman/listinfo/python-list

-- 
In the days when Sussman was a novice Minsky once came to him as he sat
hacking at the PDP-6. "What are you doing?" asked Minsky. "I am training a
randomly wired neural net to play Tic-Tac-Toe." "Why is the net wired
randomly?" asked Minsky. "I do not want it to have any preconceptions of how
to play." Minsky shut his eyes. "Why do you close your eyes?" Sussman asked
his teacher. "So the room will be empty." At that moment, Sussman was
enlightened.
 --
 2:08am up 22 days, 13:02, 6 users, load average: 0.08, 0.10, 0.09
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 196 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-list/attachments/20021124/5b64d28c/attachment.sig>