map, LC, genexp

[Alex Martelli]

Bryan <belred1 at yahoo.com> wrote:
   ...
> the slowest looping mechanism in 2.4 is 20% faster than the fastest
> looping mechanism in 2.3.  if i'm wrong about this, remembered wrong or
> did something wrong, please let me know.  here are some timeit tests i did
> on my machine.  notice that map is faster in both python versions and that
> LC24/LC23 is only a 5.5% speed improvement.

It may depend on your machine and how well you're optimizing the
building of the various version.  My sweet little laptop is no match for
the general performance of your box, but relative performances are (best
one in each case):

With 2.4:

$ python -mtimeit 'for i in (str(x)for x in xrange(100)):pass'
1000 loops, best of 3: 534 usec per loop

$ python -mtimeit 'for i in [str(x)for x in xrange(100)]:pass'
1000 loops, best of 3: 526 usec per loop

$ python -mtimeit 'for i in map(str,xrange(100)):pass'
1000 loops, best of 3: 400 usec per loop

--> genexp and LC within noise, map of built-in over 20% faster

with 2.3:

$ /usr/bin/python timeit.py 'for i in map(str,xrange(100)):pass'
1000 loops, best of 3: 540 usec per loop

$ /usr/bin/python timeit.py 'for i in[str(x)for x in xrange(100)]:pass'
1000 loops, best of 3: 741 usec per loop

2.3 way slower than 2.4 for same ops, higher map gain wrt LC.


But this is for tiny loops.  Try less-tiny ones:

$ python -mtimeit 'for i in [str(x)for x in xrange(10000)]:pass'
10 loops, best of 3: 59.9 msec per loop

$ python -mtimeit 'for i in (str(x)for x in xrange(10000)):pass'
10 loops, best of 3: 52.8 msec per loop

now difference is NOT 'in the noise': it's repeatably over 12%.

$ python -mtimeit 'for i in map(str, xrange(10000)):pass'
10 loops, best of 3: 45.5 msec per loop

Still faster (with a built-in), but slowing down like the LC, due to
similar allocation issues, while the genexp scales up better.


The genexp will really shine if your loop is huge, so the saving of
memory allocation translates into huge time savings.

But if you're REALLY in a hurry, the fastest approach is another:

$ python -mtimeit -s'from itertools import imap' 'for i in imap(str,
xrange(10000)):pass'
10 loops, best of 3: 39.1 msec per loop

As you see, even for this modest loop itertools' advantage on map is
already comparable to map's on the genexp.  AND imap scales up like the
genexp -- no big memory allocation.


Ratios very similar in 2.3:

$ /usr/bin/python timeit.py -s'from itertools import imap' 'for i in
imap(str, xrange(10000)):pass'
10 loops, best of 3: 4.91e+04 usec per loop

$ /usr/bin/python timeit.py 'for i in map(str, xrange(10000)):pass'
10 loops, best of 3: 5.58e+04 usec per loop

2.4 has an easy 20% gain, but, if you're stuck with 2.3, you can still
get some 15% gain by using itertools rather than map.  Here, too, the
same scaling considerations apply.


It _is_ peculiar that the fastest solution isn't even in the subject;-).
Lesson to retain: *** itertools roolz *** !!!-)


Alex