[BangPypers] Map vs List comprehensions (was Re: parsing xml)

[Anand Balachandran Pillai]

On Mon, Aug 1, 2011 at 7:51 PM, Noufal Ibrahim <noufal at gmail.com> wrote:

> Anand Balachandran Pillai <abpillai at gmail.com> writes:
>
> > On Mon, Aug 1, 2011 at 6:08 AM, Anand Chitipothu <anandology at gmail.com
> >wrote:
>
> [...]
>
> > It is more subtler than that.
> >
> > List comprehensions are faster than map functions when
> > the latter needs to invoke a user-defined function call or a lambda.
> >
> > Maps score over list comprehensions in most cases where the function
> > is a Python built-in and when no lambda is used.
> >
> > Example of former:
> >
> >>>> def f1(): map(sqr, range(1, 100))
> > ...
> >>>> def f2(): [sqr(x) for x in range(1, 100)]
> > ...
> >>>> mytimeit.Timeit(f1)
> > '37.91 usec/pass'
> >>>> mytimeit.Timeit(f2)
> > '37.50 usec/pass'
> >
> > Example of latter:
> >
> >>>> def f1(): map(hex, range(1, 100))
> > ...
> >>>> def f2(): [hex(x) for x in range(1, 100)]
> > ...
> >>>> mytimeit.Timeit(f1)
> > '49.41 usec/pass'
> >>>> mytimeit.Timeit(f2)
> > '55.29 usec/pass'
>
> This is confusing. Why is
>
> map(sqr, range(1, 100))
>
> faster than
>
> map(hex, range(1, 100))
>
> Assuming sqr is implemented in python, it should be slower than hex
> which is implemented in C.
>

 Maybe I should have rephrased it like this.

 - If using anonymous functions, prefer list comps over map.
 - If using built-in functions (C functions), prefer map over list comps.
- If using pythonic user functions, there is nothing much to choose among
   these two, since the difference is  not much.

The reason - List comprehensions perform best, if all the calculations
are inline within the two square brackets. Hence it scores over map
in the lambda use-case but doesn't differ much if the function is
declared outside.

>>> def f1(): [lambda x: x*x for x in range(100)]
...
>>> def f2(): map(lambda x: x*x, range(100))
...
>>> mytimeit.Timeit(f1)
'24.80 usec/pass'
>>> mytimeit.Timeit(f2)
'37.44 usec/pass'

The gain is significant.

Whereas ,

>>> def sqr(x): return x*x
...
>>> def f1(): [sqr(x) for x in range(100)]
...
>>> def f2(): map(sqr, range(100))
...
>>> mytimeit.Timeit(f1)
'37.23 usec/pass'
>>> mytimeit.Timeit(f2)
'36.72 usec/pass'

The gain is minuscule, barely noticeable.

This is generally perceived as a performance trick with Python.

http://www.ardendertat.com/2011/05/30/python-optimization/




>
> [...]
>
>
>
>
> --
> ~noufal
> http://nibrahim.net.in
>
> She used to diet on any kind of food she could lay her hands on. -- Arthur
> Baer, American comic and columnist
> _______________________________________________
> BangPypers mailing list
> BangPypers at python.org
> http://mail.python.org/mailman/listinfo/bangpypers
>



-- 
--Anand