[Python-Dev] Don't set local variable in a list comprehension or generator

[Guido van Rossum]

On Wed, May 18, 2011 at 2:34 PM, Victor Stinner
<victor.stinner at haypocalc.com> wrote:
> Le mercredi 18 mai 2011 à 16:19 +0200, Nadeem Vawda a écrit :
>> I'm not sure why you would encounter code like that in the first place.
>
> Well, I found the STORE_FAST/LOAD_FAST "issue" while trying to optimize
> the this module which reimplements rot13 using a dict in Python 3:
>
> d = {}
> for c in (65, 97):
>    for i in range(26):
>        d[chr(i+c)] = chr((i+13) % 26 + c)
>
> I tried:
>
> d = {chr(i+c): chr((i+13) % 26 + c)
>     for i in range(26)
>     for c in (65, 97)}
>
> But it is slower whereas I read somewhere than generators are faster
> than loops.

I'm curious where you read that. The explicit loop should be faster or
equally fast *except* when you can avoid a loop in bytecode by
applying map() to a built-in function. However map() with a lambda is
significantly slower. Maybe what you recall actually (correctly) said
that a comprehension is faster than map+lambda?

> By the way, (c for c in ...) is slower than [c for c
> in ...]. I suppose that a generator is slower because it exits/reenter
> into PyEval_EvalFrameEx() at each step, whereas [c for c ...] uses
> BUILD_LIST in a dummy (but fast) loop.

Did you test this in Python 2 or 3? In 2 the genexpr is definitely
slower than the comprehension; in 3 I'm not sure there's much
difference any more.

> (c for c in ...) and [c for c in ...] is stupid, but I used a simplified
> example to explain the problem. A more realistic example would be:
>
>   squares = (x*x for x in range(10000))
>
> You don't really need the "x" variable, you just want the square.
> Another example is the syntax using a if the filter the data set:
>
>   (x for x in ... if condition(x))
>
>> > I heard about optimization in the AST tree instead of working on the
>> > bytecode. What is the status of this project?
>>
>> Are you referring to issue11549? There was some related discussion [1] on
>> python-dev about six weeks ago, but I haven't seen anything on the topic
>> since then.
>
> Ah yes, it looks to be this issue. I didn't know that there was an
> issue.

Hm, probably.

-- 
--Guido van Rossum (python.org/~guido)