[Python-3000] Is this a bug with list comprehensions or not?

[Nick Coghlan]

Raymond Hettinger wrote:
> Would you be open to an alternate implementation for list/set/dict comps
> that doesn't use a genexps?  That would solve the scope problem, the 
> performance hit, and make tracing more straight-forward.
> 
> If you're open to it, I can see if the EuroPython sprinters can find an 
> implementation closer to what we had before but using the compiler to 
> create an invisible induction variable.

Georg and I tried doing it that way and had major problems trying to get 
it to work - the hard part is that the body of the list comp (which may 
include nested list comps, lambda functions and generator expressions) 
needs to be able to see the iteration variables exactly as they are, but 
the surrounding code shouldn't be able to see them at all.

By leveraging off the genexp implementation, the scoping rules we want 
"just work": they are exactly the same as if the list comp was written 
using the equivalent generator expression. [1]

As for whether or not this is worth fixing, I think getting rid of the 
current subtle differences between the scoping rules for list 
comprehensions and generator expressions is a very worthwhile outcome, 
even at the possible performance cost.

For example, why should this work:

class A(object):
    v = 8
    x = [v for i in range(6)]

And these fail:

class A(object):
    v = 8
    x = list(v for i in range(6))

class A(object):
    v = 8
    x = tuple(v for i in range(6))

> Not sure if the current implementation was really a winning trade. It may have been better to have the minor but easy to explain nuisance
> of exposing the induction variable instead of having the harder-to-explain
> subtleties associated with a slower underlying genexp. 

Finding ways to speed up all generator expression invocations would 
strike me as a more worthwhile use of sprint time rather than creating 
complicated special cases in the compiler that would benefit only list 
and set comprehensions.

Cheers,
Nick.

[1] Guido's original OK of the class variable semantic change:
http://mail.python.org/pipermail/python-3000/2007-March/006077.html

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org