[Python-ideas] For-loop variable scope: simultaneous possession and ingestion of cake

[Terry Reedy]

Greg Ewing wrote:
> There's been another discussion on c.l.py about
> the problem of

The behavior is a fact.  Calling it a 'problem' is an opinion, one that 
I disagree with.  So to me, your 'solution' is a solution to a non-problem.

>   lst = []
>   for i in range(10):
>     lst.append(lambda: i)
>   for f in lst:
>     print f()
> 
> printing 9 ten times instead of 0 to 9.

If one understands that 'lambda: i' is essentially 'def f(): return i' 
and that code bodies are only executed when called, the behavior is obvious.

> The usual response is to do
> 
>     lst.append(lambda i=i: i)

Here are 5 more alternatives that have the same effect:
(The 3rd is new since my c.l.p response)

lst = []
for i in range(10):
     lst.append(eval("lambda: %d" %i))

lst = []
def f(i): return lambda: i
for i in range(10):
     lst.append(f(i))

lst = []
def f(i):
     lst.append(lambda:i)
for i in range(10):
     f(i)

def populate(n):
   n -= 1
   if n >= 0: return populate(n)+[lambda:n]
   else: return []
lst = populate(10)

def populate(i,n,lst):
     if i < n: return populate(i+1,n,lst+[lambda:i])
     else: return lst
lst = populate(0,10,[])

> but this is not a very satisfying solution.

To you.

> For one thing, it's still abusing default arguments,

Use is a fact, abuse is an opinion.

 > something that lexical scoping was supposed to have removed the
> need for,

Lexical scoping allows reading of variables that vary (get rebound).  In 
2.6/3.0, one can also write this from within the closure.  This addition 
was anticipated from the beginning; it just took a while for Guido to 
decide on the syntax from among the 10-20 proposals.  Modifying 
func.__defaults__ is much more awkward (I somehow thought it to be 
read-only, but in 3.0 it is not.)

 > and it won't work in some situations, such
> as if the function needs to take a variable number of
> arguments.

So use another method.  Are not 5 others enough?

> Also, most other languages which have lexical scoping
> and first-class functions don't seem to suffer from
> problems like this. To someone familiar with one of
> those languages (e.g. Scheme, Haskell) it looks as
> if there's something broken about the way scoping of
> nested functions works in Python.

A respondant on c.l.p pointed out that Python works the same as C and 
Common Lisp.  There are quite a few differences between Scheme/Haskell 
and Python.  I believe neither is as widely known and used as Python.

> So I'd like to propose something that would satisfy
> both requirements:
> 
> 0. There is no change if the loop variable is not
>    referenced by a nested function defined in the loop
>    body. The vast majority of loop code will therefore
>    be completely unaffected.
> 
> 1. If the loop variable is referenced by such a nested
>    function, a new local scope is effectively created
>    to hold each successive value of the loop variable.

As I understand this, you are proposing that

for i in it:
   body

be rewritten as

def _(i):
   body
for i in it:
   _(i)

which is my third alternative above and only takes about 15 additional 
keystrokes, and only those are needed by an anti-default purist. 
Someone who wants this semantic should write it explicitly.

I believe this sort of automagic would make Python even harder to learn 
and understand.  One should be able to learn and use loops and simple 
functions before learning about nested functions and closures.

If the loop is inside a function, as is typical for real code, and the 
loop body rebinds names outside the loop, then automagic addition of 
nonlocal declarations would be needed.

> 2. Upon exiting the loop, the final value of the loop
>    variable is copied into the surrounding scope, for
>    use by code outside the loop body.

My rewrite above does not require this.

[snip]
> The benefit would be that almost all code involving
> loops and nested functions would behave intuitively,

To you, perhaps, but not to all.

> Python would free itself from any remaining perception
> of having broken scope rules,

That is a very idiosyncratic perception.

> and we would finally be
> able to consign the default-argument hack to the garbage
> collector of history.

By ruining the language? Just to save a few keystrokes? No thanks. -1000

(Overblown rhetoric meets overblown rhetoric ;-)

Terry Jan Reedy