unintuitive for-loop behavior

[Gregory Ewing]

Steve D'Aprano wrote:
> No it doesn't mean that at all. The result you see is compatible with *both*
> the "update existing slot" behaviour and "create a new slot" behavior.

We're getting sidetracked talking about slots here.
It's not really relevant. The point is that there is
only *one* binding for x here, and the second
assignment is updating that binding, not creating a
new binding. If it were otherwise, something different
would have been printed.

This is a high-level concept that has nothing to do
with implementation details. It's the *definition*
of what's meant by the terms "new binding" and
"existing binding".

> Objection: I predict that you're going to object that despite the `del x`
> and the assertion, *if* this code is run inside a function, the "x slot"
> actually does still exist. It's not "really" deleted, the interpreter just
> makes sure that the high-level behaviour is the same as if it actually were
> deleted.

Actually, it's the same as if it were *not* deleted.

Interestingly, if you run it inside a function, something different
again happens:

 >>> def f():
...  x = 0
...  f1 = lambda: x
...  del x
...  assert 'x' not in locals()
...  x = 1
...  f2 = lambda: x
...  print(f1(), f2())
...
SyntaxError: can not delete variable 'x' referenced in nested scope

So no, I'm not going to raise that objection. :-)

> In IronPython, you could have the following occur in a function locals, just
> as it could happen CPython for globals:
> 
> - delete the name binding "x"
> - which triggers a dictionary resize
> - bind a value to x again
> - because the dictionary is resized, the new "slot" for x is in a 
>   completely different position of the dictionary to the old one

Which is all completely irrelevant. The important thing is
what's seen by closures created at different times within
that function.

I don't have an IronPython handy to try it, but if you do,
I'd be interested to know the result. (I'm fairly sure it
would still show same-binding behaviour, but you never
know.)

> Your code
> works the same way whether it is executed inside a function or at the
> global top level,

I've never claimed otherwise.

>>It's not currently possible to observe the other behaviour in
>>Python, because the only way to create new bindings for local
>>names is to enter a function.
> 
> Not at all. Delete a name, and the binding for that name is gone. Now assign
> to that name again, and a new binding must be created,

No, you still don't understand what "new binding" means. If it
truly were creating a new binding, the observable behaviour
would be different, *by definition*.

> I'm especially confused because you seem to be arguing that by using an
> implementation which CPython already uses, for-loops would behave
> differently.

No, the implementation I'm talking about is definitely *not*
being used already.

> A thunk is not really well-defined in Python, because it doesn't exist, and
> therefore we don't know what properties it will have.

Just call it a function, then. Here's an example of the
transformation I'm talking about.

Before:

def f():
   funcs = []
   for x in range(3):
     funcs.append(lambda: x)
   for g in funcs:
     print g()

After:

def f():
   funcs = []
   def _body(x):
     funcs.append(lambda: x)
   for _x in range(3):
     _body(_x)
   for g in funcs:
     print g()

> In Python, the most obvious gotcha would be that if for-loops introduced
> their own scope, you would have to declare any other variables in the outer
> scope nonlocal.

I'm not suggesting they should. If the above transformation
were used as an actual implementation, the compiler would
have to introduce implicit 'nonlocal' declarations as needed
to keep the semantics of assignments within the loop body
the same.

>>(Incidentally, this is why some people describe Python's
>>behaviour here as "broken". They ask -- it works perfectly
>>well in these other languages, why is Python different?)
> 
> Define "it". Python works perfectly well too.

So you think that useless behaviour is just as good as
useful behaviour?

> It just works differently from
> what some people expect, especially if they don't think about the meaning
> of what they're doing and want the interpreter to DWIM.

The point is that if they write the analogous thing in
any of those other languages, it *does* DWTM. So I don't
blame anyone for thinking that Python has some explaining
to do.

> "The trick with cells" -- what trick do you mean?

Each time around the loop, if the loop variable is in a
cell, instead of updating the contents of that cell (as
a normal assigment would do), create a new cell.

I don't know how to explain it any more clearly than
that. I can't write it in Python, because Python doesnt
currently have any way to express that operation. I
can't even do it with bytecode, because the bytecodes
that would be needed don't exist.

-- 
Greg