[Python-ideas] Explicit variable capture list

[Nick Coghlan]

On 23 January 2016 at 14:50, Andrew Barnert via Python-ideas
<python-ideas at python.org> wrote:
> What the thread is ultimately looking for is a solution to the "closures
> capturing loop variables" problem. This problem has been in the official
> programming FAQ[1] for decades, as "Why do lambdas defined in a loop with
> different values all return the same result"?
>
>     powers = [lambda x: x**i for i in range(10)]
>
> This gives you ten functions that all return x**9, which is probably not
> what you wanted.
>
> The reason this is a problem is that Python uses "late binding", which in
> this context means that each of those functions is a closure that captures
> the variable i in a way that looks up the value of i at call time. All ten
> functions capture the same variable, and when you later call them, that
> variable's value is 9.

Thanks for that summary, Andrew.

While I do make some further thoughts below, I'll also note explicitly
that I think the status quo in this area is entirely acceptable, and
we don't actually *need* to change anything. However, there have
already been some new ways of looking at the question that haven't
come up previously, so I think it's a worthwhile discussion, even
though the most likely outcome is still "No change".

> The OP proposed that we should add some syntax, borrowed from C++, to
> function definitions that specifies that some things get captured by value.
> You could instead describe this as early binding the specified names, or as
> not capturing at all, but however you describe it, the idea is pretty
> simple. The obvious way to implement it is to copy the values into the
> function object at function-creation time, then copy them into locals at
> call time--exactly like default parameter values. (Not too surprising,
> because default parameter values are the idiomatic workaround today.)

In an off-list discussion with Andrew, I noted that one reason the
"capture by value" terminology was confusing me was because it made me
think in terms of "pass by reference" and "pass by value" in C/C++,
neither of which is actually relevant to the discussion at hand.
However, he also pointed out that "early binding" vs "late binding"
was also confusing, since the compile-time/definition-time/call-time
distinction in Python is relatively unique, and in many other contexts
"early binding" refers to things that happen at compile time.

As a result (and as Andrew already noted in another email), I'm
currently thinking of the behaviour of nonlocal and global variables
as "capture at call", while the values of default parameters are
"capture at definition". (If "capture" sounds weird, "resolve at call"
and "resolve at definition" also work).

The subtlety of this distinction actually shows up in *two* entries in
the programming FAQ.

Andrew already mentioned the interaction of loops and closures, where
capture-at-call surprises people:
https://docs.python.org/3/faq/programming.html#why-do-lambdas-defined-in-a-loop-with-different-values-all-return-the-same-result
However, there are also mutable default arguments, where it is
capture-at-definition that is often surprising:
https://docs.python.org/3/faq/programming.html#why-are-default-values-shared-between-objects

While nobody's proposing to change the latter, providing an explicit
syntax for "capture at definition" may still have a beneficial side
effect in making it easier to explain the way default arguments are
evaluated and stored on the function object at function definition
time rather than created anew each time the function runs.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia