[Python-ideas] Explicit variable capture list

[Nick Coghlan]

On 24 January 2016 at 07:16, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Guido van Rossum wrote:
>>
>> So, I don't really want to introduce "for new x in ..." because it
>> suddenly introduces a completely different concept into the language,
>>
>> What dict hold x in "for new x ..."? It would have to be considered a new
>> dict created just to hold x, but other variables assigned in the body of the
>> for loop would still be in the dict holding all the other locals of the
>> function.
>
> We could say that the body of a "for new" loop is a nested
> scope in which all other referenced variables are implicitly
> declared "nonlocal".

This actually ties into an idea your suggestion prompted: it would
likely suffice if we had a way to request "create a new scope per
iteration" behaviour in for loops and comprehensions, with no implicit
nonlocal behaviour at all.

Consider Guido's spelled out list comprehension equivalent:

    powers = []
    for i in range(10):
        def f(x):
            return x**i
       powers.append(f)

There's no rebinding of values in the current scope there - only
mutation of a list. Container comprehensions and generator expressions
have the same characteristic - no name rebinding occurs in the loop
body, so the default handling of rebinding of names other than the
iteration variables doesn't matter.

Accordingly, a statement like:

    powers = []
    for new i in range(10):
        def f(x):
            return x**i
       powers.append(f)

Could be semantically equivalent to:

    powers = []
    for i in range(10):
        def _for_loop_suite(i=i):
            def f(x):
                return x**i
           powers.append(f)
        _for_loop_suite()
        del _for_loop_suite

Capturing additional values on each iteration would be possible with a
generator expression:

    for new i, a, b, c in (i, a, b, c for i range(10)):
        def f(x):
            return x**i, a, b, c

While nonlocal and global declarations would work the same way they do
in any other nested function.

For a practical example of this, consider the ThreadPoolExecutor
example from the concurrent.futures docs:
https://docs.python.org/3/library/concurrent.futures.html#threadpoolexecutor-example

A scope-per-iteration construct makes it much easier to use a closure
to define the operation submitted to the executor for each URL:

    with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
        # Start the load operations and mark each future with its URL
        future_to_site = {}
        for new site_url in sites_to_load:
            def load_site():
                with urllib.request.urlopen(site_url, timeout=60) as conn:
                    return conn.read()
            future_to_site[executor.submit(load_site)] = site_url
        # Report results as they become available
        for future in concurrent.futures.as_completed(future_to_site):
            site_url = future_to_site[future]
            try:
                data = future.result()
            except Exception as exc:
                print('%r generated an exception: %s' % (site_url, exc))
            else:
                print('%r page is %d bytes' % (site_url, len(data)))

If you try to write that code that way today (i.e. without the "new"
on the first for loop), you'll end up with a race condition between
the main thread changing the value of "site_url" and the executor
issuing the URL open request.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia