[Python-ideas] Explicit variable capture list

[Guido van Rossum]

On Sat, Jan 23, 2016 at 7:22 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> [...]
> For a practical example of this, consider the ThreadPoolExecutor
> example from the concurrent.futures docs:
> https://docs.python.org/3/library/concurrent.futures.html#threadpoolexecutor-example
>
> A scope-per-iteration construct makes it much easier to use a closure
> to define the operation submitted to the executor for each URL:
>
>     with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
>         # Start the load operations and mark each future with its URL
>         future_to_site = {}
>         for new site_url in sites_to_load:
>             def load_site():
>                 with urllib.request.urlopen(site_url, timeout=60) as conn:
>                     return conn.read()
>             future_to_site[executor.submit(load_site)] = site_url
>         # Report results as they become available
>         for future in concurrent.futures.as_completed(future_to_site):
>             site_url = future_to_site[future]
>             try:
>                 data = future.result()
>             except Exception as exc:
>                 print('%r generated an exception: %s' % (site_url, exc))
>             else:
>                 print('%r page is %d bytes' % (site_url, len(data)))
>
> If you try to write that code that way today (i.e. without the "new"
> on the first for loop), you'll end up with a race condition between
> the main thread changing the value of "site_url" and the executor
> issuing the URL open request.

I wonder if kids today aren't too much in love with local function
definitions. :-) There's a reason why executor.submit() takes a
function *and arguments*. If you move the function out of the for loop
and pass the url as a parameter to submit(), problem solved, and you
waste fewer resources on function objects and cells to hold nonlocals.
A generation ago most people would have naturally used such a solution
(since most languages didn't support the alternative :-).

-- 
--Guido van Rossum (python.org/~guido)