[Python-ideas] Explicit variable capture list

[Terry Reedy]

On 1/24/2016 7:54 AM, Nick Coghlan wrote:
> On 24 January 2016 at 15:16, Guido van Rossum <guido at python.org> wrote:
>> I wonder if kids today aren't too much in love with local function
>> definitions. :-) There's a reason why executor.submit() takes a
>> function *and arguments*. If you move the function out of the for loop

What I've concluded from this thread is that function definitions (with 
direct use 'def' or 'lambda') do not fit well within loops, though I 
used them there myself.

When delayed function calls are are needed, what belongs within loops is 
packaging of a pre-defined function with one or more arguments within a 
callable.  Instance.method is an elegant syntax for doing so. 
functools.partial(func, args, ...) is a much clumsier generalized 
expression, which requires an import.  Note that 'partial' returns a 
function for delayed execution even when a complete, not partial, set of 
arguments is passed.

A major attempted (and tempting) use for definitions within a loop is 
multiple callbacks for multiple gui widgets, where delayed execution is 
needed.  The three answers to multiple 'why doesn't this work' on both 
python-list and Stackoverflow are multiple definitions with variant 
'default args', a custom make_function function outside the loop called 
multiple times within the loop, and a direct function outside the loop 
called with partial within the loop.  I am going to start using partial 
more.

Making partial a builtin would make it easier to use and more 
attractive.  Even more attractive would be syntax that abbreviates 
delayed calls with pre-bound arguments in the way that inst.meth 
abbreviates a much more complicated expression roughly equivalent to 
"bind(inst.__getattr__('meth'), inst)".

A possibility would be to make {} a delayed and possibly partial call 
operator, in parallel to the current use of () as a immediate and total 
call operator.
   expr{arguments}
would evaluate to a function, whether of type <function> or a special 
class similar to bound methods. The 'arguments' would be anything 
allowed within partial, which I believe is anything allowed in any 
function call.  I chose {} because expr{...} is currently illegal, just 
as expr(arguments) is for anything other than a function call.  On the 
other hand, expr[...] is currently legal, at least up to '[', as is 
expr<...> at least up to '<'.

>> and pass the url as a parameter to submit(), problem solved, and you
>> waste fewer resources on function objects and cells to hold nonlocals.

executor.submit appears to me to be a specialized version of partial, 
with all arguments required.  With the proposal above, I think 
submit(func{all args}) would work.

> Aye, that's how the current example code in the docs handles it -
> there's an up front definition of the page loading function, and then
> the submission to the executor is with a dict comprehension.

I presume you both are referring to ThreadPoolExecutor Example.  The 
load_url function, which I think should be 'get_page' has a comment that 
is wrong (it does not 'report the url') and no docstring.  My suggestion:

# Define an example function for the executor.submit call below.
def get_page(url, timeout):
     "Return the page, as a string, retrieved from the url."
     with ...

> The only thing "wrong" with it is that when reading the code, the
> potentially single-use function is introduced first without any
> context, and it's only later that you get to see what it's for.

A proper comment would fix this I think.  That aside, if the main code 
were packaged within def main, as in the following ProcessPoolExecutor 
Example, so as to delay the lookup of 'load_url' or 'get_page', then the 
two functions definitions could be in *either* order.  The general 
convention in Pythonland seems to be to put main last (bottom up, define 
everything before use), but in a recent python-list thread, at least one 
person, and I think two, said they like to start with def main (top down 
style, which you seem to like).

I just checked and PEP8 seems to be silent on the placement of 'def 
main'.  So unless Guido says otherwise, I would not mind if you revised 
one of the examples to start with def main, just to show that that is a 
legitimate alternative.  It is a feature of Python that one can do this 
without having to add, before the first appearance of a function name 
within a function, a dummy 'forward declaration' giving the function 
signature.

>> A generation ago most people would have naturally used such a solution
>> (since most languages didn't support the alternative :-).
>
> In programming we would have, but I don't think the same is true when
> writing work instructions for other people to follow - for those,
> we're more likely to use nested bullets to describe subtasks, and only
> pull them out to a separate document or section if we need to
> reference the same subtask from multiple places.

People can and do jump around while reading code for understanding. 
They can do this without markers as explicit as needed for machines. 
Current compilers and interpreters initially read code linearly, with 
only one character or token lookahead.  For Python, a def header is 
needed for forward reference, to delay name resolution to call time, 
after the whole file has been read.

> While my view is admittedly only based on intuition rather than hard
> data, it seems to me that when folks are reaching for nested
> functions, it's that "subtask as a nested bulleted list" idiom they're
> aiming to express, and Python is otherwise so accommodating of English
> structural idioms that it's jarring when it doesn't work properly. (I
> also suspect that's why it's a question we keep returning to - as a
> *programming language*, making closures play more nicely with
> iteration variables doesn't add any real power to Python, but as
> *executable pseudo-code*, it makes it a little bit easier to express
> certain ideas in the same way we'd describe them to another person).

I thought about some explicit examples and it is not necessarily clear 
how to translate bullet points to code.  But in general, I do not 
believe that instructions to another person are meant to induce in the 
mind of a listener multiple functions that only differ in a default 
argumnet object.  In other words, I do not see

for i in it:
   def f(i=i): pass

as corresponding to natural language.  Hence my initial statement above.

-- 
Terry Jan Reedy