[Python-ideas] A real life example of "given"

[Steven D'Aprano]

On Thu, May 31, 2018 at 11:03:56AM -0700, Brendan Barnwell wrote:

> 	What I don't understand is this: if we believe that, then why was 
> comprehension-leaking EVER removed?  Everything that I've seen 
> advocating for this kind of leaking seems to me like it is much more 
> logically consistent with allowing all comprehension variables to leak 
> than it is with the current behavior, in which they don't leak.

Originally list comprehensions ran in the same scope as their 
surroundings. In Python 2:

py> [1 for x in ("spam", "eggs")]
[1, 1]
py> x
'eggs'

but when generator expressions were introduced a few years later, they 
ran in their own sub-local scope. This was, I believe, initially 
introduced to simplify the common situation that you return a generator 
from inside a function:

def factory():
    x = "something big"
    return (x for x in seq)

would require holding onto a closure of the factory locals, including 
the "something big" object. Potentially forever, if the generator 
expression is never iterated over.

(I hope someone will correct me if I have misunderstood.)

To avoid that, the x in the generator was put into a distinct scope from 
the x in factory. Either way, the PEP introducing generator expressions 
makes it clear that it was a deliberate decision:

    The loop variable (if it is a simple variable or a tuple of
    simple variables) is not exposed to the surrounding function.
    This facilitates the implementation and makes typical use
    cases more reliable.

https://www.python.org/dev/peps/pep-0289/


In the case of loop variables, there is an argument from "practicality 
beats purity" that they ought to run in their own scope. Loop variables 
tend to be short, generic names like "i", "x", "obj", all the more 
likely to clash with names in the surrounding scope, and hard to spot 
when they do. They're also likely to be used inside loops:

    for x in [1, 2, 3, 4]:
        alist = [expr for x in range(50)]
        # oops, x has been accidentally overridden (in Python 2)
        
I don't *entirely* buy that argument, and I occasionally find it useful 
to inspect the loop variable of a list comprehension after it has run, 
but this is one windmill I'm not going to tilt against. Reversing the 
decision to put the loop variables in their own sublocal scope is *not* 
part of this PEP.

But this is less likely to be a problem for explicit assignment. Outside 
of toy examples, we're more likely to assign to descriptive names and 
less likely to clash with any surrounding loop variable:

    for book in library:
        text = [content for chapter in books.chapters() 
                        if (content := chapter.get_text(all=True) 
                        and re.match(pattern, content)]

Given an obvious and explicit assignment to "chapter", say, we are more 
likely to realise when we are reusing a name (compared to assigning to 
a loop variable i, say). At least, we should be no more likely to mess 
this up than we are for any other local-level assignment.



-- 
Steve