[Python-Dev] PEP 572: Assignment Expressions

[Chris Angelico]

On Wed, Apr 18, 2018 at 7:53 AM, Terry Reedy <tjreedy at udel.edu> wrote:
> On 4/17/2018 3:46 AM, Chris Angelico wrote:
>
>> Abstract
>> ========
>>
>> This is a proposal for creating a way to assign to names within an
>> expression.
>
>
> I started at -something as this is nice but not necessary.  I migrated to
> +something for the specific, limited proposal you wrote above: expressions
> of the form "name := expression".
>
>> Additionally, the precise scope of comprehensions is adjusted, to maintain
>> consistency and follow expectations.
>
>
> We fiddled with comprehension scopes, and broke some code, in 3.0.  I oppose
> doing so again.  People expect their 3.x code to continue working in future
> versions.  Breaking that expectation should require deprecation for at least
> 2 versions.

The changes here are only to edge and corner cases, other than as they
specifically relate to assignment expressions. The current behaviour
is intended to "do the right thing" according to people's
expectations, and it largely does so; those cases are not changing.
For list comprehensions at global or function scope, the ONLY case
that can change (to my knowledge) is where you reuse a variable name:

[t for t in t.__parameters__ if t not in tvars]

This works in 3.7 but will fail easily and noisily (UnboundLocalError)
with PEP 572. IMO this is a poor way to write a loop, and the fact
that it "happened to work" is on par with code that depended on dict
iteration order in Python 3.2 and earlier. Yes, the implementation is
well defined, but since you can achieve exactly the same thing by
picking a different variable name, it's better to be clear.

Note that the second of the open questions would actually return this
to current behaviour, by importing the name 't' into the local scope.

The biggest semantic change is to the way names are looked up at class
scope. Currently, the behaviour is somewhat bizarre unless you think
in terms of unrolling a loop *as a function*; there is no way to
reference names from the current scope, and you will instead ignore
the surrounding class and "reach out" into the next scope outwards
(probably global scope).

Out of all the code in the stdlib, the *only* one that needed changing
was in Lib/typing.py, where the above comprehension was found. (Not
counting a couple of unit tests whose specific job is to verify this
behaviour.) The potential for breakage is extremely low. Non-zero, but
far lower than the cost of introducing a new keyword, for instance,
which is done without deprecation cycles.

>> Merely introducing a way to assign as an expression
>> would create bizarre edge cases around comprehensions, though, and to
>> avoid
>> the worst of the confusions, we change the definition of comprehensions,
>> causing some edge cases to be interpreted differently, but maintaining the
>> existing behaviour in the majority of situations.
>
>
> If it is really true that introducing 'name := expression' requires such a
> side-effect, then I might oppose it.

It's that comprehensions/genexps are currently bizarre, only people
don't usually notice it because there aren't many ways to recognize
the situation. Introducing assignment expressions will make the
existing weirdnesses more visible.

>> Syntax and semantics
>> ====================
>>
>> In any context where arbitrary Python expressions can be used, a **named
>> expression** can appear. This is of the form ``target := expr`` where
>> ``expr`` is any valid Python expression, and ``target`` is any valid
>> assignment target.
>
>
> This generalization is different from what you said in the abstract and
> rationale.  No rationale is given.  After reading Nick's examination of the
> generalization, and your response, -1.

Without trying it or looking up any reference documentation, can you
tell me whether these statements are legal?

with open(f) as self.file: pass

try: pass
except Exception as self.exc: pass

The rationale for assignment to arbitrary targets is the same as for
assigning to names: it's useful to be able to assign as an expression.

>> Differences from regular assignment statements
>> ----------------------------------------------
>>
>> Most importantly, since ``:=`` is an expression, it can be used in
>> contexts
>> where statements are illegal, including lambda functions and
>> comprehensions.
>>
>> An assignment statement can assign to multiple targets, left-to-right::
>>
>>      x = y = z = 0
>
>
> This is a bad example as there is very seldom a reason to assign multiple
> names, as opposed to multiple targets.  Here is a typical real example.
>
>     self.x = x = expression
>     # Use local x in the rest of the method.
>
> In "x = y = 0", x and y likely represent two *different* concepts
> (variables) that happen to be initialized with the same value.  One could
> instead write "x,y = 0,0".

Personally, if I need to quickly set a bunch of things to zero or
None, I'll use chained assignment. But sure. If you want to, you can
repeat the zero. Don't forget that adding or removing a target then
also requires that you update the tuple, and that it's not a syntax
error to fail to do so.

>> The equivalent assignment expression
>
> should be a syntax error.
>
>> is parsed as separate binary operators,
>
> ':=' is not a binary operator, any more than '=' is, as names, and targets
> in general, are not objects.  Neither fetch and operate on the current
> value, if any, of the name or target.  Therefore neither has an
> 'associativity'.

What would you call it then? I need some sort of word to use.

>> When a class scope is involved, a naive transformation into a function
>> would
>> prevent name lookups (as the function would behave like a method)::
>>
>>      class X:
>>          names = ["Fred", "Barney", "Joe"]
>>          prefix = "> "
>>          prefixed_names = [prefix + name for name in names]
>>
>> With Python 3.7 semantics,
>
> I believe in all of 3.x ..

Probably, but that isn't my point.

>> this will evaluate the outermost iterable at class
>> scope, which will succeed; but it will evaluate everything else in a
>> function::
>>
>>      class X:
>>          names = ["Fred", "Barney", "Joe"]
>>          prefix = "> "
>>          def <listcomp>(iterator):
>>              result = []
>>              for name in iterator:
>>                  result.append(prefix + name)
>>              return result
>>          prefixed_names = <listcomp>(iter(names))
>>
>> The name ``prefix`` is thus searched for at global scope, ignoring the
>> class
>> name.
>
>
> And today it fails.  This has nothing to do with adding name assignment
> expressions.

Fails in what way?

> Bottom line: I suggest rewriting again, as indicated, changing title to
> 'Name Assignment Expressions'.

You're welcome to write a competing proposal :)

I'm much happier promoting a full-featured assignment expression than
something that can only be used in a limited set of situations. Is
there reason to believe that extensions to the := operator might take
it in a different direction? If not, there's very little to lose by
permitting any assignment target, and then letting style guides frown
on it if they like.

ChrisA