[Python-ideas] Fwd: Fwd: unpacking generalisations for list comprehension

[Steven D'Aprano]

On Thu, Oct 13, 2016 at 11:32:49PM -0400, Random832 wrote:
> On Thu, Oct 13, 2016, at 18:15, Steven D'Aprano wrote:
> > Consider the analogy with f(*t), where t = (a, b, c). We *don't* have:
> > 
> >     f(*t) is equivalent to f(a); f(b); f(c)
> 
> I don't know where this "analogy" is coming from.

I'm explicitly saying that we DON'T have that behaviour with function 
calls. f(*t) is NOT expanded to f(a), f(b), f(c). I even emphasised the 
"don't" part of my sentence above.

And yet, this proposal wants to expand

    [*t for t in iterable]

into the equivalent of:

    result = []
    for t in iterable:
        a, b, c = *t
        result.append(a)
        result.append(b)
        result.append(c)

Three separate calls to append, analogous to three separate calls to 
f(). The point I am making is that this proposed change is *not* 
analogous to the way sequence unpacking works in other contexts. I'm 
sorry if I wasn't clear enough.


[...]
> > Indeed. The reader may be forgiven for thinking that this is yet another 
> > unrelated and arbitrary use of * to join the many other uses:
> 
> How is it arbitrary? 

It is arbitrary because the suggested use of *t in list comprehensions 
has no analogy to the use of *t in other contexts.

As far as I can see, this is not equivalent to the way sequence 
(un)packing works on *either* side of assignment. It's not equivalent to 
the way sequence unpacking works in function calls, or in list displays. 
It's this magical syntax which turns a virtual append() into extend():

    # [t for t in iterable]
    result = []
    for t in iterable:
        result.append(t)

    # but [*t for t in iterable]
    result = []
    for t in iterable:
        result.extend(t)


or, if you prefer, keep the append but magical add an extra for-loop:

    # [*t for t in iterable]
    result = []
    for t in iterable:
        for x in t:
            result.append(x)


> > - mathematical operator;
> > - glob and regex wild-card;
> > - unpacking;
> 
> This is unpacking. It unpacks the results into the destination.

If it were unpacking as it is understood today, with no other changes, 
it would be a no-op. (To be technical, it would convert whatever 
iterable t is into a tuple.) I've covered that in an earlier post: if 
you replace *t with the actual items of t, you DON'T get:

    result = []
    for t in iterable:
        a, b, c = *t  # assuming t has three items, as per above
        result.append(a)
        result.append(b)
        result.append(c)

as desired, but:

    result = []
    for t in iterable:
        a, b, c = *t
        result.append((a, b, c))


which might as well be a no-op.

To make this work, the "unpacking operator" needs to do more than just 
unpack. It has to either change append into extend, or equivalently, add 
an extra for loop into the list comprehension.



> There's a straight line from [*t, *u, *v] to [*x for x in (t, u, v)].
> What's surprising is that it doesn't work now.

I'm not surprised that it doesn't work. I expected that it wouldn't 
work. When I first saw the suggestion, I thought "That can't possibly be 
meaningful, it should be an error."

Honestly Random832, I cannot comprehend how you see this as a 
straightforward obvious extension from existing behaviour. To me, this 
is nothing like the existing behaviour, and it contradicts the way 
sequence unpacking works everywhere else.

I do not understand the reasoning you use to conclude that this is a 
straight-line extension to the current behaviour. Nothing I have seen in 
any of this discussion justifies that claim to me. I don't know what you 
are seeing that I cannot see. My opinion is that you're seeing things 
that aren't there -- I expect that your opinion is that I'm blind.


> I think last month we even had someone who didn't know about 'yield
> from' propose 'yield *x' for exactly this feature. It is intuitive - it
> is a straight-line extension of the unpacking syntax.

Except for all the folks who have repeatedly said that it is 
counter-intuitive, that it is a twisty, unexpected, confusing path from 
the existing behaviour to this proposal.



-- 
Steve