Pythonification of the asterisk-based collection packing/unpacking syntax

[Eelco]

On Dec 27, 11:57 pm, Steven D'Aprano <steve
+comp.lang.pyt... at pearwood.info> wrote:
> On Mon, 26 Dec 2011 13:41:34 -0800, Eelco wrote:
> > On Dec 25, 6:05 pm, Steven D'Aprano <steve
> > +comp.lang.pyt... at pearwood.info> wrote:
> >> On Sun, 25 Dec 2011 07:38:17 -0800, Eelco wrote:
>
> [...]
>
> >> > How is 'head, *tail = sequence' or semantically entirely
> >> > equivalently, 'head, tail::list = sequence' any different then? Of
> >> > course after interpretation/compilation, what it boils down to is
> >> > that we are constructing a list and binding it to the identifier
> >> > tail, but that is not how it is formulated in python as a language
>
> >> I'm afraid it is.
>
> >> Here's the definition of assignment in Python 3:
> >>http://docs.python.org/py3k/reference/simple_stmts.html#assignment-
> >> statements
>
> > Que?
>
> You have claimed that "constructing a list and binding it to the
> identifier" is not how iterator unpacking is formulated in Python the
> language. But that is wrong. That *is* how iterator unpacking is
> formulated in Python the language. The reference manual goes into detail
> on how assignment is defined in Python. You should read it.
>
> > 'head, *tail = sequence'
>
> > Is how one currently unpacks a head and tail in idiomatic python
>
> > This is semantically equivalent to
>
> > 'head = sequence[0]'
> > 'tail = list(sequence[1:])'
>
> There's a reason this feature is called *iterable* unpacking: it operates
> on any iterable object, not just indexable sequences.
>
> 'head, *tail = sequence' is not just semantically equivalent to, but
> *actually is* implemented as something very close to:
>
> temp = iter(sequence)
> head = next(temp)
> tail = list(temp)
> del temp
>
> Extended iterable unpacking, as in 'head, *middle, tail = sequence' is a
> little more complex, but otherwise the same: it operates using the
> iterator protocol, not indexing. I recommend you read the PEP, if you
> haven't already done so.
>
> http://www.python.org/dev/peps/pep-3132/
>
> > But these forms are linguistically different, in too many different ways
> > to mention.
>
> How about mentioning even one way? Because I have no idea what
> differences you are referring to, or why you think they are important.

The one spans two lines; the other one. Need I go on?

> >> > We dont have
> >> > something of the form 'tail = list_tail(sequence)'.
>
> >> I'm afraid we do. See the definition of assignment again.
>
> > Que?
>
> Again, you make a claim about Python which is contradicted by the
> documented behaviour of the language. You claim that we DON'T have
> something of the form 'tail = list_tail(sequence)', but that is *exactly*
> what we DO have: extracting the tail from an iterator.
>
> Obviously there is no built-in function "list_tail" but we get the same
> effect by just using list() on an iterator after advancing past the first
> item.
>
> > My claim is that the two semantically identical formulations above do
> > not have isomorphic linguistic form. As far as I can make sense of your
> > words, you seem to be disputing this claim, but its a claim as much
> > worth debating as that the sun rises in the east.
>
> "Isomorphic linguistic form"? Are you referring to the idea from
> linguistics that there are analogies between the structure of phonic and
> semantic units? E.g. that the structures (phoneme, syllable, word) and
> (sememe, onomateme, sentence) are analogous.
>
> I don't see why this is relevant, or which units you are referring to --
> the human-language description of what Python does, high-level Python
> language features, Python byte-code, or machine code. Not that it
> matters, but it is unlikely that any of those are isomorphic in the
> linguistic sense, and I am not making any general claim that they are.
>
> If not, I have no idea what you are getting at, except possibly trying to
> hide behind obfuscation.

I wasnt too worried about obfuscation, since I think there is little
left to lose on that front. Let me make a last-ditch effort to explain
though:

When python reads your code, it parses it into a symbolic graph
representation. The two snippets under consideration do not parse into
the same graph, in the same way that insignificant whitespace or
arbitrary identifier names do, for instance. Hence, not linguistically
isomorphic.

The very function of a programming language is to transform this
representation of the code into semantically identical but different
code; (machine code, eventually). You fail to grok the difference
between semantic equivalence and linguistic equivalence. Yes, there
can be a semantic equivalence between iterable unpacking and a syntax
of the form tail_list(sequence). And python does indeed probably apply
a transformation of this kind under the hood (though we couldnt care
less what it does exactly). AND IT PERFORMS THAT TRANSFORMATION USING
THE TYPE CONSTRAINT GIVEN.

Another example: the C code 'float x = 3', how would you interpret
that? Is it 'not a type constraint on x', because eventually your C
compiler turns it into machine code that doesnt leave a trace of that
type constraint anyway? No, its a type constraint, exactly because C
uses it to emit code specific to the type specified.

> >> > Rather, we annotate
> >> > the identifier 'tail' with an attribute that unquestionably
> >> > destinates it to become a list*. It is no longer that 'tail' will
> >> > just take anything that pops out of the expression on the right hand
> >> > side;
>
> >> Of course it will. Python is a dynamically typed language. It doesn't
> >> suddenly develop static types to ensure that 'tail' becomes a list;
> >> 'tail' is bound to a list because that's what the assignment statement
> >> provides.
>
> > How python accomplishes any of this under the hood is entirely
> > immaterial. The form is that of a compile-time type constraint,
> > regardless of whether the BDFL ever thought about it in these terms.
>
> Compile-time type constraints only have meaning in statically-typed
> languages. Otherwise, you are applying an analogy that simply doesn't
> fit, like claiming that the motion of paper airplanes and birds are
> equivalent just because in English we use the word "fly" to describe them
> both.
>
> The differences between what Python actually does and compile-time type-
> constraints are greater than the similarities.
>
> Similarities:
>
> (1) In both cases, 'tail' ends up as a list.
>
> Differences:
>
> (1) Compiler enforces the constraint that the identifier 'tail' is always
> associated with a list and will prevent any attempt to associate 'tail'
> with some value that is not a list. Python does nothing like this: the
> identifier 'tail' has no type restrictions at all, and can be used for
> any object.

After it is rebound, yes. Within the context of the iterable unpacking
statement however, IT IS ALWAYS A LIST. Not only is it always a list,
PYTHON *NEEDS* THIS KNOWLEDGE ABOUT TAIL TO EMIT THE CORRECT CODE
(explicitly or implicitly; thats why I say 'it can be viewed as a type
constraint')

Just because the type constraint only applies within the statement
does not make it any less of a type constraint; just one with
different semantics, from say, C.

> (3) "Constraint" has the conventional meaning of a restriction, not a
> result; a type-constraint refers to a variable being prevented from being
> set to some other type, not that it merely becomes set to that type. For
> example, one doesn't refer to 'x = 1' as a type-constraint merely because
> x gets set to an int value; why do you insist that 'head, *tail =
> sequence' is a type-constraint merely because tail gets set to a list?

Yes, the constraint is unique in this case; that does not make it any
less of a constraint. A formula of the form 'x = 1' is called a
constraint in mathematics. Collections with one element are still
called collections. Do you have a problem with that too?

> >> > rather,
> >> > the semantics of what will go on at right hand side is coerced by the
> >> > constraint placed on 'tail'.
>
> >> But it isn't a constraint placed on 'tail'. It is a consequence of the
> >> definition of assignment in Python 3. 'tail' becomes bound to a list
> >> because that is what the assignment statement is defined to do in that
> >> circumstance, not because the identifier (symbol) 'tail' is constrained
> >> to only accept lists. 'tail' may not even exist before hand, so talking
> >> about constraints on 'tail' is an abuse of language, AS YOU AGREED
> >> ABOVE.
>
> > 'tail' is (re)declared on the spot as a brand-new identifier (type
> > constraint included); whether it exists before has no significance
> > whatsoever, since python allows rebinding of identifiers.
>
> Given the following:
>
> tail = "beautiful red plumage"
> head, *tail = [1, 2, 3, 4]
> tail = 42
>
> is it your option that all three instantiations of the *identifier*
> 'tail' (one in each line) are *different* identifiers? Otherwise, your
> statement about "brand new identifier" is simply wrong.
>
> If you allow that they are the same identifier with three different
> values (and three different types), then this completely destroys your
> argument that there is a constraint on the identifier 'tail'. First
> 'tail' is a string, then it is a list, then it is an int. If that is a
> type constraint (restriction) on 'tail', then constraint has no meaning.
>
> If you state that they are three different identifiers that just happen
> to be identical in every way that can be detected, then you are engaged
> in counting angels dancing on the head of pins. Given this idea that the
> middle 'tail' identifier is a different identifier from the other two,
> then I might accept that there *could* be a type-constraint on middle
> 'tail', at least in some implementation of Python that hasn't yet been
> created. But what a pointless argument to make.
>
> >> > *(I call that a 'type constraint', because that is what it literally
> >> > is;
>
> >> No. It is literally a name binding of a dynamically typed,
> >> unconstrained name to an object which happens to be a list.
>
> > Let me take a step back and reflect on the form of the argument we are
> > having. I claim the object in front of us is a 'cube'. You deny this
> > claim, by countering that it is 'just a particular configuration of
> > atoms'.
>
> No no no, you have utterly misunderstood my argument. I'm not calling it
> a configuration of atoms. I'm calling it a pyramid, and pointing out that
> no matter how many times you state it is a cube, it actually has FOUR
> sides, not six, and NONE of the sides are square, so it can't possibly be
> a cube.

For that to be true, you would have had to explain to me somewhere
what it is you mean by a type constraint, and how this does not fit
the bill. Granted, you did that for the first time in this post; and I
added further detail why I think your particular usage of the term is
too narrow; C does not have a unique claim to the concept. Or
whereever you picked up your use of the term. Like I said, I mean it
in its literal sense (and in the sense that happens to match the first
hit on google; probably no coincidence).

> > 'look at the definition!', you say; 'its just a list of coordinates, no
> > mention of cubes whatsoever.'.
>
> > 'Look at the definition of a cube', I counter. 'This particular list of
> > coordinates happens to fit the definition, whether the BDFT intended to
> > or not'.
>
> But you are wrong. It does not fit the definition of a type-constraint,
> except in the vacuous sense where you declare that there is some
> invisible distinction between the identifier 'tail' in one statement and
> the identical identifier 'tail' in another statement.

Its not vacuous; its essential to the mechanics behind the
transformation of the iterable unpacking statement to its compiled
form. But its meaning does not carry beyond that, no.

> > You are correct, in the sense that I do not disagree that it is a
> > particular configuration of atoms. But if you had a better understanding
> > of cubes, youd realize it meets the definition;
>
> I might not share your deep and expert understanding of cubes, but I
> understand what "type-constraint" means, and I know what Python does in
> iterator unpacking, and I know that there is no sensible definition of
> type-constraint that applies to iterator unpacking in Python.

You seem to know what a type constraint is in a language like C, and
you seem to model your complete understanding from that particular
instance of its use, rather than look at the actual definition of the
term, and seeing where it might apply.