[Python-ideas] lazy tuple unpacking

[Andrew Barnert]

On Tuesday, July 8, 2014 1:31 AM, Todd <toddrjen at gmail.com> wrote:


>Besides the issues others have discussed, another issue I see here is that you are basically copying the iterator.  In the case of this, where "gen_a" is a generator :
>
>>>> a, b, c, *others = gen_a
>
>"others" should be the same as "gen_a" in the end (i.e. "others is gen_a == True").  This seems redundant, especially when we have the itertools "take" recipe which can be used to retrieve the first "n" values of an iterator, which can then be unpacked in whatever way you want.

I don't think the issue he's trying to solve is that others is gen_a, but just that gen_a is not exhausted and copied into a list. If others were a wrapper around gen_a instead, I think that would solve all of the interesting use cases. (But I don't want to put words in Paul Tagliamonte's mouth here, so make sure he confirms it before replying in depth…)

>However, there might be an alternative.  You could have something where, if you are unpacking an iterable to N variables, you can tell it to just unpack the first N values, and the iterable then remains at position N (similar to "take", but integrated more deeply).  For the case of something like a list or tuple, it will just unpack those variables and skip the rest.  Maybe either a method like this:
>
>>>> a, b, c = gen_a.unpack()
>
>Or some sort of syntax to say that the remaining values should be skipped (although I don't really know what syntax would be good here, the syntax I am using here is probably not good):

>
>
>>>> a, b, c, [] = gen_a


I think the obvious way to write this is a bare *:

>>> a, b, *, c, d = range(10)
>>> a, b, c, d
(0, 1, 8, 9)

>Of course with "take" so simple to implement, this is probably way overkill.  I also don't know if it is even possible for the right side of the expression to know how the layout of the left in that way.


There was a thread a few months back about enhancing the unpacking protocol by asking the iterable itself to do the unpacking (possibly feeding it more information about what needs to be unpacked, something like an __unpack__(self, prestar_count, star_flag, poststar_count)), which would allow the flexibility you're looking for.

I don't want to repeat someone else's use cases and arguments out of my own faulty memory; if you're interested in following up, search the python-ideas archive.

Anyway, your explicit method version could be written today. Obviously it wouldn't work on generators or other arbitrary iterators, but you could write a simple wrapper that takes an iterator and returns an iterator with an unpack method, which I think would be enough for experimenting with the feature.

Meanwhile, in the case of a non-iterator iterable, what would you want to happen here? Should others end up as what's left over from the iterator created by iter(iterable)? In other words:

>>> a, b, *others = [1, 2, 3, 4, 5]
>>> others
<list_iterator at 0x12345678>
>>> next(others)
3