[Python-ideas] Saving state in list/generator comprehension

[Steven D'Aprano]

Andrew Carter wrote:
> Forgive me for any problems in this e-mail as I'm new to this mailing list.
> 
> I thought it might be nice to be able to somehow save a state in
> list/generator comprehensions,
> a side effect of this (although not the intended goal) is it would make
> reduce feasible in a clean manner as the final result would just be the
> state.

reduce already exists; in Python 2, it is a built-in available at all times, 
in Python 3 it has been banished to the functools module.


What is your use-case for this? "Saving state" is a means to an end. The 
beauty of list comprehensions and generator expressions is that they are 
intentionally quite simple and limited. If you need something more complex, 
write a function or generator. Not everything has to be a (very-long and 
unreadable) one-linear.

reduce already exists, but if it didn't, you could write it quite easily. 
Here's a version with optional starting value which yields the intermediate 
results:

import itertools
_MISSING = object()  # sentinel value

def foldl(func, iterable, start=_MISSING):
     # foldr is left as an exercise :-)
     if start is _MISSING:
         it = iter(iterable)
     else:
         it = itertools.chain([start], iterable)
     a = next(it)  # raises if iterable is empty and start not given
     try:
         b = next(it)
     except StopIteration:
         yield a
         return
     a = func(a, b)
     yield a
     for b in it:
         a = func(a, b)
         yield a


Modifying this to return just the last value is easy, and in fact is simpler 
than the above:

def foldl(func, iterable, start=_MISSING):
     if start is _MISSING:
         it = iter(iterable)
     else:
         it = itertools.chain([start], iterable)
     a = next(it)
     for b in it:
         a = func(a, b)
     return a



[...]
> Some of the pros of doing it this way is that because with/as are already
> keywords in python backwards compatibility shouldn't be an issue,

That's not an argument in favour of your request. That's merely the lack of 
one specific argument against it. There are an infinite number of things which 
could be done that won't break backwards compatibility, but that doesn't mean 
we should do them all. What positive arguments in favour of your proposal do 
you have? What does your proposal allow us to do that we can't already do, or 
at least do better?



> p.s. Is there a built-in way to get the last element from a generator
> (perhaps even with a default) a quick google search did not reveal one?

The same as you would get the last element from any iterator, not just 
generators: iterate over it as quickly as possible, keeping only the last 
value seen. Because generator values are generated lazily as needed, there's 
no direct way to skip to the last value, or get random access to them.

In pure Python:

for x in iterator:
     pass

This may be faster:

collections.deque(iterator, maxlen=1)[0]

Of course, both examples assume that the iterator or generator yields at least 
one value, and is not infinite.



-- 
Steven